The more we know the less we understand. Nowhere is this more true than on the Social Network, where volume, velocity, volatility and variability are increasing on a daily basis. Those 4 V’s are part of a definition of big data, which includes both structured and unstructured data. We may have a reasonable chance of obtaining valuable information from the structured data population. That depends, of course, on the extremity of any single one or combination of the 4 Vs, yet author, time stamp, location or any other tag that accompanies a communication is easily identifiable. Howerver unstructured data poses a challenge several orders of magnitude greater. Structured data benefits from data models, data definitions and rules that enable us to extract reports and analyses even to the point of discovering new relationships and information from the regimented data. To do so, we need to nurture and maintain these structures, to prevent a degradation of data quality and avoid conflicts and loss, a goal that often eludes the best efforts even in mature IT shops. However this is not the case for unstructured data.
In general there are no data models, no data definitions, no rules and no discipline of housekeeping for unstructured data in Social Media. At least nothing that is commonly held. Individually, of course, we have an idea of what we are communicating, and we probably use both our own data definitions as well as those we assume are being used by others in any conversation; but these are amorphous concepts and certainly nothing that can be referenced by others or by cyber analysis. The same is true to a lesser degree in IT organizations and the worlds behind the firewalls. At least in those environments best practices such as change management and planned organization of unstructured data (viz Sharepoint) should ensure some semblance of control and order if not insights into hidden information.
We do however have some rudimentary tools at our disposal, but like early man our technical bows and arrows are a poor match against the stampeding herd of beasts that is the social network stream. So like our ancient ancestors we have to develop strategies and skills that help us survive and thrive in this world of pervasive communications. Tony Wagner, author of “The Global Achievement Gap” identified three such skills that he believes are fundamental for us to foster and teach. He calls them the “three C’s – critical thinking, effective oral and written communication, and collaboration.” He also believes that this should be the prime focus of our educators, and that we should establish “a new National Education Academy, modeled after our military academies, to raise the status of the profession and to support the R and D that is essential for reinventing teaching, learning and assessment.”
Knowing how to perform the three C’s is therefor one of the keys to success. Being able to put this knowledge into practice, and bring organization and governance to bear on the resources and data requires additional skills if enterprises plan to approach and consume the labor and thoughts of distributed social resources.
Taking these observations a little further I believe the following 5 components are necessary in order to navigate, participate and collaborate in world of social information.
1. Understanding – we need a better understanding of what we are dealing with in the social media so that we can properly distinguish and farm target crops whether they are preferences, demographics, opinions, gossip, information, knowledge, wisdom. or something altogether different. However to improve that comprehension we need to be more aware of the dynamics of how we think, analyze, and communicate effectively. What, for example, is a thought, and what are the attributes of thought that make it consumable? We have a notion of answers to those questions but they are personal and subjective. Yet we cannot rely solely on subjective interpretation, so we need a shared and objective framework or model of knowledge. Knowledge is the loadstone of the social community, and the more we understand it, its nature, behaviors and properties the more we can improve the discovery, sharing and use of valued information in the social stream.
Peter Drucker(1909-2005), one of the most respected commentators on management theory and practice, believed that “knowledge worker productivity” would be the next frontier of management. Drucker was also famous for his quote “If you can’t measure it, you can’t manage it”, to which I would add the following prefix, “f you cant understand it, you can’t measure it.” Building a common understanding and framework(s) for knowledge management is essential in determining meaning, relevancy, relationship or other characteristics of information within contextual and cultural settings. We need to be able to detect when ambiguities and obfuscations are intended and make a documented judgement on meaning when they are not.
2. Networking – it might be stating the obvious to point out that people, individually and collectively, lie at the heart of the global social community. And it stands to reason that knowing who is who, and what they know is another fundamental layer needed for success. The size and complexity of big social data demands a superior set of skills that can identify, analyze, classify and then connect individuals to each other and their knowledge sets. I described this in my previous post Network Weavers which attempted to define the needed attributes (acquisition: filtration/review: association: curation: construction). As the dimensions of the network, the participants and their contributions grow so will the level of skills, and proficient network weavers will become more of a premium resource than they are today. It is likely that networkers will depend on directories, personal or even corporate at first, but increasingly the directories will become more public and entries will contain more social information such as skills, contributions, preferences and factors that others will be able to use to determine relevancy and fit for purpose.
3. Analytics -With improved understanding of knowledge and how we use and abuse it, we can approach analysis with a higher level of confidence in the accuracy of our observations. There are techniques and technologies that attempt to extract meaning from unstructured data but they still fall short of the human computer that is the brain when it comes to analyzing written and visual communications. As with humans machine semantics are bounded by self imposed rules and definitions, and like humans, communication is improved if there is an agreed set between participating bodies. If those rules and definitions remain hidden and obscured then the output can only be regarded as personal opinion. Rating the relevancy or social worthiness of an individual or entity against undisclosed rules and definitions has as much value as the street corner tipster who whispers a sure fire winner for any given horse race. Consequently social media demands semantic definitions that are shared amongst correspondents and a semantic analysis engine with the flexibility to parametrize selected characteristics so that relevancy can be tuned to group or community objectives.
4. Curation – In an earlier post, Curation – In Need of a Cure I raised the need for knowledge workers to approach the care and maintenance of Social Media information in the same way that enterprises manage their data through Information Lifecycle Management. It is not enough just to store knowledge as we do currently with Pinterest, Tumblr, scoop.it and others: beyond catching the item in our personal butterfly net, our efforts resemble little more than childhood scrapbooks of things that caught our interest and appetites. Curation is an excellent term for the housekeeping that needs to be performed on the captured knowledge data. In museums and art galleries curation is a highly sophisticated skill set that seeks to first isolate the item of knowledge, then to expand it with information about its provenance (where it came from) and pedigree (eg what school of thought), augment it with related content (supporting and detracting) and finally exhibit it to educate and edify an interested audience. Curation is an essential component in building a rich and relevant knowledge base, and can and often does lead to new insights and innovations.
5. Collaboration – Unlike “Field of Dreams” you can’t just build a field and expect the games to begin. All the understanding, networking, analyzing and curating will bring but small value if you keep it all to yourself. The key to success lies in participation. The more you contribute, the greater value you generate both for yourself and for your correspondents. The root of the word collaboration is “labor” , meaning work or effort, and the prefix “Co” means sharing. The more you share and contribute the more you will be rewarded by your involvement with the social network. You will be further rewarded as others do the same, whether its contributing common rules and definitions, understanding of knowledge and thought, the names and skills of great social network participants, or exemplary curation of well defined and related content. It is the act of collaboration that provides the secret sauce of success and bridges the resources and knowledge in the social stream. This is not theory: this is proven without any shadow of doubt by the open source community. If you get the opportunity, interact with an open source contributor, and ask them for guidance; they have been doing it effectively, efficiently and profitably for more than a decade.
WARNING: Please don’t attempt any of the steps above without clear and careful planning
- Search is Not Enough: Using Solr for Analytics (architects.dzone.com)
- Examples to help clarify what’s unstructured data and what’s structured? (parasdoshi.com)
- Tackling that unstructured data mess, practically (infocus.emc.com)
- Visualising The Future – New Techniques will revolutionise understanding and interpretation of ‘big data’ (blog.bt.com)