You are currently browsing the category archive for the ‘Technology Tools’ category.

data graphic

First published in Internet Media Labs Blog – 27th October 2012

We are amassing data at an unprecedented rate.  In the course of a day the internet handles more than 1,000 Petabytes of data (2011 figures) and is projected to double in less than three years.  That’s a million terabytes or a billion gigabytes  just on the public internet alone.   Granted there is a lot of duplication and the amount of image and video content is greatly contributing to the accelerated growth. Furthermore our growing dependency on mobility demands even greater participation and production that further magnifies digital traffic.

That is a lot of data and a very large amount of noise carrying a decreasing ratio of signal.  How do we operate in such an environment and meet our objectives for education, career, parenting, healthcare, community participation, consumerism and entertainment? How do we locate and recognize the availability and qualities of resources that will help us live our lives productively and successfully?

A complex question no doubt, but one that highlights the current capabilities and shortcomings of the network today.

The short and most common answer would be search engines.  To a degree that is a reasonable response, but given the immensity of available data it is woefully short of satisfying anything but the last two on my list of objectives (consumerism and entertainment).

The issue starts with search engines and the demands of commercialism.  Commerce sustains our civilization and provides the impetus for innovation and discovery.  But it also dominates the way we create and prepare content, and the way we search for information.  We are also largely dependent on a single search engine, which is still evolving though firmly rooted in textual analysis. Yes there are other search options but the majority of us use Google.

Search technology is beginning to branch out as witnessed by Google’s goal of producing a knowledge graph. Currently it has the ability to determine sentiment which is the first step in semantic analysis.  Yet there is a long way to go before search can provide an accurate return on how, what and who we are searching for.

Google spends a lot of capital on developing and improving search algorithms, which are obscured to prevent gaming the system. Those algorithms perform a large number of calculations that include the analysis and synthesis of web content, structure and performance.

Providers of content and information are aware that they can improve the ranking of their published material by optimizing their web site  through Search Engine Optimization (SEO), Conversion Rate Optimization (CRO) or improving the quality and attractiveness of their content. In addition the search engine vendor(s) provide consulting services to assist content providers in achieving approved “white hat” SEO status as opposed to “black hat” SEO which is risky, unapproved, and has the potential to be banned.

Any search results in an index of entries ranked by how well they have been produced and optimized.  The more content humankind produces the more commercial entities will spend in order to ensure high ranking so that we consume their products or services, after all few consumers go beyond the first page of search results.  Hence my assertion above that consumerism and entertainment (which for sake of argument includes news and events) are the principal beneficiaries of the current solutions. And that’s great if you are catching up on news, wish to be entertained or shopping either actively or casually.  The ranking system will give you the most up to date, the most popular and the most advertised consumables.

However the ranking system doesn’t scale down for the individual, the community or small businesses or enterprises, unless predetermined keywords are used in the content and search.  A small voice cannot be heard where shouting is encouraged even demanded.  The more we use search engines the louder that shouting becomes.  Furthermore the ranking system doesn’t really scale economically for SEO content as globalization will introduce more competition for the coveted top ranked entries, demanding increased effort and optimization.

But this post is not about search engines and optimization of content.  It’s about locating resource and identifying quality and relevancy that will help in collaboration; finding people, ideas, material, skills and availability so the other objectives on my list can be fulfilled.

We need something more than simple signposts or lists, valuable as they are.  We need a capability that will not only locate a resource, but one that will also provide us with much needed information about the resource, its properties, location, status, history and relationships to other resources. In short we need directories, repositories of resources and their attributes that are easily accessible and extensible.

Directory databases have been around for a long time and are currently in operation in most large enterprises,  most commonly behind corporate firewalls.  They meet many of the requirements outlined above, although their use has been necessarily constrained to a management and security function. In most implementations they perform that function well.  That style of directory is also appropriate beyond the firewall, especially when authentication amongst diverse communities and populations needs to be supported.

Yet we can do so much more with directories, especially if we liberate their extensibility and open them up to collaborative contributions and housekeeping.  Today we keep our own lists and collaborate on those in communities of interest. There are several listing applications on Social Media such as list.ly, Twitchimp or the late lamented Formulists.  These are great applications and no social media maven can exist without one.  But they are only lists and they only carry a small number of entries and attributes.

Open collaborative directories will be able to scale to support large numbers of entries and attributes, including attributes that are determined by the participants and their communities. In other words directories will carry the hard facts about a resource as well as attributes that are determined by those who use and collaborate with those resources.

This is very similar to Facebook’s like, (and imaginary don’t like), but applied to the performance or quality of resource as experienced in collaboration.  Such peer review and measurement lies at the heart of Open Source development, a meritocracy where your contributions are evaluated by peers to determine your value and position within the group.  Such information will prove invaluable to those seeking knowledge and the resources to get things done.

And why stop at people? Open Collaborative Directories can support any resource be it curated knowledge bases, dictionaries, almanacs and compendiums.

As long as they are open and accessible they will serve and be served by the communities that need them. Because directory searches have little need for ranking they will be the first port of call for those who want more than the latest news or consumable.

Data image via Tom Woodward in Flickr Creative Commons

Enhanced by Zemanta

Originally posted on Internet Media Labs Blog – September 6th, 2012

Sometimes  it is the little things that are the most useful in life: using a paperclip to retrieve a disc locked in a computer or as emergency back up when the hems on your clothing are in disrepair.

One virtual paperclip that has huge potential for Social Media is versioning.

Versioning, or more accurately, Version Control System (VCS),  is the secret sauce that keeps agile development agile and multi-threaded tasks in synch.  Versioning maintains content and context for any given artifact and is most commonly used in software development – in particular maintaining code bases or code trees.

Versioning is much more than a way to ensure edits and changes are not lost and can be tracked.

Version Control Systems have evolved to enable a protected, searchable environment, allowing individuals to create separate branches and then merge their modifications or augmentations back into the base.  Each version can be searched and reconstructed, providing both stability and maintainability.

The quality of code is improved as bugs can be traced back to the time of their introduction.  Quality can be further improved by including relevant comments and logs, all of which help provide richer history and valuable context when revisions or replacements for the code base are being considered.

While this is all very useful – I would suggest essential – for application development, versioning has even greater potential to support and improve the quality of most, if not all, collaborative projects.

Like the paperclip, VCS can be applied to any creative activity where content changes frequently,  particularly where multiple contributors are involved.  VCS allow contributors to create and evolve their own branches which can then be merged back to become the latest version.  Using a VCS is so much simpler than using “track changes” in an office productivity document, which does not support multiple branches nor keep each saved change.

Reconstruction of office productivity documents case tempts the patience of even the most tolerant of individuals.

Let’s look at a few cases  where the approach would be integral to effective effort and overall success of a collaboration.

Case 1: Collaboration Dictionary:

Standard Definitions and Terms are easy to establish when co-workers are part of a specific group.   Common vocabularies usually develop in most communities, but writing down the words and their definitions is critical to ensuring that there are no ambiguities or misinterpretations.

When co-workers belong to different groups with their own vocabularies, the challenge becomes larger and the value of a dictionary rises.

As the group’s reach continues to expand, so too does the potential for miscommunication and misunderstanding.  Authoring and maintaining the dictionary can  be onerous, especially where it is approached from within a hierarchy, where one group or individual controls the content and holds the sole authority to augment, modify and publish.

Opening up the effort to joint collaboration is both expedient and efficient, providing there is sufficient control to ensure integrity and maintainability.  A version control system will allow co-workers to define their respective  areas of the dictionary, treating each term or collection of terms as a branch of the information base.

The VCS will facilitate the merging of the branches, as well as the ability to roll back to any version should it be required.
Case 2: Risk Assessment

Risk assessment is another key part of planning and demands copious amounts of input, discussion, review and revision.  Similar to the Dictionary case above, risk assessment is relatively easy when performed in a small discrete group.  Again when the scope of the project extends to other groups the complexity and effort required increases factorally.

Collaboration can ameliorate these difficulties,  dependent on good governance and control.  In this case VCS offers a bonus benefit, which is a full context of the discussions and determinations made during the lifetime of the risk that is being assessed.

Before VCS Risk Assessment documents were static and usually represented the final summary of assessment.  But VCS allows that assessment to continue as a living artifact, providing historical context when new events and conditions demand a fresh analysis of the solution and its environment.

Case 3: Curation

I have often stressed the need to treat curation, and especially organizational curation, as a form of Information Lifecycle Management.

Organizational curation means that information is not just a publication, with fresh content for every issue.  Information needs to be cultivated, nurtured, refreshed and made available when and where it may be needed.

Old information never dies, it awaits to inform future consumers of ideas and knowledge.  So content is more than the data presented either visually or verbally, it is augmented by meaning and context, both of which can be accommodated in a versioning approach.

External Collaboration

The cases above are fairly common, but are usually contained within a particular organization or enterprise –  in other words behind the corporate firewall.

Generally. in these cases the individuals, are part of the same organization (at least for the project at hand) and in efficient companies experience a common purpose, culture, and set of standards and policies.

The ever-increasing possibility of external collaboration on projects makes the value of a Version Control System reach the level within Software Development – i.e. Essential.

Moving Version Control to the cloud and enabling a distributed model makes the “essential” desirable.  DVCS (Distributed Version Control System) removes the need for centralized management and the dilemma of either supporting every known platform and stack, or limiting the number of contributors to those that comply with corporate standards.

Distributed Version Control opens the door to wider communities, unrestricted by culture, location or time.

It proves the paperclip that keeps collaborative efforts organized, manageable and crowdsourced.

Photo by Tyler Howarth via Flickr Creative Commons
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Enhanced by Zemanta

Radar identifying objects of interestIn a previous post I presented the challenges involved in deriving value from Big Data and in particular unstructured Big Data, which increasingly dominates the Social sphere.  The tools that will enable us to make sense of the plethora of conversations, contributions and observations are still in their infancy and the likelihood that we will have to rely on human scanning and analysis for the short to mid term appears inevitable.

In his book, “The Shallows“, about the effect of the internet on our brains, author Nicholas Carr  discusses the demands on our working memory or, as he puts it, “our mind’s scratch pad”.  Working memory is our CPU and the agent that identifies and transfers information to our long term memory, which is our data store.  It is therefore a pivotal component in our ability to seek out and retain knowledge.

The information flowing into our working memory at any given moment is called our “cognitive load“.  When the load exceeds our mind’s ability to store and process the information… we are unable to retain the information or to draw connections with the information already stored in our long-term memory”

The internet and particularly the social web is constantly bombarding our working memory with stimuli that are intentionally distracting (this post is a representational example with multiple hypertext links in the first two paragraphs).  The pervasiveness of mobile connectivity means that we are always on-line, and never at rest from the interrupting nature of the medium. This suggests we need assistance in processing the social media stream, firstly in being able to recognize important and relevant information, and secondly to earmark that information for further analysis, refinement or augmentation.  The first requirement is for a social media radar, the second for a social information refinery.

Search remains our favorite tool that we use to seek out information.  Google dominates with 4.7 billion searches a day, but Twitter is not far behind (considering its size) at 1.6 billion, and both services are growing fast (Google at 30% increase per year – Twitter at 50% per year). However there are certain limitations in both search functions evidenced by the changes announced by both companies in the past 12 months. Google has recognized the effects of Search Engine Optimization and the fact that we demand results that are more contemporary.  Both companies have added semantic search elements to their  armory, an acknowledgement that searches need to be relevant to time, location, context and searcher’s intent.  However such elements are far from comprehensive. Furthermore because the definitions, rules and algorithms are unpublished the searcher is dependent on Google’s and Twitter’s interpretation of what was intended.  We are still a code generation or two away from being able to parametrize semantic search using our personal or group definitions and meanings.

Turning now to the need for refinement, which is the ability to analyze what we have found, understand its value and relationship to other captured information, and to provide single or collaborative commentary on the discovery itself.  Once again the technology has provided some rudimentary tools, commonly called curation tools.  Related to museum curation, these tools capture and display information of interest.  There are over 40 such applications and each provides a web page in which curators can display their captured content.  In many cases the tools allow for comments to be added as separate components, listed in historic order with the most recent first.  Some curation tools, such as Pearltrees,  support content linking, allowing curators to provide insight into relationships between islands of information.  Many tools provide a browser add-on that will enable the curator to save browsed/searched content to the curated web site.  The tools are improving but there is still a small disconnect between the radar and the refinery functions.

Until now.  SeeSaw is still a fledgling product yet it offers to bridge that gap between scanning and curation, and holds great promise in being able to map  content relevancy  and provide a lens on both active streams and refined content.  What is particularly appealing about SeeSaw is that it is built for visual scanning. as opposed to lexical scanning.  “See”, the radar component of the tool, filters live social media streams and displays the visual content of links and embedded graphics.  For Twitter this is a vast improvement on current viewing dashboards such as Hootsuite and Tweetchat. where speed reading is essential in keeping up with fast moving activities and events such as chatrooms.  SeeSaw not only displays the images but can also stream the video links within individual messages.

This visual facility has three immediate advantages.  Firstly the participant can remain in the chat stream without having to hyperlink to a new page, which has been a major distraction and an extended opportunity for further interruptions and distractions.  Staying in the same window enables continuous contact with the flow of active conversations.   Secondly it enables the viewer to see trends within the stream, connections (ie who is talking to whom) and tangential conversations.  Thirdly, and most importantly, it enables the participant to save active content in the stream to the Saw side of the product, and the bridge to the product’s curation functionality.  A simple toggle button allows the reader to change between the active stream (See Board) and the curated site (Saw Board). In practice this allows the participant to remain in contact and context with the conversation, as opposed to the multi-windowed, heavy interrupt laden environment to which we have been constrained.  Reflection and analysis can now occur after the scan or chat, ensuring that focus and attention can be appropriately applied to both.

SeeSaw is more than welcome to my active toolkit, it allows me to “Embrace, Extend and Expand” (via EMC, via Microsoft) my social media environment.  It is more than a helpmate – it has the potential to be my primary Social Media Assistant.

The See part of Saw – watching the Social Stream for relevant information

Enhanced by Zemanta
Taking the Plunge into the Social Media Swim

Image: vorakorn kanokpipat / FreeDigitalPhotos.net

It takes about 6 months of immersion and splashing about  in the waters of Social Media  to feel  comfortable and confident enough to not just tread water but make strokes that might lead to reaching one of the other sides.  That is of course if any of the other sides are visible, as the social network is far larger than any public amenity previously encountered by a an order of magnitude.  For those about to jump in for the first time – do not be afraid – the temperature is fine and its predominantly shallow water.  You wont get into difficulties and the worst thing that can happen is a little bit of personal embarrassment, but then nobody is really watching that closely.

During those months of familiarization, learning how to engage with Facebook, posting to your wall, writing up your “resume on steroids” aka Linkedin, pinning your curated content on Pinterest or taking the bolder step of blogging or tweeting you may have wondered a little bit about this medium, the proximity of millions of people all swimming about much the same as you.  What is it?  Rushing through the list of possible descriptors it’s a club, a hangout, its a place to share your thoughts and life with friends , it’s a knowledge pool, it’s the maker and breaker of news. it’s the university of life, the crucible of change even revolution, it’s a marketing paradise.  Finally a reliable source on how commercial brands can track the public success of and reaction to their products and identity.  And that is how it is viewed today.   At a recent Social Media Week event in New York the main topic of conversation was how to use the social network for competitive advantage, with the focus almost exclusively on the marketing advantage.   One of the biggest discussions of the week was who owned Social Media – marketing or PR, and ancillary to that was the often expressed sentiment that CMOs owned Social Media and the strategy around it.  Furthermore that ownership and the technical budget that accompanies it indicates that the CIO should now report to the CMO.

Social Media spoils, Marketing versus PR, or Google versus Facebook or Apple

But before anyone stakes a claim on Social Media, shouldn’t we first understand more about Social Media, what it is, how it is structured (or non-structured) and what are the economics of the model(s), who are the principle players, what are the constituent parts, and what are the appropriate standards and rights that should attend the use and access to this massive data universe.  And the questions do not end here.   Most importantly what is the value of the Social Network to the individual, the private citizen, the corporate or public sector employee, the community, the enterprise, the nation, mankind?  Without doubt it is worth far more than the sum of its parts, but the largest opportunity of all lies in collaboration.  Being able to reach out and share is the fundamental behavior of the social network, and millions upon millions of people do so every day.  They participate, contribute and collaborate freely and willingly, and it is manna from heaven for the marketing profession, who understandably have sharpened their knives to capture the likes and dislikes of the masses.  But if that energy and effort is properly channeled, if tasks can be performed by social network teams, if thoughts and ideas can be evolved and extended to stimulate innovation then marketing will be just the tip of the value iceberg.

In the end no-one can or should own the social network, money will be made from it, reputations will be won or lost, but ultimately the social network benefits and belongs to all of us, to Everyman.  We are all contributors, we are all curators, we are all custodians, and our first task as stewards is to define and describe the infant that is our charge, so that we may nurture and care for its health and growth.  This blog will attempt to start that process, and with a lot of help from others (both a plea and an invitation)  hope to bring some focus and understanding to the ever expanding pool of knowledge and resource that is the social network.

.

Enhanced by Zemanta