You are currently browsing the tag archive for the ‘Lists’ tag.

data graphic

First published in Internet Media Labs Blog – 27th October 2012

We are amassing data at an unprecedented rate.  In the course of a day the internet handles more than 1,000 Petabytes of data (2011 figures) and is projected to double in less than three years.  That’s a million terabytes or a billion gigabytes  just on the public internet alone.   Granted there is a lot of duplication and the amount of image and video content is greatly contributing to the accelerated growth. Furthermore our growing dependency on mobility demands even greater participation and production that further magnifies digital traffic.

That is a lot of data and a very large amount of noise carrying a decreasing ratio of signal.  How do we operate in such an environment and meet our objectives for education, career, parenting, healthcare, community participation, consumerism and entertainment? How do we locate and recognize the availability and qualities of resources that will help us live our lives productively and successfully?

A complex question no doubt, but one that highlights the current capabilities and shortcomings of the network today.

The short and most common answer would be search engines.  To a degree that is a reasonable response, but given the immensity of available data it is woefully short of satisfying anything but the last two on my list of objectives (consumerism and entertainment).

The issue starts with search engines and the demands of commercialism.  Commerce sustains our civilization and provides the impetus for innovation and discovery.  But it also dominates the way we create and prepare content, and the way we search for information.  We are also largely dependent on a single search engine, which is still evolving though firmly rooted in textual analysis. Yes there are other search options but the majority of us use Google.

Search technology is beginning to branch out as witnessed by Google’s goal of producing a knowledge graph. Currently it has the ability to determine sentiment which is the first step in semantic analysis.  Yet there is a long way to go before search can provide an accurate return on how, what and who we are searching for.

Google spends a lot of capital on developing and improving search algorithms, which are obscured to prevent gaming the system. Those algorithms perform a large number of calculations that include the analysis and synthesis of web content, structure and performance.

Providers of content and information are aware that they can improve the ranking of their published material by optimizing their web site  through Search Engine Optimization (SEO), Conversion Rate Optimization (CRO) or improving the quality and attractiveness of their content. In addition the search engine vendor(s) provide consulting services to assist content providers in achieving approved “white hat” SEO status as opposed to “black hat” SEO which is risky, unapproved, and has the potential to be banned.

Any search results in an index of entries ranked by how well they have been produced and optimized.  The more content humankind produces the more commercial entities will spend in order to ensure high ranking so that we consume their products or services, after all few consumers go beyond the first page of search results.  Hence my assertion above that consumerism and entertainment (which for sake of argument includes news and events) are the principal beneficiaries of the current solutions. And that’s great if you are catching up on news, wish to be entertained or shopping either actively or casually.  The ranking system will give you the most up to date, the most popular and the most advertised consumables.

However the ranking system doesn’t scale down for the individual, the community or small businesses or enterprises, unless predetermined keywords are used in the content and search.  A small voice cannot be heard where shouting is encouraged even demanded.  The more we use search engines the louder that shouting becomes.  Furthermore the ranking system doesn’t really scale economically for SEO content as globalization will introduce more competition for the coveted top ranked entries, demanding increased effort and optimization.

But this post is not about search engines and optimization of content.  It’s about locating resource and identifying quality and relevancy that will help in collaboration; finding people, ideas, material, skills and availability so the other objectives on my list can be fulfilled.

We need something more than simple signposts or lists, valuable as they are.  We need a capability that will not only locate a resource, but one that will also provide us with much needed information about the resource, its properties, location, status, history and relationships to other resources. In short we need directories, repositories of resources and their attributes that are easily accessible and extensible.

Directory databases have been around for a long time and are currently in operation in most large enterprises,  most commonly behind corporate firewalls.  They meet many of the requirements outlined above, although their use has been necessarily constrained to a management and security function. In most implementations they perform that function well.  That style of directory is also appropriate beyond the firewall, especially when authentication amongst diverse communities and populations needs to be supported.

Yet we can do so much more with directories, especially if we liberate their extensibility and open them up to collaborative contributions and housekeeping.  Today we keep our own lists and collaborate on those in communities of interest. There are several listing applications on Social Media such as list.ly, Twitchimp or the late lamented Formulists.  These are great applications and no social media maven can exist without one.  But they are only lists and they only carry a small number of entries and attributes.

Open collaborative directories will be able to scale to support large numbers of entries and attributes, including attributes that are determined by the participants and their communities. In other words directories will carry the hard facts about a resource as well as attributes that are determined by those who use and collaborate with those resources.

This is very similar to Facebook’s like, (and imaginary don’t like), but applied to the performance or quality of resource as experienced in collaboration.  Such peer review and measurement lies at the heart of Open Source development, a meritocracy where your contributions are evaluated by peers to determine your value and position within the group.  Such information will prove invaluable to those seeking knowledge and the resources to get things done.

And why stop at people? Open Collaborative Directories can support any resource be it curated knowledge bases, dictionaries, almanacs and compendiums.

As long as they are open and accessible they will serve and be served by the communities that need them. Because directory searches have little need for ranking they will be the first port of call for those who want more than the latest news or consumable.

Data image via Tom Woodward in Flickr Creative Commons

Enhanced by Zemanta

Twitter Updates