I’m stepping back, for the moment, from the ongoing analyses of taxonomical vs ontological structures (for example), to concentrate on what works and what doesn’t. Further, I speak, as always, from my belief that open/free are not only compatible with commercial viability, but further, that what we should wish to survive and evolve long term requires a marriage of the two.
To explain what I’m seeing, I’ll posit that one of the most vigorous successes on the web to date is still eBay. The site may be tired, with many segments in serious decline, and they may, perhaps, have wandered off base from a business growth perspective, yet the dynamic architecture has no equal or better after an entire decade. I have stated, in talking to web developers, that eBay has superior search, and been told that, of course they do since they’ve millions of dollars to spend on it. This is really an irrelevant fact out of context, since the company in question has some 11,000 employees and a dynamic database of tens of millions of pages today.
The size of the operation may require a sizable investment in maintaining and improving search performance and database management, but I believe that the original information architecture of eBay remains the primary basis for its success. If you’ve never analyzed the site or actively sold there, check out the all categories page, which can be found on the Site Map, which is accessible from any and every location. As recently as 2-3 years ago, all categories were listed (as links) on this one page, with no requirement to drill down further, and without those rather unfriendly search options on top.
Every item you will find in every category there was placed by an individual user. The number of categories has expanded dramatically over the years in direct (more or less) response to user demand, and the company has frequently worked with community groups in specialized collectibles fields towards more effective categorization.
The database structure is strictly hierarchical, with sellers and collectors using the equivalent of tags in listing titles. In practice, a buyer/collector chooses a sub or sub-sub category to search in and usually enters one or more tag equivalent descriptive terms combined with at least one identifying noun. An example is ‘anthropomorphic salt & pepper’. Go ahead and search it and, yes, go figure, there is a strong constituency of avid collectors in this little kitsch niche.
Now, most eBay sellers are the furthest thing from techies or geeks, and I’d also guess that they run the gamut across average in terms of education, erudition, etc. Sure, there were highly motivated early adopters at the beginning, and the profile altered as the constituencies grew, but the important point here is that a typical non-techie user is perfectly capable of navigating hundreds of categories and sub categories and choosing the appropriate ones in order to correctly list the items she’s cleaning out of her closet (to make room for new things, of course). She further goes on to easily use ‘tag equivalents’ to communicate and connect …by putting words such as Pink Cashmere Sweater in her title.
The basic concept of classification is easily grasped by grade schoolers, and indeed many of our children determine by themselves where to find a category they’re interested in, whether in a public library or in a department store, before formal schooling has even begun.
Why then, when we have hundreds of thousands of authoritative bloggers, website operators, etc., is there still so much difficulty connecting billions of pages of content to billions of users?
There is no centralization on the web, no way to collate the endlessly different data structures, written in many different languages, found via the internet. This is, in fact, a naturally resulting state of affairs representing the very basis of open enabling upon which has grown such a vast, multidisciplinary and interestingly vigorous network.
Our need and desire to navigate efficiently, to seek and find effectively, however, does not diminish as the internet expands, and as the web becomes more of a mess, but grows keener in response to both an increase in quality and the overload of quantity.
All of us, from bloggers to researchers to shoppers to the advanced users (who are more likely to be point and click developers rather than virtuoso codeslingers), want more than anything to find and, more recently, to be found. It is this last, the widespread desire to be found, that has taken us across an economic threshold into a realm where communal architecture, I believe, can become a reality.
There are tens of millions of users (at least), who now have a stake in terms of optimizing both a web presence and their computer time.
Up until now, serious search capability has been the mysterious province of wizards such as those at Google. Fairly common familiarity with metatags notwithstanding, the complexity of search algorithms, secretive and constantly changing in a battle against those who would take advantage of and abuse our attention, has made SEO a highly specialized and full time endeavor. Unfortunately, the other side of that coin is that finding what we seek can require almost as much specialized knowledge and experience as it takes to be found.
As a blogger (for example), even if I knew not a single character of basic html, I could still define or select categories and topic tags for my posts far beyond the maddeningly elementary list offered by almost every online blog directory. On most of these lists, one chooses between a handful of categories such as technology, business, academic, women’s issues, etc. As a reader of blogs, I would welcome not only more subcategories within which to search, but also further options such as the ability to search in more than one sub-category at once.
It’s time for a collaborative search engine, and likely a related series. I can easily contribute either a for or non profit business plan and plenty of preliminary ideas for development, but I’m not a coder, and my business interests are tech supported rather than tech based. Are there open source community leaders already at work on such collaborative concepts?
More importantly, how can tech and non-tech online communities intersect more fully towards revealing and achieving this elusive level of connectivity?
I wonder, also, whether further enabling a billion users, eventually, is a sexy enough idea to engage the brainpower that could accomplish it. Although the long term project potential could be both deeply satisfying and related opportunities dazzling, it holds little likelihood of instant gratification.
The competing motivators are strong. Media and money are creating hothouse environments for coders building components such as recommendation engines. Biomedical and nanotech projects lever billions of dollars often sweetened by fantastical premises that capture the science fiction fed imagination. The brilliant coders I have met or observed may not be primarily swayed by dollars, but little can compete with the opportunity to live in that intensely paced environment where non-tech issues are almost non-existent. The more such environments flourish, the less likely it is that the internet will remain a mostly public domain.
The biggest obstacle I see to implementing collaborative organization and search is the simplicity of the primary foundation required. On the surface, the structures would appear rudimentary, and the problem solving of the most annoying variety to elite coders, the variety supplied by civilians. Despite the illusion of simplicity, the successful reality requires brilliant design, in order to achieve stability and flexibility at the same time. I’d also expect it to demand evolution to the sophistication of a multi dimensional structure in the longer term.
The obstacles and de-motivators all beg the question that many believe already answered, or at least inevitably pre-destined. The question of who will exploit, control and benefit most from the internet and our use of it.
