By BoLOBOOLNE payday loans

Democratization of Information

Remember what internet search was like back in the pioneer days — say 1998 or 1999?  There were lots of bad ones out there, but I’ll talk about three representative ones.

There were lots of page-search engines along the lines of Alta Vista.  They crawled the web and indexed the contents of each web page.  They would try to figure out which web page best matched your search keywords based entirely on the contents of the pages themselves.  This didn’t work very well since spammers could fill their pages with keywords they liked that didn’t necessarily add any value to you.

Yahoo searched a database that was built manually instead of by crawling.  Yahoo’s staff would catalog thousands of web sites and categorize them according to keywords.  This provided a higher level of quality since a human reviewed every entry, but they were having trouble keeping up with the explosive growth of the net.

Then there was these two punk Stanford kids with their upstart Google.  Google was a lot more like Alta Vista than Yahoo in that they automatically crawled and indexed the entire web.  But they judged which pages were useful not based on what was on the page itself, but on other pages on the net that link to it.

We all know who won.  But I’d like to share a perspective on why that uses a political analogy. Google democratized search.  Yahoo was based on a communist model.  Alta Vista was complete anarchy.  Democracy won because it gives power to the people, and the aggregate opinion of millions of people is almost always better than even a carefully chosen set of experts.

Yahoo’s model was analogous to having a central politburo that makes all decisions.  Provided the politburo is skilled and benevolent, this can be a great solution.  But if the system they control gets too big, it just won’t work.  Alta Vista’s ranking system gave everybody speaking equal say in what happened, which amounted to total anarchy.  Google allowed every web page on the net to cast a vote on which pages were the most important ones.  (Beyond that, the pagerank system iterates so that some votes coming from more important pages count more than other votes — the details of implementation are always key.)  Information democracy is achieved by giving everybody a say in what’s important and aggregating the reults.

There are numerous examples of democratic services completely obsoleteing services based on communist editorial systems.  Wikipedia democratized the encyclopedia and has replaced Encarta.  Youtube democratized internet video clips and replaced iFilm.  In many more cases, the democratic service hasn’t replaced the centrally-controlled services, but provides a strong alternative.  E-Bay democratized shopping.  Blogs have democratized news.  Open-source software has democratized software development.

Clearly democratization isn’t a silver bullet for every problem.  Expertise is much more rare and valuable in some fields than others.  But if your business today is based on having a database that your staff maintains, take note!  Somebody’s probably out there right now figuring out how to build a competing business where anybody in the world can contribute to their database.  And pretty soon they’re gonna be taking pot-shots at your market.  Managing user-generated content is really hard.  Counting votes is really hard.  But if it’s done well, it will dominate any system based on central editorial control.  With the help of computers, groups of people can solve problems far more effectively than individuals can.  This truth will not change.

(I brought this idea up at the Decibel Festival talking about music editorial systems and received a lot of positive feedback about it, so I wanted to post it.  But I must give credit for the vocabulary to my good friend Ramez Naam — I first heard it when he was critiquing a startup‘s business model as being communist.  At first I laughed, but later I appreciated his wisdom.)

  1. […] Google has done a lot to democratize the internet — notably by democratizing search through PageRank which allows anybody to implicitly vote on the relative merit of a web page.  They have also democratized the way some product development choices are made through through their policy of encouraging developers to build whatever they want in 20% of their time.  The result is that everything you can possibly imagine is probably being worked on by at least one googler, and the ideas with merit gain momentum and get built into real services.  But before they get launched to the public they still must be approved by a central authority.  Sure Google does A/B testing like everybody else, which is great for UI tweaks and to verify that new services won’t crash when hit with massive traffic.  But it’s extremely difficult to do A/B testing on major changes to functionality.  For example, it’s hard to imagine testing a change to how g-mail delivers mail through this kind of test.  Moreover, depending on how the test goes, the change is either rolled out to the entire user base or not at all. […]