Embracing Chaos

Back in 2007, I predicted that Apple would launch a subscription music service probably around 2010. My logic was based on how long it would take to get enough connected iPods into the world. Having spent a bunch of time with an unconnected mp3 player with a subscription music service I knew this was necessary. I had been using a Sansa mp3 player, which was playing content from Rhapsody’s subscription service. The device was designed to essentially brick itself every 30 days unless you plugged it into a PC. This was necessary to ensure that you were still paying for the music that it had stored, since it couldn’t connect itself. The experience sucked.  Jobs would never let this fly. But now there’s a whole slew of media devices (iPhones, iPod touches, and the new slate) which have their own connection to the outside world and wouldn’t need to be plugged in every month to verify that you’ve paid up.

iSlate is rumored to have a bunch of new content associated with it.  Particularly print content.  Print publishers will probably want consumers to sign up for subscriptions.  So Apple’s probably going to be introducing people to the concept of content subscriptions on their portable devices, likely with iPhone OS 4.0 which probably will run the iSlate and old iPhones and iPod touches too.  So I wouldn’t be surprised if you can get an all-you-can-eat music subscription service available too.  We’ll see.  It’s pure speculation, but it would make sense.  I’d be particularly tickled if my off-the-cuff prediction of dates from 2007 turned out to be right.

An example of the trend towards information democracy is the democratization of culture.  ”Participatory Culture” is the modern trend of many individuals contributing to the mass of popular culture rather than culture being broadcast from a small elite of performers.  By analogy, Hollywood’s hegemony over movies and television represented a communist politburo where a small group had the power and responsibility to control the cultural experiences of the masses.  Today’s information technology is tearing down this monopoly that broadcasters held, and thus democratizing culture through three mechanisms: easier content creation, distribution, and a better editorial process.  We’ll look at each of these three aspects after a brief review of other aspects of the democratization of information.

Broadly, the concept of information democracy is that an increasingly large number of people are able to influence how information is aggregated.  Wikipedia is a clear and simple example of allowing anybody to contribute to what used to be authored by a select few — “The Encyclopedia.”  Google’s Pagerank algorithm democratized web search.  Today’s most successful software is democratizing the feature set by allowing users to vote on how they want to use it.  The general principal is that large numbers of individuals can together make better decisions than any small group.  Applying this principal to culture, we can predict that a cultural democracy will produce “better culture” than what was available before.

Information technology makes it cheaper and easier to both create and to distribute culture.  With the right software, any laptop today has all the power of a professional music or video studio.  Sure the quality won’t be as good without professional inputs (microphones, cameras, etc) but the cheap stuff is good enough for a lot of things.  Obviously the internet makes distribution of this content trivially easy, which is disrupting traditional media businesses.  Easy creation and distribution of cultural content is an important part of creating a cultural democracy, but it is not the critical enabling step.

The key to democratizing culture is in the editorial process.  If everybody is contributing cultural content that is easily distributed, but there’s still a small group deciding which pieces everybody watches, we’re still in a cultural dictatorship.  Enabling the mass public to “vote” on content is the democratizing step.  That enables the collective intelligence of all media consumers to help choose what should become part of mass culture.  So instead of some programming executive trying to guess what will be popular, the question almost becomes moot — whatever is popular becomes popular culture.  Actually making this work is not at all straightforward.  I’ll save a full description of the necessary ingredients for another post, but we can look at a couple examples.

Youtube does this quite well.  It blurs the line between sharing a video clip with your friends and publishing it as a piece of mass culture.  Any video that isn’t marked private is submitted into a kind of massive popularity contest.  Videos that get millions of views are undeniably bits of popular culture.  For music, last.fm does a good job of being inclusive, but hasn’t quite taken off.  When I started building social features into Rhapsody I hoped they could democratize the music editorial process but that hasn’t happened yet.  Like many things in social media there’s a chicken and egg problem with scale which Youtube has clearly gotten past, but music is still struggling with.

Cultural Democracy is “retro”?!

This post is inspired by a recent story by Heather Chaplin that NPR aired describing participatory culture in video games.  The surprising part of the story for me was the assertion that this trend is not modern but in fact “retro.”  The story points out that before analog broadcast media, most culture was participatory — singing, dancing, crafts, etc.  Analog technology created the possibility of cultural hegemonies, and digital technology is breaking them down. A fine point, implying that the 20th century will likely be unique as the only period in human history when popular culture was dictated by an elite group of editors.  Thanks for the interesting tidbit.

Excuse my newbie exuberance, but OMG Rhapsody.com finally launched profile pages!!!  They’ve been up for a while now, which makes me think they’re for real this time.  A couple of you might remember that this feature was live for something like a week in early 2007.  But it was very slow and didn’t live long.  Sniff.

I worked hard to make this feature possible when I was working at Real.  The fact that I couldn’t get it re-launched was a big motivator for me to move on to greener pastures.  I saw making Rhapsody social as an important evolution of the music catalog’s organizational schema.  It’s also an attempt to bring the product into what Tim O’Reilly would call Web 2.0.  Tim’s canonical essay is long-winded, but I really liked how he summarized it in a recent interview on NPR — basically the product gets better as people use it.  The millions of people who use Rhapsody are an asset that has been almost completely unused, except to take their money.  I saw it as a way to take on one of the product’s biggest shortcomings.

Rhapsody has tons of music.  TONS.  Rhapsody almost certainly has something you want to listen to right now, regardless of who you are or what your current mood or situation is.  It’s a strong statement, but there really is that much music.  The problem is figuring out what you want to listen to.  Rhapsody has a great categorical index of music, so if you know you want to listen to D&B or Emo or Vocal Jazz, no problem.  Or if you know specifically the name of something you want to listen, just search for it.  Other than that, you can take the homepage recommendations, browse the catalog manually, or sift through Playlist Central, a dumping ground for unvetted playlists that is a case study in how not to use user-generated-content (UGC) on a website.

Picking good music is difficult.  This is what DJ’s get paid for.  I originally wanted this feature to be called "DJ Pages."  The idea was to give a voice to the small fraction of Rhapsody users who are fanatical about the product.  People who are serious music buffs love Rhapsody, and if given a voice would and still might add tremendous value to the music catalog.  Right now the editorial voice in Rhapsody is controlled by a politburo of paid editors.  They’re really good, but they’re just a handful of hands.  DJ Pages would democratize the music editorial process so anybody with an opinion can contribute.  The social graph becomes the voting process to select who’s worth paying attention to, just like with pagerank.  What Tim calls Web 2.0, I like to refer to the democratization of information.  Partly because it’s fun to call people Communists when they cling to control of information, but mostly because the analogy is apt and helpful.

The Rhapsody team has made an important step in this direction of openness.  I hope they keep running with it.  If you want to see what’s been playing on my Sonos at home, check out my profile page.  But most importantly, I’d like to express my CONGRATULATIONS to everybody who made this possible again and the first time!!!!11!!1

[I wrote this for my excellent class on Open Innovation.  With mere weeks to go until I finish my MBA, I haven't found much time to write original stuff for this blog, so I'm recycling a bit.]

The music recording industry is in trouble.  Disruptive changes in music playback technology have seriously reduced demand for their mainstay business, physical CD sales.  CD sales comprise 80% of the industry’s total revenue, but have dropped sharply in recent years.  Last year sales dropped by 19%, and the channel is in danger of freefall as retailers start to re-allocate store space currently assigned to CDs.  The industry’s hopeful replacement revenue stream, digital downloads, looks like it will only replace a fraction of the loss.  What went wrong?  How did an entire industry fail to keep up with technological innovation?

The recording industry’s value in the economy comes from providing consumers access to great music.  The value chain includes discovering talent, developing the talent to create and record great music, and distribution of that music to consumers.  The early stages of the pipeline have remained about the same for decades.  But technology has permanently changed how music is distributed to consumers.  This fact was driven home to EMI management when a group of teenagers were invited to take as many free CDs as they wanted after participating in a focus group, and they didn’t take a single one!  The recording industry has acted as a manufacturer of physical goods.  But really their business is in licensing Intellectual Property (IP).  When it was inconvenient for consumers to reproduce high-quality recordings the distinction was unimportant.  But today physical distribution of recorded media provides a tiny fraction of the value in the music value chain.

Music IP is legally controlled by copyright.  Digital Rights Management (DRM) technology has been used to enforce licensing agreements on digital recordings files.  Until 2007, the recording industry only sold digital music with DRM, in an attempt to control copyright violations.  The great irony of DRM that has prevented its acceptance by consumers is that by restricting the use of the legally distributed digital music, DRM makes the legal product lower quality than the illegal product.  The lack of consumer incentive to use a lower quality product, combined with the impracticality of enforcing copyright agreements on individual consumers makes the appropriability regime in the distribution of music to consumers very weak.

We can think of innovation in this content space as the creation of compelling new music.  A hot young band with a new album or style of music has an innovation they want to commercialize.  As discussed earlier, the appropriability regime with consumers is quite weak.  The value of the labels’ distribution assets are waning, putting the band in the position of the attacker’s advantage according to Gans’ and Sterns’ innovation framework.  The band should go it alone and seek novel distribution techniques, ignoring the incumbent labels.  The appropriability regime is less clear with respect to incumbent labels – the album itself is well protected by copyright law since the legal recourse is straightforward against a large recording company, but a novel style of music is unprotectable.  So a promising band considering partnering with an incumbent label should consider how easily the value of their art could be expropriated.

The recording industry has focused too long on a part of the value chain that is no longer economically relevant.  They should look to other industries for inspiration as to how to create value in an environment where content and innovation are created more openly.

Many times I’ve been asked about the possibility of Apple offering a subscription music service for iPods and iTunes.  Here I’ll lay out why I think this will happen, what the timeline is for it, how that relates to the future of DRM, and what impact it would have on the competitive landscape.

First off, I am confident Apple will launch a subscription music service.   As every Rhapsody fan and many industry analysts agree, subscription services are the best way to consume music.  Just like Hotmail moved email into the sky, and Google Docs are doing the same for office productivity applications, music can and will go the same way.  Being tied to a specific piece of hardware to enjoy your information services is so 20th century.  The reason we’re not there yet is that it’s not easy to provide a great experience.  And considering people’s long-standing investments in legacy music media like CD’s, non-hosted music services actually provide a smoother transition.

When I worked for Real people generally spoke of Apple launching a subscription service with fear.  I argued that it would actually be one of the best things for the company.  The reason being that even modern electronic music consumers don’t understand what a music subscription service is.  If Apple started spending their quarter-billion dollar per year marketing budget to explain this to consumers, it would do wonders for Rhapsody.  Especially considering the low-quality, poorly-funded advertising campaigns Real has traditionally engaged in.  I wish I could find some of the infomercial-style TV ads they used to run.  Glaser built Real Player without advertising and still believes all internet services should be able to bootstrap themselves.  Maybe the alliance with MTV will help there. 

Also, managing a multi-million song library is not easy.  Rhapsody does a pretty great job of it.  Although they’re going to get obsoleted unless they can figure out how to democratize the music editorial process.  But they’re still way better at it than Apple, who has frankly never been very skilled at online services.  So if Apple were to start spending their huge marketing budget tomorrow to explain why it’s not important to own your own music, it would be a huge boost to Rhapsody.

It won’t happen tomorrow though.  My guess is that within 5 years iTunes will offer all-you-can-eat music for a recurring monthly fee.  The timing depends on a couple of key factors:

• Uptake of network-enabled iPods
• Availability and quality of wireless net access

Before the iPhone, Apple could not launch a subscription music service for one simple reason.  If you stop paying your monthly fee, your subscription tracks need to be disabled from your portable device.  Otherwise somebody could pay the fee for a single month, go on a shopping spree and load up their device with all the music they’ve ever wanted, and never pay another dime.  So even though DRM is going away for track purchases, it has to stick around for subscription models, at least until many other things change.  How does this limit Apple’s ability to launch a subscription service?  As anybody who has used a portable music device with a subscription service can tell you, it is incredibly frustrating to pull your mp3 player off the shelf only to see a message that says it won’t play any of your music because your licenses expired and you need to plug it into a computer to verify that you have been paying your bills.  Even if you are paying, you need to constantly tend to your device or else it bricks itself after a few weeks, by design!  Steve Jobs would never allow his iPods to do this.  The solution is to enable the device to check your subscription entitlement itself — wirelessly, in the background, automatically. 

That’s exactly what the iPhone and iPod touch can do with their built-in networking stacks.  Even a slow network like AT&T’s EDGE network is good enough to verify that the monthly fee has been paid up.  Or for the wifi-only Touch, at least once per month you need to pass by an open hotspot or be in your house where it knows how to connect and it keeps working.  Not a serious burden.

So once there is a sufficiently large installed base of connected iPods, Apple will start selling a subscription service.  If I had more motivation to figure out the timing of when this would happen, I’d look at adoption/saturation curves for iPods and typical turn-over rates for such consumer electronic devices.  Other factors include the financial and market success of competing services.  I leave all this as an exercise to the reader for those of you working in this challenging industry.  My gut says it’ll be in 2010.

I’m glad the music industry is finally allowing legal sales of music online without DRM.  Before this, the situation was absolutely screwball.  Consumers had three choices for getting music onto the electronic devices:

1. Buy the CD and rip it
2. Illegally download it through a peer-to-peer network or sketchy Russian service
3. Buy the DRM’d track legally

The first option sucked because it either involved driving to a brick and mortar store or waiting for somebody else to drive the CD to your house.  There’s no instant gratification.  Then there’s the hassle of converting the CD to electronic format.

The biggest problems with the second option are things like not knowing how to do it.  There’s also some risk of viruses, etc by trawling shady parts of the net as you try to figure it out.  And there’s a minimal risk of the RIAA suing you.  But once you overcome these fears and startup costs, you end up with exactly the product you want.

The third option kinda sucked because modern DRM systems don’t work very well.  I think there’s nothing intrinsically wrong with DRM.  Enforcing copyrights through technical means is a good part of a multi-layer security and incentive system to promote the creation of valuable intellectual property.  But modern DRM systems are incompatible with each other across brands and devices and often they just plain break.  So as a music consumer, if you do the morally correct thing and pay for your music, you end up with an inferior product.

Now with DRM-free track sales, consumers can do the right thing and get the best possible product.  How this fact has been lost on the RIAA for so long amazes me.

This is just one example of why the digital music industry is a horrible one to be in right now.  I wrote a thorough analysis of the industry for my strategy class at school will likely post more of it as time goes on, but I wanted to start here.

At long last, the world’s best digital music system has fixed a glaring UI hole.  With today’s release of v2.5 of their software, Sonos controllers (both hardware remotes and PC/Mac based software) can search for music by artist, composer, album, or track.  This feature works within your own local library or within music services such as Rhapsody. 

Up until now if you wanted to listen to an artist in Rhapsody that you hadn’t previously bookmarked, you would need to guess what top-level genre they were categorized under and then scroll through an enormous list to try to find the artist.  How many times have I scratched my head asking questions like "Is Pink Floyd Rock/Pop or Alternative/Punk?"  Much easier was to find a web browser, pull up http://www.rhapsody.com/pinkfloyd and bookmark music into your library.  That human-writable URL scheme is still one of my favorite accomplishments in the last several years.

I started beta testing this release last week.  As always, the update was fast, easy and works flawlessly.  My biggest complaint is that the search is not interactive.  Considering how fast results typically come back, I would much prefer to have a type-ahead style search where results start to appear as you type.  This would be especially useful considering the somewhat painful scroll-wheel-alphabet typing interface they provide.

Sonos is a great company that makes fabulous products.  They continue to advance the state-of-the-art in digital music systems.  By adding Napster support they have taken another step to commoditize Rhapsody’s music subscription product.  They’ve also released a new product called a ZoneBridge which acts as a WiFi range extender which would address one of my biggest complaints about the system.

A little while ago I heard an interesting story on NPR about collaborative music software.  They described a series of websites that empower geographically separated musicians to create music collaboratively.  Using sites like ejamming, Musicians can find additional band members, share tracks and mix your own tracks with those of your partners across the net.  They even hint at being able to practice with each other live, although I’ve never tried it.

All this reminds me of the story behind the fabulous first album by The Postal Service, Give Up.  For those who don’t know the story, this fabulous album was created by two musicians living in different cities who sent tapes back and forth by mail to create the music. 

Now sites like JamGlue and SpliceMusic make this kind of collaboration possible for anybody musically inclined.  It’ll be fun to hear the first big successes from this new kind of band.  You might even call them a transhuman bands since they’ll using modern technology to overcome human geographic limitations to creating music.

I’ve made some updates to the Rhapsody Greasemonkey Script I mentioned earlier.  The script scans your web pages for the names of the most popular 1,000 or so artists and marks up the page with links the Rhapsody Online for playback.  So anytime you’re reading a web page that’s talking about popular music, the names of the musicians will be hyperlinks that when you click them will let you listen to the artists’ music.

The biggest change from the previous version is that instead of running the regex on the HTML of the doc, it just runs on the text nodes of the DOM.  This fixes the bug that would result in broken half-finished HTML tags in your page if the regex found the name of an artist in a URL or somewhere else in the middle of an HTML tag.  Previously, firefox would also get fairly confused if the script found an artist name in the middle of a link since nested hyperlinks aren’t allowed in HTML for some reason.

If you’d like to try it, you can download and install the new and improved Rhapsody Artist-linker Greasemonkey Script.  (If you haven’t already, you’ll want to install greasemonkey.)  For those of you who don’t have greasemonkey installed, or are still using aaaayyeeee for browsing (why??), here are some examples of what it does.  Here’s a chunk from random friendster profile, before and after applying the artist-linker script:

and after…

And here’s an entertainment news story after getting marked up:

Those are all hyperlinks that when you click on them will start playing key tracks by those artists on Rhapsody online.  Like usual no account is needed for full length high-quality tracks, but available in the US only.

Hope you enjoy it!

After many months of talking and thinking about it, I finally wrote a greasemonkey script to annotate web pages with Rhaplinks.  The script scans web pages looking for the names of musicians and when it finds them, links them to Rhapsody.com so you can listen to music by the named artist.

This simple idea is actually tricky to implement properly.  Rhapsody has a lot of music and a lot of artists.  So many that keeping the entire list in a javascript program is impractical, as is downloading the entire list from the server.  So I took the most popular 50-100 artists in each primary genre and combined them into a single manageable list of about 1,000 names.

This idea is made practical by one of my favorite features of Rhapsody.com — human-writable URLs.  Assuming your browser is set up properly (install plugin, enable pop-ups), opening http://play.rhapsody.com/Morcheeba causes Morcheeba to start playing.   This API (can URL’s be API’s?  I think so!) accepts punctuation too — http://play.rhapsody.com/R.E.M. will play R.E.M.  And thanks to a generous interpretation of the HTTP spec by just about everybody, http://play.rhapsody.com/The Postal Service actually works too.  (Note the technically illegal spaces in the URL.)  What this means is that my script just needs a list of the names of the artists, and doesn’t need corresponding ID values to generate the playback URL.  In fact, you can browse www.rhapsody.com for quite a while before ever seeing a database ID in your address bar.  Which brings me to the interesting part of this post.

Computer Science Interlude

Javascript is a slow, interpreted language.  The straightforward way to write this script would be to loop through a list of artist names, replacing each one in the document.  Something like this:

var artists = ['The Postal Service', 'Morcheeba', 'Massive Attack', 'Madonna', 'Tosca', 'Underworld' ];  // The actual list is much
longer...

for(var i=0; i<artists.length; i++) {
   document.body.innerHTML = document.body.innerHTML.replace( //... some regular expression
   );
}

This script would run very slowly.  To scan an HTML document with N characters for M artist names this way would take O(N*M) time.   Instead I wrote the script in just 2 lines as follows:

var regex = /\b(The Postal Service|Morcheeba|Massive Attack|Madonna|Tosca|Underworld)\b/gi;  // The actual list is much longer...

document.body.innerHTML= document.body.innerHTML.replace(regex,"<a href=\"http://play.rhapsody.com/$1\" title=\"Play $1 on Rhapsody\" >$1<img src='http://www.rhapsody.com/favicon.ico' alt=\"Play $1 on Rhapsody\"/></a>");

This might look like a cop-out — a cheezy easy way to do this.  But it’s actually much faster.  This will run in about O(N) time (assuming N>>M).  The single giant regular expression looks for any of the artist-name-keywords and applies it to the whole HTML document at once.  Firefox’s highly-optimized C++ regular expression engine compiles the big artist list into a single state-machine which is applied to the HTML much faster than anything I could possibly write in javascript.  Regular expression interpreters are brilliantly efficient.  Check out Jeffrey Friedl’s excellent Regular Expressions book if you want to know more about this highly practical topic.  The result is that the script can parse a document for a large number of artist names in a totally tolerable amount of time.  There’s a short delay when the page loads, but it’s still faster than browsing in IE.

Enough Theory.  Let’s get down to practice!

The script isn’t perfect, but it’s pretty neat to use it to browse Myspace or Facebook and have a lot of the music people mention be instantly playable. 

If you’d like to play with it, install Greasemonkey, and then install the Rhapsody Artist Linker script here.

[Update 5/4/07: a new and improved script is available here.  Read about the changes.]