Here’s how that math works, Kurzweil explains: The design of the brain is in the genome. The human genome has three billion base pairs or six billion bits, which is about 800 million bytes before compression, he says. Eliminating redundancies and applying loss-less compression, that information can be compressed into about 50 million bytes, according to Kurzweil.

About half of that is the brain, which comes down to 25 million bytes, or a million lines of code.

This reasoning is IMHO flawed and overly optimistic.  It’s an interesting idea to compare the complexity of these two systems by comparing their bit representations.  I think the idea has merit, at a very rough level — that is I think you can compare the complexity of a genome to the complexity of a piece of software on a rough order-of-magnitude scale. The biggest flaw in Kurzweil’s argument is that he magically throws in a factor of 16x improvement in his favor by saying the genome can be “compressed.”  Well, software executables can be compressed too, a fact that Kurzweil conveniently ignores.  So I’d follow his reasoning to say that a human brain simulator probably needs about 10 – 100 million lines of code.  (I’m deliberately including 0 significant digits here to indicate the roughness of this approximation.)  This puts a human brain simulator on par with the some of world’s most sophisticated software projects so far, which seems about right, at least to an order of magnitude or so.

Strong reactions

PZ Myers published a wrathful condemnation of Kurzweil’s argument titled “Ray Kurzweil does not understand the brain.”  If you sift through the name-calling you see that Myers assumes a specific tactic in building the brain simulator: starting with the human genome and deriving the brain’s functionality from it.  This strategy will certainly work, once we have solved the protein-folding problem, and more generally have the ability to do quantum chemical simulations of kilogram-sized masses of organic chemicals.  Which is to say it’s theoretically possible (we might be living in a software simulation of our universe for all we know), but completely intractable with current technology.  For comparison, our best quantum chemical simulations if you push them top out at maybe a dozen atoms right now.  So being able to simulate an entire kilogram of organic matter is nowhere in sight.

Tactics to simulation

I agree with Myers that we are nowhere near being able to interpret the genome well enough to understand how it makes a brain.  But we probably don’t need to in order to simulate a brain.  By analogy, consider the Super Nintendo (SNES) Emulator, which is another kind of simulator many of us have experience with.

SNES emulators let you play all the old Nintendo games but on a modern computer instead of original SNES hardware.  Let’s say somebody handed you a box and a stack of cartridges and told you to build a Nintendo simulator.  What would you do?  Well, clearly you could open up the SNES box and reverse engineer the circuit boards to figure out all the wiring.  You’d probably figure out that the CPU was important — a variant on the 65816, which was essentially the 16-bit version of the 6502 some of us grew up with in our Commodore 64s and Atari 800s.  So you could (theoretically) crack open the 65816 CPU chip itself, put it the through an electron microscope and understand every transistor it used to interpret the instructions. In this way you could reliably create an emulator which completely replicated every aspect of the SNES. Such a simulation would replicate all of its bugs, timing quirks and everything, but it would work and be extremely expensive to simulate.

This is analogous to the tactic PZ Myers seems to be assuming Kurzweil would take to simulating a human brain. But Kurzweil would actually start at a much higher level of abstraction.  Simulating every protein in every neuron is like building an SNES emulator by simulating every transistor in the original Nintendo’s hardware. The key to getting those SNES games to work does not lie in replicating the design of the CPU which interprets the instructions.  The key is figuring out how to run those instructions on modern hardware.  By moving up through levels of abstraction, we can simulate the system much more cheaply and easily, although there’s a chance edge-case behavior won’t be captured properly.  (What if our world is a simulation and we bump into the edge-cases?)

Similarly, the key to simulating a human brain anytime soon does not lie in understanding every chemical pathway in human neurons.  Although if we did understand neurons at this level, we would have a great head start at simulating a brain.  Success in simulating a human brain will come by recognizing higher levels of abstraction in neuronal function.  We have known for a very long time that neurons communicate by “firing” electrical signals which are transmitted chemically at synapses.  The details of these behaviors are complex and determined by a great many interdependent chemical systems, but it seems highly likely that we can replicate the key behaviors of human neurons at this level of abstraction without needing to understand everything underneath supporting them.  If we can replicate the firing behavior of neurons in sufficient detail, we don’t care what the proteins underlying them are doing.  The key question here of course is what is “sufficient detail.”  I expect that question is one that researchers who are genuinely interested in reverse-engineering the brain will actually focus their attention on.

Once we can simulate the firing behavior of neurons, simulating a brain becomes much more of an engineering problem than a scientific one.  Still it’s going to be a massive engineering challenge, and gathering the input data will probably require a bunch of new science.  Then the philosophers can debate the meaning of free-will if our brains are Turing-complete.

To get across the chasm, your technology really needs to work well.  Apple seems to have done that with FaceTime.  But there’s more than just having it work — you also need to explain to the public why it’s important.  Here, Apple is just paying for that with traditional advertising.  They’re putting out lots of touching, heartwarming commercials showing the value of video chat.  Apple is spending millions of dollars to explain to people why they’ll like video chat. Primary demand stimulation.  They’re working to overcome people’s biases against  the technology or the idea — that it’s clunky, or the extremely common “I don’t want to see the people I’m talking to” reaction, which is really pretty funny when you think about it.  Of course, there will be some times when you will prefer audio only, but that’s going to be the exception when the technology is good enough.

It’s all playing out in a textbook fashion.  The geeks are all crying that there’s nothing new here, that this technology has been around forever, and they don’t understand why it’s such a big deal.  But it is a big deal, because video chat is finally entering the mainstream.

Some side-effects of this that I pointed out before, but are perhaps more clear now as the story unfolds, are that this will benefit existing established video chat vendors.  Apple is explaining to people that video chat matters.  This will help Skype and Google and Cisco with their products.

Speaking of Cisco, there’s another prediction coming true: Cisco is pushing into consumer video chat.  I had guessed 2012, but barely more than a week after my last post, they announce Cius, a video chat terminal.  Kinda like a Flip phone fused with a linksys router, but running Android and in a pretty nice looking case.

Video chat is in a similar place today to where mp3 players were 10 years ago.  There are lots of video chat solutions out there on the market.  Skype is the most well known.  I helped launch Google’s video chat system across Gmaill, iGoogle and Orkut during my tenure there.  It definitely is one of the best on the market, and it’s still only appealing to early adopters.  I mean “early adopters” in the classic Everett Rogers sense, which is to say folks for whom the extra value of the new technology outweighs the hassles of using it.  This is a step beyond the “innovators” category, who are willing to bend over backwards debugging a brand new product just because they understand that it will be important later.  Video chat has been available to innovators for a great many decades.

With iPhone 4, Apple will push video chat to the early majority category. Apple has a history of sitting on potential technologies until all the bugs are worked out, which is fundamentally what’s needed to appeal to more than just early adopters.  I’m pretty sure FaceTime will be no exception.  In 6 months, video chat won’t be this geeky thing that people put up with out of desperation.  It will begin to be integrated into normal culture.  We’ll start to see television dramas and movies incorporate it as just a way some people communicate, rather than as a way to demonstrate how high-tech somebody is.  People who aren’t geeks will start to use video chat.

Video chat really matters

Those of us who have lived deeply with video chat understand its value.  There is a ton of additional content transmitted in video that helps communication on many levels.  It allows for a more nuanced informational discussion, but more importantly IMHO, it allows distant communication to be much more personal and emotional.  Anybody who has tried professional collaboration with another team that is thousands of miles away knows that this level of communication is at least as important for business uses as it is for social communication.  The first time you meet your collaborators in person, they become more real, more trustworthy, easier to talk to, especially about difficult subjects like problems that might arise in a project *gasp*.  Video chat is certainly not as good as meeting people in person, but it is a huge step above email, IM or phone.  (Getting drunk together I believe represents the highest professionally-accepted level of humanization.)

Human-to-human communication has always been the killer feature of computer technology. Video chat makes synchronous communication fundamentally better, and as such will become a major part of everybody’s life in the developed world in the years to come.

What does this mean for everybody else?

The history of technology innovation tells us that a couple things typically happen when an emerging technology pushes into the majority segments.  First, established players will all get a boost.  Apple will be doing a huge favor to Skype and Google Video Chat by removing the veil of geekiness from their products.  Apple’s huge investment in making this product work well will make all consumers more willing to try alternatives.

Another common side-effect is that the space will get more difficult for new entrants.  This usually happens as the technology standardizes.  There becomes a “normal way of doing things” that people start “to get.”  Before a technology can reach the majority, it will typically bounce around dozens of different modalities as everybody tries to find a way of doing it that resonates with the market.  This uncertainty represents a clear opportunity for start-ups and the subsequent standardization is the closing of that opportunity.  Another reason the space usually gets harder for startups is that economies of scale start to kick in as production levels ramp up to meet the larger demand.  This naturally favors large companies, since it raises the amount of investment needed to compete.

Which Social Graph?

Another reason the space might get harder for newcomers is the natural monopoly of social graphs — consumers are better off if there is a single definitive place to keep track of their contacts rather than having to replicate and maintain a different list for each service.  As such, social graphs are important assets to anybody in this space.  But if Tim O’Reilly gets his way (which I hope he does) and we end up with a loosely-coupled internet OS, this won’t be a problem for startups, as they’ll just be able to draw from an openly available graph, say from a Google or Facebook.

I’m really not sure how this aspect will play out.  My guess is that Apple will rely on the de-centralized social graph which is the contact list built into every iPhone.  It’s a less useful corporate asset than if it were properly cloud-hosted, which will make it harder for them to expand the service to OS X machines.  Perhaps they’ll make something useful out of mobileme here, but I have my doubts.  But given the revenue they get from App Store sales, it’s not clear that the OS X machine is even a major part of their consumer strategy going forwards.  If so, this would likely be a strategic shift for them, as the inclusion of web-cams on essentially every OS X machine for years was probably done in anticipation of making a major push into video chat at some point.

Don’t forget Cisco

In addition to the obvious players like Google and Skype, this is also incredibly important for Cisco too.  Cisco has long been interested in video chat.  Why?  The same underlying reason Intel has been investing in multi-media since the late 1980’s.  Multimedia on PCs needs lots of CPU power, and video chat needs lots of bandwidth.  It’s called “primary demand stimulation.”  To be very clear: Cisco wants everybody to use video chat because video chat uses lots of bandwidth, and when people are using lots of bandwidth, Cisco sells more big routers.

Cisco is in the final phases of buying Tandberg, who is the biggest supplier of video-chat hardware for businesses.  Their aquisition of Flip last year seems strategically odd in isolation, but in this context makes perfect sense.  They are building (buying) expertise in consumer-electronics which can handle high quality video.  Take a Flip camera, add a network (like a linksys wifi box) and you’ve got a video chat terminal.  I predict we’ll see such a toy out of Cisco in about 2012, as video chat fills the early majority segment and edges against the late majority.

I set about answering this question with data.   I quickly found a list of oil spills on wikipedia.  A quick pass through google spreadsheets and a few regexs later, and I’ve got the data in a form that it can be graphed with protovis, a wonderful web-based visualization package.  An initial look shows some interesting trends. (Sorry IE users – modern browser required.)

First, it’s important to emphasize that this is a log-scale graph.  Given the dynamic range of the input, it’s the only reasonable way to visualize what’s there, but if you’re not used to reading log-scale graphs, the data will be deceptive.  In short, being a little higher on the graph means that the spill is a lot larger.  In fact if would be very reasonable to only include the spills near the top of the graph when thinking about “big spills.”  But I wanted to present the entire data set for completeness and analysis.

It’s interesting to note the general downward trend at the bottom of the graph.  I believe this is not a real effect at all, but a result of selective memory.  The smallest spill on this graph was only a couple months ago — the Great Barrier Reef spill in April.  Are we to believe that in the preceeding 100 years of oil exploration there had never been a spill of less than 10 tons of oil, and only a single other spill of less than 100 tons?  Of course not.  I bet spills of this size have happened dozens if not hundreds of times, but 50 or 100 years ago nobody bothered writing them down.  Or if they did write it down, the event has been filtered out of our collective historical memory before making it into wikipedia.  The Lakeview gusher in 1909 is another interesting example of this effect.  This certainly wasn’t the only oil production accident before 1930, but it was clearly an important major accident, and so has been remembered far better than others.

I’ve highlighted a few other spills because of their historical interest.  The Gulf War oil spill (purple dot) of 1991 is exceptional in that it was not an accident, but a deliberate act of war.  As such, it should not be considered in answering the question of whether oil exploration has been getting safer.  The Exxon Valdez spill (light blue dot) in 1989 is large in our memory, but in context we can see that it was not at all a large spill by historic standards.  But clearly the Deepwater Horizon spill (green dot) is huge, ranking as one of the largest spills ever and certainly the largest spill in quite some time.  But aside from this current mess, there does seem to be a real trend towards increased safety in oil exploration.

Again, the log-scale graph makes this somewhat hard to read intuitively.  Because the spills near the top are so much larger than the ones below them, a fair approximation of the sum of all spills can be found by simply considering the points along the top envelope, which is generally decreasing.  Looking just at the last several decades on a linear scale, this trend becomes more clear: since about 1980, serious oil spills have been getting smaller / less frequent. Now we see visually that the majority of spills listed are tiny compared to the few big ones.  I scaled the graph so only the bottom of the uncertainty bar for the gulf war oil spill.  Also note that I’ve kept the middle dots at the geometric average of the low and high estimates, which works visually on the log-scale graph, and makes logical sense given the nature of the problem.

Another factor to consider is that the total amount of oil being produced during this time period has been generally increasing.  I’ve overlaid data from the US Energy Information Administration on global oil production rate, scaled to the average amount produced each hour, to get it to show up on the same scale of this graph.   Another interesting comparison which I haven’t included is the average size of each well, or the number of wells being drilled per unit time.  My understanding is that oil exploration has been getting more difficult over time in that we’re having to drill deeper to get at relatively smaller oil deposits.  Again, this reinforces the idea that we have been getting better and safer — we’re spilling less even though we’re drilling more holes.  Except for the Deepwater Horizon.

Feel free to browse the javascript source code of the graphs for further details, inspiration, double-checking, or whatever.

Simultaneously, technological advances have allowed advertising to progress along a different axis – to become more targeted.  Advertising used to only be broadcast widely through newspapers and television shows.  The best an advertiser could do to ensure their message reached the right kind of people was to select the aggregate demographics of everybody who read a particular magazine.  Now the internet allows ads to be targeted as precisely as you’d like.  Today, Google lets you get your message only in front of people who are about to buy a product like yours.  The ability to connect to people who have expressed an intention to “buy digital camera” is a literal gold-mine, making billionaires out of Larry, Sergey and Eric.  As effective as it is, targeted advertising won’t replace broadcast advertising, because there is still value in abstract brand-building.  Rather, the two will complement each other.

Enter Social Media

Social media has been all the buzz recently.  At its core it’s just a more convenient way for friends to communicate.  The “killer app” for computers has always been helping people communicate, and this is just another chapter in that book.  With this new communications medium comes a new opportunity for organizations to tell their stories.  In fact, I believe that social media will bring another tectonic shift in the entire marketing industry, possibly as important as search-based advertising.  As consumers have gotten more and more sophisticated at filtering out advertising from broadcast media, advertisers have gotten more and more desperate in their attempts to connect with people.  Social media marketing offers a new path – instead of hearing about products and services through ads, people can hear about products and services from their own friends.  Exactly how this will play out through Twitter/Facebook/Foursquare/whatever is not at all clear to me right now, but I fundamentally believe this change is coming, and it will take the entire marketing industry with it.  Klein and her fans are free to unplug from popular culture in order to avoid the onslaught of brand advertising, but they would be foolish to stop talking to their friends just because their friends are happy with things they’ve bought.

This vision is one of the main things that prompted me to jump off the comfy Google cruise liner and start paddling hard in Banyan Branch’s crowded dinghy.

Is marketing intrinsically evil?

I sometimes feel a need to justify this line of work to those who think that marketing is inherently dirty.  I admit that I’m more of a capitalist than many of my friends, but I certainly recognize that capitalism has its limits.  The vast majority of economic transactions are both consensual and mutually beneficial, and I will argue vigorously that there is nothing wrong with an economic system consisting of these transactions.  The biggest exception to this happens when transactions are not mutually beneficial because one party is not fully informed.  But what we’re doing is helping people share honest opinions and feedback about the things they buy and use.  By lubricating the flow of information between real people, I believe social media will reduce the effectiveness of deceptive marketing.  Moreover, it will help companies connect to their customers and hone their goods to people’s real concerns and desires.  It will help hold companies accountable for their mistakes, and enable companies to better make things that make people happy.

Additionally, I will point out that my employer represents no small amount of “pure good” for the world, including organizations such as The Bill & Melinda Gates Foundation and Vittana, helping them tell their stories.

Taking a chance on a startup

Why did I choose this opportunity out of the sea of possibilities?  I evaluated the landscape as an investor would, since I am investing no small chunk of my life in this effort.  From my entrepreneurial training and experience, I know that smart investors care more about the people than the specific business plan.  The plan will almost certainly change, but the key management will not.  Having known one of the founders of Banyan quite well for a number of years, I am certain that many key elements for success are in place.  The corporate culture and governance will be solid.  I will be working in an environment where I am supported, and where I can learn and grow as a manager and a technologist.

Exactly what will I be doing or building?  I admit I’m not sure yet, but I have some very interesting ideas that I won’t be sharing here anytime soon.  I am sure that my work is very well positioned to be a part of a major shift in an entire industry — a rare opportunity.  Whether or not my work will play a key role in this shift is somewhat out of my hands — these things are always a roll of the dice.  But in another sense, it’s entirely within my control, and this is what I love about working in a small company.  There’s almost nothing but work between me and effective execution of our ideas.  Many people tend to exaggerate the importance of the idea itself, forgetting that it is incredibly important to execute well on whatever ideas you have.  I’ve heard people say that they had the idea for YouTube years before YouTube did.  How quickly we forget the dozens of other companies all working on the same problem in 2006, which almost all fell by the wayside because they didn’t execute as well as YouTube did.  Ideas matter for sure.  But hard work is critical.

If you’d like to jump on this raft and start paddling too, get in touch with me.  I need a few key rock-star developers who are’t scared of chaos and can think creatively about business problems.

In Walmart’s case, it’s generally suppliers who get squeezed.  Walmart demands that manufacturers of goods produce them at the lowest possible price so that Walmart can charge the lowest prices in their stores.  They really do try hard to pass the savings on to you.  Another case that is less well known is with so-called “interchange” fees for debit and credit cards, charged by the card networks like Visa and Mastercard.  Back in 2003, Walmart pushed hard on Visa and Mastercard to charge less for debit card transactions since they are both lower risk (because of pin-code use) and cheaper to process (verifying signatures is expensive).  The cynical will point out that with lower fees, Walmart just gets to keep more profit.  Which is true.  But they are genuinely motivated to lower prices for consumers, since that’s their main selling point.  So it’s a win-win – Wal-Mart’s motivations to lower costs are closely aligned with consumer’s desires to pay less.

Apple has similar desires for their network-connected gadgets like iPhones and iPads.  Apples wants people to be able to connect their devices to the network for as little as possible. Apple has clearly negotiated very hard with AT&T to demand low monthly rates on data plans for these devices.  Next month you’ll be able to buy an iPad with a 3G data plan for just $15 / month.  That is basically unheard of in the US.  For people on a limited budget, the iPad is the cheapest way to get online.  Compare this to other data plans available from major U.S. carriers:

The Apple / AT&T rates are the lowest in each of their categories, except Verizon’s smartphone data plan which ties the AT&T iPhone plan.  The iPad rates are extremely low compared to data plans for laptops, and also when when you consider that tethering plans or phone data plans require paying an extra $30/mo – $50/mo for a voice plan.  The unlimited iPad plan is literally half what it costs to get 3G on any other laptop, and it doesn’t come with the 5 GB limit that other plans do.  You might argue that the iPad can’t do as much as a full laptop, which is true.  So you might then argue that iPad won’t tax the network as much as a laptop, which I doubt considering the propensity to consume video on such a device.  So you can’t trade torrents on an iPad, which from an Intellectual Property perspective is just fine with me.

My guess (and this is pure speculation) is that Apple negotiated these rates by offering AT&T a share of the revenues generated through App Store purchases.

Again, the cynical will point out that Apple is just trying to grab the lion’s share of economic surplus for itself, which is true.  But nonetheless, this is a case where Apple’s desires and our desires as consumers line up well.  In a very real way, Apple is fighting on our behalf for lower prices from AT&T.

Apple products are expensive. Apple gets high margins on its hardware, allowing it to recoup large investments in NRE (non-recurring engineering) to design the hardware and its accompanying software.  This is a great place to be from a competitive standpoint, because as a company they don’t need to squabble over the cheapest parts to try to deliver the best prices to consumers.  So long as they can maintain a sufficiently large customer base to support the practice, it is an easy place to defend against competition from.  Certainly a lot easier than being Dell or HP, who struggle with operational efficiency to compete on price, and try to innovate within a very narrow window defined by their platform.

Apple’s success at selling high-end products has secondary benefits for the rest of the ecosystem.  Because the products are expensive, they tend to be purchased by people with more disposable income. So the segment of the computer market which buys Apple products self-selects to be very attractive demographic for many other reasons.  Advertisers love to get their products in front of people who are more-willing-than-most to buy something expensive / unnecessary / fun.

Similarly, app developers know that if they write an app for iPhone / iPad, the people who are able to buy it are much more likely to be willing to pay a couple bucks for something silly than, say, somebody who bought the cheapest smartphone they could afford because they felt they really need that functionality.  I had previously speculated that Apple’s platform play required a very large distribution base to attract developers, which is not quite correct.  The strategy is successful even with a relatively small market, provided that the market is segmented properly.  Which in this case it clearly is.

iSlate is rumored to have a bunch of new content associated with it.  Particularly print content.  Print publishers will probably want consumers to sign up for subscriptions.  So Apple’s probably going to be introducing people to the concept of content subscriptions on their portable devices, likely with iPhone OS 4.0 which probably will run the iSlate and old iPhones and iPod touches too.  So I wouldn’t be surprised if you can get an all-you-can-eat music subscription service available too.  We’ll see.  It’s pure speculation, but it would make sense.  I’d be particularly tickled if my off-the-cuff prediction of dates from 2007 turned out to be right.

Unless Apple has come up with a better way to do soft keyboards, that is.  When I say “soft keyboard” I mean the kind of keyboard that appears on a touch screen and has no physical keys.  I’ve complained about the iPhone’s keyboard for a while.  While it’s true that people do get better at using these, I still don’t think I’ll ever be nearly as fast or accurate (even with smart correction) with a soft keyboard as I was with my blackberry.  I think that’s probably true on average for most people.  The basic reason is the lack of tactile feedback.  With a physical keyboard, if my fingers are slightly off target, they are guided to the right place by feel.

I’ve argued for some time now that the way to solve this is by figuring out how to make a touch-screen display with tactile feedback.  How would such a device work?  Physically I couldn’t tell you.  But what we’d need would be a way to electronically manipulate texture in a clear material.  A plastic with a matrix of cells that could expand or contract under electronic control.  So the software could create bumps where each of the keys are.  This would allow a software-reconfigurable gadget that could be almost as usable as a dedicated-purpose device.

This is very different from what is commonly referred to as “haptic feedback” on some of today’s gadgets like the Nexus One.  Here, the phone’s vibrator pulses a bit when you press a soft key.  This is a kind of feedback which is tactile in that you feel it, and it gives you information about your interaction with the device without having to look at the screen.  It certainly helps.  But it is not going to improve basic typing for a critical reason — it can’t help guide fingers to the right place.  The basic act of positioning fingers on controls is still basically open loop, feed forward, without guidance.  What I’m referring to as tactile feedback helps the fingers find the right spots to press without looking.  Today’s haptic feedback can’t do that.

To be clear, true tactile feedback like this almost certainly doesn’t exist yet.  This kind of pure technological innovation basically always starts in universities or government run labs.  The ROI on pure research into unproven technology is so low that it doesn’t make sense for companies to invest there.  Even if a company proved this was possible (which AFAIK hasn’t been done yet) they’d need to figure out how to manufacture it at scale before they could sell a device with it.  Last time I predicted it would be about 2012 before we saw these.  Even though Jobs almost certainly foresees the value of such a system, Apple’s expertise is not in material science.  Wired speculated about such a keyboard based on Apple’s patent filings, but what they describe seems a bit too sci-fi for me to believe.

If they have come up with something new and cool, it’s going to be a smarter way to use basically existing hardware.  I’m gonna guess it’s probably something like a touch screen which is pressure sensitive, so you can rest your fingers on it without indicating a “button press”, making typing more natural.  You could combine this with fixed, transparent dimples on the screen under the positions where the keys are, and you’d do pretty well.  Restrict the keyboard to only work in landscape mode and you only need one set of dimples.  This would be a huge improvement in usability and the biggest technological breakthrough would be the ability to distinguish a soft push from a hard push on a capacitive touch-screen.  Like by how much surface of your finger is on it.

Regardless of what Apple’s actually managed to achieve, I wish them the best.  They’re really pushing the envelope on human-computer interactions.  If they’ve done anything significant to improve soft keyboards, they will have once again done something that the entire rest of the industry will want to emulate, and I’ll tip my hat to them.

Broadly, the concept of information democracy is that an increasingly large number of people are able to influence how information is aggregated.  Wikipedia is a clear and simple example of allowing anybody to contribute to what used to be authored by a select few — “The Encyclopedia.”  Google’s Pagerank algorithm democratized web search.  Today’s most successful software is democratizing the feature set by allowing users to vote on how they want to use it.  The general principal is that large numbers of individuals can together make better decisions than any small group.  Applying this principal to culture, we can predict that a cultural democracy will produce “better culture” than what was available before.

Information technology makes it cheaper and easier to both create and to distribute culture.  With the right software, any laptop today has all the power of a professional music or video studio.  Sure the quality won’t be as good without professional inputs (microphones, cameras, etc) but the cheap stuff is good enough for a lot of things.  Obviously the internet makes distribution of this content trivially easy, which is disrupting traditional media businesses.  Easy creation and distribution of cultural content is an important part of creating a cultural democracy, but it is not the critical enabling step.

The key to democratizing culture is in the editorial process.  If everybody is contributing cultural content that is easily distributed, but there’s still a small group deciding which pieces everybody watches, we’re still in a cultural dictatorship.  Enabling the mass public to “vote” on content is the democratizing step.  That enables the collective intelligence of all media consumers to help choose what should become part of mass culture.  So instead of some programming executive trying to guess what will be popular, the question almost becomes moot — whatever is popular becomes popular culture.  Actually making this work is not at all straightforward.  I’ll save a full description of the necessary ingredients for another post, but we can look at a couple examples.

Youtube does this quite well.  It blurs the line between sharing a video clip with your friends and publishing it as a piece of mass culture.  Any video that isn’t marked private is submitted into a kind of massive popularity contest.  Videos that get millions of views are undeniably bits of popular culture.  For music, last.fm does a good job of being inclusive, but hasn’t quite taken off.  When I started building social features into Rhapsody I hoped they could democratize the music editorial process but that hasn’t happened yet.  Like many things in social media there’s a chicken and egg problem with scale which Youtube has clearly gotten past, but music is still struggling with.

Cultural Democracy is “retro”?!

This post is inspired by a recent story by Heather Chaplin that NPR aired describing participatory culture in video games.  The surprising part of the story for me was the assertion that this trend is not modern but in fact “retro.”  The story points out that before analog broadcast media, most culture was participatory — singing, dancing, crafts, etc.  Analog technology created the possibility of cultural hegemonies, and digital technology is breaking them down. A fine point, implying that the 20th century will likely be unique as the only period in human history when popular culture was dictated by an elite group of editors.  Thanks for the interesting tidbit.

]]> http://www.embracingchaos.com/2009/12/participatory-culture-and-the-democratization-of-information.html/feed 1 http://www.embracingchaos.com/2009/05/uaw-vs-chrysler-friends-at-last.html http://www.embracingchaos.com/2009/05/uaw-vs-chrysler-friends-at-last.html#comments Wed, 20 May 2009 16:00:13 +0000 leodirac http://wp.embracingchaos.com/2009/05/uaw-vs-chrysler-friends-at-last.html I’d like to share a couple thoughts on Detroit — a couple ideas that
I’m not hearing in the popular or business press, but are important to
understand.

Chrysler goes bankrupt

First some background.  Chrysler is being restructured under
bankruptcy.  This doesn’t mean they’re going out of business.  It means
that they owe more money than they have or will be able to pay.  So
with the help of a judge, they’re sitting down with everybody they owe
money to and telling them frankly “you’re not getting everything we owe
you.  Sorry, but there just isn’t enough to go around.”  So everybody
has to compromise.  The idea is that by striking some bargains to
reduce debt the company can get back in the game and become profitable
again.

UAW owns Chrysler

One of the biggest debts Chrysler has is to the UAW, the United Auto
Workers.  This is the labor union which represents all the
“blue-collar” workers who actually make the cars.  Chrysler owes them
benefits like pensions and health benefits.  Part of the settlement is
that the UAW will own 55% of Chrysler stock.  That’s a majority.  So
the workers will own the company.  Personally I think this is great and
makes a ton of sense, and I’ll tell you why.  But not everybody does.

If you’re lucky enough to be blissfully unaware of labor relations in
Michigan, this is downright bizarre.  Chrylser corporation is
“management.”  UAW is “labor.”  These two groups traditionally have not
gotten along.  I don’t think the word “hate” is out of place.  People
say the UAW will try to unwind this position as fast as they can.  I
heard one “expert” say that the UAW is placed in a position of conflict
of interest representing both Chrysler stockholders and UAW workers.
Why?  Because their responsibility to stockholders is to increase the
value of the company, but their responsibility to the union is to save
jobs, and these two goals are diametrically opposed.

Cooperation is the only way

Hold on.  The goals of the workers and the goals of the company are
diametrically opposed? This kind of adversarial thinking underlies how
Detroit got into trouble in the first place.  In truth the UAW’s goals
and Chrysler “management” goals are very strongly aligned.  This
painful truth of this fact is excrutiating today.  Chrysler and GM are
on the verge of ceasing to exist.  If and when this happens, the UAW
workers will lose their jobs.  What’s bad for management is bad for
labor.  But figuring out how to keep Chrysler building cars that can
compete with Japan and everybody else is a really hard problem.  Solve
it and both labor and management win.  If ever there was a time for
labor and management to come together and cooperate it’s now.  To be
extremely blunt for those still harboring grudges: if you two don’t
figure out how to play nicely together, you’re both doomed.

Historical tensions caused these problems and SUV’s too

Conventional wisdom cites two reasons for why Detroit is in this mess:

• They only built big gas-guzzling cars as consumer preferences shifted towards smaller, more fuel-efficient vehicles
• Union labor costs for things like pensions and health care are so
  high compared to foreign competition that the company just can’t compete

I believe both these are true.  But more interestingly (and something
I’ve never heard reported in the press) I believe there’s a causal link
here.  It is precisely because of these high labor costs that Detroit has focused on building gas-guzzlers. Smaller cars are cheaper and are subject to more intense price competition, meaning the margins are lower.  In business school
we learn about two basic types of product strategies: low cost and
high-end.  In the low-cost strategy you try to be more efficient than
your competitors.  You do things cheaper and still maintain a good
enough product.  This is what Japan did with cars.  But because UAW
kept labor costs high, Detroit couldn’t go this direction.  Their small
lower-end cars would just cost more because of the higher input costs.
So they had to go after a high-end strategy where they made bigger,
more expensive vehicles that came with higher profit margins.

Sophie’s Policy Choice

So UAW workers collectively bargained their way out of jobs.  That is,
they bargained up their salaries beyond what their labor is actually
worth in the modern economy.  So what should we do?  Let the market
correct itself so many of them lose their livelihoods?  Or sustain them
publicly somehow?

There is no easy answer to this question from a policy perspective.
China is facing this same question with hundreds of millions of
uneducated peasant farmers.  A relatively modest investment (on the
national scale) in farm machinery could replace a good fraction of
their output.  But the economically efficient choice comes with a high
human cost.  In this country we believe governments exist to serve the
people.  We’ll see how it does.

Background on SLRs and DSLRs

(If you what “f-stop” means, feel free to skip ahead to the next section.)

SLR stands for Single Lens Reflex.  Practically speaking it refers to a camera where you can change the lens.  You look through the same lens that actually takes the picture, letting you put any lens from an ultra-wide angle fisheye to a telescope-length zoom lens.  You can also put filters on the front like star filters or color shifters or polarizers.  Imagine a classic 35mm camera — like what a P.I. would carry to snap pictures of your wife having an affair — that’s an SLR.

SLR’s require a mirror that physically moves to divert the light into one of two places — your eye, or the film / CCD. The mirror was important when the only technology for capturing images was chemical film.  But nowadays we have various electronic devices like CCDs that digitize an image.  DSLR cameras use a CCD to get many of the benefits of digital imaging, but still have the same physical form factor as an old chemical-film SLR.  They can use the old lenses, which is one of their big appeals.  But so many things about these cameras just don’t make sense.

The problems with DSLR cameras

First there’s the noise. The sound of the mirror slapping against its stops as it switches positions is very recognizable. We used to accept sounds like that as a necessary part of taking
pictures.  Today it just annoys me.  Especially when I’m at a small
event and some photographer is there making loud clicking noises all
the time while I’m trying to enjoy whatever it is they’re digitizing
with their dinocam.  In 99% of all use cases, it’s totally unnecessary.  CCDs can continuously capture images and display them on a screen, creating a digital light path that doesn’t require loud expensive mechanical assemblies.  These displays aren’t as good as what a human eye can pick out, so this doesn’t work all the time.  But if you don’t need interchangeable lenses, then the camera can have a second optical path just for the eye, which doesn’t need to be as good.

One argument against a separate optical viewfinder is that youc can’t put filters in front of the lens.  This is very true, but filters are also obsolete.  With few exceptions, everything that a physical filter does can be done later in photoshop with more control and accuracy.  Color tinting, sparkle, gradients, soft, mist, etc — these all used to be rendered in physical glass out of necessity.  Polarizing filters are probably the most important exception to this — since CCD’s don’t record a light’s polarization state, it can’t be adjusted later.  But for the most part, filters aren’t necessary anymore, meaning you don’t need the whole single-lens thing.

But what about interchangeable lenses?  Isn’t it useful to have the same camera body and be able to change lenses?  (I hear you cry.)  Yes, sorta.  There are definitely situations where one lens won’t be able to do everything you want.  But those situations are getting rarer and rarer.  And in the few exception cases, I’ll argue that interchangeable lenses aren’t the right solution.  The reason these cases are getting less and less common is that zoom lenses are getting better.  When SLR cameras first came on the scene zoom lenses basically didn’t exist because they sucked when they did.  You needed a different lens for each amount of magnification you wanted, so people had lots of lenses.  But with computers to help us design the lenses, and vastly improved manufacturing processes, zoom lenses are getting better all the time.  Nowadays a lens with a huge 10x zoom can even win accolades from camera snobs.  And lenses as versatile as 26x cover every situation most of us would ever want, and at a quality we’ll be thrilled with.  So for almost all situations, a single zoom lens is good enough today.

What about the situations where that’s not quite good enough?  Where you need that 14mm fisheye that captures people standing immediately to the left or right side of the lens?  Or that 8000mm super- long telephoto telescope?  It turns out in either of these challenging cases, getting the lens to fit the standard SLR form factor becomes the hardest part.

Why SLR’s cripple even the extreme lens cases

With ultra-wide fisheye lenses, the problem is the space reserved for that stupid mirror.  In this case, the focal length is very short, so as a
lens designer, you’d naturally want the focal plane to be very close to
the glass.  (Like about 14mm.)  But the place where the lens attaches to the camera body necessarily needs to be a certain distance away from the imaging plane.  That distance was determined by the size of the mirror, which was determined by the size of your chemical film — 35mm, which is more than you’d really want for a 14mm lens.  Even on today’s 2009 DSLR cameras, that distance is exactly the same as it was a generation ago in order to ensure backwards compatibility with old lenses.  The literal tons of carefully polished glass represent a very real barrier to improvement since people have invested lots of money in them.

So if you really want a camera that’s good at taking super-wide angle pictures, you don’t want your lens to have to be that far away from the imaging plane.  You’re better off with a specially built camera.  The lens will be simpler, cheaper and higher quality.  But super-wide starts to look funny, no matter what.  Funny meaning
distorted, because if your eye is more than a couple of inches away
from the reproduced super-wide image, then it won’t look right.  And it’s not super useful to capture 360 degrees in one shot — you can shoot a dozen pictures and stitch them together later in software, and it’ll look more natural.  This is all why people don’t pay a lot of attention to how super-wide lenses get anymore.

On the super-telephoto side of things, the SLR legacy is even worse.  To get a super-long telephoto lens you need lots of big glass.  This gets expensive quickly simply because it’s a large mass of carefully manufactured stuff.  The amount of glass you need for a lens is proportional to the cube of the length of your imaging plane, which for legacy chemical-film is 35mm. But CCD’s just don’t need to be that big.  On almost every DSLR they’re only about 20mm across, and on high-quality non-SLR cameras are as typically about 6mm across.  So that size legacy means you would need literally 200x  the almost 40x the amount of physical glass to make a good telephoto lens for an SLR vs a non-SLR camera.  This ridiculous discrepency is just going to get worse.

CCD’s are silicon devices, so they share manufacturing improves along with CPU’s and follows a Moore’s law-like improvement curve for performance.  A key way they improve is in pixel density, but also by simply getting smaller.  As they get smaller, high-quality zoom lenses get smaller and cheaper too.  But only if the lenses are specifically designed for the new smaller CCD’s.  With an SLR system they can’t be — the size must be fixed in order to maintain backwards compatibility.  So while sensor technology improves at Moore’s law speed, lenses for non-SLR cameras improve as well, but SLR lenses do not.  Expensive zoom lenses for modern cameras just don’t need to be that big or expensive — It’s like having to build a cell-phone big enough to hold floppy disks.

To illustrate this point, consider the popular Canon SX10IS camera which does not feature interchangeable lenses.  It features a zoom lens that goes from pretty wide (28mm equivalent) to really very far zoom (560mm equivalent).  Because its CCD is only 6mm across, it can do all this for under $400 and weigh in under a pound for the whole camera.  For comparison, a comparable SLR lens weighs in at over 11lbs and costs upwards of $7,000, just for the lens.  No doubt this lens can take better pictures than the tiny Canon, but a smaller lens built for a modern CCD could take pictures that are every bit as good for a fraction the price.

I would be remiss if I didn’t mention the noise floor on these sensors.  When the scene is dark, you need more light to get a good image.  A bigger hunk of glass captures more light.  This all makes intuitive sense and is mostly accurate.  CCD sensors can take more accurate pictures in low light when they are bigger.  But the limits here are electronic noise, which is also improving.  At some point we’ll hit some other barrier like the thermal noise in the sensor, although a piezo cooler could work around that.  Ultimately there’s the the quantization of photons, but if you’re taking pictures in a scene that dark, you probably can’t see what you’re pointing at anyway.  My point is that while there are advantages in low light for larger glass and sensors, technology is erroding away at those too.  We’re seeing ISO equivalents of 6400 as fairly common in cameras these days, with an economic competitive pressure to improve that.

In summary, the problems with the SLR format are that it ties its owner to a physical legacy that denies them the advantages of advancing technology.  There are cases where specialized lenses are still important.  But those cases are dwindling.  Personally, I’m going to be happier carrying around a full featured small camera that can transform itself into whatever I want without needing interchangable parts than a bag full of bits that were standardized before email.

Lessig’s book sounds interesting, and since I’m tearing through non-fiction right now, I ordered a copy. I was very surprised to see that the inside flap declares “Copyright © Lawrence Lessig, 2008 All rights reserved.” Below that it says:

Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), without the prior written permission of both the copyright owner and the above publisher of the book.

(As I write this I half wonder if I have violated the stated copyright by typing that in. But seriously I think it’s a clear of fair use.) I expected the book to be released under a Creative Commons license, as Lessig espoused in his interview. I recently started reading Cory Doctorow’s Little Brother which is available for free download from his website under a Creative Commons Non-Commercial Share-Alike license.

What’s going on? Could it be that Doctorow is ahead of Lessig in the practicalities of modern book licensing? Or was it that the publishers were enforcing something? I bought a physical copy of Little Brother, and saw that it too has a traditional Copyright note at the front: “Copyright © 2008 by Cory Doctorow. All rights reserved, including the right to reproduce this book, or portions thereof, in any form.” Okay that just doesn’t make sense. I can download the book under CC, but the print edition is All rights reserved. What gives?

Maybe I’ll have a better answer after I’ve read Lessig’s book. Or maybe Lawrence can explain himself. His website also says that “The book will be available under a Creative Commons license from Bloomsbury Academic. Stay tuned for launch.” I’m waiting.

Diversifying from hardware

Since Jobs pulled the company out of the doldrums, Apple has been a manufacturer of high-end hardware.  This is a very nice place to be. Their margins are very high.  If you compare the hardware bits that go into a Mac to those in any PC, Apple’s prices are much higher.  PC manufacturers squabble over low single digit margins because their product is almost completely commoditized. HP, Dell and IBM struggle to differentiate themselves in the market.
Meanwhile Apple can charge a hefty premium for good industrial design and software that is slicker than windows.  iPod followed in this tradition of high margins by setting the bar for usability in portable
media devices and following up with fantastic marketing.

Being a niche retailer of high-end products is a comfortable and stable place to be.  However, having a single line of revenue isn’t good for a large company, so diversifying makes a lot of sense.  Thus Apple’s current push into services.

The iTunes music store was an important pre-cursor to the current push into services.  Even though iTunes moves a massive volume of music, if you work through the accounting confusion, they’re not actually making much money there.  Because the established music industry giants are completely fucked and have no idea what to do, they drive the margins of all big online retailers down to a pittance.  But iTunes did something very important for Apple: it established a billing relationship with customers. Cellular operators have this kind of ongoing billing relationship with their customers which enables them to push high margin, low utility products like ring-tones. Similarly, people are used to spending small quantities of money in iTunes to get music. So iTunes is the perfect precursor for an App Store.  In this way, Apple doesn’t even need to rely on the carrier’s billing relationship
to build a services business.  This will be very important for Apple in coming years as carriers increasingly become just another provider of wireless bandwidth.  In the coming decades, the value will not come from piping bits around.  It will come from the services built on those pipes. 

MobileMe is a very Apple service

MobileMe is a cloud-hosted email, contact and calendaring solution. There’s nothing revolutionary about this.  Google offers all of these services for free.  By charging for these services, Apple is implicitly promising to provide a better solution.  Considering their vertical integration into popular hardware, it’s not hard to imagine that they will succeed at this.  Google will probably remain committed to supporting open standards for working with hardware.  By using proprietary protocols, Apple can provide a higher-quality product and support it better on the few platforms they care about.  It’s a classic story we’ve seen in this industry before.

MobileMe’s launch was a disaster.  This isn’t surprising as Apple is yet to demonstrated great skill in online services.  Personally I believe they’ll figure it out, because it’s important to their long-term objectives, and they’re a smart company, and the skills to do this well are getting easier and easier to find. But from a marketing perspective, MobileMe is nothing new for Apple.  Because MobileMe is a premium service for which there is a very good free alternative, MobileMe still targets Apple’s classic market segment. They’re still targeting people who are willing to pay extra to have something really polished.  They can stick to their classic bag of tricks, like the ads that make fun of people who aren’t as cool as the mac devotees.

App Store is new: a platform play

App Store on the other hand is going to be much harder for them to pull off.  For App Store to succeed, their primary challenge is not to attract paying customers, but developers.  Because independent software developers (ISVs) are the ones who are actually creating value in the App Store.  App is just a distributor taking a cut on that.  So what attracts developers to the App Store?  Customers do.

It’s a real chicken and egg problem.  Nobody’s going to build software unless there are customers to buy it, and it’s hard to get customers without cool apps.  Well right now Apple has the whole farm, but it’s a very small farm.  There are millions of iPhones out there, representing potential customers.  Moreover, the iPhone SDK is very rich and capable when compared to its competitors.  Qualcomm’s BREW, Windows Mobile, Palm OS, Blackberry and Symbian have all faltered for one reason or several.  Google’s Android holds much promise and hope, but at this
point it’s complete vapor-ware.  So for now, Apple has almost all the mindshare of mobile application developers.

But how long will this dominance last?  Software platforms are natural monopolies, meaning the economics tend to be winner-take-all.  The chicken and egg “problem” can easily turn into a virtuous cycle, pushing a winner to the top. Application developers are fickle and will code to whatever platform has the best distribution.  iPhones are very popular, but they are still only used by a small fraction of all mobile subscribers.  Until the distribution of iPhones reaches a critical mass, their dominance as a mobile application platform is very shaky.

The mobile app challenger is HTML

But the laundry list of alternatives shows that the competition is fragmented.  What could unseat Apple?  IMHO it’s not another application platform, open or proprietary.  It’s the web.  Every high-end phone can display web pages, and increasingly they’re using high-quality javascript engines that can run real web applications. Webkit, the super-fast open-source HTML/JS engine behind Safari is showing up in Symbian devices, Blackberries and Windows Mobile phones.

Apple’s challenge is to make the proprieray iPhone SDK compelling to developers.  The alternative is to build a web application that works well on webkit, and works on every high-end phone.  They can attract developers in two basic ways — make the native features of their SDK more compelling, and provide a large market for distribution of the applications.

Charging for distribution of these applications is a gimick that won’t last long.  Soon all the interesting applications will be free, but tied to cloud services that have their own business models independent of the mobile client.  Premium applications will start to seem a lot like premium ringtones pretty fast.  Still, it will help bootstrap this market for Apple so long as there are no serious competitors.

To stay on top, iPhones need distribution quickly

One thing that ties all these points together is that Apple’s continued success with App Store hinges on having wide distribution of iPhones. They are currently throttling the distribution of iPhones for some reason, possibly
because of software glitches.  But the aggressive $199 pricing is clearly aimed at attracting a new larger customer base that will help maintain their dominance in the mobile application space.

Long term they might be happy getting by offering premium versions of applications that are freely available on the web.  But something tells me they’re actually trying to break open the mass market on this one. This is Jobs’ big play.  It’ll be really interesting to see how it works out once Android hits the streets.

New iPhones are hard to get

The blogosphere is full of speculation about whether or not Apple deliberately made the iPhone scarce on opening day and since then.  Most assume that this is deliberate on Apple’s part for a variety of reasons, mostly to attract more attention, increase demand, etc.  I assume most of these rants are from bloggers who want their new iPhones but haven’t overcome the barriers to obtain one yet.

But if Apple’s goal was purely to meter out their distribution, why not sell them online?  To get a phone you need to place an order for one, wait a week or two, and then you can get it.  This seems reasonable in conditions of scarcity.  But to get an iPhone 3G, you need to walk into an at&t store to place your order, and then walk into the store again to pick it up.  Think about this.  If the limitation was purely lack of supply then there are several ways this could be easier for customers:

1. You could order a phone online to be delivered to your house.
2. You could order a phone to be delivered to your nearest at&t store.
3. You could call the nearest at&t store to place your order, but still have to walk in to pick it up.

Try asking them why you can’t do any of these things and they will answer with one word: policy.  Clearly Apple & at&t have gone out of their way to make it difficult for people to get their hands on a phone.   This goes above and beyond just preserving a limited supply.  You have to work to get an iPhone 3G.

New iPhones have Issues

From all the reports I’ve read, the problems with the new iPhone are in the software not the hardware.  I conclude this because my friends with first generation iPhones are experiencing the same problems as those with the new 3G iPhones.  Moreover everybody seems to agree that these problems only showed up after they upgraded their iPhone software.  Problems include:

• Frequent crashes of applications, especially Safari
• Increased lag in common operations
• Significant problems with large contact lists (>200 contacts)
• Extended delays before placing a call

Apple is legendary for their high quality software.  People buy Macs because they "just work."  It’s really not like Apple to release a buggy piece of software.  But it sure seems that they did in this case.  Why?  Obvious answers of fierce competition for high-end smartphones.  The more interesting question for me is "How did they get away with it?"  Which it sure seems they are.

Escalation of Commitment: The Hush-factor

There’s a well-document psychological principal at play which prevents people from objectively critiquing things that they are personally invested in.  Sometimes called escalation of commitment, or irrational escalation, the idea is the same.  If somebody works really hard to obtain something, they will blind themselves to its faults.  Imagine this conversation:

"Dude, I can’t believe you waited in line for hours to get that phone.  What do you think of it?"

"Actually, it’s just okay.  The applications crash a lot.  And it’s not nearly as fast as I’d hoped it would be — sometimes it just hangs for like 10 seconds.  But at least it’s pretty."

Very few people have the objectivity to imply that their personal sacrifice was not worth while.  This effect is commonly observed in people who buy high-end items. 

The flip side of this effect is buyer’s remorse.  But since the phone itself is not actually at all expensive (when compared to the monthly fees), that’s unlikely.  Also, it has become a positional good, whereby it has value simply because other people don’t have one.  That fact remains regardless of how unreliable it is.

Speculative Conclusion

I posit that Apple knew about the software problems with the iPhone 3G before launch.  They did manage to iron out all the performance and stability problems they encountered before launch.  They felt they needed to launch it this summer to get ahead of other notable smartphones like the Blackberry Bold, HTC Touch, and Android which are hot on their heels.  So they rushed it out the door at sub-standard quality.

In order to partially cover for this mistake, they have made this device especially hard to get.  This covers their tracks in two ways: people make even more noise about scarcity.  And those who do jump through the whoops to obtain one are far less likely to complain about it.

Open your fridge.  If you’ve lived with that fridge for a while, there’s a good chance it looks something like mine does.  Shelf upon shelf of half-used bottles and jars of long-lasting meta-foods.  Condiments, salad dressings, jellies, beverages, chutneys, nut butters, salsas, pickled vegetables, etc.  We expect our fridges to be full of food, so this doesn’t in itself challenge the evolutionary principal of selection.  But taking an inventory shows that there is a strong bias towards foods we don’t actually like.  In fact, the typical selection process for foods in our refrigerators tends to concentrate foods we don’t like, thus running backwards to what should intuitively evolve towards a selection of our favorite foodstuffs.  But for a couple very understandable reasons, that just doesn’t happen.

Consider salad dressings.  Most of us like to have some choices when we’re topping our raw vegetables.  So when we’re at the store, we don’t just buy the one salad dressing we like, but will often try a new variety.  There’s a documented psychological principal called Variety Seeking that encourages diversity in buying because people want to explore different choices.  But what happens when we buy a variety we don’t particularly enjoy?  Like that orange blossom vinaigrette or the honey mustard that’s just a bit too thick and sweet.  We try it once, form an opinion, and the next time we have salad we go for the old-reliable Goddess dressing.  So it lingers.  But we don’t throw it away.  Because there’s nothing WRONG with it.  Besides, one day when we have guests over they might prefer a syrupy honey-mustard dressing.  Or maybe we could dip chicken knuckles into it or something.  Plus the combination of preservatives, low-temperature and food that doesn’t promote bacterial growth in the first place means it can stay edible for years.  So their continued presence provides some small marginal benefit of choice.  The only real alternative is throwing them away  (which makes us feel guilty) since there’s no secondary market for used condiments.

Beyond choice, they do provide marginal benefit in terms of ballast for heat capacity.  Refrigerators run more efficiently when they’re full since there’s a larger thermal mass which is more stable.  But this assumes the fridge has ample space for the food that is being cycled through and consumed.  In many households the need to find space for food you’re actually going to eat creates a selection pressure to remove such undesirable foods.  But the door of the fridge is a niche environment that isn’t very well suited to large, short-lived main courses and thus things like eleven different varieties of mustard tend to thrive.

What’s the take-home lesson here?  How do we fight this scourge on our pallets?  Actually I don’t think it’s that big of a problem.  When we need space in the fridge, we find it.  But otherwise we collect things like Mang Thomas All Purpose Sauce, and pickled cherry peppers.   If clutter bothers you, resist the temptation to try something new and stick with something you know you’ll use.  Heck, get a really big bottle.  Or look for similar reverse-evolutionary processes in your medicine cabinet, liquor shelf, or office supplies, and be conscious that you have the power to change things.  Or just accept that sometimes human nature tends to concentrate our surroundings with things we don’t actually like.

But there’s a dark side to this rescue that I haven’t seen anybody discuss.  The reason the guerrillas allowed those hostages to get on that helicopter without firing a shot because they thought it was operated by a humanitarian group.  It’s true that the operation relied on intercepted communications and a spy in the FARC’s command structure.  But the operation relied on a having military helicopter painted white and its crew claiming to be apolitical.  The press even describes the acting lessons the soldiers took to pretend to be NGO workers.  Oh those foolish rebels who fell for such a simple trick by trusting aid workers.  What dupes!

Now look at this from another angle.  Imagine you really are an NGO worker, trying to provide some kind of support service to remote Columbia.  How does knowledge of an operation like this make you feel?  Scared, probably.  From now on, rebels are going to doubt the legitimacy of all NGO workers.  They might think you’re in the Columbian military trying to take advantage of them again.  They might even start shooting down Red Cross helicopters.  The negative externality of this rescue is that all legitimate humanitarian work in the area has just gotten a lot more difficult and dangerous.

So as Santos brags that this rescue "will go down in history for its audaciousness and effectiveness" he ignores the fact that he just cashed in a bunch of good will to make this happen.  This stuff doesn’t grow easily like coca plants.  I’m glad those people have their lives back, but I am in no way convinced it was worth the sacrifice.  What’s going to happen next time there’s a public health crisis in the area?  The moral calculus is undoubtedly complex.  But ask yourself, would you trade the freedom of a dozen captives (including three Americans) for risking the well-being of many thousands of needy individuals?  How about for the lives of a half dozen International Red Cross workers murdered by suspicious rebels?

Several startup companies have sold e-book readers in the past, but none successfully.  Sony is the only other large company to have tried.   Assurance that a risky new technology is backed by a company that won’t disappear is important for mass-market adoption, giving Sony and Amazon an advantage.  This is especially important for devices that consume media, as the device’s utility dwindles without new content.  Amazon is especially well positioned to offer media for Kindle through its complementary assets.

Amazon’s established relationships with book publishers are extremely valuable to Kindle.  Book publishers control e-book content.  Amazon’s history of selling physical books has earned them the trust of almost every publishing house, ensuring easy access to electronic versions of books. In addition to existing e-books, Amazon’s scale gives them leverage to encourage publishers to release electronic versions of books.

Beyond that, Amazon has rare technology to make electronic versions of books available with far less work on the publishers’ parts.  Amazon has spent years scanning physical books to enable a feature called “Search Inside This Book” on their website.  Along with Google, they have one of the only large archives of scanned physical books in the world.  This enables selling e-books for books that publishers don’t even have original electronic copies of, with rights negotiations as the only remaining barrier.

Innovators have been jibbing together their own e-book readers out of laptops and PDF files for years.  Early-adopters look for concrete advantages like the ability to search books.  Med-students give Kindle rave reviews for this capability.   The easy availability and portability of dozens of books appeal to the small segment of truly voracious readers.  Kindle seems to serve these early segments well.  To cross the chasm into the mass market of the early majority, Kindle must make the experience simple and reliable.  Kindle’s wireless data connection sets it apart from all previous e-book readers.  By leveraging Sprint’s nation-wide 3G cellular data network, Kindle can load content without the operator even owning a computer.  Thus Kindle dodges the inevitable complexity that arises anytime a PC is involved.  This, along with Amazon’s well-established customer service, promise to make Kindle much easier for the early majority to accept.

Kindle seems well positioned for acceptance by the mass market.  If successful, Amazon will need to balance publishers’ need for DRM against consumers’ desire for open content.  The music industry has exposed these issues but certainly not solved them.

[This is another recycled homework assignment.  Something to keep y'all entertained while I'm in New Zealand!]

The music recording industry is in trouble.  Disruptive changes in music playback technology have seriously reduced demand for their mainstay business, physical CD sales.  CD sales comprise 80% of the industry’s total revenue, but have dropped sharply in recent years.  Last year sales dropped by 19%, and the channel is in danger of freefall as retailers start to re-allocate store space currently assigned to CDs.  The industry’s hopeful replacement revenue stream, digital downloads, looks like it will only replace a fraction of the loss.  What went wrong?  How did an entire industry fail to keep up with technological innovation?

The recording industry’s value in the economy comes from providing consumers access to great music.  The value chain includes discovering talent, developing the talent to create and record great music, and distribution of that music to consumers.  The early stages of the pipeline have remained about the same for decades.  But technology has permanently changed how music is distributed to consumers.  This fact was driven home to EMI management when a group of teenagers were invited to take as many free CDs as they wanted after participating in a focus group, and they didn’t take a single one!  The recording industry has acted as a manufacturer of physical goods.  But really their business is in licensing Intellectual Property (IP).  When it was inconvenient for consumers to reproduce high-quality recordings the distinction was unimportant.  But today physical distribution of recorded media provides a tiny fraction of the value in the music value chain.

Music IP is legally controlled by copyright.  Digital Rights Management (DRM) technology has been used to enforce licensing agreements on digital recordings files.  Until 2007, the recording industry only sold digital music with DRM, in an attempt to control copyright violations.  The great irony of DRM that has prevented its acceptance by consumers is that by restricting the use of the legally distributed digital music, DRM makes the legal product lower quality than the illegal product.  The lack of consumer incentive to use a lower quality product, combined with the impracticality of enforcing copyright agreements on individual consumers makes the appropriability regime in the distribution of music to consumers very weak.

We can think of innovation in this content space as the creation of compelling new music.  A hot young band with a new album or style of music has an innovation they want to commercialize.  As discussed earlier, the appropriability regime with consumers is quite weak.  The value of the labels’ distribution assets are waning, putting the band in the position of the attacker’s advantage according to Gans’ and Sterns’ innovation framework.  The band should go it alone and seek novel distribution techniques, ignoring the incumbent labels.  The appropriability regime is less clear with respect to incumbent labels – the album itself is well protected by copyright law since the legal recourse is straightforward against a large recording company, but a novel style of music is unprotectable.  So a promising band considering partnering with an incumbent label should consider how easily the value of their art could be expropriated.

The recording industry has focused too long on a part of the value chain that is no longer economically relevant.  They should look to other industries for inspiration as to how to create value in an environment where content and innovation are created more openly.

Why bigger is better for online advertisers

The reason lies in network effects of the online search + advertising industry.  Imagine you’re an ad buyer which is to say you have a service you want consumers to find online.  Unless you’re a huge company, you have limited energy to expend buying your ads.  So rather than buying and managing separate ads from each Microsoft, Google and Yahoo, you’re likely to just deal with a single ad publisher.  The sensible ad buyer will choose the ad publisher which gives them the most value for limited effort. 

Right now the clear choice for an online advertiser is Google.  Because they have the most search traffic, they are best able to reach customers.  Combined with their adsense network, Google clearly has the largest inventory for an ad buy making them the natural choice for anybody not willing to spend a lot of energy managing their online advertising.  This logic underlies the recent acquisitions of Doubleclick, AvenueA/Razorfish/whomever, and now Yahoo!  Network effects in advertising mean that the largest network will be the most sucessful.  So the mergers will continue as far as the anti-trust regulators allow them to until a handful of bitter enemies remain.

This much might be obvious to some of my readers.  But I felt like sharing this analysis since I’ve read nothing in the common press that explains the basic economic motivation of this deal.

Wrinkles, twists

An irony of the network effect comes from the auction nature of keyword buys.  Advertisers bid for the right to get their message in front of customers.  When more advertisers are competing for the targeted eyeballs of consumers, the prices for advertising go up.  This means that prices will tend to be higher on the larger ad networks.  So bargain seekers can get more advertising for their dollar by seeking out smaller networks.  This appears to contradict the logic that bigger is better for ad networks.  But many advertisers are limited not so much by budget but by the ability to reach highly qualified customers.  If you are selling poodle tattooing services in the pacific northwest, odds are you will not hit max out your advertising budget on any of the ad networks simply because not that many people are searching for your services.

I could probably fill pages with speculation about the culture clash between Microsoft and Yahoo and other reasons why it will or won’t work.  But if you’re interested in that stuff, I’m sure you’ll have no problem finding it in the backwaters of the blogosphere.  I can’t help but drop a couple relevant ideas though.  First, from what I hear, the executive management at Microsoft is so dysfunctional right now, Yahoo will provide fertile new ground for their turf wars.  If the top bosses are adept, they will use the many iterations of re-orgs to sluff off ineffective execs to projects where their overall damage can be minimized.  Second, I think I hope Microsoft has evolved enough humility to understand that they’re better off simply shutting down Yahoo’s services than forcing everything to port over to NT servers.  Right, guys?

Disclaimer

I feel compelled to point out that the opinions expressed here are mine and mine alone.  In no way does this article reflect any official position of my employer.  This is my personal analysis of the economics behind the industry I work in.

But as communist governments age they tend to become corrupt and dictatorial.  Why?  Unchecked power.  Without elections or a free press, there is essentially no way to remove a bad leader.  Highly secretive control structures like the politburo tend to select for leaders that can amass and wield power by any means possible.  These traits tend to become much more concentrated than any traits related to good governance.  This was the undoing of the Soviet block and hopefully soon will dismantle North Korea.  How China managed to avoid this state I won’t ponder here.

Despite what the US State Department would like you to believe, Cuba’s government has not (yet) devolved to this state.  Cuba is still highly egalitarian where top government officials only earn twice what a factory worker might make — not the wretched excess of a corrupt system.  They probably get to drive cars, but are required to pick up all hitchhikers since really it’s the people’s car.  For the most part Cubans trust their government and with good reason.  Cuba is not corrupt.  Of course if you’re smart and ambitious you have much more to gain in a capitalist system which leads to justifiably frustrated opponents.  While their lifestyles are anything but luxurious, Cubans are generally fairly happy, healthy and well educated.  Happiness is subjective, but statistics clearly show Cuba to be on par if not better than the US for healthcare and education in most measures.  It’s not a bad place to live.

The success of Cuban communism should not come as much of a surprise considering what we’ve discussed so far.  Castro was idealistic when he led the revolution, and he’s still on power.  Corrupt leaders have not been able to take control yet.  A problem with communism is that what happens next will depend very strongly on one individual.  Will they be more like Fidel or Kim Jong-Il.  We’ll have to wait to see.

First off, I am confident Apple will launch a subscription music service.   As every Rhapsody fan and many industry analysts agree, subscription services are the best way to consume music.  Just like Hotmail moved email into the sky, and Google Docs are doing the same for office productivity applications, music can and will go the same way.  Being tied to a specific piece of hardware to enjoy your information services is so 20th century.  The reason we’re not there yet is that it’s not easy to provide a great experience.  And considering people’s long-standing investments in legacy music media like CD’s, non-hosted music services actually provide a smoother transition.

When I worked for Real people generally spoke of Apple launching a subscription service with fear.  I argued that it would actually be one of the best things for the company.  The reason being that even modern electronic music consumers don’t understand what a music subscription service is.  If Apple started spending their quarter-billion dollar per year marketing budget to explain this to consumers, it would do wonders for Rhapsody.  Especially considering the low-quality, poorly-funded advertising campaigns Real has traditionally engaged in.  I wish I could find some of the infomercial-style TV ads they used to run.  Glaser built Real Player without advertising and still believes all internet services should be able to bootstrap themselves.  Maybe the alliance with MTV will help there. 

Also, managing a multi-million song library is not easy.  Rhapsody does a pretty great job of it.  Although they’re going to get obsoleted unless they can figure out how to democratize the music editorial process.  But they’re still way better at it than Apple, who has frankly never been very skilled at online services.  So if Apple were to start spending their huge marketing budget tomorrow to explain why it’s not important to own your own music, it would be a huge boost to Rhapsody.

It won’t happen tomorrow though.  My guess is that within 5 years iTunes will offer all-you-can-eat music for a recurring monthly fee.  The timing depends on a couple of key factors:

• Uptake of network-enabled iPods
• Availability and quality of wireless net access

Before the iPhone, Apple could not launch a subscription music service for one simple reason.  If you stop paying your monthly fee, your subscription tracks need to be disabled from your portable device.  Otherwise somebody could pay the fee for a single month, go on a shopping spree and load up their device with all the music they’ve ever wanted, and never pay another dime.  So even though DRM is going away for track purchases, it has to stick around for subscription models, at least until many other things change.  How does this limit Apple’s ability to launch a subscription service?  As anybody who has used a portable music device with a subscription service can tell you, it is incredibly frustrating to pull your mp3 player off the shelf only to see a message that says it won’t play any of your music because your licenses expired and you need to plug it into a computer to verify that you have been paying your bills.  Even if you are paying, you need to constantly tend to your device or else it bricks itself after a few weeks, by design!  Steve Jobs would never allow his iPods to do this.  The solution is to enable the device to check your subscription entitlement itself — wirelessly, in the background, automatically. 

That’s exactly what the iPhone and iPod touch can do with their built-in networking stacks.  Even a slow network like AT&T’s EDGE network is good enough to verify that the monthly fee has been paid up.  Or for the wifi-only Touch, at least once per month you need to pass by an open hotspot or be in your house where it knows how to connect and it keeps working.  Not a serious burden.

So once there is a sufficiently large installed base of connected iPods, Apple will start selling a subscription service.  If I had more motivation to figure out the timing of when this would happen, I’d look at adoption/saturation curves for iPods and typical turn-over rates for such consumer electronic devices.  Other factors include the financial and market success of competing services.  I leave all this as an exercise to the reader for those of you working in this challenging industry.  My gut says it’ll be in 2010.

• Pickup trucks — many models, small and large
• Full-sized vans — think church-group van or delivery van, not soccer-mom minivan
• Mercedes — 3 models: E-class sedans, R-class station wagons, and GL-class or M-class SUVs.  All $45k+
• Volkswagen Taureg 2 — VW’s SUV has a diesel option starting at $68k
• Jeep Grand Cherokee — starting at $37k for diesel

Color me underwhelmed.  I might have missed something, but as far as I can tell there is exactly one non-SUV non-pickup diesel car on the market in this country: the Mercedes E-class.  Yowza.   Seriously, what gives?

In Europe, diesel cars are totally common-place.  While here we’re all abuzz about our fancy hybrids that can get 40+ mpg, Europeans can choose cars like the Citroen C4 which gets 46 38 mpg city and 71 59 mpg on the highway!  [Correction: These are per imperial gallon, which are 1.2 US gallons.]  I drove a Citroen (might even have been a C4) from Paris to Tuscany and back a couple of summers ago.  Let me assure you these are not stereotypically crappy French-engineered clunkers, but actually pretty nice cars, and not old-world tiny either. 

That number bears repeating.  71 59 miles per gallon on the highway.  When is this country going to get it together and raise the CAFE standards in a meaningful way and not just for show?  It’s for everybody’s good.

[Photo courtesy of Robert Candelori]

Economically, companies should be valued at the present value of their free cash flows.  That is to say, project forward all the possible ways the company might behave, and take a probability-weighted average (expectation value) of the total dividends the company would pay in each of these scenarios.  Discount these cash flows by an appropriate discount rate and you’ll get a fair market value for the company.  This is called fundamental analysis.

Now anybody who’s tried their hand at such financial calculations will know there’s a lot of judgement calls involved.  Small differences in numbers like discount rate or growth rates have huge effects on the results, and these numbers are hard to judge.  So it’s definitely possible to come up with a believable (by some) model of future cash flows that will value any currently successful company at whatever huge valuation you want.  But that doesn’t make it correct.  Is Facebook worth $300 per user?  It’s not possible for me to click on a $10 CPM ad every day for 100 years, but maybe they can add more users to grow into that?  Maybe?  It sure seems high.  I think there’s something else going on.

For context, think back to March of 2005 when Yahoo bought Flickr.  IMHO that made Google feel bad because Picassa wasn’t doing so well.  I think they saw this as a big missed opportunity to help organize the world’s photos.  I think this was big on their minds when they paid too much for YouTube.  And Google is still very far from monetizing this investment.  But they now control the dominant way that videos are communicated on the net.  This has to help them feel good about getting closer to their corporate mission of organizing the world’s information.  Since it’s not clear right now how they’re going to achieve that goal for photos.

Now consider Facebook.  Left and right, Facebook’s internal applications are surpassing total usage of th best dedicated net applications.  Their invitation app gets many times more usage than evite, and I believe their photos app is actually well beyond flickr in terms of usage too.  I don’t know where they stand for videos right now.  But it’s clear that they are a force to be reckoned with.  As I’ve written before, their application platform is potentially game-changing because it’s very attractive for information service developers and democratizes the process of product development in a novel and powerful way. 

For all these reasons, I think Facebook has the potential to dislodge Google as king of the hill.  No, Facebook isn’t going to become the dominant search engine, or even the dominant deliverer of internet advertising.  But I think Facebook could become the dominant way the humans communicate with each other using computers.  This could be the leverage they need to claim the crown of innovative thought leader on the internet.  If I were running Google, I’d be concerned about this possibility.  If I were running Microsoft, I’d be excited to get a piece of this.  Any piece.  Because even a tiny piece (like <2%) means that Google can’t take control of Facebook.  And yesterday, Microsoft got their foot in that door.  So, the game is on.  It’s gonna be fun.

Research consistently shows that most corporate mergers are unsuccessful.
That is to say, when two companies combine the value of the merged
company rarely even equals what the individual companies were doing by
themselves.  The reasons for this are numerous.  Organizations face
diseconomies of scale as they grow.  A simple example of such a
diseconomy of scale is increasing communications overhead — the number
of possible communication paths grows as N^2 for the number of people
in an organization, so if there is any cost associated with such paths
then the org will become less efficient as it grows past a certain
point.  A typical example of this cost would be in trying to find the
right (best) person to handle a specific issue or task.  At some point
finding the best person becomes too costly and workers instead choose
to duplicate the effort or work with a sub-optimal colleague.  For
these and many other less quantitative reasons like cultural mismatch,
merged companies are generally less efficient.

So why do they keep happening?  There’s an agency problem in upper management.
Everybody involved in this kind of M&A deal is personally
incentivized to see the deal go through.  Investment bankers of course
want to see their commissions.  But the bigger problem is that senior
management is typically compensated based not just on the economic
success of their organization, but also based on its pure size.  That
is to say, the CEO of a company with 10,000 employees and a 5% ROE
(return on equity) will probably get paid more than the CEO of a
company with 5,000 employees and a 7% ROE.  The 7% ROE company is
better run, more efficient, and doing better for its investors.  But executives
have this perverse motivation to decrease the economic effectiveness of
their organizations in order to grow their empires.  It’s a classic
agency problem whereby managers have a conflict between their personal
interests and their responsibilities as agents for stockholders.

Mergers only make sense when there is synergy.  This term gets thrown around loosely quite often but it has a specific meaning.  It refers to one company being able to make more effective use of its own resources when combined with resources from the other merged company.  For example one
company has a product that they don’t have enough manufacturing
capacity to meet demand for, and the other company has excess
manufacturing capacity.  Or one company has a new product that can be
sold more effectively at minimal marginal cost through an established
distribution channel that the other company has.  In these rare cases, mergers make fundamental economic sense.

Stay tuned for more specific thoughts on the big players buying bits of each other…

Expectation Value of Money

A basic analysis looks at what your expected income will be over the next 3 years.  Big company will clearly pay $300k.  Startup will pay $150k in salary.  A 10% chance of $3m is worth $300k in terms of its "expected value" following standard statistical methods.  So by this logic, startup is a better bet at $450k over the 3 years.  This analysis is correct and totally appropriate for how a large institution would analyze the trade-off.  But it has a problem when applied to individuals: it ignores the differences between utility and money.

Utility != Money

Different quantities of money have different relative values to people.  Economists express this in terms of "utility."  Most people will find $20 about twice as useful as $10.  Because you can do twice as much with $20 vs $10, we say that $20 has twice the "utility" of $10.  My favorite econ professor refers to utility as "jollies."  Utility is in arbitrary units, but here let’s say $1 is worth about 1 jolly, more or less.  So far this should be totally intuitive.

Things get a little stranger when we’re talking about large quantities of money.  For example, most people would not find $2 billion twice as useful as $1 billion.  For me, getting a gift of one billion dollars or two billion dollars would have almost exactly the same effect on my life — either way I’d be rich beyond my dreams.  The marginal utility of that second billion dollars is worth far less than 1 billion jollies — maybe only another million jollies over the first billion?  So for me, the graph of utility vs. money becomes nearly flat at these extremely high dollar values.

The shape of this curve at "intermediate" dollar values varies a lot from person to person, as does what intermediate would mean.  For example, consider two people — one who is working a minimum wage
job and the other who just made their first million dollars.  A $1,000
bonus would be extremely useful to the minimum wage worker, and almost
ignored by the millionaire.  But to the millionaire, the difference
between $400k and $800k is much more meaningful than it is to the
minimum wage worker.  For them, that much money sounds more like a billion dollars would to me.  Most everybody has a range of money that is too small to matter to them
and a range of money which is so large that differences don’t really
matter either.  Here are some factors that can affect the shape of a person’s utility curve:

• How much disposable income they are used to having
• How much money they have saved
• What expectations they have about future earnings
  • Career growth path
  • Are they expecting inheritance?
  • Pension plan or 401k?
• How much money would be needed to reach financial goals that can markedly affect their quality of life, like
  • Buying a car or house
  • Getting out of debt
  • Putting children through college

As promised, figuring out the shape of this curve is extremely personal.  I’ll describe a technique for doing so numerically at the end, and then I’ll duck.  Here is a spreadsheet {link fixed} that unscientifically attempts to graph what a utility curve might look like for somebody who is financially comfortable, but not exactly wealthy — like somebody who might be considering such a career decision.  And here’s the graph:

Applying Utility to the job choice

You might think that the next step in the analysis is to figure out how many jollies you’d get for the $300k from Big Company and compare that to the $450k from Startup.  Using the above data, this works out to about 649 kilojollies for Big Company and 907 killojollies for Startup, and Startup clearly wins.  These values are the utilities of the expectation values of money, which sounds impressive, but really isn’t very useful.  For uncertain outcomes, you should figure out the utility value of each financial possibility before calculating the expectation value.

Using the above utility curve, the correct analysis involves figuring out how many jollies you would get for each of these three possible outcomes:

• Work at Big Company ($300k = 649 kilojollies)
• Work at Startup with no bonus ($150k = 351 kilojollies)
• Work at Startup with big bonus ($3,150k = 3.25 megajollies)

The appropriate comparison is between 649 kilojollies for Big Company and (0.90 * 351kj) + (0.10 * 3246kj) = 641 kilojollies for Startup, which is comparing the expectation values of utility.  Expectation values are calculated by adding up the products of the probability of each event with the utility of that event.  See how this is different from the utility of the expectation value of money?  You apply the utility transformation before applying the probability weights.  Because the utility transform is non-linear, the operations are not associative, and order matters.  So, the important question is how valuable would that $3 million dollar bonus be to you, and is a shot at it worth the loss of guaranteed salary?

By this analysis, it’s not worth it for our hypothetical job seeker to take the chance.  The 649kj from Big Company is slightly higher than the 641kj from Startup, although it’s really close.  The closeness shouldn’t invalidate the result though.  You might want to check the assumptions used to derive this, but ultimately we have to make choices based on limited information.  We can understand this result intuitively because the sweet-spot on the money-utility graph is in the hundreds of thousands of dollars, not the millions.  This is the order of magnitude of money that would change this person’s life in the most meaningful way.

Tautological Conclusion

This kind of "what if" exercise is an accepted technique for calculating the shape of a person’s utility curve.  Would you rather have $10 or a 10% chance at $100?  Would you rather have $100 or a 10% chance at $1k?  Maybe I’ll put together a spreadsheet that actually uses this technique to find an accurate utility curve.  But you can see that this entire analysis boils down to a realization that you must decide for yourself if the reward is worth the risk.

Haha!  ;^) 

/me ducks

(Hopefully this discussion gives a framework for thinking about such decisions.)

We’re quickly approaching a world where we’re always connected to the net in some manner or another.  As we all know, the bandwidth with which we can communicate with the net varies tremendously between locations and situations.  It might be
as slow as AT&T’s EDGE network, or as fast as a dedicated office
line with many Gbps of throughput.  But when we’re in the office, the speed of our pipe to the net isn’t the limiting factor.  Usually it’s the servers on the other end which limit how fast we can get things done.  Even when I’m on my DSL line at home, Gmail is so slow that my pipe isn’t the limiting factor.  Effective bandwidth is limited by the smallest pipe in the series from your brain to the information service.

Sometimes the smallest pipe isn’t a network layer at all.  If you’re using your iPhone on the office’s WiFi network, the network will all run super fast.  But your effective speed will be the iPhone’s virtual keyboard, and there are many small devices which are way harder to use than the iPhone.  There are multiple places the communications pipeline can get clogged:

1. The physical Human-Computer Interface of your device
2. The UI of the software on the device
3. The local processing power of your device
4. The direct connection from your device to the series of high-speed routers and fiber known as "the net"
5. The processing power of the servers running the information service you’re using

Laptops have totally caught up with desktops in terms of #2 and #3, but not #1.  The reason to use a desktop machine is that you can trick out its Human-Computer Interface to be super high bandwidth.  You can get yourself a really nice ergonomic keyboard, multiple high-resolution monitors, and a real mouse.  A friend of mine even built himself a foot-mouse.  Pretty soon your desktop will start to look like Lain’s Navi.  (Pictured above for those not familiar with it — go watch it.  It’s rad.)

You can do some of this with a laptop docking station if
available, or by manually plugging and unplugging things.  Many laptops
support 2 monitors, but generally one of them needs to be the internal
monitor, which won’t match the second one.  A USB port multiplier can
handle all your input devices which is nice.  So if you’re happy with
just two displays, a laptop can probably get enough HCI bandwidth today.

Looking further down the line, someday Apple will extend the iPhone’s multi-touch UI to iMacs and give us the Minority Report interface.  This will offer far more Human-Computer bandwidth than we’ve ever seen before.  This trend will continue towards direct Computer-Brain Interfaces at which point the line between our biological brains and our "exocortex" will get very blurry indeed.  I can hardly wait.

In this environment it’s tempting to design products that assume customers will always be well connected.  It is certainly easier to build compelling services to users that have a good pipe to the net on them at all times.  So this begs the question: If customers will soon always have good broadband net access, why do we need a client-side data store like Google Gears?  For example, somebody working on a subscription music service might conclude that it’s a waste of time building portable mp3-players with local storage since soon enough everyone will have broadband access everywhere, so why not just stream the music off the net?

There are several good reasons why client-side storage is still important and will continue to be important into the future:

• Wireless net access sucks down battery.  Always will.  It’s physics.  Local access to data will always cost less battery.  This won’t change no matter how pervasive broadband is.
• Pervasive net access is expensive. Arguably we’re already in a world where some people have pervasive net access.  Verizon EVDO cards do pretty darned well in this country, for $60/month.  But it will be a long time before most people have it.  Higher speeds will always demand a premium.
• Net access is unreliable.  Especially wireless access, but wired too.  Packets collide.  Transmission patterns have nodes.  Routers flap.  Cables get unplugged.  Laptops wake up and can’t figure out where they are for a while.  Something gets misconfigured.  If your software is designed to gracefully degrade when the network is unreliable, your customers will be happier, because it’s going to happen.  Remember what Outlook/Exchange was like when the entire Outlook UI would freeze while waiting for the Exchange server to respond to any request?  Please don’t do that to your users.

Once web applications are fully embracing it, Google Gears will close most of the functionality gap between native-client applications and web applications.  I believe it’s really important, and I’m really glad that there’s industry consensus around Google Gears and that other offline browser storage projects have deferred to it.  I’d hate to see web app developers trying to choose between several different client-store plugins.

• Build your app without social networking data
• Start from scratch with your own social network
• Integrate your app with Facebook

The third choice is so simple, it is the obvious best choice for most new information services.  As I see it, this is the fundamental power of the Facebook platform and why they’re going to go very, very far.

What I’ve been thinking about building in my copious spare time is a web-based feed-reader that assumes you over-subscribe to feeds.  That is, it expects you to "subscribe" to more feeds than you can fully consume.  These days many of the most popular feeds on the web meet this criterion even if that’s all you subscribe too.  I don’t have time to follow any one of TechCrunch, Scobleizer, Engadget, or even Radar in their entirety — I generally don’t even get to skim all their headlines.  But I know people in my social network do, and when they do it would be a small extra effort for them to help me identify the posts that are worth me reading.

This could be done by explicitly recommending articles to friends, or by tagging, or rating, or any of a number of well-understood-yet-often-poorly-implemented mechanisms.  Additionally, I could subscribe to a meta-feed coming out of a single-friend or a set of people in the social network graph that could expand several levels.  And of course there would be meta-feeds covering the aggregate opinions of all users.  The result would be that I could "express mild interest" in a feed by "subscribing to it" and the system would help me figure out which of the voluminous posts were actually worth reading.  Or if other users tagged posts, I could find good posts on a particular topic.  It would encompass a lot of the utility of digg, techmeme and link blogs all at once.  Another step in the process of democratizing information consumption.

I’ve been talking with friends about building this in the context of a facebook app for reasonably obvious reasons.  I’d call it "the outside world" as a reference to the fact that college kids are generally so isolated from external news, and this would be a social way for those few who do read the traditional-news to share good things with their friends.  Facebook’s restrictions on apps processing social networking metadata would make somegood features difficult, but the advantages in marketing and lower barrier to entry probably outweigh that.  Now my idea is out there for the world, so I’m not getting a jump on anybody.  If anybody wants to take this idea and run with it, drop me a line and I’d be happy to help advise.  I might just do it anyway because the Facebook market and the Google Reader market are both healthy and the basics just aren’t that hard.

But it sounds like you’ll have stiff competition.  Quoting from Blogoscoped’s analysis of the video:

Google’s recent big social effort is called Mocha-Mocha (or
Mocka-Mocka?), and will become the infrastructure for all social stuff
across all of their applications. As a part of this, a new
feature called Activity Streams will be introduced or at least
implemented in Reader this quarter. This will be comparable to
Facebook’s News Feed (Minifeed?) feature, and integrate Gmail’s
addressbook and contact list.

Also there will be some other Gmail and Orkut integration, but this might just mean there will be links to Reader.

Hearing that Brad Fitzpatrick has joined Google and because it’s the kind of thing I do, I’ve been putting some thought into how Google could reasonably add social networking features to their services.  I’ve been talking to folks about how Facebook is currently Google’s biggest strategic threat because they’ve done such a good job integrating the social network into new feature development, and in doing so have democratized new feature development in a way the world has never before seen.  This need struck me as a good way to start integrating social networking features into Google. 

Orkut is and only ever will be a toy IMHO.  Let the Brazilians keep playing with it and don’t push it on the rest of us.  Between contacts and knowledge about whom we chat and e-mail with, gmail has vastly more meaningful set of social networking data.  As we’ve learned watching LinkedIn and okcupid and other social networks thrive side by side, it makes sense to have different social networks for different purposes.  Orkut is a toy network and should not be the basis of anything more meaningful.  Sorry, Orkut.

Simplified Simulation or Complete, Accurate Model?

The simulations Bostrom describes would not be precise to the subatomic level, but rather use abstractions to simplify the computation.  Instead of simulating every electron, proton, neutron, quark, etc in each person’s body and everything around us, it might only simulate synapses and neurons in our brains.  Such short-cuts would be extremely useful to accomplish the goals he describes of virtually resurrecting ancestors.  (A convenient version of heaven.)  Just simulating the brains of the inhabitants of a virtual world is drastically easier than accurately simulating an entire universe down to the subatomic level.  For many purposes, including the ones we are likely to engage in anytime soon, it is sufficient. 

The software to run a simplified simulation like this would put its designer in an interesting predicament whenever the simulatees decide to build a new particle accelerator or perform some other experiment that pushes the limits of their understanding of fundamental physics.  Would a dialog box appear on the simulation screen asking the designer to make decisions about how to treat a new class of quark that had never been observed?  Then once the designer answers this question the simulation moves on?  Moreover, so many trappings of modern life are the result of applications of scientific breakthroughs like this?  For example, we could have never built semiconductors and thus computers without a solid understanding of quantum mechanics since they take advantage of quantum effects.  So closing the dialog box would require not only require describing the results of this experiment, but also coding up a bunch of new high-level abstractions that represent things like semi-conductors.  The simulation would need to know when it could use the molecular mechanics model, and when it would have to substitute a more detailed model or a coding abstraction that simplifies the results of more base laws.

If we lived in such a simplified simulation, it seems likely that chinks in the armor of reality would periodically appear.  Modern science has few inconsistencies like this.  (The big bang and quantum randomness being the two biggest two exceptions IMHO.)  I would wager that if we live in a simulation it is a completely accurate physical model that started with the big bang and covers the entire universe including our own evolution from primordial soup.  It’s not clear to me whether or not our universe has enough matter/energy to build a computer powerful enough to run such a simulation.  I should dig up my notes from Yael Maguire’s excellent talk at Foo Camp on the fundamental limits of computation to be sure, but I know it would chew through at least solar systems worth of our universe if not galaxies or more to simulate a comparable universe.  It seems more likely to me that if our world is simulated then the “host world” is governed by a different set of physical laws.  This point is debatable and important, but I’ll assume from here that the host world is governed by different laws.

Motivations of the Simulation Designers and Implications for Personal Morality

As the NY Times article points out, the simulators might just be bored, doing the equivalent of playing video games with us.  Or they might be scientific researchers investigating how changes to fundamental laws affect how worlds evolve.  Whatever their goals are in running a simulation of this scale, they are almost certainly interested in the complexity that we are creating here and now.  But how should we behave?

Robin Hanson suggests that as individuals living in a simulation we should try to lead the most interesting, impactful lives that we can.  This goal attempts to optimize for the case that the simulators will pick individuals from this simulated society to do something special with.  I think it extremely unlikely that the designers care about individuals at all.  If they’re looking at anything, I’d bet it’s entire societies.  So, if we are living in a simulation, I argue that we should do our best to advance technology as an insurance policy against extinction.  I have written a fair bit about the transhuman morals that such a guiding principal implies, but basically it boils down to being a geek and/or a hippie – advance technology as fast as possible and conserve natural resources so that the world doesn’t end before we reach the next level of technology.  Thinking that somebody might hit the “stop” button on the entire simulation puts a new twist on the idea of the world ending because as a society we failed to reach a certain level of technological sophistication.

A Simulation Argument for Truth in Mathematical Beauty and Simplicity

If our world is a simulation running inside a massive computing device, then something must have programmed this simulation.  The programmers of the simulation chose the physical laws that we live by, perhaps to see what would happen.  This puts an interesting spin on evaluating fundamental physical laws.  Which of these two equations below is more likely to be an accurate representation of the way the simulation designer wrote the code?  These are two different mathematical representations of P.A.M. Dirac’s eponymous equation, which is AFAIK believed to be a completely accurate representation of our physical world.

By this logic, the second one is almost certainly closer to how the simulation programmer understood the concept.  This perspective puts an interesting twist on Occam’s razor – the principal that the simpler explanation is probably true.  My grandfather believed that the simpler a physical law was, the more likely it was to be correct.  In this way he saw a certain beauty in math and physics.  If our world exists only as a simulation, then the simpler a physical law is, the more likely it is to be an accurate representation of the way the simulation was coded.

Amazon makes extensive use of so-called A/B testing to try out new UI’s and optimize the user flow.  This works very well for them because their end goal is very well defined: they want people to buy stuff.  They are facing a very hard optimization problem, but their objective function is clear and easy to measure.  So they can try out new UI’s for 1% of users, and if it does well according to this well-defined metric, roll it out to a broader audience.  This is essentially best practice for any modern successful online company.

Google has done a lot to democratize the internet — notably by democratizing search through PageRank which allows anybody to implicitly vote on the relative merit of a web page.  They have also democratized the way some product development choices are made through through their policy of encouraging developers to build whatever they want in 20% of their time.  The result is that everything you can possibly imagine is probably being worked on by at least one googler, and the ideas with merit gain momentum and get built into real services.  But before they get launched to the public they still must be approved by a central authority.  Sure Google does A/B testing like everybody else, which is great for UI tweaks and to verify that new services won’t crash when hit with massive traffic.  But it’s extremely difficult to do A/B testing on major changes to functionality.  For example, it’s hard to imagine testing a change to how g-mail delivers mail through this kind of test.  Moreover, depending on how the test goes, the change is either rolled out to the entire user base or not at all.

Facebook’s platform offers another alternative.  ISV’s have the opportunity to offer major new kinds of functionality to Facebook users in a very democratic way.  Users can try out the new features, and if they like it, they’ll tell their friends about it, and the feature will spread.  Some features which are only appropriate for a certain segment of a user base can naturally find that segment.  This mechanism doesn’t really lower the cost of adding new functionality compared to how Google does it — Google is always launching new features that you’d never know about without reading their dozens of product blogs.  But it democratizes the process of figuring out which of these new features are valuable enough for a mass audience.  To continue with the democracy analog, these decisions are still made by a communist-style central-planning committee in Google’s world, whereas Facebook users can vote with their keyboards on what features are worth using.  This will make the Facebook platform very competitive in the arena of user’s attention.