Here’s how that math works, Kurzweil explains: The design of the brain is in the genome. The human genome has three billion base pairs or six billion bits, which is about 800 million bytes before compression, he says. Eliminating redundancies and applying loss-less compression, that information can be compressed into about 50 million bytes, according to Kurzweil.

About half of that is the brain, which comes down to 25 million bytes, or a million lines of code.

This reasoning is IMHO flawed and overly optimistic.  It’s an interesting idea to compare the complexity of these two systems by comparing their bit representations.  I think the idea has merit, at a very rough level — that is I think you can compare the complexity of a genome to the complexity of a piece of software on a rough order-of-magnitude scale. The biggest flaw in Kurzweil’s argument is that he magically throws in a factor of 16x improvement in his favor by saying the genome can be “compressed.”  Well, software executables can be compressed too, a fact that Kurzweil conveniently ignores.  So I’d follow his reasoning to say that a human brain simulator probably needs about 10 – 100 million lines of code.  (I’m deliberately including 0 significant digits here to indicate the roughness of this approximation.)  This puts a human brain simulator on par with the some of world’s most sophisticated software projects so far, which seems about right, at least to an order of magnitude or so.

Strong reactions

PZ Myers published a wrathful condemnation of Kurzweil’s argument titled “Ray Kurzweil does not understand the brain.”  If you sift through the name-calling you see that Myers assumes a specific tactic in building the brain simulator: starting with the human genome and deriving the brain’s functionality from it.  This strategy will certainly work, once we have solved the protein-folding problem, and more generally have the ability to do quantum chemical simulations of kilogram-sized masses of organic chemicals.  Which is to say it’s theoretically possible (we might be living in a software simulation of our universe for all we know), but completely intractable with current technology.  For comparison, our best quantum chemical simulations if you push them top out at maybe a dozen atoms right now.  So being able to simulate an entire kilogram of organic matter is nowhere in sight.

Tactics to simulation

I agree with Myers that we are nowhere near being able to interpret the genome well enough to understand how it makes a brain.  But we probably don’t need to in order to simulate a brain.  By analogy, consider the Super Nintendo (SNES) Emulator, which is another kind of simulator many of us have experience with.

SNES emulators let you play all the old Nintendo games but on a modern computer instead of original SNES hardware.  Let’s say somebody handed you a box and a stack of cartridges and told you to build a Nintendo simulator.  What would you do?  Well, clearly you could open up the SNES box and reverse engineer the circuit boards to figure out all the wiring.  You’d probably figure out that the CPU was important — a variant on the 65816, which was essentially the 16-bit version of the 6502 some of us grew up with in our Commodore 64s and Atari 800s.  So you could (theoretically) crack open the 65816 CPU chip itself, put it the through an electron microscope and understand every transistor it used to interpret the instructions. In this way you could reliably create an emulator which completely replicated every aspect of the SNES. Such a simulation would replicate all of its bugs, timing quirks and everything, but it would work and be extremely expensive to simulate.

This is analogous to the tactic PZ Myers seems to be assuming Kurzweil would take to simulating a human brain. But Kurzweil would actually start at a much higher level of abstraction.  Simulating every protein in every neuron is like building an SNES emulator by simulating every transistor in the original Nintendo’s hardware. The key to getting those SNES games to work does not lie in replicating the design of the CPU which interprets the instructions.  The key is figuring out how to run those instructions on modern hardware.  By moving up through levels of abstraction, we can simulate the system much more cheaply and easily, although there’s a chance edge-case behavior won’t be captured properly.  (What if our world is a simulation and we bump into the edge-cases?)

Similarly, the key to simulating a human brain anytime soon does not lie in understanding every chemical pathway in human neurons.  Although if we did understand neurons at this level, we would have a great head start at simulating a brain.  Success in simulating a human brain will come by recognizing higher levels of abstraction in neuronal function.  We have known for a very long time that neurons communicate by “firing” electrical signals which are transmitted chemically at synapses.  The details of these behaviors are complex and determined by a great many interdependent chemical systems, but it seems highly likely that we can replicate the key behaviors of human neurons at this level of abstraction without needing to understand everything underneath supporting them.  If we can replicate the firing behavior of neurons in sufficient detail, we don’t care what the proteins underlying them are doing.  The key question here of course is what is “sufficient detail.”  I expect that question is one that researchers who are genuinely interested in reverse-engineering the brain will actually focus their attention on.

Once we can simulate the firing behavior of neurons, simulating a brain becomes much more of an engineering problem than a scientific one.  Still it’s going to be a massive engineering challenge, and gathering the input data will probably require a bunch of new science.  Then the philosophers can debate the meaning of free-will if our brains are Turing-complete.

To get across the chasm, your technology really needs to work well.  Apple seems to have done that with FaceTime.  But there’s more than just having it work — you also need to explain to the public why it’s important.  Here, Apple is just paying for that with traditional advertising.  They’re putting out lots of touching, heartwarming commercials showing the value of video chat.  Apple is spending millions of dollars to explain to people why they’ll like video chat. Primary demand stimulation.  They’re working to overcome people’s biases against  the technology or the idea — that it’s clunky, or the extremely common “I don’t want to see the people I’m talking to” reaction, which is really pretty funny when you think about it.  Of course, there will be some times when you will prefer audio only, but that’s going to be the exception when the technology is good enough.

It’s all playing out in a textbook fashion.  The geeks are all crying that there’s nothing new here, that this technology has been around forever, and they don’t understand why it’s such a big deal.  But it is a big deal, because video chat is finally entering the mainstream.

Some side-effects of this that I pointed out before, but are perhaps more clear now as the story unfolds, are that this will benefit existing established video chat vendors.  Apple is explaining to people that video chat matters.  This will help Skype and Google and Cisco with their products.

Speaking of Cisco, there’s another prediction coming true: Cisco is pushing into consumer video chat.  I had guessed 2012, but barely more than a week after my last post, they announce Cius, a video chat terminal.  Kinda like a Flip phone fused with a linksys router, but running Android and in a pretty nice looking case.

Video chat is in a similar place today to where mp3 players were 10 years ago.  There are lots of video chat solutions out there on the market.  Skype is the most well known.  I helped launch Google’s video chat system across Gmaill, iGoogle and Orkut during my tenure there.  It definitely is one of the best on the market, and it’s still only appealing to early adopters.  I mean “early adopters” in the classic Everett Rogers sense, which is to say folks for whom the extra value of the new technology outweighs the hassles of using it.  This is a step beyond the “innovators” category, who are willing to bend over backwards debugging a brand new product just because they understand that it will be important later.  Video chat has been available to innovators for a great many decades.

With iPhone 4, Apple will push video chat to the early majority category. Apple has a history of sitting on potential technologies until all the bugs are worked out, which is fundamentally what’s needed to appeal to more than just early adopters.  I’m pretty sure FaceTime will be no exception.  In 6 months, video chat won’t be this geeky thing that people put up with out of desperation.  It will begin to be integrated into normal culture.  We’ll start to see television dramas and movies incorporate it as just a way some people communicate, rather than as a way to demonstrate how high-tech somebody is.  People who aren’t geeks will start to use video chat.

Video chat really matters

Those of us who have lived deeply with video chat understand its value.  There is a ton of additional content transmitted in video that helps communication on many levels.  It allows for a more nuanced informational discussion, but more importantly IMHO, it allows distant communication to be much more personal and emotional.  Anybody who has tried professional collaboration with another team that is thousands of miles away knows that this level of communication is at least as important for business uses as it is for social communication.  The first time you meet your collaborators in person, they become more real, more trustworthy, easier to talk to, especially about difficult subjects like problems that might arise in a project *gasp*.  Video chat is certainly not as good as meeting people in person, but it is a huge step above email, IM or phone.  (Getting drunk together I believe represents the highest professionally-accepted level of humanization.)

Human-to-human communication has always been the killer feature of computer technology. Video chat makes synchronous communication fundamentally better, and as such will become a major part of everybody’s life in the developed world in the years to come.

What does this mean for everybody else?

The history of technology innovation tells us that a couple things typically happen when an emerging technology pushes into the majority segments.  First, established players will all get a boost.  Apple will be doing a huge favor to Skype and Google Video Chat by removing the veil of geekiness from their products.  Apple’s huge investment in making this product work well will make all consumers more willing to try alternatives.

Another common side-effect is that the space will get more difficult for new entrants.  This usually happens as the technology standardizes.  There becomes a “normal way of doing things” that people start “to get.”  Before a technology can reach the majority, it will typically bounce around dozens of different modalities as everybody tries to find a way of doing it that resonates with the market.  This uncertainty represents a clear opportunity for start-ups and the subsequent standardization is the closing of that opportunity.  Another reason the space usually gets harder for startups is that economies of scale start to kick in as production levels ramp up to meet the larger demand.  This naturally favors large companies, since it raises the amount of investment needed to compete.

Which Social Graph?

Another reason the space might get harder for newcomers is the natural monopoly of social graphs — consumers are better off if there is a single definitive place to keep track of their contacts rather than having to replicate and maintain a different list for each service.  As such, social graphs are important assets to anybody in this space.  But if Tim O’Reilly gets his way (which I hope he does) and we end up with a loosely-coupled internet OS, this won’t be a problem for startups, as they’ll just be able to draw from an openly available graph, say from a Google or Facebook.

I’m really not sure how this aspect will play out.  My guess is that Apple will rely on the de-centralized social graph which is the contact list built into every iPhone.  It’s a less useful corporate asset than if it were properly cloud-hosted, which will make it harder for them to expand the service to OS X machines.  Perhaps they’ll make something useful out of mobileme here, but I have my doubts.  But given the revenue they get from App Store sales, it’s not clear that the OS X machine is even a major part of their consumer strategy going forwards.  If so, this would likely be a strategic shift for them, as the inclusion of web-cams on essentially every OS X machine for years was probably done in anticipation of making a major push into video chat at some point.

Don’t forget Cisco

In addition to the obvious players like Google and Skype, this is also incredibly important for Cisco too.  Cisco has long been interested in video chat.  Why?  The same underlying reason Intel has been investing in multi-media since the late 1980’s.  Multimedia on PCs needs lots of CPU power, and video chat needs lots of bandwidth.  It’s called “primary demand stimulation.”  To be very clear: Cisco wants everybody to use video chat because video chat uses lots of bandwidth, and when people are using lots of bandwidth, Cisco sells more big routers.

Cisco is in the final phases of buying Tandberg, who is the biggest supplier of video-chat hardware for businesses.  Their aquisition of Flip last year seems strategically odd in isolation, but in this context makes perfect sense.  They are building (buying) expertise in consumer-electronics which can handle high quality video.  Take a Flip camera, add a network (like a linksys wifi box) and you’ve got a video chat terminal.  I predict we’ll see such a toy out of Cisco in about 2012, as video chat fills the early majority segment and edges against the late majority.

I set about answering this question with data.   I quickly found a list of oil spills on wikipedia.  A quick pass through google spreadsheets and a few regexs later, and I’ve got the data in a form that it can be graphed with protovis, a wonderful web-based visualization package.  An initial look shows some interesting trends. (Sorry IE users – modern browser required.)

First, it’s important to emphasize that this is a log-scale graph.  Given the dynamic range of the input, it’s the only reasonable way to visualize what’s there, but if you’re not used to reading log-scale graphs, the data will be deceptive.  In short, being a little higher on the graph means that the spill is a lot larger.  In fact if would be very reasonable to only include the spills near the top of the graph when thinking about “big spills.”  But I wanted to present the entire data set for completeness and analysis.

It’s interesting to note the general downward trend at the bottom of the graph.  I believe this is not a real effect at all, but a result of selective memory.  The smallest spill on this graph was only a couple months ago — the Great Barrier Reef spill in April.  Are we to believe that in the preceeding 100 years of oil exploration there had never been a spill of less than 10 tons of oil, and only a single other spill of less than 100 tons?  Of course not.  I bet spills of this size have happened dozens if not hundreds of times, but 50 or 100 years ago nobody bothered writing them down.  Or if they did write it down, the event has been filtered out of our collective historical memory before making it into wikipedia.  The Lakeview gusher in 1909 is another interesting example of this effect.  This certainly wasn’t the only oil production accident before 1930, but it was clearly an important major accident, and so has been remembered far better than others.

I’ve highlighted a few other spills because of their historical interest.  The Gulf War oil spill (purple dot) of 1991 is exceptional in that it was not an accident, but a deliberate act of war.  As such, it should not be considered in answering the question of whether oil exploration has been getting safer.  The Exxon Valdez spill (light blue dot) in 1989 is large in our memory, but in context we can see that it was not at all a large spill by historic standards.  But clearly the Deepwater Horizon spill (green dot) is huge, ranking as one of the largest spills ever and certainly the largest spill in quite some time.  But aside from this current mess, there does seem to be a real trend towards increased safety in oil exploration.

Again, the log-scale graph makes this somewhat hard to read intuitively.  Because the spills near the top are so much larger than the ones below them, a fair approximation of the sum of all spills can be found by simply considering the points along the top envelope, which is generally decreasing.  Looking just at the last several decades on a linear scale, this trend becomes more clear: since about 1980, serious oil spills have been getting smaller / less frequent. Now we see visually that the majority of spills listed are tiny compared to the few big ones.  I scaled the graph so only the bottom of the uncertainty bar for the gulf war oil spill.  Also note that I’ve kept the middle dots at the geometric average of the low and high estimates, which works visually on the log-scale graph, and makes logical sense given the nature of the problem.

Another factor to consider is that the total amount of oil being produced during this time period has been generally increasing.  I’ve overlaid data from the US Energy Information Administration on global oil production rate, scaled to the average amount produced each hour, to get it to show up on the same scale of this graph.   Another interesting comparison which I haven’t included is the average size of each well, or the number of wells being drilled per unit time.  My understanding is that oil exploration has been getting more difficult over time in that we’re having to drill deeper to get at relatively smaller oil deposits.  Again, this reinforces the idea that we have been getting better and safer — we’re spilling less even though we’re drilling more holes.  Except for the Deepwater Horizon.

Feel free to browse the javascript source code of the graphs for further details, inspiration, double-checking, or whatever.

Having been writing code for 25+ years, the differences between programming languages fade away in my mind.  I know that I, like any good software engineer, can be productive in basically any language.  Certainly within object-oriented languages, (which is where essentially all serious software engineering happens these days) the differences in the language itself are IMHO small compared with other factors in choosing a platform.  Other important factors I consider include:

• Quality and availability of libraries.
• Quality of tools like IDEs, debuggers, automation systems.
• Size and healthy of the active community using the framework.  (i.e. If I run into a problem, how easy is it to google the answer?)
• Ease of hiring people who already know the platform.

All of these considerations are IMHO more important than compiled vs. interpreted or run-time performance or whether the language is statically typed or dynamically typed, or even if it’s open source vs. proprietary technology.  But there is a single common factor which directly feeds into all four of the criteria I list above: How many people are actively using the platform?

The Candidates

The last time I was really writing much code was back in 2007.  At the time Ruby on Rails was the coolest thing since sliced bread.  I played around with it, was amazed by how easy it was to quickly put together simple database-driven websites, and got very frustrated with it when I wanted to color outside of its lines.  A major concern of mine was the level of “magic” that happens behind the scenes — this makes Rails beautiful and elegant when it works, but difficult to debug or extend.  I blogged about my experiences with rails and in particular my conclusion that at the time Rails was not ready for large, complex projects, partly because of a lack of good tools, libraries and sensible error messages, all of which can be fixed by more users.

Around then I started working for Google and stopped writing code.  The next year or so saw a couple big things happen for Ruby on Rails.  Twitter famously had trouble maintaining even 99% uptime (fail whale anybody?), and everybody knew they were running on Ruby on Rails.  Also, Google launched AppEngine, which supported Python, not Ruby, and pointed people like me to an alternative high-level web framework: Django.

So now, in 2010, I return to the fray, and I’m trying to decide between the two frameworks.  There are of course other alternatives, but for the purpose of brevity, I’ll leave out my process of reducing my choices to these two: django or rails?

Measuring community activity

I often use Google Trends to measure relative interest in technologies.  My time is very valuable, so even downloading something and reading its documentation is an investment I’d rather shortcut if I can.  This crude measure of relative search activity can actually be quite telling, and has saved me a bunch of time in choosing packages.

The relative values are hard to take literally since “django” as a single word search query will naturally be higher than a 3-word query like “ruby on rails”, but the search for “rails” by itself will clearly have lots of irrelevant searches.  Likewise, some people searching for “Django” won’t be looking for the web framework, but rather the guitarist or the movie or what-have you.

But it is clear that search activity for Ruby on Rails peaked in 2007-2008 and has been declining since then, while Django has been on a steady upwards trend.

Elsewhere I can find evidence that Rails is still a more used platform.  Stackoverflow has twice as many questions about Ruby on Rails vs Django: 13,882 versus 7,496.  To me this indicates pretty clearly that Rails is more active than Django.  Either that or Rails is more confusing and people ask more questions about it, but I doubt that.

The Tiobe Index attempts to objectively measure activity across programming languages.  By its easily manipulated measure, Python has twice the activity of Ruby, independent of the django / rails frameworks, with both languages in decline.

Meanwhile it’s easy to find comparisons on the net between the two.  Everything from content-free videos to people declaring rails dead, and everything in between.

On balance, I find the stackoverflow numbers the most compelling, unbiased indication that Ruby on Rails has more activity than Django / Python.  Even though I’ve been frustrated by it in the past, by my own objective criteria, that seems to make Rails a better choice for building a new web application.

What’s your opinion?

If you have experience with both frameworks, I’d love to hear your experiences.  Please leave a comment.

Simultaneously, technological advances have allowed advertising to progress along a different axis – to become more targeted.  Advertising used to only be broadcast widely through newspapers and television shows.  The best an advertiser could do to ensure their message reached the right kind of people was to select the aggregate demographics of everybody who read a particular magazine.  Now the internet allows ads to be targeted as precisely as you’d like.  Today, Google lets you get your message only in front of people who are about to buy a product like yours.  The ability to connect to people who have expressed an intention to “buy digital camera” is a literal gold-mine, making billionaires out of Larry, Sergey and Eric.  As effective as it is, targeted advertising won’t replace broadcast advertising, because there is still value in abstract brand-building.  Rather, the two will complement each other.

Enter Social Media

Social media has been all the buzz recently.  At its core it’s just a more convenient way for friends to communicate.  The “killer app” for computers has always been helping people communicate, and this is just another chapter in that book.  With this new communications medium comes a new opportunity for organizations to tell their stories.  In fact, I believe that social media will bring another tectonic shift in the entire marketing industry, possibly as important as search-based advertising.  As consumers have gotten more and more sophisticated at filtering out advertising from broadcast media, advertisers have gotten more and more desperate in their attempts to connect with people.  Social media marketing offers a new path – instead of hearing about products and services through ads, people can hear about products and services from their own friends.  Exactly how this will play out through Twitter/Facebook/Foursquare/whatever is not at all clear to me right now, but I fundamentally believe this change is coming, and it will take the entire marketing industry with it.  Klein and her fans are free to unplug from popular culture in order to avoid the onslaught of brand advertising, but they would be foolish to stop talking to their friends just because their friends are happy with things they’ve bought.

This vision is one of the main things that prompted me to jump off the comfy Google cruise liner and start paddling hard in Banyan Branch’s crowded dinghy.

Is marketing intrinsically evil?

I sometimes feel a need to justify this line of work to those who think that marketing is inherently dirty.  I admit that I’m more of a capitalist than many of my friends, but I certainly recognize that capitalism has its limits.  The vast majority of economic transactions are both consensual and mutually beneficial, and I will argue vigorously that there is nothing wrong with an economic system consisting of these transactions.  The biggest exception to this happens when transactions are not mutually beneficial because one party is not fully informed.  But what we’re doing is helping people share honest opinions and feedback about the things they buy and use.  By lubricating the flow of information between real people, I believe social media will reduce the effectiveness of deceptive marketing.  Moreover, it will help companies connect to their customers and hone their goods to people’s real concerns and desires.  It will help hold companies accountable for their mistakes, and enable companies to better make things that make people happy.

Additionally, I will point out that my employer represents no small amount of “pure good” for the world, including organizations such as The Bill & Melinda Gates Foundation and Vittana, helping them tell their stories.

Taking a chance on a startup

Why did I choose this opportunity out of the sea of possibilities?  I evaluated the landscape as an investor would, since I am investing no small chunk of my life in this effort.  From my entrepreneurial training and experience, I know that smart investors care more about the people than the specific business plan.  The plan will almost certainly change, but the key management will not.  Having known one of the founders of Banyan quite well for a number of years, I am certain that many key elements for success are in place.  The corporate culture and governance will be solid.  I will be working in an environment where I am supported, and where I can learn and grow as a manager and a technologist.

Exactly what will I be doing or building?  I admit I’m not sure yet, but I have some very interesting ideas that I won’t be sharing here anytime soon.  I am sure that my work is very well positioned to be a part of a major shift in an entire industry — a rare opportunity.  Whether or not my work will play a key role in this shift is somewhat out of my hands — these things are always a roll of the dice.  But in another sense, it’s entirely within my control, and this is what I love about working in a small company.  There’s almost nothing but work between me and effective execution of our ideas.  Many people tend to exaggerate the importance of the idea itself, forgetting that it is incredibly important to execute well on whatever ideas you have.  I’ve heard people say that they had the idea for YouTube years before YouTube did.  How quickly we forget the dozens of other companies all working on the same problem in 2006, which almost all fell by the wayside because they didn’t execute as well as YouTube did.  Ideas matter for sure.  But hard work is critical.

If you’d like to jump on this raft and start paddling too, get in touch with me.  I need a few key rock-star developers who are’t scared of chaos and can think creatively about business problems.

In the summer of 2007, as I was finishing up my MBA, I was faced with a choice of where to work.  I had this job at Google, and was also considering spinning up a startup on my own.  I had spent the last several years studying entrepreneurship and wanted to put that training into action.  One thing that really pushed me to Google was some advice from a very wise friend who was also looking for a new gig.  He said something like this.

“So many people have adjustable rate mortgages, it’s ridiculous.  They’re going to reset and people won’t be able to afford to pay them any more.  I think a lot of people are going to default on their mortgages and lose their houses.  I bet that will drag the entire economy down.  That’s why I’m looking for a job in a nice big stable company.”

He went to Amazon.  Having tried to start a small venture in a recession previously, I decided that Google would be a great place to hide out.  I also figured (correctly!) that I could learn a lot about how an extremely well-run company works from the inside.

Bottoms-Up Management

I learned how bottoms-up management works.  The idea of giving as much control as possible to the individual contributors who are working on the problems themselves.  Google exemplifies the saying “let 1,000 flowers bloom” with teams seemingly running in every direction at once.  It’s worked very well for them, and I can see a lot of value in this structure.  But I also see its limitations.

Having spent a while working on building infrastructure projects at Google, I experienced first-hand the limitations of bottoms-up management.  Individuals are empowered to make their own decisions, and they optimize for what’s best for themselves, which means their careers and their projects.  It’s easy to optimize towards a local-minimum which is great for your project, but not necessarily ideal for the company as a whole.  Similarly, top-down directives are extremely difficult to enforce.  Executives can make decrees about how things should work, and it just doesn’t happen.  I remember walking out of a review with Larry, Sergey and Eric and hearing one of the Directors respond “well that’s Sergey’s opinion.”

I spent a bunch of time trying to encourage teams to standardize parts of their backend infrastructure on shared components.  Shared technology is rarely the easiest or fastest thing for an individual project, so let me tell you the road was challenging.  I’ll refrain from pointing out the holes in Google’s products that I was trying to plug which are still open.  (To be clear, I’m referring to missing features and UX annoyances, not security holes.)  But I also learned some important lessons in how to design such infrastructure so that it is a win-win-win for all the teams involved.

Alignment with Customers

The best thing about working at Google was the clear alignment with customers.  Because Google is so strong in web search, they can afford to invest heavily in primary demand stimulation.  Like Campbell’s paying for an ad campaign that says “Soup is good food”, it makes financial sense for Google to work very hard just to get people to use the web more.  It goes beyond the motto “focus on the user” — Google really does want whatever the user wants, at least 99% of the time.

This makes huge classes of problem solving so much easier, because there’s no cognitive dissonance trying to remember the difference between a great user experience and a sound business decision.  This particular aspect of the business was an extremely refreshing change coming from Real Networks.  I remember working hard on re-designing the Real Player installer experience and tackling that specific problem head on.  The company had become addicted to the pocket change they could scrape out of their unsuspecting customers by surreptitiously taking over their computers when the Player was installed.  None of that BS at Google.

Looking forwards

Google faces some serious challenges in the years ahead.  For many of the reasons I pointed out in 2007, Facebook continues to be a serious threat to Google’s primary business.  Android faces an uphill battle against Apple’s dominance in the all-important mobile space.  Amazon seems to be squarely in control of the hosted cloud computing space.  And Microsoft seems to be pulling their head out of the sand for office productivity software.  None of these problems will be easy for Google to solve.  But I’m confident the amazing folks I worked with there will put in very strong showings.

As for me, I’m going to work in social media.  I’m very excited about my new role which I’ll write more about soon.

I was patiently waiting for the feature to be turned for my account, which wasn’t happening.  I kept getting the same old spreadsheet program.  But then I found the setting to enable it.  So I’m sharing with y’all an explanation of how you too can get these great new features.

Go to http://docs.google.com/ and click the select “Settings” link in the upper-right hand corner, and choose “Documents Settings”.

Then choose the “Editing” tab and check the checkbox next to “New version of Google documents”.

Now any new documents you create will use the new real-time editor.  And anybody you share them with will get the new features without having to set this up for their account.

In Walmart’s case, it’s generally suppliers who get squeezed.  Walmart demands that manufacturers of goods produce them at the lowest possible price so that Walmart can charge the lowest prices in their stores.  They really do try hard to pass the savings on to you.  Another case that is less well known is with so-called “interchange” fees for debit and credit cards, charged by the card networks like Visa and Mastercard.  Back in 2003, Walmart pushed hard on Visa and Mastercard to charge less for debit card transactions since they are both lower risk (because of pin-code use) and cheaper to process (verifying signatures is expensive).  The cynical will point out that with lower fees, Walmart just gets to keep more profit.  Which is true.  But they are genuinely motivated to lower prices for consumers, since that’s their main selling point.  So it’s a win-win – Wal-Mart’s motivations to lower costs are closely aligned with consumer’s desires to pay less.

Apple has similar desires for their network-connected gadgets like iPhones and iPads.  Apples wants people to be able to connect their devices to the network for as little as possible. Apple has clearly negotiated very hard with AT&T to demand low monthly rates on data plans for these devices.  Next month you’ll be able to buy an iPad with a 3G data plan for just $15 / month.  That is basically unheard of in the US.  For people on a limited budget, the iPad is the cheapest way to get online.  Compare this to other data plans available from major U.S. carriers:

The Apple / AT&T rates are the lowest in each of their categories, except Verizon’s smartphone data plan which ties the AT&T iPhone plan.  The iPad rates are extremely low compared to data plans for laptops, and also when when you consider that tethering plans or phone data plans require paying an extra $30/mo – $50/mo for a voice plan.  The unlimited iPad plan is literally half what it costs to get 3G on any other laptop, and it doesn’t come with the 5 GB limit that other plans do.  You might argue that the iPad can’t do as much as a full laptop, which is true.  So you might then argue that iPad won’t tax the network as much as a laptop, which I doubt considering the propensity to consume video on such a device.  So you can’t trade torrents on an iPad, which from an Intellectual Property perspective is just fine with me.

My guess (and this is pure speculation) is that Apple negotiated these rates by offering AT&T a share of the revenues generated through App Store purchases.

Again, the cynical will point out that Apple is just trying to grab the lion’s share of economic surplus for itself, which is true.  But nonetheless, this is a case where Apple’s desires and our desires as consumers line up well.  In a very real way, Apple is fighting on our behalf for lower prices from AT&T.

Apple products are expensive. Apple gets high margins on its hardware, allowing it to recoup large investments in NRE (non-recurring engineering) to design the hardware and its accompanying software.  This is a great place to be from a competitive standpoint, because as a company they don’t need to squabble over the cheapest parts to try to deliver the best prices to consumers.  So long as they can maintain a sufficiently large customer base to support the practice, it is an easy place to defend against competition from.  Certainly a lot easier than being Dell or HP, who struggle with operational efficiency to compete on price, and try to innovate within a very narrow window defined by their platform.

Apple’s success at selling high-end products has secondary benefits for the rest of the ecosystem.  Because the products are expensive, they tend to be purchased by people with more disposable income. So the segment of the computer market which buys Apple products self-selects to be very attractive demographic for many other reasons.  Advertisers love to get their products in front of people who are more-willing-than-most to buy something expensive / unnecessary / fun.

Similarly, app developers know that if they write an app for iPhone / iPad, the people who are able to buy it are much more likely to be willing to pay a couple bucks for something silly than, say, somebody who bought the cheapest smartphone they could afford because they felt they really need that functionality.  I had previously speculated that Apple’s platform play required a very large distribution base to attract developers, which is not quite correct.  The strategy is successful even with a relatively small market, provided that the market is segmented properly.  Which in this case it clearly is.

Preparing for this talk has been a lot of fun.  I’m guessing it will be fun to deliver as well.  (I’m writing this to be posted immediately after my talk, so I can’t know for sure yet!)  The whole process reminds me how much I love my career — the huge impact I can have on making people’s lives better.  This particular talk was a very good reminder to me how much I rely on my training as a scientist to perform this job.

This kind of information and much more is available from a cool web site called Space Weather.  For example, did you know that just a week ago an asteroid got as close to the Earth as the moon is?  It was just a 27 m in size, so not earth destroying.  But these encounters are happening all the time.  Browsing around SpaceWeather a bit more and you’ll find great pictures of the sun like this one.  Or the current interplanetary magnetic field measured in nanoTeslas.

All in all, fun stuff.

I don’t want to give too much away, but from watching the trailer you can tell it’s gonna be good.  The story explores “emoticlips” which are a way to digital encode and transmit emotions, like a podcast.  Drama heats up when a viagra ad shows up, something about blackmail.  And my favorite line asks if you’ve tried Googling “omnipotent self-aware botnets.”

I just saw a fun trailer for the play on YouTube, shared here for your convenience…

You can get your tickets now from Brown Paper.

In the course of learning about people’s personalities for helping them get dates, they also gather deep demographic data.  Recently they analyzed these data with respect to politics.  They show a series of graphs that would make Edward Tufte proud, analyzing people’s preferences along the political plane defined by permissiveness vs government control over economic and social issues.  They look at how these preferences change with age, and how relatively important each axis is.  Their conclusions match exit poll data quite nicely and demonstrate analytically that the Republican party is much more focused in the issues it cares about, while the Democratic party draws in people whose opinions are much more diverse.

It’s a long article, but I heartily encourage you to read it, or at least skim the diagrams and play with the animation on the 6th chart.  Nice analysis.  Very nice presentation.  Nice job, folks.

The Blue Brain Project is the first comprehensive attempt to reverse-engineer the mammalian brain, in order to understand brain function and dysfunction through detailed simulations.

Others have described it as “a step toward the superbrain” or even ”the most interesting project in human history.”

I agree that this project is extremely important.  In my mind, this line of research is the only reliable way to achieve AI – artificial intelligence.  There’s little doubt in my mind that a sufficiently powerful computer could simulate in detail the operation of every neuron in a human brain.  Someday we’ll get there.  If you buy the argument so far, then the only thing standing between us and strong AI is an extremely detailed scan of a brain — down to the sub-neuron level.  In principal this should be possible by freezing a brain, slicing it up into very thin sections, and scanning them.  More sophisticated 3-D imaging techniques might even make this possible without having to cut somebody’s head off, but I’m not necessarily counting on that.  Besides, for the purpose of creating artificial intelligence, we just need to do this once, and it doesn’t even need to be perfect.

In the past, I’ve explored the question of whether or not such an AI would have free will.  This fascinating question about the nature of consciousness is fairly abstract, along with other ones related to the nature of consciousness, and what destructive uploading means to the person whose head gets cut off and sliced up.  (Do you just take a nap when they cut your head off, and then wake up inside the computer?  What happens when they spin up a second process?  Etc. Etc. Etc.)  But all of this philosophizing pales when I realize they’re actually doing it!

IBM is supplying one of their Blue Gene/L “supercomputers” with 8,000 processors — definitely cool hardware.  I’m not sure what supercomputer really means in the era of massively parallel cloud computing, but maybe this is exactly it.  Read more in their FAQ.  Or watch this TED talk:

Thanks to Joe Duck for digging this one up.

Scores of developers have had iPad’s for weeks now. They’ve had to sign non-disclosure agreements, and have the iPad locked in a separate room that random employees couldn’t access. And even that wasn’t enough. The iPads are literally chained to the desk with steel cable and a lock. Apple comes by the office with a suitcase, installs the iPad in a bolted case, chains it to the desk and locks it there. And they they do occasional surprise visits just to make sure it’s still there.

What’s more, Apple has told developers that they are monitoring the location of the device as well.

Apple.  Gotta love them.

Mike says he can type 50 wpm on it.  That’s really quite cool.

I’ll be developing marketing and distribution programs to broaden access to clean water for families living on as little as a few dollars a day.  Every year over 1.6 million people, largely children under the age of 5, die of diseases caused by unclean water and poor sanitation.

I really admire Jeremy for his can-do attitude, working on very important problems that are not at all close to home.  I invite all of you to follow along on his blog Clean Water For All and if you feel so moved to help them out.

UniXKCD command line console

My favorite webcomic, Randall Monroe’s brilliant XKCD, is running a command-line version of itself today.  A few commands you might want to try include:

• find
• wget http://xkcd.com/
• Make me a sandwich
• go west

Google renames itself to Topeka

In honor of Topeka, Kansas renaming itself Google in a bid to get ultra-high-speed broadband installed, Google has renamed itself Topeka today.  Although Google is well known for April Fool’s jokes I believe this is the first time any have been on the homepage.

YouTube’s TEXTp mode

YouTube has the option to render most any of its videos in ASCII by adding the &textp=fool parameter onto the URL.  Looking at bandwidth graphs I can’t tell if they’re actually sending ASCII over the wire, or doing the conversion client-side.  Fun trick though.

Bing’s funny cows

Bing has one of their defining pastoral pictures, this time literally bucolic, but with fake cows.  They’ve supposedly been bread to only make non-dairy creamer.  Glad you’re trying, folks.

Great job everybody!  It’s awesome to see what you can do without leaving your browser.

The big problem I have with the current variations of the CC license is the “who knows where else” part.  Folks are free to use my content, so long as they attribute it to me, without ever letting me know about it.  And that removes a big part of the fun for me — I get a huge kick when somebody tells me they want to publish my photo.  As a matter of politeness, I’ve taken to notifying content creators when I use their CC license, so they can get the joy.  But the license variations can’t require that.

What I’d like to see is a “Notification Required” variation of the Creative Commons License.  It would go alongside the current variations:

• Attribution
• Share-alike
• Non-commercial
• No derivative
• Notification (not yet implemented)

It would be executed similarly to the Attribution requirement — somewhere the content creator needs to explain to re-users how to attribute, and in this case how to notify.  In fact, I’ve updated the CC notice by my photos requesting notification.  I’m asking people to jump through some small hoops to contact me as a form Turing Test.  But if you understand English, they’re really pretty easy.

I know others would like to make similar requests when their content is re-used, so I think this should become a part of the standard array of options for CC licenses.

I’m quite excited about being invited to speak in this forum.  Science and physics in particular have always been very close to my heart.  Although I spend my days working on what I consider very practical problems compared to pure science of physics, I always enjoy musing on the underlying rules that govern our universe.

The conference is in Portland at the Pacific University of Oregon on April 16th and 17th.  If you have reason or occasion to attend, I encourage you to do so.

The author of the book is Arthur Benjamin.  He gave a demonstration of his mad skillz at TED a while back, which I’m embedding here because it’s awesome.

Migrating this blog has been fun because it’s forced me to look over a lot of the old content I’ve written.  A couple years ago I found Benjamin’s Ted talk, which has inspired all this craziness.  I think it’s good to keep the brain fresh by taxing skills that one might not have used in a while.

iSlate is rumored to have a bunch of new content associated with it.  Particularly print content.  Print publishers will probably want consumers to sign up for subscriptions.  So Apple’s probably going to be introducing people to the concept of content subscriptions on their portable devices, likely with iPhone OS 4.0 which probably will run the iSlate and old iPhones and iPod touches too.  So I wouldn’t be surprised if you can get an all-you-can-eat music subscription service available too.  We’ll see.  It’s pure speculation, but it would make sense.  I’d be particularly tickled if my off-the-cuff prediction of dates from 2007 turned out to be right.

Unless Apple has come up with a better way to do soft keyboards, that is.  When I say “soft keyboard” I mean the kind of keyboard that appears on a touch screen and has no physical keys.  I’ve complained about the iPhone’s keyboard for a while.  While it’s true that people do get better at using these, I still don’t think I’ll ever be nearly as fast or accurate (even with smart correction) with a soft keyboard as I was with my blackberry.  I think that’s probably true on average for most people.  The basic reason is the lack of tactile feedback.  With a physical keyboard, if my fingers are slightly off target, they are guided to the right place by feel.

I’ve argued for some time now that the way to solve this is by figuring out how to make a touch-screen display with tactile feedback.  How would such a device work?  Physically I couldn’t tell you.  But what we’d need would be a way to electronically manipulate texture in a clear material.  A plastic with a matrix of cells that could expand or contract under electronic control.  So the software could create bumps where each of the keys are.  This would allow a software-reconfigurable gadget that could be almost as usable as a dedicated-purpose device.

This is very different from what is commonly referred to as “haptic feedback” on some of today’s gadgets like the Nexus One.  Here, the phone’s vibrator pulses a bit when you press a soft key.  This is a kind of feedback which is tactile in that you feel it, and it gives you information about your interaction with the device without having to look at the screen.  It certainly helps.  But it is not going to improve basic typing for a critical reason — it can’t help guide fingers to the right place.  The basic act of positioning fingers on controls is still basically open loop, feed forward, without guidance.  What I’m referring to as tactile feedback helps the fingers find the right spots to press without looking.  Today’s haptic feedback can’t do that.

To be clear, true tactile feedback like this almost certainly doesn’t exist yet.  This kind of pure technological innovation basically always starts in universities or government run labs.  The ROI on pure research into unproven technology is so low that it doesn’t make sense for companies to invest there.  Even if a company proved this was possible (which AFAIK hasn’t been done yet) they’d need to figure out how to manufacture it at scale before they could sell a device with it.  Last time I predicted it would be about 2012 before we saw these.  Even though Jobs almost certainly foresees the value of such a system, Apple’s expertise is not in material science.  Wired speculated about such a keyboard based on Apple’s patent filings, but what they describe seems a bit too sci-fi for me to believe.

If they have come up with something new and cool, it’s going to be a smarter way to use basically existing hardware.  I’m gonna guess it’s probably something like a touch screen which is pressure sensitive, so you can rest your fingers on it without indicating a “button press”, making typing more natural.  You could combine this with fixed, transparent dimples on the screen under the positions where the keys are, and you’d do pretty well.  Restrict the keyboard to only work in landscape mode and you only need one set of dimples.  This would be a huge improvement in usability and the biggest technological breakthrough would be the ability to distinguish a soft push from a hard push on a capacitive touch-screen.  Like by how much surface of your finger is on it.

Regardless of what Apple’s actually managed to achieve, I wish them the best.  They’re really pushing the envelope on human-computer interactions.  If they’ve done anything significant to improve soft keyboards, they will have once again done something that the entire rest of the industry will want to emulate, and I’ll tip my hat to them.

• Keep all blog posts and comments
• Keep all posts at their original URLs
• Maintain all category pages at the same URLs

The first one’s easy.  Google released some migration tools which cover that quite well.  But, at least when importing from typepad / movable type, they don’t preserve permalink URL’s.  So anybody who followed a linked to a specific page on my site would get a 404 page.  Weak.

I spent a lot of time on this.  Basic problem is that Typepad doesn’t include URL information in their export file format. It would be very easy for them to do this, but then why would they want to make it easy for you to leave?  Actually the answer there is easy.  Because by trying to lock in users, they create angry vocal opponents of their service.  I’m not angry, but I would advise against anybody considering Typepad as a blog host, specifically because of their tendency to lock people in.

<rant> Don’t keep my data hostage.  It’s my content.  I created it.  You’re just delivering it.  Do not try to lock me into using you as a service provider.  You might get some more money out of me, but every dollar I give you after I want to leave will contribute to my dis-liking you.  As the internet matures and consumers become more sophisticated and better able to share their experiences with each other, they will increasingly choose the service providers who are open.  (Echoing Jonathan Rosenberg’s recent diatribe on openness.)  I really appreciate Google’s commitment to Data Liberation.  My current provider, Dreamhost, also does a splendid job of giving me control over my data.</rant>

It turns out that getting a full-fidelity export out of typepad is possible with some work.  I followed these instructions from FolioVision which provides a custom export template that does include URL’s.  If your blog has more than 100 posts, then you need to change the first line to

<MTEntries lastn="100">

<MTEntries lastn="100" offset="100">

<MTEntries lastn="100" offset="200">

etc. and merge all these files together into one big export file that has URL’s.  Then I tried to get blogger to honor the import file with permalinks but I couldn’t.  I do believe blogger is capable of doing this, but what ultimately turned me away from it was that it doesn’t seem to offer any way to honor links like www.embracingchaos.com/humor for category listings.  Which I like and get a lot of visitors on.  So I went with wordpress.

FolioVision helpfully posted a custom wordpress import plugin to match their typepad output template, which makes it all go.  Once that’s done, you have to move all the attachments hosted at typepad, and then there’s a bunch of wordpress configuration, and moving your analytics and favicons and finally switching DNS.

So here we are.  Please tell me if you notice anything amiss with the new site.

Broadly, the concept of information democracy is that an increasingly large number of people are able to influence how information is aggregated.  Wikipedia is a clear and simple example of allowing anybody to contribute to what used to be authored by a select few — “The Encyclopedia.”  Google’s Pagerank algorithm democratized web search.  Today’s most successful software is democratizing the feature set by allowing users to vote on how they want to use it.  The general principal is that large numbers of individuals can together make better decisions than any small group.  Applying this principal to culture, we can predict that a cultural democracy will produce “better culture” than what was available before.

Information technology makes it cheaper and easier to both create and to distribute culture.  With the right software, any laptop today has all the power of a professional music or video studio.  Sure the quality won’t be as good without professional inputs (microphones, cameras, etc) but the cheap stuff is good enough for a lot of things.  Obviously the internet makes distribution of this content trivially easy, which is disrupting traditional media businesses.  Easy creation and distribution of cultural content is an important part of creating a cultural democracy, but it is not the critical enabling step.

The key to democratizing culture is in the editorial process.  If everybody is contributing cultural content that is easily distributed, but there’s still a small group deciding which pieces everybody watches, we’re still in a cultural dictatorship.  Enabling the mass public to “vote” on content is the democratizing step.  That enables the collective intelligence of all media consumers to help choose what should become part of mass culture.  So instead of some programming executive trying to guess what will be popular, the question almost becomes moot — whatever is popular becomes popular culture.  Actually making this work is not at all straightforward.  I’ll save a full description of the necessary ingredients for another post, but we can look at a couple examples.

Youtube does this quite well.  It blurs the line between sharing a video clip with your friends and publishing it as a piece of mass culture.  Any video that isn’t marked private is submitted into a kind of massive popularity contest.  Videos that get millions of views are undeniably bits of popular culture.  For music, last.fm does a good job of being inclusive, but hasn’t quite taken off.  When I started building social features into Rhapsody I hoped they could democratize the music editorial process but that hasn’t happened yet.  Like many things in social media there’s a chicken and egg problem with scale which Youtube has clearly gotten past, but music is still struggling with.

Cultural Democracy is “retro”?!

This post is inspired by a recent story by Heather Chaplin that NPR aired describing participatory culture in video games.  The surprising part of the story for me was the assertion that this trend is not modern but in fact “retro.”  The story points out that before analog broadcast media, most culture was participatory — singing, dancing, crafts, etc.  Analog technology created the possibility of cultural hegemonies, and digital technology is breaking them down. A fine point, implying that the 20th century will likely be unique as the only period in human history when popular culture was dictated by an elite group of editors.  Thanks for the interesting tidbit.

]]> http://www.embracingchaos.com/2009/12/participatory-culture-and-the-democratization-of-information.html/feed 1 http://www.embracingchaos.com/2009/12/2009-a-year-of-commitments.html http://www.embracingchaos.com/2009/12/2009-a-year-of-commitments.html#comments Fri, 18 Dec 2009 04:44:49 +0000 leodirac http://wp.embracingchaos.com/2009/12/2009-a-year-of-commitments.html As the year wraps up, I'd like to share some of the major events that have happened in my life recently.  Many of my readers will be well aware of these events, but I recognize that personal news travels through a variety of channels, and all of those channels are unreliable.  (I'll save the diatribe on why Facebook is a horrible way to keep up with friends for another day.)  For readers who are looking for insightful analysis of technology, my apologies.  Note the "ego" tag.  This is a personal update but does contain a little insight into real-estate finance.

December is often a time of reflection, with good reason.  It's a natural opportunity to consider how things are progressing on a longer time-scale than we often do.  For me, 2009 was a year of making long-term commitments.  I made two huge ones, and I'm extremely happy with both of them.  The process of making these commitments kept me quite busy for almost the entire year.

Most significantly, I married the most amazing woman I know.  Maegan Ashworth and I permanently committed ourselves to each other on September 19th.  Our promises to each other were conversational, humorous, long-winded, personal and deadly serious.  We made them in the most public way we could manage, and were still sad to miss the company of many important people in our lives.  I could fill a book with everything I love about Maegan, but that's even more self-indulgent than I'm willing to be right now.  Suffice to say I am confident this will turn out to be one of the most important positive changes in my life ever.

The real planning for our wedding was compressed into just a couple months because it was difficult to focus on the ceremony while the other major event of the year was uncertain.  But in July we moved into a new house, ending 8 months of ambiguity about where we'd call home.  The process started in November 2008 when we first became interested in the house.  (Just before Maegan and I left for our bicycle tour across Vietnam, where we got engaged.)  It took months to reach agreement with the sellers and then months more to finish the process.  

I went in with a group of friends to buy the house together.  For years we had dreamed of living together in something like an "urban kibbutz".  I've liked that phrase ever since I read it applied to Barack & Michelle's early domestic life.  But for a more complete description of our situation, see our co-habitation blog.  (currently unpublished.  sorry.)

Getting a mortgage was particularly complicated.  The global financial crisis obviously did not help, but our situation was especially difficult.  Living comfortably with lots of good friends requires a big house, which means an expensive house.  In real-estate, expensive is also referred to as "jumbo" meaning that it's too much for any kind of government guarantee.  So banks would either need to make a long-term commitment to us themselves (a so-called "portfolio loan") or re-sell the mortgage to another bank on the secondary market.  We learned that the secondary market was "frozen" to use the popular vernacular, probably at about the same time as one particular bank which had all but committed to giving us a loan.  Another complication was that we needed 3 unrelated applicants to demonstrate our collective ability to pay back the debt, which was unusual enough to make many mid-crisis banks feel extra skittish.  I spent a large part of 2009 working on different aspects of how to finance this house.

Happily the stars aligned one evening when I was walking over to the house of my then-future, now-current roommates.  It was quite common for me at the time to walk those several blocks to sign yet another thick stack of papers to give to some agent or broker or other helpful professional.  Along the way I noticed a four-leafed clover in the grass, and picked it up.  In grade school I spent a surprisingly large amount of my recesses scanning the lawn for these botanical mutants, and once had quite an eye for finding them.  So it wasn't an unusual or significant event for me, but it had been years since I'd found one.  We taped the clover onto the application-du-jour which was going to a small local bank, in an act that signified frustration, exhaustion and powerlessness more than hope.  This bank ended up financing our house.

So that took up most of my year.  Trying to buy a house for about the first half, with moving and settling.  Then a wedding followed by a fabulous honeymoon.

I'm also giving an Ignite talk.  I wanted to make it something of a motivational speech.  Encourage people to think about their own roles in helping bring about the robot revolution.  I also wanted an excuse to share some of my thoughts on how to build an alarm bed.  I'll post my slides after the conference, or at least link to somebody else who does.  But for now, I've got the credits and content licensing posted.

Chrysler goes bankrupt

First some background.  Chrysler is being restructured under
bankruptcy.  This doesn’t mean they’re going out of business.  It means
that they owe more money than they have or will be able to pay.  So
with the help of a judge, they’re sitting down with everybody they owe
money to and telling them frankly “you’re not getting everything we owe
you.  Sorry, but there just isn’t enough to go around.”  So everybody
has to compromise.  The idea is that by striking some bargains to
reduce debt the company can get back in the game and become profitable
again.

UAW owns Chrysler

One of the biggest debts Chrysler has is to the UAW, the United Auto
Workers.  This is the labor union which represents all the
“blue-collar” workers who actually make the cars.  Chrysler owes them
benefits like pensions and health benefits.  Part of the settlement is
that the UAW will own 55% of Chrysler stock.  That’s a majority.  So
the workers will own the company.  Personally I think this is great and
makes a ton of sense, and I’ll tell you why.  But not everybody does.

If you’re lucky enough to be blissfully unaware of labor relations in
Michigan, this is downright bizarre.  Chrylser corporation is
“management.”  UAW is “labor.”  These two groups traditionally have not
gotten along.  I don’t think the word “hate” is out of place.  People
say the UAW will try to unwind this position as fast as they can.  I
heard one “expert” say that the UAW is placed in a position of conflict
of interest representing both Chrysler stockholders and UAW workers.
Why?  Because their responsibility to stockholders is to increase the
value of the company, but their responsibility to the union is to save
jobs, and these two goals are diametrically opposed.

Cooperation is the only way

Hold on.  The goals of the workers and the goals of the company are
diametrically opposed? This kind of adversarial thinking underlies how
Detroit got into trouble in the first place.  In truth the UAW’s goals
and Chrysler “management” goals are very strongly aligned.  This
painful truth of this fact is excrutiating today.  Chrysler and GM are
on the verge of ceasing to exist.  If and when this happens, the UAW
workers will lose their jobs.  What’s bad for management is bad for
labor.  But figuring out how to keep Chrysler building cars that can
compete with Japan and everybody else is a really hard problem.  Solve
it and both labor and management win.  If ever there was a time for
labor and management to come together and cooperate it’s now.  To be
extremely blunt for those still harboring grudges: if you two don’t
figure out how to play nicely together, you’re both doomed.

Historical tensions caused these problems and SUV’s too

Conventional wisdom cites two reasons for why Detroit is in this mess:

• They only built big gas-guzzling cars as consumer preferences shifted towards smaller, more fuel-efficient vehicles
• Union labor costs for things like pensions and health care are so
  high compared to foreign competition that the company just can’t compete

I believe both these are true.  But more interestingly (and something
I’ve never heard reported in the press) I believe there’s a causal link
here.  It is precisely because of these high labor costs that Detroit has focused on building gas-guzzlers. Smaller cars are cheaper and are subject to more intense price competition, meaning the margins are lower.  In business school
we learn about two basic types of product strategies: low cost and
high-end.  In the low-cost strategy you try to be more efficient than
your competitors.  You do things cheaper and still maintain a good
enough product.  This is what Japan did with cars.  But because UAW
kept labor costs high, Detroit couldn’t go this direction.  Their small
lower-end cars would just cost more because of the higher input costs.
So they had to go after a high-end strategy where they made bigger,
more expensive vehicles that came with higher profit margins.

Sophie’s Policy Choice

So UAW workers collectively bargained their way out of jobs.  That is,
they bargained up their salaries beyond what their labor is actually
worth in the modern economy.  So what should we do?  Let the market
correct itself so many of them lose their livelihoods?  Or sustain them
publicly somehow?

There is no easy answer to this question from a policy perspective.
China is facing this same question with hundreds of millions of
uneducated peasant farmers.  A relatively modest investment (on the
national scale) in farm machinery could replace a good fraction of
their output.  But the economically efficient choice comes with a high
human cost.  In this country we believe governments exist to serve the
people.  We’ll see how it does.

Background on SLRs and DSLRs

(If you what “f-stop” means, feel free to skip ahead to the next section.)

SLR stands for Single Lens Reflex.  Practically speaking it refers to a camera where you can change the lens.  You look through the same lens that actually takes the picture, letting you put any lens from an ultra-wide angle fisheye to a telescope-length zoom lens.  You can also put filters on the front like star filters or color shifters or polarizers.  Imagine a classic 35mm camera — like what a P.I. would carry to snap pictures of your wife having an affair — that’s an SLR.

SLR’s require a mirror that physically moves to divert the light into one of two places — your eye, or the film / CCD. The mirror was important when the only technology for capturing images was chemical film.  But nowadays we have various electronic devices like CCDs that digitize an image.  DSLR cameras use a CCD to get many of the benefits of digital imaging, but still have the same physical form factor as an old chemical-film SLR.  They can use the old lenses, which is one of their big appeals.  But so many things about these cameras just don’t make sense.

The problems with DSLR cameras

First there’s the noise. The sound of the mirror slapping against its stops as it switches positions is very recognizable. We used to accept sounds like that as a necessary part of taking
pictures.  Today it just annoys me.  Especially when I’m at a small
event and some photographer is there making loud clicking noises all
the time while I’m trying to enjoy whatever it is they’re digitizing
with their dinocam.  In 99% of all use cases, it’s totally unnecessary.  CCDs can continuously capture images and display them on a screen, creating a digital light path that doesn’t require loud expensive mechanical assemblies.  These displays aren’t as good as what a human eye can pick out, so this doesn’t work all the time.  But if you don’t need interchangeable lenses, then the camera can have a second optical path just for the eye, which doesn’t need to be as good.

One argument against a separate optical viewfinder is that youc can’t put filters in front of the lens.  This is very true, but filters are also obsolete.  With few exceptions, everything that a physical filter does can be done later in photoshop with more control and accuracy.  Color tinting, sparkle, gradients, soft, mist, etc — these all used to be rendered in physical glass out of necessity.  Polarizing filters are probably the most important exception to this — since CCD’s don’t record a light’s polarization state, it can’t be adjusted later.  But for the most part, filters aren’t necessary anymore, meaning you don’t need the whole single-lens thing.

But what about interchangeable lenses?  Isn’t it useful to have the same camera body and be able to change lenses?  (I hear you cry.)  Yes, sorta.  There are definitely situations where one lens won’t be able to do everything you want.  But those situations are getting rarer and rarer.  And in the few exception cases, I’ll argue that interchangeable lenses aren’t the right solution.  The reason these cases are getting less and less common is that zoom lenses are getting better.  When SLR cameras first came on the scene zoom lenses basically didn’t exist because they sucked when they did.  You needed a different lens for each amount of magnification you wanted, so people had lots of lenses.  But with computers to help us design the lenses, and vastly improved manufacturing processes, zoom lenses are getting better all the time.  Nowadays a lens with a huge 10x zoom can even win accolades from camera snobs.  And lenses as versatile as 26x cover every situation most of us would ever want, and at a quality we’ll be thrilled with.  So for almost all situations, a single zoom lens is good enough today.

What about the situations where that’s not quite good enough?  Where you need that 14mm fisheye that captures people standing immediately to the left or right side of the lens?  Or that 8000mm super- long telephoto telescope?  It turns out in either of these challenging cases, getting the lens to fit the standard SLR form factor becomes the hardest part.

Why SLR’s cripple even the extreme lens cases

With ultra-wide fisheye lenses, the problem is the space reserved for that stupid mirror.  In this case, the focal length is very short, so as a
lens designer, you’d naturally want the focal plane to be very close to
the glass.  (Like about 14mm.)  But the place where the lens attaches to the camera body necessarily needs to be a certain distance away from the imaging plane.  That distance was determined by the size of the mirror, which was determined by the size of your chemical film — 35mm, which is more than you’d really want for a 14mm lens.  Even on today’s 2009 DSLR cameras, that distance is exactly the same as it was a generation ago in order to ensure backwards compatibility with old lenses.  The literal tons of carefully polished glass represent a very real barrier to improvement since people have invested lots of money in them.

So if you really want a camera that’s good at taking super-wide angle pictures, you don’t want your lens to have to be that far away from the imaging plane.  You’re better off with a specially built camera.  The lens will be simpler, cheaper and higher quality.  But super-wide starts to look funny, no matter what.  Funny meaning
distorted, because if your eye is more than a couple of inches away
from the reproduced super-wide image, then it won’t look right.  And it’s not super useful to capture 360 degrees in one shot — you can shoot a dozen pictures and stitch them together later in software, and it’ll look more natural.  This is all why people don’t pay a lot of attention to how super-wide lenses get anymore.

On the super-telephoto side of things, the SLR legacy is even worse.  To get a super-long telephoto lens you need lots of big glass.  This gets expensive quickly simply because it’s a large mass of carefully manufactured stuff.  The amount of glass you need for a lens is proportional to the cube of the length of your imaging plane, which for legacy chemical-film is 35mm. But CCD’s just don’t need to be that big.  On almost every DSLR they’re only about 20mm across, and on high-quality non-SLR cameras are as typically about 6mm across.  So that size legacy means you would need literally 200x  the almost 40x the amount of physical glass to make a good telephoto lens for an SLR vs a non-SLR camera.  This ridiculous discrepency is just going to get worse.

CCD’s are silicon devices, so they share manufacturing improves along with CPU’s and follows a Moore’s law-like improvement curve for performance.  A key way they improve is in pixel density, but also by simply getting smaller.  As they get smaller, high-quality zoom lenses get smaller and cheaper too.  But only if the lenses are specifically designed for the new smaller CCD’s.  With an SLR system they can’t be — the size must be fixed in order to maintain backwards compatibility.  So while sensor technology improves at Moore’s law speed, lenses for non-SLR cameras improve as well, but SLR lenses do not.  Expensive zoom lenses for modern cameras just don’t need to be that big or expensive — It’s like having to build a cell-phone big enough to hold floppy disks.

To illustrate this point, consider the popular Canon SX10IS camera which does not feature interchangeable lenses.  It features a zoom lens that goes from pretty wide (28mm equivalent) to really very far zoom (560mm equivalent).  Because its CCD is only 6mm across, it can do all this for under $400 and weigh in under a pound for the whole camera.  For comparison, a comparable SLR lens weighs in at over 11lbs and costs upwards of $7,000, just for the lens.  No doubt this lens can take better pictures than the tiny Canon, but a smaller lens built for a modern CCD could take pictures that are every bit as good for a fraction the price.

I would be remiss if I didn’t mention the noise floor on these sensors.  When the scene is dark, you need more light to get a good image.  A bigger hunk of glass captures more light.  This all makes intuitive sense and is mostly accurate.  CCD sensors can take more accurate pictures in low light when they are bigger.  But the limits here are electronic noise, which is also improving.  At some point we’ll hit some other barrier like the thermal noise in the sensor, although a piezo cooler could work around that.  Ultimately there’s the the quantization of photons, but if you’re taking pictures in a scene that dark, you probably can’t see what you’re pointing at anyway.  My point is that while there are advantages in low light for larger glass and sensors, technology is erroding away at those too.  We’re seeing ISO equivalents of 6400 as fairly common in cameras these days, with an economic competitive pressure to improve that.

In summary, the problems with the SLR format are that it ties its owner to a physical legacy that denies them the advantages of advancing technology.  There are cases where specialized lenses are still important.  But those cases are dwindling.  Personally, I’m going to be happier carrying around a full featured small camera that can transform itself into whatever I want without needing interchangable parts than a bag full of bits that were standardized before email.

Creative Commons licenses can either allow somebody to make commercial use of your material, or not, at your discretion, assuming you’re the one who created the license.  Independently, you can allow anybody to modify, adapt, or remix the content.  Or not.  Or you can allow modification so long as the modified content shares the same license, a so-called “Share Alike” license.  Here’s a nice page that shows you the options and allows you to pick a license appropriate for your material.

A CC attribution license is in many senses more realistic in the modern world than an all-rights-reserved license.  It is practically impossible to stop people from using or distributing your work.  The all-rights-reserved license is a threat to take legal action to prevent somebody from using your work.  But suing somebody is such a hassle that it almost never happens for personal content.  Asking somebody to put a link to your website is a pretty reasonable thing and easy to accomplish.   An all rights reserved copyright is for most individuals a bluff.

CC content is also easier to use.  Negotiating terms of licensing under a traditional copyright is daunting.  It necessarily requires a back and forth with the author and probably a whole lot.  The underlying mindset is that content costs money, so if you’re going to use my content, then you’re going to sell it and I deserve some of that money.  As the music industry is slowly, painfully learning, in modern times this model doesn’t work so well.  Access to information is generally free, and those who are making money here are doing so by providing value-added services on top of merely distributing the information.  (Think ads by Google, or concerts for music.)  With a CC license the terms of use of the license are right there.  No need to negotiate.  Just follow the attribution instructions and do what you will.  Instead of requiring negotiation and payment in the traditional economy, this is payment in the nascent reputation economy.

The first full biography of Paul Dirac, the greatest British physicist
since Newton – and one of the strangest geniuses of the twentieth
century, who may have suffered from autism.

Paul Dirac was a
pioneer of quantum mechanics and was regarded as an equal by Albert
Einstein. He predicted, purely from what he saw in his equations, the
existence of antimatter. The youngest person ever to win the Nobel
Prize for Physics, he was also pathologically reticent, strangely
literal-minded and almost completely unable to communicate or
empathise. His silences were legendary and when he spoke, he betrayed
no emotion. Through his greatest period of productivity, his postcards
home contained only remarks about the weather. He is said to have cried
only once, when his friend Einstein died.

I’m very much looking forward to reading it, mostly because somebody
wrote a whole book about somebody in my family. I recently met Francis Crick’s granddaughter and she said how fun it
was to read her grandpa’s biography and wished somebody would write
them about all of her relatives!  I’m waiting for Amazon to ship me my copy, but they say it’ll still be a couple of weeks, although apparently I can get it faster from Amazon.co.uk so I might just do that.  I’ve had a few chats with Graham Farmelo, the author, over the last few years as he’s been working on it, but I hadn’t been in touch with him recently and was tipped off to its publication by the Economist’s book review.

I’m also very happy to see that Graham is being upfront about the possibility that Autism or Asperger’s was at the root of his strangeness.  Many of us in the family suspected this, but it hasn’t been talked about publicly much if at all.  I’m happy to see this out in the open especially with the dramatic rise of Autism in the world today.  When people hide or just don’t talk about medical conditions, it creates a stigma that makes them that much harder for the afflicted to deal with.  Moreso, my grandfather can be a role model of what is possible to accomplish even with a potentially debilitating condition like that.

I’m also happy that it will provide authority to improve his wikipedia page.  I’ve tried making corrections and additions myself in the past, but I quickly learned that wikipedia’s editorial policy does not allow me to include anything I know about my grandfather in the article, until it has been "published" by somebody else, otherwise it’s "original research."  I include the quotations because the definition of publication is rapidly becoming less clear these days — is this blog published?  How about an IM conversation in a chat room that is persisted at a public URL?  But I digress — this policy is big part of why wikipedia is the important modern reference that it is, so I can’t really begrudge it.  And now that Faber & Faber has blessed Graham’s work into dead trees, wikipedia’s policy will allow his extensive research to be included on their summary.

http://www.yellowpagesgoesgreen.org/stop-yellow-pages/

and stop the unsolicited deliver of unwanted phone books.  Even if recycled, these books waste resources through paper processing, transportation and the recycling process which produces a lower quality paper, supported by inefficient advertising.

While I’m on the subject, if you haven’t tried Google SMS, it’s a great way to look things up.  Just send a text to 466453 ("GOOGLE") with the name of the business you want, and a location specified in writing or zip-code and it’ll respond with what you’re looking for.  It does all sorts of other good things too.  Works on all phones.  I’m a big fan.

Lessig’s book sounds interesting, and since I’m tearing through non-fiction right now, I ordered a copy. I was very surprised to see that the inside flap declares “Copyright © Lawrence Lessig, 2008 All rights reserved.” Below that it says:

Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), without the prior written permission of both the copyright owner and the above publisher of the book.

(As I write this I half wonder if I have violated the stated copyright by typing that in. But seriously I think it’s a clear of fair use.) I expected the book to be released under a Creative Commons license, as Lessig espoused in his interview. I recently started reading Cory Doctorow’s Little Brother which is available for free download from his website under a Creative Commons Non-Commercial Share-Alike license.

What’s going on? Could it be that Doctorow is ahead of Lessig in the practicalities of modern book licensing? Or was it that the publishers were enforcing something? I bought a physical copy of Little Brother, and saw that it too has a traditional Copyright note at the front: “Copyright © 2008 by Cory Doctorow. All rights reserved, including the right to reproduce this book, or portions thereof, in any form.” Okay that just doesn’t make sense. I can download the book under CC, but the print edition is All rights reserved. What gives?

Maybe I’ll have a better answer after I’ve read Lessig’s book. Or maybe Lawrence can explain himself. His website also says that “The book will be available under a Creative Commons license from Bloomsbury Academic. Stay tuned for launch.” I’m waiting.

• http://movabletype2blogger.appspot.com/
• http://wordpress2blogger.appspot.com/
• http://livejournal2blogger.appspot.com/

Well I tried this with my blog, and the resulting file it spit out was almost empty.  I think my blog is just a bit too large, since when I ran it on my local machine it came out to 1.03 megs.  So if your blog is smaller than mine you can probably use the online tool.

After a couple of bX-xji785 errors, the file imported into blogger about as well as could be expected, which is to say okay but not great.  The blog is mostly there.  Feel free to take a peak at http://leo-embracingchaos.blogspot.com/ but please don’t make any permanent links to that URL as it’s really just for testing.  The posts and drafts all made it with the right dates and times, along with the comments and tags.  But as previously noted, the TypePad export format does not include URLs.  So if I were to actually use this conversion, all the inbound links to pages other than the homepage of my blog would break, which is totally unacceptable for me.

I started a thread on the discussion group if you’d like to follow along.

If you’d like to hear more about BNCO, consider coming to their annual fund-raising Gala on February 21st.  Guaranteed good food and entertainment on top of all the worthwhile stuff.  Or read the Times story.  It mentions my wonderful fiance (did I mention that I’m engaged?) and her ongoing efforts to support the clinic.  I’m really looking forward to seeing it in person when it’s built and operating!

I worked hard to make this feature possible when I was working at Real.  The fact that I couldn’t get it re-launched was a big motivator for me to move on to greener pastures.  I saw making Rhapsody social as an important evolution of the music catalog’s organizational schema.  It’s also an attempt to bring the product into what Tim O’Reilly would call Web 2.0.  Tim’s canonical essay is long-winded, but I really liked how he summarized it in a recent interview on NPR — basically the product gets better as people use it.  The millions of people who use Rhapsody are an asset that has been almost completely unused, except to take their money.  I saw it as a way to take on one of the product’s biggest shortcomings.

Rhapsody has tons of music.  TONS.  Rhapsody almost certainly has something you want to listen to right now, regardless of who you are or what your current mood or situation is.  It’s a strong statement, but there really is that much music.  The problem is figuring out what you want to listen to.  Rhapsody has a great categorical index of music, so if you know you want to listen to D&B or Emo or Vocal Jazz, no problem.  Or if you know specifically the name of something you want to listen, just search for it.  Other than that, you can take the homepage recommendations, browse the catalog manually, or sift through Playlist Central, a dumping ground for unvetted playlists that is a case study in how not to use user-generated-content (UGC) on a website.

Picking good music is difficult.  This is what DJ’s get paid for.  I originally wanted this feature to be called "DJ Pages."  The idea was to give a voice to the small fraction of Rhapsody users who are fanatical about the product.  People who are serious music buffs love Rhapsody, and if given a voice would and still might add tremendous value to the music catalog.  Right now the editorial voice in Rhapsody is controlled by a politburo of paid editors.  They’re really good, but they’re just a handful of hands.  DJ Pages would democratize the music editorial process so anybody with an opinion can contribute.  The social graph becomes the voting process to select who’s worth paying attention to, just like with pagerank.  What Tim calls Web 2.0, I like to refer to the democratization of information.  Partly because it’s fun to call people Communists when they cling to control of information, but mostly because the analogy is apt and helpful.

The Rhapsody team has made an important step in this direction of openness.  I hope they keep running with it.  If you want to see what’s been playing on my Sonos at home, check out my profile page.  But most importantly, I’d like to express my CONGRATULATIONS to everybody who made this possible again and the first time!!!!11!!1

• Better WYSIWYG editing in Blogger.  (I can’t change any font characteristics in Typepad without going into raw HTML. Ugh.)
• Typepad’s lack of support for Chrome, Google’s awesome new browser
• The awkward and limiting way Typepad assigns human-readable URLs to posts
• Blogger is free, while Typepad costs >$100 per year
• Cool new features being added to Blogger like Followers
• Adding new features in Typepad is a p.i.t.a. (i.e. I’m too old to be messing with perl-based templating languages.  Did more than my share of that in 1996.)

Typepad has long supported an export feature, which dumps out all the posts, drafts, comments, etc into a text file.  And now blogger has an import feature, which takes an XML file that looks kinda like an Atom feed.  I looked around for a tool to convert between the two and found only others asking the same question. 

So I thought here’s a chance for me to contribute something to
society and help myself at the same time.  Converting one text file to
another shouldn’t be that hard.  So I rolled up my sleeves and started
playing with it.  I realized one problem fairly quickly — the typepad export format
doesn’t include the URL for each post.  I really don’t want to break
all the previous inbound links because
that’s how people get to my content.  But I’m going to need to crawl
the old blog to get those old links.

Since the blogger format looks like a regular Atom feed, I thought I’d try to just grab the Atom feed off the blog and import it into Blogger.  Well to that, blogger said:

Blogger does not currently support import files generated by TypePad.

Fine.  But what are the differences?  The blogger format includes a bunch of fake "entry" elements which are really configuration — like a giant "layout" node and a bunch of other settings.  So I tried grafting the two together — just taking the legitimate "entry" nodes from the typepad feed and putting them into the blogger export that includes all the layout and settings.  By converting everything to look like the blogger format, it imports, but that also loses information.  So I set about looking for the minimal set of changes that will work.

One that definitely breaks it is a side-efect of being hooked-up to feedburner.  There’s extra stuff in there like the feedburner:origLink tags:

    <feedburner:origLink>http://www.embracingchaos.com/2008/12/repairing-a-deg.html</feedburner:origLink>

These break the import, which shouldn’t be much of a surprise since the feedburner: namespace isn’t defined in the Blogger export.

I gave up looking for today because blogger was having too many server errors on import.  They’re intermittent, so I was happy to refresh through them for a while.  But at some point 90% of my imports gave me funny error codes like bX-gxxw6w so I’m giving up for now. 

I wrote a stub of a python program to do the merge.  I called it typepad2blogger.py and happily donate it to the public domain.  It’s definitely not done and is useful only insofar as it shows one way to approach the problem and has enough stuff in place that somebody else should be able to get a running start.  I’ll hopefully continue work on this at some point.  But even a couple hours of work on it has triggered my RSI so I’ll have to take a break.  If you want to pick this up, let me know and we can coordinate efforts.