Complexity of Intelligence: Kurzweil v. Myers Redux

PZ Myers doesn't like Ray Kurzweil very much. He came out swinging with a scathing critique of some of Kurzweil's statements, which Kurzweil fired back at in a similarly hostile manner. Somehow I don't think these guys are going to be having dinner together any time soon...

To decide the winner in this little sparring match (tl;dr summary: they're both wrong, in a way), we've really got to get the argument straight.

First, Myers was upset by Kurzweil's supposed claim that we could reverse engineer the human brain in a decade. Now, it turns out that this wasn't what Kurzweil actually claimed, but a reference to Henry Markram of the Blue Brain project's beliefs - his own thoughts are that it will take more on the order of 20 years. Point to Kurzweil (though it should be mentioned that it was Gizmodo that got that wrong, not Myers, at least directly).

But that's not what Myers was really jazzed up about. Here's what got his goat for reals:

Here's how that math works, Kurzweil explains: The design of the brain is in the genome. The human genome has three billion base pairs or six billion bits, which is about 800 million bytes before compression, he says. Eliminating redundancies and applying loss-less compression, that information can be compressed into about 50 million bytes, according to Kurzweil.

Myers took particular issue with the statement "The design of the brain is in the genome", and that's, in my opinion, the most important part of this dispute.

At some level, the design of the brain is quite trivially in the genome, because it is constructed from it. But Myers, and most biologists, are quick to point out, correctly, that the developmental road from DNA to a living, functioning brain is exceedingly complex, and that the complicated physics, chemistry, and biology that would be required to simulate all of that would probably take a helluva lot of bytes to simulate properly. Way more than the mere DNA string.

Point Myers.

Kurzweil fights back, though: he's not talking about literally reverse engineering the brain in this number of bytes, he's talking about the information content that it takes to create the brain.

If we ignore the fact that Kurzweil has literally advocated for such an explicit reverse engineering of the brain as the easiest route to AI, then I suppose we can give him that point, or at least split the difference. Even if, at that point in his rebuttal, he meanders into the usual "you guys just don't get exponential growth!" spiel, which I'm sure we're all tired of by now.

Break. We'll call it a draw, for now - I'm not sufficiently convinced by either side to care too much about what they're saying. Especially since most of the argument seems to be a matter of interpreting exactly what it is that Kurzweil is claiming, and he won't really pin it down very precisely.

These two are getting a little bit emo for my tastes, so let's break it down to something that we can specify precisely.

What can we infer from the fact that the brain is coded up (even if we don't understand the entire decoding framework) in somewhere around 50 million bytes? Specifically, how does this figure affect our estimates about what it would take to create an AI that was as intelligent as a human? [Note: I'm very deliberately not specifying here that it should function exactly the way a human does, just that it's "as intelligent", for whatever that means...]

My impression is that Myers would say that we can infer almost nothing: the dynamics of the translation from DNA -> cellular functionality destroy any information theoretic content of any DNA based estimate. In other words, because things that do very complex tasks can be specified in short strings of DNA, there's an unpredictable amount of information that sneaks its way in, and we can't use genome length as a proxy for information content in any meaningful way.

Kurzweil, on the other hand, would probably argue that the genome length is a reasonable estimator for the length of a program that we could hope to produce that would achieve the same function.

Here's where it gets fun: in a general sense, applied to all the things that DNA encodes, Kurzweil is wrong. Even if we restrict to just the construction of the brain, he's probably wrong - a working simulation of the brain would likely require more than that 50 million bytes, for many of the reasons that Myers gave.

But - and this is a big but - if we are very careful about what we're looking for, that 50 million byte estimate does give us an upper bound on something.

It turns out that we can, with extremely high probability, assume that we can create an algorithm that can do roughly the same thing as the human brain, within 50 million bytes or less, based on Kurzweil's argument.

Notice the crucial difference in the claim: I'm not saying that we can simulate the brain, I'm saying that we can roughly achieve its abilities. This might be a big difference to Myers, but to anyone that cares about AI, the difference is all but trivial.

Here's the way the proof goes - it's remarkably simple. We simply compare the classes of information processing algorithms that can be implemented via development from DNA - call these U(DNA), the universe of DNA-based algorithms, and the algorithms that we can implement on a computer - U(CPU). Unless you subscribe to some dualist hoohah, you'll likely accept that there's a subset of each of these sets of algorithms that would qualify as "intelligent".

Now, it turns out that the information content necessary to specify a working algorithm to solve a task within a certain "language" (universe, here) is inversely related to the number of possible algorithms that exist in that language that solve that problem. So comparing information content is really a question of percentages: what percent of algorithms in U(DNA) are intelligent, and what percent in U(CPU) are? That will tell us how many bits it takes to code up an intelligent algorithm in each.

The real guts of my claim (a revised version of Kurzweil's) follows: if we know that a problem can be solve in N bytes in universe A, then as a rule, we can expect it to take roughly N bytes or less to solve in universe B, as long as there's nothing "special" about universe A that makes this problem particularly easy to solve (or that makes it especially difficult in universe B).

Myers' claim (or rather, my implied extension of it) is, in fact, that there's something "special" somewhere along the line between the string of DNA that is used to build the brain and the working lump of goo inside your head, that makes the directive "solve intelligence" much easier to solve than it should be.

What we should be asking: what computational advantage does the DNA->protein->neuron process offer that makes it so well suited to computing an "intelligence function", to the extent that it can express such a function so much more easily than a bunch of highly flexible computer code?

Most of the instances where we see special complexity reduction are extremely obvious and low level: DNA is very well suited to the "compute protein folding" problem, because it has a direct physical representation. Similarly, it solves the "arrange atoms in physically stable configurations" problem, and the "create an object that self-replicates" problem extraordinarily efficiently in bit-wise terms, because the continued existence of the structure literally depends on solving such problems. DNA would not stably exist if it couldn't solve those things, so we shouldn't be surprised that it's able to.

Conversely, "do intelligence" is neither a low-level side-effect of the way cells work, nor required for their continued existence. So absent some other evidence, we would be extremely surprised to find that DNA was particularly well suited to create it (or in other words, that a higher percentage of DNA-spawned algorithms lead to intelligence than the corresponding percentage of machine code algorithms).

We have yet to see many (if any!) instances where the low level functionality of DNA, proteins, or cells make high level computations much easier than they would otherwise be, and for very good reason - most "easy" constructs in biology are borne of emergence, not evolution or intelligence, so they are not very powerful things, and they just don't do very much! On the other hand, evolutionary constructs can be more powerful, and intelligently designed constructs an order of magnitude more powerful than even evolutionarily designed ones. But all of evolution's work ends up writing bits into DNA, and would thus be captured by the raw complexity estimate that Myers so abhors.

So in the end, Myers might be right: the design of the brain is not exactly in the genome. We do need a massive machinery to dissect, interpret, munge, and fiddle with the base pairs before we end up with a brain, and that brain has a lot of dynamics of its own that we'd be foolish to think we could replicate without a truly heroic effort.

But absent some really extraordinary evidence, we have no reason whatsoever to believe that any of that extra machinery actually helps the goal of creating intelligence: yes, we'd need to simulate it to simulate a brain, but as long as our goal is not to explicitly simulate a brain, we can probably discard most of what it does as irrelevant implementation detail. If we could figure out the algorithm, in all likelihood we could fit it within the space of the brain's part of the human genome.

So I'm calling this one a draw. We might not be able to simulate the brain within 50 million bytes, but it's overwhelmingly probable that we can solve the same problem that the brain does in that amount of space. To me that's the goal that we're shooting for, we don't need to worry about biological accuracy - I don't want to create yet another human, we've got plenty already, I just want a smarter computer!

Kurzweil may not understand the brain (and who does, really?), but Myers doesn't understand that nobody working on AI gives one solitary shit about the physical brain; we're after intelligence. And Kurzweil's argument more or less stands up as long as that's where we constrain our focus.

Now if only I could figure out the right 50 million bytes to code, I should have this thing cracked in no time at all...there's only 2^(400 million) possible algorithms, it should be easy, right? :)

In fact, I believe we can (will, and should!) get there with orders of magnitude less code, but that's another post for another day...