Having just read Jeff Hawkins’ On Intelligence, Danny Hillis’ PhD thesis The Connection Machine and Peter Norvig’s Artificial Intelligence: A Modern Approach, I have been having some strange ideas about intelligence.
What got me thinking about intelligence in the first place was the observation that many of the tasks that seem to be difficult for computers, but relatively easy for biological brains, are most naturally thought of as NP-hard optimization problems. Basically anything that involves complex pattern matching–recognizing speech, inference, relational database search, vision, and learning, for example.
Another thing that seems interesting is this: take any algorithm that scales linearly with input size. For the problem this algorithm solves, can you think of a single example where a human could beat a computer? I can’t think of one.
Finally: biological brains operate with small amounts of input data (five senses). For example if we look at a photograph the total data we receive is quite small.
Is it possible that the notion of complexity classes is important for asking the right questions about intelligence? Here’s a rough outline of an idea.
- Categorize all of the problems that biological brains have to solve as well-posed computational problems.
- The subset of these problems that can be solved with algorithms that scale polynomially can always be done better with silicon than with bio-brains. Note that this isn’t true in general–it requires the observation that the input problem instance size is small for bio-brains (# of pixels in a photograph eg.).
- There are problems that have survival value to solve that are harder than P.
- Brains evolved excellent heuristics to provide quick approximate solutions to the problems harder than P, and currently it is that subset where bio-brains beat silicon.
- In a hierarchy of difficulty, some problems will be too hard for bio-brains to evolve heuristics for. This means that the primary differentiator in this picture of things will be the “easiest hard problems”. The hard hard problems are too hard for evolution to create hardware heuristics for.
- The easiest hard problems (the group most likely to have good hardware heuristics and bad software heuristics) are NP-hard optimization problems.
In this view of the world, the key to mimicking bio-brains (at least from the computational viewpoint) is being able to quickly solve (in the heuristic sense) the types of NP-hard problems generally associated with intelligence. This is an algorithmic issue, and not so much a hardware capability issue. This argues against Ray Kurzweil’s implicit assumption that hardware capability is sufficient for intelligence to arise in machines. I tend to think now that breakthroughs in machine intelligence are going to come from algorithms–either new classical algorithms, or the capability to run quantum algorithms on quantum hardware, and not on Moore’s Law advances.