There seems to be a strong inverse correlation between claims of having read the book and actually having read the book. I haven't read the whole thing, but I've read enough to know that this particular Zuse-head is bullshitting.
I'll list just a couple of errors that would be impossible to make if one had read even most of the book. Wolfram often pedantically reiterates the same points, so keep in mind that these things are hard to miss:
1) "The Principle of Computational Equivalence" does not state that all-is-computation. It states that whatever 'objective' means we use to quantify computational complexity, we will discover that all computations are either trivial or of equal complexity. I.e. computational complexity (where this is crucially left undefined) "saturates" very quickly in the world of natural computations, no matter how you decide to measure complexity.
2) SW's discovery of universality among the simplest CAs is not a triviality, because unlike what this guy says, the dovetailer is not a simple program -- it is explicitly set up to be universal (in a manner). Its Turing machine rule number is probably in the trillions or higher. Whereas the surprise is that even amongst the very simplest programs, universality is easy to find.
To use an analogy, string theorists would cry with joy if it turned out that there was some small number of "simplest natural string theories" and one of them gave us all the known particles of the Standard Model.
3) Asymptotically optimal program search, in practice, isn't the way you would hunt for universes, and it is relatively easy to see why (TL;DR for now). Schmidhuber's academic work is of no practical relevance to the chapter on physics, although its cool from a math geek perspective. Same with maximally rational agents.
And the main idea here is just Occam's razor, not some arcane formulation of maximum predictive accuracy under a strange universal prior of symbol sequences, as cool as that sounds.
4) Wolfram doesn't propose the universe is a discrete CA, although everyone seems to think this. He makes all the obvious points about why it is unlikely to be so, and goes on to propose a graph automata model as being a suitable generalization of space and time.
So yeah, don't trust every well written review you read on Amazon.
As for not referencing people enough, I have sympathy with this criticism. On the other hand, as the book delves into a million and one different domains, the inquisitive reader would get extremely bogged down if he were to descend into the jargon of each individual field. And you would need to descend into jargon to say anything other than light summarizations of what has come before.
But these light summarizations do exist, in the extensive notes. In fact they're often not so light -- for example there is quite an interesting discussion of why the Pressburger arithmetic and the theory of intermediate degrees isn't a contradiction of the principle of computational equivalence.
Many times when one first thinks that Wolfram is being simplistic or naive, it turns out that he's gone into a lot more depth in the notes (I assume to avoid getting bogged down in the main text).
He really does know his shit.
Disclosure: I work on Wolfram|Alpha. But I have a brain, and I can think for myse.... ALL HAIL THE HYPNOTOAD
I assume you've read GEB. That wanders all over the map, and is fascinating for it.
"Complexity" by Roger Lewin is a sort of journalistic take on the early history of the slightly vague field of complexity science. But its fairly interesting.
"The Computational Beauty of Nature" by Microsoft R&D dude Rob Flake might also be a good candidate.
"The Jaguar and the Quark" by Gell-Mann, complexity theorist and Feynman nemesis, is enjoyable too.
A complexity theorist friend of mine also recommended Rudy Rucker's "The Lifebox, The Seashell, and the Soul" to me, but I haven't read it.
"Darwin Among the Machines" by Freeman Dyson's son (!) is frigging great, but that's now getting off topic.
