That said the data is pretty useless for many Android phones. For example the international S3 has a quad-core A9 based SoC while the US S3 has a dual-core krait based SoC. Also consider that both overclocking and underclocking is quite common amongst the android crowd that bother running geekbench.
The average across a product name is actually a really poor metic to use here. Well, geekbench scores in general are a really poor metric to use...
The results on the chart aren't averaged by name; they're averaged by name, by processor, and by processor frequency. We're aware that many Android devices with the same name use different processors, and of the fact that many Android devices are both over- and under-clocked by enthusiasts.
Sorry if this wasn't clear from the preamble on the chart; I'll have to update it.
I'm confused then. The US S3 with the dual-core krait SoC doesn't appear on the list at all but searching for the S3 shows results for a 2 core version, strangely a 3 core version, and the dual-core version doesn't have the expected clock speed of the krait cpu (1.5ghz).
Are these just the quad-core model with cores disabled or something? If so are they included in the average? They appear to be based on a cursory look through the data as 4 core @ 1.4 seems to mostly score 1700-1900 but the displayed average is obviously much lower.
I also don't get the One X results. Its like a 50/50 split of scores of ~1500 or ~600 and the difference seems to be android 4.0.4 vs 4.0.3. No idea what the difference was but almost tripling the score on a minor OS version seems odd.
Sorry but I'm still saying the results here aren't very useful, there is far too much variation in these tests to assign any significance to them.
Not all handsets are included in the chart. The Geekbench Browser has a list of handsets which it uses to build the benchmark chart. This list contains model and processor information and is manually maintained; if a handset isn't in the list, it's not included in the chart. I thought all of the S3 models were included in this list, but apparently I'm wrong. I'll make sure this list is up to date.
Geekbench is built with the NDK (since all of the benchmarks are written in C or C++). There was a bug in Android 4.0.3 and earlier that caused Android to select the ARMv5 libraries instead of the ARMv7 libraries (which caused a massive drop in performance). This was fixed in 4.0.4 which is why there's a huge jump in performance between the two versions on the One X.
Shouldn't results from the One X on 4.0.3 (or even all results from 4.0.3) be excluded from the averages then? In the case of the One X the average is ~1000 but without the ~600 scores in there the average would be much closer to ~1500.
Even leaked versions of the 4.0.4 update to the WWE One X were in demand because of the large performance improvement on benchmarks and smoother interaction. That's actually a true case. Although partially due to things like less 3D in the launcher.
I don't know what was done in this case, but I do think I remember seeing a statement by NVIDIA on early benchmarks at release that they still had optimizations to do. So that could be involved.
It hasn't been unknown in the mobile world for wacky stuff to get done, like having a dual core processor where you almost never use the second core to save batteries as well. So software can make a big difference.
Those numbers are barely better than the SIII and Nexus. I would have expected more, given that Androids much faster than these are coming out any day now.
I'm always surprised how many shipping Apple products are beaten in benchmarks by about-to-be-maybe-shipped-but-don't-yet-exist products from competitors. You think they'd do better than that!
The reason that there is only one iPhone release a year. So it better be much faster than the competition at release so as not to get totally whopped 4 to 6 months down the line. iPhone 4S sales started softening in the 3rd quarter itself instead of the usual 4th quarter before the new iPhone comes out.
iOS devices have never really been on top for CPU benchmarks though have they?
For example the Tegra 2 based non-prime Transformer and A5 based iPad 2 were released about the same time (March 2011) and that Transformer score 35% higher on Geekbench but the A5 GPU benchmarks were way way ahead and have stayed competitive with today's competition [1].
"so as not to get totally whopped 4 to 6 months down the line. iPhone 4S sales started softening in the 3rd quarter itself instead of the usual 4th quarter..."
Does anyone really think that iphone sales lagged more than previously because of CPU benchmarks?
I would guess only indirectly as better performance on Android has helped make the experience better. But there's all kinds of other things improving the Android experience (e.g. better apps, better content experience, ICS and Jellybean hitting more devices).
Even with Android improving relative to iOS over the year I'd say a more reasonable reason for Apple's Q3 lag was the fact that they rolled out globally in 2 quarters instead of, historically, adding more countries and carriers well into Q3 and sometimes Q4 of launch. The iphone 4 also had the midseason CDMA and the much delayed white iphone launches midyear.
You need to remember the iPhone 5 uses a different architecture (armv7s) that Geekbench currently isn't compiled for. It's highly likely there wil be a number of optimisations to be had from moving between armv7 to armv7s, so the benchmark score for the iPhone 5 is highly likely to increase once the binaries are re-made.
Why? Apple's devices have rarely focused on raw performance though their access to the latest and greatest mobile GPUs has been way ahead of the curve in the last couple of years. Using a 4S it's actually a little hard for me to comprehend my device being any faster. Everything is already so fast and smooth. This may be the first generation of iPhone where the performance increases are barely noticeable to users.
The score of 629 for the iPhone 4S and 766 for the iPad is "calibrated against a baseline score of 1,000 (which is the score of a single-processor Power Mac G5 @ 1.6GHz)" (see http://browser.primatelabs.com/ios-benchmarks) — not that big a difference, thus.
It seems a good chunk of that is not coming from the processor itself, but from other stuff like how fast is the memory, the storage, etc. So it might not have a better processor than Galaxy S3 or Nexus 7, but it may have slightly better storage and RAM memory.
Anandtech is reporting that the iPhone 5 is using dual-channel 32bit LPDDR2 running at 1066MHz, about 33% more peak memory bandwidth than the iPhone 4s.
Out of curiosity I built Geekbench with Xcode 4.5 (it's not available on the App Store yet) and took a look at the code generated for the armv7 and the armv7s architectures. Surprisingly there weren't a lot of differences between the two. The biggest difference I saw was that Xcode uses conditional VFP instructions (e.g., vaddeq.f64) for the armv7s but doesn't use them for the armv7 despite the fact that these instructions are supported by the armv7. My guess is that the A6 implementation of these instructions is much faster, but I won't know for sure until I can run benchmarks on the iPhone 5 myself.
Also, I could only find two instructions (sdiv and udiv) that Xcode generates for the armv7s architecture that aren't supported by the armv7 architecture.
Googling yields a 6.4 GB/s figure for the single channel LPDDR3 supported by Tegra 3. It doesn't sound like the
most likely bottleneck for a CPU benchmark given that those A9 cores plod along at about 1/10 the perf of a desktop chip.
What sort of bandwidth do other mobile chips provide?
That page is a chart of results for many of the previous models. As one can see, the iPhone 5's results are 2.55x better than the iPhone 4S. However, that should be taken with a grain of salt since we only have one iPhone 5 result.
• 2004 Power Mac G5 - 1571
• Samsung Galaxy S III - 1560
• Asus Nexus 7 - 1591
• iPad (3rd Generation) - 766
• iPhone 4S - 629