"This makes it extremely difficult to evaluate products shipped in such incomplete form. "
No it doesn't. If that's what they give you, that's what you review. If x is missing, you say the product lacks x. If the manufacturer is not happy with that, then maybe they shouldn't release the product until it's fully functional.
If anyone is hoping for Samsung to value perfection at launch over being one of the first-to-market in every Android form factor, they're holding out for a Galaxy far, far away.
But it seems to be working well (enough) for Samsung. I agree, though, that reviews should always be based on what the reviewer has in-hand. Every review I've read for non-Apple tablets has offered some "benefit of the doubt" that the iPad never got... or needed.
We currently have no way of knowing how Samsung's software customizations will impact the Tab 10.1's performance, reliability, or battery life when TouchWiz eventually gets rolled out to consumers.
...don't ignore the possibility that things might get worse when the manufacturer's "enhancements" and "features" are added, and readers who go by a review of the product in its current state might be disappointed with the product they get next month.
If that's what they give you, that's what you review. If x is missing, you say the product lacks x.
While that would be satisfying for the writer, the reader wants to know what it will be like to own one. They may buy one several months from now, and keep using it for years. The author is saying they can't evaluate the dominant experience of actual users.
Wait, are you saying that it is impossible to write a review that is useful for the reader unless it is a review of a product which is not going to change in the future?
Change is fine if it is aimed at making the overall experience better. From what we have seen in the past, any UI layer an OEM adds above stock Android just makes the overall experience worse, performance wise mainly. The reviews would never be able to foresee that.
I'd be curious to know the longer-term traffic data for sites which do these types of reviews. Folks like us are quick to read these reviews, but I wonder how many regular users are reading them, say, four months down the road. I know when I'm researching a purchase about which my knowledge is nothing noteworthy, the immediate reviews (like this one from Ars) are beneficial to me when compared to later reviews because it enables me to see how the product has developed over time.
I think long term traffic is a major focus for them all - the products become released, they probably have great success sending people straight from reviews to vendors through advertisements - if they can secure good positions in search engines. Content farms like Engadget (Ars are much higher quality) try pretty hard to corner Google rankings for products, manufacturers and everything else.
- 18 internal links to their own garbage articles and tag pages which only exist for search engines
- 20 tags that cover everything they could think of that you might put into Google: android 3.1, android tablet, Android3.1, AndroidTablet, galaxy tab, Galaxy Tab 10.1, GalaxyTab, GalaxyTab10.1, google, honeycomb, review, Samsung, Samsung Galaxy Tab 10.1, SamsungGalaxyTab10.1, slate, tab, tablet, tablet pc, TabletPc, video
- no external links that might accidentally boost someone else's SERPs
For software that the developer can update instantly (an OS update doesn't count), then yes, this is a great methodology to live by. Hardware is a completely different beast, and costs significantly more for the purchaser and should be tested to extreme levels before launching.
I've yet to be won over to the fast-iteration model of software (or product) development, mostly because I think that presenting an unfinished, or unpolished, product to a customer is usually a bad idea. Even if you get it right later, the customer is still going to be left with a poor first impression.
Apple, despite their faults, is a great example of this: the majority of their products are carefully polished and finished, with a lot of attention to detail.
Obviously a company doesn't want to sit on a new product for too long, trying to perfect it before releasing it, and obviously a company should continue to improve its products and services continuously, preferably at a nice, steady rate. But, I'm steadfastly opposed to shipping unfinished products. I think it's bad advice every time I hear it or read it, and I think the negative parts of this review are a good example of why.
I know you mentioned fast-iteration, but you also mentioned MVP in your earlier comment, so I thought I refer to that.
When Apple first released iPhone, there wasn't an SDK publicly available. There wasn't copy and paste. There was also only EDGE/GPRS support, no multitasking (of any form except for a few built-in apps), no third party apps, no MMS, no SMS forwarding, no group SMS. It was only available in the US and only for AT&T subscribers. In many sense of the term, it is an MVP and it worked. It's just an MVP that was very polished.
As it rolled out releases, there were both enhancements and spots where they aren't as polished which was improved in a later release. Push Notifications in 3.0 + the coming Notifications center in 5.0 comes to mind.
So, they did an MVP with iPhone, initial features were minimum, but a selected subset of them are extremely polished. In fact if you look at many successful iPhone apps, this seems to be a very successful model.
Perhaps this approach only works on Apple's "target market". But I'd like to think this is an excellent approach when working with mass market consumers.
This link where Steve Jobs commented about Segway's marketing and launch is not totally relevant, but still interesting - http://hbswk.hbs.edu/archive/3533.html
"ship ship ship" is hard when it comes to hardware.
As for the software side of things, I'd say "QA", but I find the dev/release cycles at these major phone/tablet manufacturers to be unbelievably slow. As in, I really don't understand what they're doing a lot of the time. 5 volunteers working on CM can fix and implement features in the time it takes Motorola's team to do a `git merge` on Android and push it out to their phones (yes, even non-Blur phones).
I wonder if a lot of what we read about software development methodology is to blame for this? I would expect that individual and freelance developers, and developers at small companies, are more likely to practice "cowboy programming" -- use frameworks, glue them together, write code, handle (most) edge cases, exercise the most basic of best practices -- while larger companies, like Motorola, are more likely to practice something that a manager read about in a book or a consultant sold them on -- "waterfall development", "test-driven development", etc.
(Not that I have anything against those methods, but they would tend to slow down development in a corporate environment.)
I'm not sure it's so cut and dry. If you look at how CM releases are managed... they're very well done. Nightly build servers, good communication on their GitHub and Gerrit, very stable builds considering their nightly builds, all the dozens of phones exist as the larger CM project. I know you were speaking more generally, but I don't know that one or the other is always better, whether it's speed or style or development workflow.
I got a Galaxy Tab (the old one) over the weekend, overall I find the experience pretty disappointing:
- I never knew how much I loved hating iTunes, until suddenly everything had to be done manually
- it really sucks to buy a game, fire it up and discover it's a shitty mobile version stretched to fit your screen. I'm especially disappointed in PopCap for doing that w/ PvZ!
The Tab 10.1 is a much more credible product than the Xoom, but it's not quite competitive with the iPad. If Google wants to compete, it still needs to build a vibrant third-party application ecosystem in order to make Android tablets a good option for regular users.
Hm, I'm curious... what do people want to see Google do in order to convince them? People are developing for Android in droves on mobile phones because they finally (and I do mean finally, the writing on the walls was clear before any major Android app explosion)... do they not assume that a similar effect will happen for Android tablets?
It's off to a pretty slow start. Android on phones was coming from nothing. On tablets it's different - I expected a much faster takeup with at very least a huge number of developers doing minimal tweaks and updating graphics just so they could be in the small pool of tablet apps and get more exposure by calling themselves "HD" or "Honycomb compatible". But if I understand right (and I'm not able to test this directly) there is still only a very small number of apps specialised for tablets.
I think part of the problem was the very anemic Xoom launch which simply didn't get many devices into developers hands, especially outside the US, and the quite unusable Honeycomb emulator which also turned off many devs from even trying to support Honeycomb. Google tried to fix that at Google IO but they should do much more.
It's worth noting another part of this is confusion about how Android works - there is no reason to create a "honeycomb specific app" on Android. The whole point of Android's layout and fragment infrastructure is to let you easily make an app that seamlessly caters to any number of form factors. So the idea of looking for and counting apps that are dedicated to Honeycomb is silly - it's not surprising there aren't many of those, and those that do have specific support for Honeycomb aren't necessarily labeled that way in the market like they are in iTunes (which is a problem with the market).
It hasn't happened yet and reviewers don't have a time machine. What should they write? "Well, the app ecosystem sucks but might well improve in the future, maybe"?
Uh, I was asking what will motivate developers. I agree with the consensus here. If it's not shipping with apps, and it's shipping with features missing... it should be reviewed that way.
And besides, saying "Oh, 3g sucks but you can mail it off for a week to get LTE" is 100% completely different than saying "We expect the Android tablet ecosystem to grow".
Do you think I bought my D1 thinking that I'd be stuck with a trash Facebook app and basically no other apps to speak of? Again, I don't expect or even have a desire for reviewers to go out of their way to make such excuses... my entire post was just questioning WHY the ecosystem is so dry, even though the Honeycomb SDK has been available (albeit in a flawed state initialy) for so long.
Samsung looks like they are just trying to be the first and grab market share with the first Honeycomb tablet. Looking at this and the iPad, my question still is: what do I use this for? On the go my smartphone handles all my computing tasks and at home I have several boxes for at home computation. Where is this supposed to fit in?
Tablets seem to be the perfect kitchen table computing device for when you just want to look up something really quick, but don't want to walk down the hall to the office and wake up the computer from sleep.
Some use cases:
- Recipes using Epicurious - who wants a full blown computer in the kitchen?
- Browsing instantwatcher.com and adding movies to your Netflix queue from the sofa.
- Reading your newsfeeds while you eat breakfast.
- Any browsing related task where you don't want to leave the room (checking weather forecast, google searches, booking an airline ticket, reading email).
Really, the wife and I have had an iPad for 1+ years and we still get a few good hours a day of use out of it. I've had a Sharp Zaurus and other devices before and they all ended up in a closet or drawer after a few weeks. The tablet PC is very useful because it does fill that niche in between wanting to go to the office and get on the desktop tower or trying to look something up on your smartphone.
I've always thought that companies should have emphasized how perfect it is for college students.
Use cases:
-Torrent textbooks. Now you have all ten of your heavy
textbooks in one thin compact device. Also, they're now free.
-Sitting in lecture and the professor says something that you'd like to wikipedia. Why haul out a clunky, sleepy laptop when you can do it quickly on your tab. Also, you can read hacker news quite conveniently when the professor is 5 minutes late.
-Write notes. Also, there's got to be some app that let's you draw on your screen so you can record diagrams as well.
-Physically convenient. You can almost completely replace a full backpack with a one-handed 7-inch slab. It also plays music so you can turn on that rain ambience mp3 while you lay on the campus lawn reading any of the millions of ebooks that are now at your whimsical disposal.
IMO the top use of the iPad is for little kids (think 10 and younger). It's the modern Gameboy.
The other big reason is as an eReader.
I just don't find the web, virtual keyboard, and other apps to be interesting enough to grab me if there is a desktop/laptop alternative. And their too large for me to take everywhere I take my phone.
With that said, I have little doubt that touch-based computers will come to dominate, although I think convertibles will actually turn out to be quite popular. Unlike cellphones, an actual keyboard on a tablet is HUGE win.
"Samsung looks like they are just trying to be the first and grab market share with the first Honeycomb tablet"
If that's their plan, it is going to have to involve a time machine because there are at least 3 other Honeycomb tablets that have been released already: Xoom, Eee Pad Transformer and the Acer Iconia.
Tablets are good for restaurants, stock taking and other specialised uses. Though they will soon fight the connected TV in the lounge room, right now they are a good option to have about the house if you just want to read or browse and remain mobile. Depending upon how their price and applications develop (very much in flux right now, it seems) against other platforms (PC, mobile, connected TV) I expect we'll see a leveling out in tablet growth. One thing's apparently for sure: they've killed the underpowered 'netbook' concept (finally).
I'm with you. The only thing I'd imagine is the ASUS Transformer if it had a full copy of Chrome. Then it would just be a nice netbook with all of Android and an even more compact form factor for when it might make sense.
I think it's fair to say that will be available soon. I think dual boot to ubuntu on the ee-pad transformer is coming within the next couple of weeks and chromium os shouldn't be far behind.
Excuse my ignorant question: if I buy one of these is there any (vaguely legal) way I can use it to watch movies on long flights?
edit: Never mind, I found out for myself. Rip from DVD to some wacky format, obtain app to play it. Perhaps I'll consider it, now I find myself flying (bleh) United with its severe lack of video.
No it doesn't. If that's what they give you, that's what you review. If x is missing, you say the product lacks x. If the manufacturer is not happy with that, then maybe they shouldn't release the product until it's fully functional.