Hacker News new | past | comments | ask | show | jobs | submit login
The Washington Post's bestseller lists have been wrong (washingtonpost.com)
160 points by gruseom on July 29, 2019 | hide | past | favorite | 88 comments



Here's how "Best Sellers" become a thing:

Publishers count the number of books they sell.

This seems pretty simple, right? It is, but as a consequence, easily gamed. Publishers don't sell to you and me, they sell to book stores. So, you create some marketing to get people to preorder your book from a lots of different book stores. Book sellers inform the number of copies based, in part, on preorder numbers. There are ways that you can get book stores to order many more books than there is demand for by fraudulently (not sure if it's legally fraudulent, but maybe?) pre-ordering books, and stocking shelves with books that no one ever picks up. But boom, there you go - best seller.

If you want to cut out the middle man, and you're rich enough, you can just _be_ the middle man. Buy all the books from the publisher, and resell them. The publisher still gets their money, so they don't care. Again, now you have a best-seller.

And then, of course, there are legitimate sales that the best seller lists market themselves as measuring.

Book sellers don't want to be stuck with unsold inventory, so there's something of an arms race between book sellers and publishers/authors.

After seeing this game first-hand, I no longer believe anything on a best-seller list. As Amazon continues their vertical integration, it's not clear whether this problem will be addressed or not. The system can still be gamed, especially if it's run by algorithms.

So, basically, I'd argue that it doesn't matter whether WaPo is "right" or not because the input data is likely of poor quality anyways.


You left out my favorite trick: politics! For example, say that you're running for President, and you want your campaign book to be a "top 10 list" best seller. Well, just have your campaign pre-order 4,000 or so copies of the book. That plus 1000 or so real sales will be enough to get you into the top 10 list, and at only about $55,000, it's practically a steal. Even better, the proceeds come right back to you because you're sure as heck not gonna give your campaign a discount.


Scientology did one better - they bought their own books at retail, then fed them back into the distribution chain. It was common for "new" books to arrive with price tags still on them.


I have a story about a well-known "best selling" author

Some years ago received a random package in the mail - turns out it was 40+ copies of this authors new book. Had no idea what to do with them so they just sat around, I never even read it

A few days later I notice some blogs running giveaways for the same book - figure they got deliveries of it as well. Go to a conference and in conversation find that a bunch of people also got the same package

I didn't put together until years later that this is all part of self-sales to boost best seller rankings.

Apparently if you know which book stores are surveyed by the NYTimes you can call them up and order a handful of copies of your own book and boost your own sales

It's no coincidence that this author heavily pitches themselves as a multiple-NYTimes best seller. It likely pays off because I have a good idea of what their appearance fee is and it's mid-5 figures+


The same thing used to happen in the UK music charts. Sales were tracked in a small number of stores and then interpolated.

So of course record companies and sometimes artists would make sure there were big orders at those stores. And a lot of very bad records became unexpected chart toppers.


This reminds me of the early 90's when Billboard's music sales charts switched from subjective reported to SoundScan (actual scanned sales).

That truth was the breakout tipping point for country music. It also makes you wonder how legit/truthful any charting previous to SoundScan actually was.


> As Amazon continues their vertical integration, it's not clear whether this problem will be addressed or not.

Amazon might make things a lot worse. The “recommended by amazon” windows 10 keys are mostly fake ripoffs at this point with pages of complaints in the reviews

Edit: added quote


I'm not familiar with this, what's the sellers business model then? Consumers easily get full refunds on fakes, no?


Book sellers are rarely stuck with unsold inventory. They almost always have the ability to send unsold inventory (though not magazines) back to the publisher for full credit.


To remainder paperbacks, instead of sending the entire book back for credit they'd rip off the covers, send those instead, and dumpster the book. I don't know if they do this anymore. When I grew up in the 80s, I read an enormous number of coverless books:)


This was SOP for me when I worked as the receiver in a supermarket from ~2007-2011.

It’s a shame, I must have thrown thousands and thousands of books away, instead of donating them to a library or something.

But then again, I guess the library would only accept so many copies of the same nascar-themed trashy romance novel.


That's still SOP.


So then if you have sufficient capital, you park it into your own book for a few weeks and then return it all later once the best-selling status is achieved?

I can't imagine publishers would let this happen more than once or twice before revoking this ability for the seller. Inventory handling is expensive.


Publishers need booksellers more than the reverse.


I know of one publisher who massively overprinted a title. Most of the run didn't even make it to the stores.

Because of a worker error, the copies were left under a leak in the warehouse roof.

I understand there was an insurance claim. It was all very unfortunate.


Amazon's bestseller ranking is famously gameable by having a flash mob order the item all on the same day


It seems like amazon has access to more real metrics like total kindle time per book.

But perhaps it doesn’t serve them to diverge from the bestseller model.


Kudos to the Post for attempting to produce an objective list. The New York Times doesn't:

> In other words, The New York Times best-seller list is not a best-seller list -- which even The New York Times once acknowledged. In the early 1980s, William Peter Blatty, author of the monumental best-seller "The Exorcist," sued The New York Times for only listing his novel on the list one time, even though it sold in the millions. In defending itself before the court, as reported by Book History, the annual journal of The Society for the History of Authorship, Reading and Publishing (Penn State University Press), The Times said, "The list did not purport to be an objective compilation of information but instead was an editorial product."

https://townhall.com/columnists/dennisprager/2018/04/17/the-...


"The New York Times' Hopeful to be Sellers List"

Doesn't seem to have the same ring :)


The Times makes it clear what their methodology is: https://www.nytimes.com/books/best-sellers/methodology

One anecdote posted on a conservative website is hardly a debunking of The Times' 75+ year history of best sellers lists.


Seems unfair to dismiss it that way. Here is the decision, Blatty vs New York Times Company, decided December 29, 1986:

https://caselaw.findlaw.com/ca-supreme-court/1843273.html

The Times won on First Amendment (freedom of the press / speech) grounds. Blatty couldn't win, since the list would need to be "of and concerning" him:

> "To begin with, the list does not expressly refer to Blatty or his novel. Nor does he contend otherwise. Quite the contrary: the failure of Legion to appear on the list is the very basis of his action."

Perhaps most interesting for me is that the Times survey of bookstores included a pre-written list of the books they want sales numbers for:

> "To obtain sales figures from bookstores, [the Times] sent to the bookstores forms which [it] prepared; the forms for works of fiction contain and contained a printed list of 36 ‘bestselling’ books which has and had the effect of encouraging reports as to sales of books listed on the forms and discouraging reports as to sales of books not listed on the forms”


> Institutional, special interest, group or bulk purchases, if and when they are included, are at the discretion of The New York Times Best-Seller List Desk editors based on standards for inclusion that encompass proprietary vetting and audit protocols, corroborative reporting and other statistical determinations.

So basically, they use editorial discretion when ranking bulk sales. Which means they are prone to exclude books they don’t like and include books they do like. Excluding some bulk sales but not others isn’t objective. Institutional and bulk sales should all be excluded from the counts.


Does that apply to the rest of paper as well?


Taking your question at face value, yes, and it’s true of every private news company. They are first and foremost entertainers, and even those who hold themselves out as agents of change require a large audience.


Read Noam Chomsky's "Manufacturing Consent" if you want to know what motivates media companies. (Spoiler: Advertisers first and foremost)


It applies to nearly every organization that's been advertising itself as news for easily the last 20+ years.

They are all - every last one of them - suddenly nothing but editorial opinion and entertainment as soon as they show up in a courtroom.


Not sure why the downvotes. If a news provider can’t promise to try to tell the truth in a legally binding way, but just claims to be editorializing, it’s no wonder there is so much public distrust of the news.


> If a news provider can’t promise to try to tell the truth in a legally binding way

which news provider does this?


You see the point then...


> to tell the truth in a legally binding way

feels like these two are mutually exclusive


These lists can be manipulated. There was a case a few years back involving bogus bookstore sales of Lani Sarem’s "Handbook for Mortals" to trick the New York Times:

Stamper and other YA writers, including Jeremy West, began to investigate. Stamper shared messages he had received from bookshop staff who said they had been contacted to see if their store was an NYT-reporting shop – the paper’s lists are collated from information supplied by a confidential group of stores – before a bulk order was placed. Another bookshop shared similar information with West, while Publishers Weekly reported that a shop outside Las Vegas had a customer who ordered 87 copies after learning it was an NYT-reporting shop.

(Source: https://www.theguardian.com/books/2017/aug/25/handbook-for-m...)

I'm also skeptical of data from Amazon, whose KDP Select/Kindle Unlimited ebook platform is notorious for scams designed to ensure rank or extra payouts for authors. See http://www.annchristy.com/ku-scammers-on-amazon-what-you-nee... for an explanation of how one scam worked, and the HN discussion: https://news.ycombinator.com/item?id=11520212.


A friend used to work for an organization whose function was to buy books from NYT-reporting sellers for purposes of inflating the sales numbers of a given title, to manipulate it onto the bestseller list. They were usually paid by the publishers, but sometimes directly by the authors themselves.

This has been going on for a very long time.

EDIT: Phrasing.


WOW, can I ask what the most common genre was?


I didn't ask, but in the conversations we had on the topic, I never got the sense that was a thing that mattered to them.


The process was broken for seven months before anybody noticed?

If I were the Washington Post, I'd wonder why nobody brought up discrepancies with other lists during that time.


I misread it the first time as one month (May->June) but it was May 2018 to June 2019(!).


Someone probably did, and the person they told didn't take it any farther. The chain broke somewhere.


[flagged]


That was dismissed recently, with a handy summary from the judge: The judge who oversaw the case, William O. Bertlesman, said in his dismissal, "The Court accepts Sandmann's statement that, when he was standing motionless in the confrontation with Phillips, his intent was to calm the situation and not to impede or block anyone. However, Phillips did not see it that way. He concluded that he was being 'blocked' and not allowed to 'retreat.' He passed these conclusions on to The Post. They may have been erroneous, but as discussed above, they are opinion protected by the First Amendment. And The Post is not liable for publishing these opinions."

https://www.cnn.com/2019/07/27/media/washington-post-defamat...


The dismissal has issues and contradictions that several major lawyers and even other judges have pointed out, for example it was dismissed before any evidence was even adduced. Many of the claims the dismissal relies are on exactly the claims which would've been tested in court.

I highly recommend this video as a good overview of the issues: https://www.youtube.com/watch?v=iSRVnvfD6Cc


It'll certainly be interesting to see where the appeal goes.

I need to read the opinion, but my initial inclination is that the judge should have let it go to trial.


Wins against newspapers for libel are extremely rare in First Amendment country.


But we've also never been at a point in history where a "respectable" publication can blast erroneous information out at the scale the Post did.

They might not lose the in a trial, but there needs to be reconning of some sort which breaks the assumed trust we've developed with antiquated publications.


You should consider the probability that in the past when newspapers deceived the public, the deception went largely unnoticed (at least at the time.) For instance, there is the time American newspapers almost single-handedly started a war with Spain.

Or, even worse, the infamous case of Pulitzer Prize winner and genocide denying communist sympathizer Walter Duranty using the New York Times to promote Soviet propaganda during the Holodomor. The damage there is still ongoing, despite the NYTs coming clean half a century later. When HBO's Chernobyl mentioned it by name, that was the first time many Americans ever heard the word.


FYI there is a memorial outside Union Station for visitors to Washington DC.

https://parkplanning.nps.gov/projectHome.cfm?projectID=39537


This is very, very incorrect. The term "yellow journalism" exists today because of really nasty work done in the late 1800s. Nobody at the Washington Post--or any other American newspaper today--is operating at nearly the same level of toxicity as William Randolph Hearst did then.


That is what the internet is for.


America loves its fake news, that's why Alex Jones was popular for years. He was able to lie freely about anything and anyone until finally libelling the Sandy Hook survivors was too much.


As other replies stated, the suit was dismissed and for the reasons stated. However, this is how journalists do hit pieces: find someone who will say what they want to say then quote them and thus is slander converted into "news". In this case they wanted to hold up for public hatred a kid who was just waiting for his bus. Why else would a kid waiting for a school bus be news?


Frivolous lawsuit, you mean. Perhaps you need to do some fact-checking.


My comments are merely opinion, fact checking not needed.


As was the claim that was made in Washington Post, according to the judge.

Edit: I initially said the claim was made by Washington Post, but it wasn’t their claim.


I'm not sure why the reply to you (from someone else) has vanished; perhaps that poster deleted it to redraft it. But your description of the case is, well, a little facile. This is the case of the Trump-supporting high school student who felt he was being unfairly portrayed in a confrontation with an American Indian activist in January, and indeed, it was dismissed last week.

https://www.pressdemocrat.com/news/9844670-181/judge-dismiss...

The original reply to you said it was dismissed because the statements accused of being defamatory were mostly opinions, which is true, but in context, they weren't even the Post's opinions: they were statements by others, including the activist, the Post was reporting as having been said. The statements by the activist that he felt threatened are protected because, as opinion, they're not subject to libel laws; the Post's reporting on the activist's statements appear to be accurate.


They should just get rid of bestseller lists. They exist purely for marketing and for tricking chumps into buying mediocre books.


Discovery is challenging, how would you propose finding something new to read?


A lot of influential figures publish their reading lists, like Bill Gates [0], Elon Musk [1], and Patrick Collision [2]

[0]: https://www.gatesnotes.com/About-Bill-Gates/Summer-Books-201...

[1]: https://medium.com/swlh/elon-musks-reading-list-every-book-h...

[2]: https://patrickcollison.com/bookshelf


That seems like an even more questionable tool for deciding what to read than a best-seller list. Those guys are renowned for starting tech companies that made lots of money, not for their outstanding taste in books.


There are numerous other people who make book lists. Authors write book lists called "bibliographies".


They are also very smart, which made them read/like interesting books.

If they read books ofc, in these cases they obviously do


>They are also very smart, which made them read/like interesting books.

This logic doesn't take us anywhere because there existed equally intelligent people who either worked in farm or meatshop or even might be working as a janitor instead of launching companies worth billions and recommending othes books.

And does being intelligent means not being evil? What if they posted these books to distract you while they go around reading something else which helps them build competitive advantage against masses.

I won't blind accept a reading suggestion from someone even if they happen to be commerically successful or they happen to be smarter.


So people post book reviews to distract you from doing something. Lol.

I still have a bigger respect for people that started a multi billion dollar company than a janitor. It doesn't mean I have no respect for them, but just a bigger one for someone that did something unique.


Why ask three people when you can ask a million?


Because my taste and interests are more aligned with person X than with the general populace.


Are you sure it's that and not that you self-identify as being person X? Just how well do you know X apart from X's name as a brand?


While your question is interesting as a psychological thought experiment, it does not matter pragmatically.

If I am interested in, say, sci-fi, or futurism, and so is person X - or, person X's marketing image - then person X will bolster their brand by posting sci-fi and futurism books.

Therefore, I still get recommendations in the genres I'm interested in, and whether those recommendations are accurate portrayals of person X's interests is irrelevant.


Because the opinion of the three might be worth more.


i highly recommend this as well. have a list of people you admire and monitor/check their twitter, blog, website, etc. they usually post amazing books they have read and all you have to do is add these books to your wishlist (if possible, or bookmark them). see how long they stay there and how your interest in those books changes.

to be honest, books recommended by, say Bill Gates, are often in other spheres of influence and interest than i am currently part of and feel i can do anything about. but any technical book from John Carmack is a go for me.


Why should I care what those people are reading or think. I'm not them and they're not me and while some of my interests may overlap with them, it doesn't mean I want to, or will enjoy, reading what they read.


...okay? With the exception of tailored algorithms, i.e. Pandora, every media service or list is going to be the opinions of one or more persons.

What's on cable is relegated by Neilson ratings, what's on the radio is relegated by what's popular / the DJ's favorite songs, what's in a bookstore is relegated by what sells well (or, by what doesn't sell poorly). Everything is curated.


Why should you care about the bestseller lists? This is an alternative if you want to get some insight into the thought process of these (terrible) people.

Also these three people are examples; the point is that some public people that you may care about recommend books, and that is a way to discover books.


>thought process of these (terrible) people

Whoa that is some casually thrown-in hardcore judgement.


I only said it because I meant it. Booklists from people one admires (or detests) are a good place to check to discover new books, though.

I also suggest bibliographies and references in things that you are currently reading. That's usually where I find the next things to read. Especially with journal articles, because you can find other things that cite a paper you enjoyed pretty easily on the web.


> They exist purely for marketing


And then we can aggregate those, put it on a website and post affiliate links gradually becoming the go to source for book lists and then sell spots on the list and then people start publishing their own reading lists and someone aggregates those and...


browse goodreads! go to local author talks! go to a local book club ran by someone well-read! totally agree with the above supporting long form review publications like the new york/ london/ paris review of books; they typically will write about a general topic whose sources are 1-3 books that are listed. If the long form piece peaks your interest you know exactly where to dig in for more information.

Best seller lists skew towards "branded" writers typically pushing (in my opinion) a diluted version of an existing thought that is re-marketed for our time. There is nothing new under the sun, so stick to the classics. Life is much too short to read mediocre books. (:


NLP text analysis by NNs to identify features that humans can't perceive but are shared across genres that are liked by the user.

(I'm only half joking - book discovery is a pain point in my life and I fantasize about this every time looking for a new read)


Browse a book store is the obvious answer.

But this makes sense, because a store will generally buy items that it thinks it can sell, so it'll come down to humans making choices about what's a good book. Of course, publishers will be pushing their latest items through their marketing chains, and so new books will make it to stores based on marketability to the store owner.

In the digital realm, this is less of an issue, and can accept more books, but makes the whole shopping experience very noisy imho.

However, I personally like to read those little recommendation cards that the shop assistants make. I've bought a few books outside my normal genre that way, and haven't been let down yet.


By sticking to the classics and letting time act as your filter.


This is by far the most insightful comment on this thread. Especially for fiction and poetry, which like opera and classical music, the we're presently in a period of creative lullness, and seemingly nothing that compares with the grandeur of works from the 19th century ever comes up these days.


Oddly enough, that's not been my problem for some sime now. Note that I largely read nonfiction....

1. Find a topic of interest.

2. Start reading at some arbitrary entry point, though generally favour texts with references.

3. Branch from there.

- Find the authors you most respect / who most make you think, and read their publications, and track down their sources and references, or where they're cited. Notes, footnotes, references, bibliographies, and citation indices are gold.

- Find the topics, questions, ideas, methods within the topic that you're most interested in and drill down on those.

- If you find yourself needing new skills in the course of explorations, pick up on those.

- Don't hesitate to ask for recommendations.

- If there's a concept that seems unclear, chase that down to its origins, and trace variants as the concept evolves

- Incidental histories and biographies can often provide further illumination.

- Eventually you should start constructing your ow structure or modle of the field or space (or subsets / components of it/them). This can further drive your exploration.


I think the key is looking for longevity. A book at the top of a bestseller list for a week is a mere blip. Books start catching my attention when I seem them out there for about 3 months- its not some flash in the pan or selling just because the author previously wrote a good book. Around 6 months, you start getting stuff that is almost certainly going to at least be entertaining. If something has been on the list a year or more, this is likely going to be a thought provoking book and something highly enjoyable. This scale has never really let me down- except maybe in the case of The Alchemist. That was such a miss and the language just so... vapid... that I actually downloaded a copy off the internet just to see if I got a bootleg mistranslated version or something. I read tons of books, 99/100+ times a book on the NYTimes Bestseller list for more than 6 months has been worth its time to read.


Despite the obvious conflict of interest, you can get pretty far by talking to bookshop staff. They’re around the books all day, so it’s pretty much inevitable they’ll leaf through them and form their own opinions.


Read quality book reviews. I would especially recommend the New York Review of Books and the London Review of Books, but there are dozens of long-form journals of this kind.

Most newspapers also have book review sections, as do many academic journals.

Presumably if you're interested in someone or something, you can also simply peruse the internet to see what's out there, i.e. you can start with your interests rather than with what's popular or well-advertised.


I find Goodreads recommendations pretty good, which are presumably based on a souped-up version of "people who liked $YOUR_LAST_BOOK also liked X".


If it's on a bestseller list, it's already discovered. It's also going to be pushed by every book retailer.


That’s securities fraud


For which security?



I actually wondered whether it was a reference to Matt Levine. Does literally every HN reader also read his news-letter?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: