Years ago I worked as a frontend dev for Newsweek & The Daily Beast and saw the other side of this. It boiled down to money, brand, advertising and engagement.
Wrapping the content in an iframe allowed the host to monetize our content by decorating it with ads. This would dillute the CTR on our ads, resulting in less money. Big sites enter into contracts with major brands for full page advertisements, which could include contractual requirements that theirs is the only ad on the page. Serving and decorating a site would create headaches for us to explain why their ads weren't alone.
The iframe also reduced engagement as reader would only read the embedded article and move on. An actual visitor of the site had a far higher likelihood to navigate to other pages. Higher engagement resulted in more money and some brand loyalty. This also dramatically hurt sharing as the browser URL wasn't ours, but the hosts (which I also hate about AMP).
Your site messed with a lot of the ways their business generated revenue, in an industry that was already struggling to adapt to "the internet".
I think what the "legacy" news media failed at more than adapting was effectively communicating the time and effort that goes into serious news gathering and reporting. I regularly get the impression a certain slice of HN commenters think "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees.
Most software engineers innately know that taking an open-source project, stripping the authorship, putting it your own site and running ads next to it is unethical, regardless of the licensing etc. Yet, when it comes to capturing news stories of equivalent production effort, "information wants to be free". I think that may largely be a failure to understand the work that real reporting takes.
Ironically, as prestige newsrooms have shrunk, the percentage of high-quality stories has gone down because revenue has declined and those stories are expensive. So in fact we do perversely end up with more click-bait as Google et al engineered systems to optimize for attention and click-through (the opposite of a multi-part investigation).
> I regularly get the impression a certain slice of HN commenters think "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise.
That's not just an impression, it's a fact. More than once have I seen commenters suggest (with what I assume is a straight face) that amateur bloggers can effectively replace "the media." A similar position I've seen frequently is that science reporting is worthless, and individuals should just sift through the all raw papers instead (which not even actual scientists do).
The best bloggers can do is substitute for opinion writers, since having and expressing an opinion is really a solo endeavor, regardless of where you do it. Pretty much everything else needs a bigger team and more support structure.
I think OP was trying to make a point that the entire investigative journalism industry can't be replaced by freelance bloggers, not that freelance bloggers don't add value.
> I think OP was trying to make a point that the entire investigative journalism industry can't be replaced by freelance bloggers, not that freelance bloggers don't add value.
That's exactly it. I elided some thoughts about sustainability from my comment, but I basically think reporting is so much more labor intensive than opinionating that no configuration of amateur or part-time bloggers can substitute for all the vital civic functions of a newspaper.
That's not to say that bloggers can't to real reporting, but when they do, by necessity it will have a far narrower topical focus and will either burn out an amateur blogger or force that blogger to professionalize and evolve their blog into something a lot like a traditional media organization.
Given how much easier opinionating is, I'd wager 99%+ of bloggers will stick to it (and other easy things like link aggregation) 99%+ of the time. If you skim the cream, you could probably come up with something more-or-less comparable to a newspaper opinion page.
Agree completely. I couldn't figure out how to both save democracy and pay my bills. And I'm not well suited to the fund raising, running a non-profit org type stuff.
Only notion I have for addressing this is some kind of grant or fellowship, meaning just give prospects cash.
Your incredulity isn't much of an argument. A blogger is usually some person at a keyboard, typing. A journalist is the same.
The major difference is people going in to journalism tend not to be domain experts or leaders in their field. That means bloggers can easily substitute for investigative journalism - university professors, for example, are usually better researchers than news reporters.
In fact, in this rather divided era, it is fair to question character of most journalists and ask exactly what is positive about their contribution. Good intentions do not make good outcomes, and for most things I care about the formal media is more effective at silencing real concerns than amplifying them. Bloggers sometimes get upset about issues that are important to me and tend to sustain the coverage for years. Consider https://en.wikipedia.org/wiki/Groklaw . No competition I'm aware of in the media circuits for that sort of work. I'll take the bloggers thanks.
Amateur journalism came into my awareness through reading about Lovecraft. My impression is that people engaged in journalism for various hobby/interest groups, clubs through at least the 1920s and considered themselves amateurs in the sense of practice for the love of the activity itself. I can see a similar strand in 'zines and later web logs and blogs.
A rather older sense of "profession" (maybe concurrent with people calling themselves amateur journalists) meant any job which entailed a primary responsibility to the public, like a doctor, a lawyer, or a journalist. Unlike the other professions, one didn't need any educational credentials, bonding or special license to be a journalist.
Journalism is unique among the professions: in at least some sense of the word, if one considers oneself a journalist, one is a journalist.
> In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise.
Investigative journalism is 1 or 2% of the budget of most media companies. Saying that running a media company is expensive because of investigative journalism is like saying that running a tech company is expensive because of the free snacks.
Sadly this. A lot of the "old" media companies saw the "success" of free online content, be it digital-only-ad-based publishers, be it social media, and started to mimic that, and in the process not only desperately pivoted into a market that in fact isn't as successful as they thought but also burnt a lot of their reputation in the process.
Right now it seems back to paid models for a lot, but I wonder if it's too late now with all the damaged reputation and the shift in consumer expectation (that they help create) that news and journalism should be free. I'd guess for a lot of those newspapers and magazines it will be too late, while a few will survive the onslaught and be able to concentrate the willing-to-pay consumers.
Youtube personalities are aggregating and editorializing their content and making millions to pay for a staff of 1.
Others are just going and doing the reporting themselves and doing similarly -- paying for the camera crew and legal defense out of their own pockets.
Those youtube personalities dwarf the reach and viewership that all but the clickbaitiest media companies are getting.
That's what's satisfying the market need right now. That's what media companies have to compete against. They'll never get back the dominant position that they once had and ads will never work for their business again. Why would I pay for a display ad on your site when I can pay for a video ad that has much better targeting?
This is the issue that most people have missed. Advertising dollars flow to where they are most efficient. News site display ads are a shotgun whereas youtube channel, facebook page, news aggregator ads are a sniper. If any periodical is going to succeed, they're going to need to cater to a hyper specific niche that has cash.
Err no, it might look that way sometimes so that it seems authentic, but most major youtubers have staff. They are way more lean on expense than the TV pundits but still!
The Joe Rogan Experience podcast has only one full time employee beyond the host, plus a few part timers for special projects. It is hugely profitable with a valuation far higher than some whole media companies with orders of magnitude more employees.
> the shift in consumer expectation (that they help create) that news and journalism should be free
OK but they created that expectation in the 1890s. The Internet isn't the cause of the media companies' problems, it's just beating them at their own game.
Not really. There has been a long tradition of mostly shit quality journalism available for "free", along with some refurbished agency reporting, true. However, the quality journalism for the most part was still paid, as in buy a paper or magazine or a subscription, if you want reliable and regular in depth journalistic reporting and/or investigation (with some exceptions, of course). That really only shifted when publishers en masse put stuff on the web for "free", financed by ads.
Well it's not like they didn't know that people hate ad networks, hate getting tracked the shit out of them, hate downloading 5MB of mysterious JS to read a 2kB article.
The sliminess of the online advertising industry wasn't a secret 10 years ago or something.
So don't talk about it as if they ever were an ethical source of business income. (Not just talking about all the lying that is inherent in advertising, print ads suffer from that too)
They never were, they're not like inert pieces of your magazine. They're little pieces of software doing whatever the fuck it likes, control that you offer to semi-anonymous parties over your readers.
Imagine if a print newspaper didn't just sell ad space in their print, but whoever buys the ad also determines what ink it is printed with. And the ink can be any substance! Sometimes it's ink, sometimes it's poop, sometimes it's smallpox, sometimes it's radioactive.
I mean, MOST of the time it's really just ink, with only a tiny amount of aerosolized heroin, just your average respectable ads. But every once in a while, and there is really nothing to do but hope it happens to the other paper first, it'll really harm the reader.
I mean you can't really expect a news magazine to check all the ink its printed with right, just doesn't scale.
From a business perspective, perhaps. From a social benefit perspective, getting rid of corrupt politicians, exposing pollution, and giving citizens the ability to understand what the powerful are doing has massive benefit. Think of all the fraud and waste that has been uncovered over the years, recouping that money doesn't go to the newspaper, but it may benefit society more broadly. Simple fact is places with less free media have more corruption and misuse of public funds.
A shockingly high number of people who would have been journalists 20 years ago now work in Public Relations. Many journalism school graduates try to find work in the trade and end up as corporate mouthpieces.
well I wouldn't say corporate mouthpieces, but journalism is mostly rewriting news from bigger agencies (like the dpa from germany) or from freelancers. I mean it's basically impossible to fill a daily newspaper with just own articles/personal. so newspaper call agencies to fill a lot of stuff. and they bring mostly a few original stories.
I mean literally people who are trained journalists, and sometimes even experienced ones, going to work for a company or agency and writing press releases.
A more interesting number would be the proportion of the budget that goes on journalism full stop. "Investigative journalism" can't be cleanly separated from other kinds. It also costs money to give an informed take on a bill going through Congress, the latest iPhone, or the Covid stats.
Do you have a link to that percentage of the budget? And also what comprises the investigative journalism budget - if it is primarily for the wages of employees doing the investigating part it seems not that bad in salaries are generally not the largest part of most companies' operating costs (although generally significant)
I would suppose the offices, computers, health benefits for family etc. might not count as part of the investigative journalism budget as well. Or they might, accountants of respective media companies would know I guess.
Speaking as somebody who has been involved in tech sales for a long time, 'communicating' is not the main problem. You need a monetization model that channels users into paths that oblige them to pay for content. And it needs to work at scale.
$120 a year for a subscription might seem iniquitous to HN readers. From the business point of view you need over 1500 subscription to pay for a single dev working at non-large-city rate, e.g., in Richmond Virginia. That's just to run the website, not to generate content. It's a big hill to climb.
My estimate assumes a $150/year salary + benefits, $120 subscription, and 20% customer acquisition cost for each subscription.
My point was more about software engineers having trouble grasping effort/production costs involved in media outside of their immediate domain, despite having the power to dramatically affect revenue calculations in other domains.
This! I doubt whoever is running the website for the Richmond Times-Dispatch is making more than $75k/year. They're probably also maintaining sites for others papers in their group, and maybe even working on sites for local businesses too.
Pick your salary. With benefits and overhead it's not going to be far off even in Richmond assuming you can find the right person. Most big media outlets work in more expensive markets.
My point is that you have to make real money to support these businesses, which means figuring out a monetization strategy that results in 10s or even hundreds of thousands of subscribers.
> I think what the "legacy" news media failed at more than adapting was effectively communicating the time and effort that goes into serious news gathering and reporting.
Among many parts of the public, the perception (that IMO is correct) is that serious news gathering and reporting is the rare exception, not the norm.
It certainly seems like most news pieces these days take the form of uncritically reporting anything the government proclaims as fact, thinly veiled activism, and meaningless outrage click farming.
The respectable journalists out there with a shred of integrity and effort put into their reporting are very rare. Maybe a Glenn greenwald (who is an idealogical opposite of mine) is a rare exception. But I’d struggle to name more than like 5 journalists i think are even largely interested in truth, willing to take some risks, and have a bit of integrity.
The business model of journalism has changed (outrage based click farming) and the profession’s respectability (to me) has changed with it. Journalism was respectable before you could measure clicks and therefore profit.
> Most software engineers innately know that taking an open-source project, stripping the authorship, putting it your own site and running ads next to it is unethical, regardless of the licensing
only "stripping the authorship" makes it unethical, all the other things are totally fine and approved in the open source world.
Make an entire copy of debian software repository? People will thank you for another "mirror".
create a webpage which hosts existing docs (like man pages or software package documentation)? Totally fine as long as the docs are not modified (so any AUTHOR section is intact)
create another "how to get started with ZZZ" website, which is three quarter official docs? Ok as long as license allows remixing. Many do.
start a business which took over many project's discussion forums and centralized information in one place? And then the said business gets millions from ads and kills many previous forums? People love this, some even say that Stack Overflow is "the best thing that happened to programming"
and so on... so for people used to open source model, news aggregators are totally fine, as long as content is not modified and there is a clear attribution. The GP comment mentions "iframes", so both of those would have been true - it's not like you can blank out author name from a third-party iframe.
Yes, the journalism is different and the open source rules do not apply to it. But hopefully you can see where does this "information wants to be free" mindset is coming from.
This makes sense for big, well researched investigative journalism. But that is not the case for the other 95% of the drivel on most news sites -— the engagement driving crap that keeps the lights on.
Lots of people want to see sports scores, news, and commentary. I'm not a "sports guy", but those types of benign tribal affinities can help community cohesion and also generate revenue for more expensive forms of journalism. I don't think it's a bad thing, either for people who get excited to "root for the home team" and get that from a newspaper, or the newspaper reaping that reward and reallocating it elsewhere in the business.
> I think what the "legacy" news media failed at more than adapting was effectively communicating the time and effort that goes into serious news gathering and reporting
I think what they failed at was actually putting in the time and effort. I read news stories every day that fail to include basic pieces of information. How many news stories have you read that say "The Republicans proposed X dollars and the Democrats proposed Y dollars" that don't make any sort of historical or international comparison? It seems to me that "serious news gathering and reporting" would include comparing the amount of money to similar situations in the past or to similar situations in other countries, and yet, I almost never see that. The word "stenographers" has become more and more common for a reason - news stories, particularly political news stories, increasingly include quotes from the few active actors on the topic today, but no serious research or serious context outside of the people who literally called the reporter.
YES! This! I worked for a higher-up “legacy”(?) news paper for three years and what people don’t understand is that “the news” isn’t just some objective collection of facts. It’s work to collect, analyze, verify, source, and interpret those facts. That’s why it’s always so sigh-worthy when I see some other startup crop up in the HN-esque circle that’s “going to fix news” when these people have essentially 0 actual journalists as their friends or even as consultants on their product. Everyone wants to complain about paywalls and say their undemocratic or even say they violate “free speech” (misunderstanding both words in that right) and then offer no other alternative other than “trust us, we’ll do it better.” The fundamental problem is that a contrarian take on pop culture gets vastly (3-10x) more views than something which took several months and dozens of sources to put together. I could go on and on but point being: Y-E-S, us tech people cannot “fix” other industries. We can help, but cannot lead.
Even worse is the amount of time a publisher maintained their status as the "exclusive" source for a story used to be a least a couple days - meaning a subscription to an outlet that regularly has an edge is valuable (particularly in finance journalism, which has fared better than the industry as a whole). Now, a newsroom spends a year preparing a story and somebody summarizes the key points, devoid of context, in a Twitter thread and nobody sees the actual story. Why pay for getting the story first if you can just wait a couple minutes to get the free summary?
> "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees.
I don't think there is any evidence to suggest deep, unbiased, evidence-based journalism ever dominated the field, or that the "crass commercial" interests didn't drive the news rooms.
See the "yellow journalism" of Joseph Pulitzer's era, for example.
Further, "expensive, pulitzer level" work may also be agenda-driven. Walter Duranty's bias and agenda, for example, aren't really in doubt anymore.
> I don't think there is any evidence to suggest deep, unbiased, evidence-based journalism ever dominated the field, or that the "crass commercial" interests didn't drive the news rooms.
> See the "yellow journalism" of Joseph Pulitzer's era, for example.
> Further, "expensive, pulitzer level" work may also be agenda-driven. Walter Duranty's bias and agenda, for example, aren't really in doubt anymore.
Those examples are stale. It's like saying there's no evidence that English language literature was ever common in any library, and citing the contents of some British library circa 1100AD as proof. Pulitzer's yellow journalism phase spanned from the late 1800s to about 1900. In the last decade of his life, he rejected it an apparently turned his paper into a respectable publication [1]. Duranty's reporting occurred in the early 1930s, and has since been scathingly criticized and rejected by the paper he published it in [2].
I chose historic examples specifically because they are now universally recognized as biased, and to show that problems of sensationalism and bias in journalism are not new.
More recent examples of bias tend to be still politically controversial, like the years of coverage of Trump-Russian collusion in the 2016 election, which resulted in 2 Pulitzers:
For deeply sourced, relentlessly reported coverage in the public interest that dramatically furthered the nation’s understanding of Russian interference in the 2016 presidential election and its connections to the Trump campaign, the President-elect’s transition team and his eventual administration.
But the Mueller investigation, which had to stick to what could be proven, in spite of any personal agenda, concluded:
Ultimately, the investigation did not establish that the Campaign coordinated or conspired with the Russian government in its election-interference activities.
There is also good circumstantial evidence connecting the Russian actors to the Russian government, but Mueller couldn't subpoena or extradite across the Atlantic, and declined to follow the money trail.
This undercuts your point somewhat, but on the other hand perhaps you are being intentionally self-referential.
Maybe, but just because it's impossible doesn't mean there shouldn't be an effort to try or that we shouldn't appreciate better journalism.
Biased isn't a boolean, it's a scale. Something that is a little bit biased is more useful for trying to understand all the issues at hand. Something that's very biased is still useful, but only for telling you what the author (and their readership) thinks.
It would seem like the content creator has the right to make their site as easy or hard to aggregate as they want. If you’re in the aggregation business, you have to convince them of the benefits to make it easier. If you can’t, you’re not adding value (to them) for what you’re taking.
There was a time when corps pushed the idea that you'd need permission to link to their content, where does this begin/end? seems to me there should be some control on behalf of the user wrt to how data is displayed: It maybe your content, but it's running on my hardware; you say "3rd party not allowed to run ads, only us" I say "my hardware, you aren't allowed to run ads either".
If a brick and mortar store refuses services and kicks you out for disruption... breaking in at night, taking what you want, and leaving money is still illegal.
I love web scraping (it and tampermonkey basically got me into CS) but lets not act as if this behavior has no moral precedent.
You really shouldn't expect to be able to put up a public website and then get upset when people start connecting to it using a standard, open protocol to retrieve the information you're deliberately exposing to the world.
Ah, the old "unauthorized content handling is the same as theft" nonsense, because a more fitting metaphor wouldn't justify police-state copyright enforcement.
Also, what do you mean by "disruption"? Using "their" content in a way they don't like? content manipulation on my machine is my house, not theirs.
At the end of the day you can't legislate everything to the letter if the people just don't want to do it. If people want to steal content the content creators will suffer and we as a society will lose high quality content. I think that some sort of NPR or BBC patron approach is the only way out, i.e. sponsor a high quality production organization as a charity. Communist countries did this too, though it ended up quite biased.
> If people want to steal content the content creators will suffer and we as a society will lose high quality content
This sounds like a PR copypaste. I don't want to "steal" content, I want to restrict the number of ways I can be influenced to view it. And "high quality content" is often not the case here; I have to get through a lot of crap to get to real news in a lot of cases. There is also the problem of who gets to judge quality - often those that tell people what they want to here are judged so - Reality has a subjective bias.
My sense is if you put it on the public internet, by default you are giving people the ability to link to it. It’s just a matter of how hard or easy you choose to make it.
We are getting further from my expertise. My inclination is on the public internet it’s ok to make an ad blocker and ok to make something that thwarts the ad blocker. Different on a private network.
Most of the major news websites still offer RSS feeds, although you do have to search quite a bit to find them. NYTimes, WSJ, Vox Media, The Atlantic, Time Magazine, etc etc all have them. Those last two are even full-text feeds, which is super lovely!
I have a suspicion that journalists rely on RSS more than the general population, which gives the technology support within news organizations themselves.
I can speak first hand to that, in Canada. RSS is actively supported. Specific RSS feeds are even run to (web, Apple News) aggregators after agreements are made to syndicate the content.
And we maintain feeds for the Canadian government archives.
I can only imagine American outlets do something similar.
If this is a case of one party trying to throw more ads at the "user", and being miffed about earning less of the ad network tracking points than the other (meanwhile the ad networks win either way), then what are you defending?
Because in that case there is no good guy, no "other side", just garbage.
Stuffing the WWW with ad networks isn't exactly saving it.
I'm not really defending anything, just explaining. But I do sympathize with any content creator trying to get paid for their work even if it is a large organization.
Wrapping the content in an iframe allowed the host to monetize our content by decorating it with ads. This would dillute the CTR on our ads, resulting in less money. Big sites enter into contracts with major brands for full page advertisements, which could include contractual requirements that theirs is the only ad on the page. Serving and decorating a site would create headaches for us to explain why their ads weren't alone.
The iframe also reduced engagement as reader would only read the embedded article and move on. An actual visitor of the site had a far higher likelihood to navigate to other pages. Higher engagement resulted in more money and some brand loyalty. This also dramatically hurt sharing as the browser URL wasn't ours, but the hosts (which I also hate about AMP).
Your site messed with a lot of the ways their business generated revenue, in an industry that was already struggling to adapt to "the internet".