I don't trust Ahrefs at all. The whole SEO industry is tainted, and Ahrefs is not a good part of it.
I don't want a search engine that pays "creators" (a search engine finds things, if they want to monetize it it is after it is found). I want a search engine that runs exactly as many ads as are needed to pay for itself, nothing more. Preferably with an option to pay for myself as an option. I'm guessing that means https://kagi.com/ but I would like to see some more competition.
If you go to the about section of Yep.com, you can see this is powered by the "Ahrefs" bot (not just for sourcing crawler data but also run by the same people).
Ahrefs is a sleazy "SEO tool" ( https://ahrefs.com/pricing ) that basically exists to exploit weaknesses in search engine algorithms and other shady things (and make lots of money doing it -- $99/mo to $999/mo).
It sounds like you never actually used ahrefs, or even know what it does. You just read “SEO” and sent straight to “these are the people spamming Google with content marketing fluff posts when I search for a product review!”
What it does is showing you errors on your page 404, slow urls, pages that link to your site, missing tags, images that are too big and load slowly, keyword research and other things.
I use it even for my small projects (the free version) because it helps me fix broken links or improve the page speed which is all good for users.
Been using ahrefs for years, not sure what is sleazy about it.
It audits your site for issues, shows Google rankings for keywords, estimated volume for keywords, backlinks to specific sites, backlinks to specific pages...etc.
It doesn't do anything for SEO (no content, no links...etc), it simply reports on the state of affairs.
SEO itself is sleazy. Any gaming of search is sleazy, whether active or passive. The NSA is sleazy, even though they simply report on "the state of affairs".
Hi, I have a small SEO consultancy. We work with tech companies (mostly B2B saas), many you’ve heard of.
You’re right that some approaches to SEO are sleazy. They are a bit old fashioned at this point, and usually result in short term results which crash and burn. Google does actually care if users don’t find a good result; they are working on improving the results every day.
But you’re wrong in the sense that SEO done correctly is just a good way to get in front of potential customers who are actively searching for information by proving value to them up front.
Let’s say you run a company and I tell you customers are searching for your product 1000 times a day. What are you going to do? Not invest in SEO?
So, to demystify it, here’s what we do:
1. Identify topics that people are searching for, prioritize by search volume and conversion potential
2. Create content which is going to best satisfy that person’s search intent. We don’t want them clicking the back button and choosing another result, that would hurt our rankings in the long run
3. Try and link to the new content from other pages on our own site
4. Identify any technical issues which would prevent a search engine crawler from being able to access or understand our content
There's another side to the coin that I think you're, understandably, overlooking. I don't think either side alone is the full story. It's a big grey ball of tradeoffs, no matter how it all falls.
Even with just your own description of your activities you can find perverse incentives and negative externalities.
You aren't exactly incentivized to just create content that satisfies common queries. More fully, you're incentivized to create content that makes people think (rightly or wrongly) their query is being answered while also marketing your goods/services to them as much as possible.
Even assuming the best intentions to actually answer the query - be it altruism, discerning searchers, fear of Google, or something else - the content itself is hardly unbiased. This isn't inherently some kind of moral failure on your part, but it's hardly negligible either for the searcher. Everybody is going to have different thresholds here for what crosses the line, but I think it's understandable that some people are just unhappy having to even put in the time and energy to disentangle this aspect in the first place.
And this dovetails into one of the obvious externalities. Who is doing SEO, and who isn't? That totally unaffiliated small-time blogger that just happened to write up a bit of info after having the same problem as you probably isn't doing a whole lot of SEO. At least, they aren't hiring you to help increase their hosting costs. Somebody trying to find less biased views ends up having to wade through all the sites making enough money to pay for SEO first, to actually find what they're looking for. It should be pretty obvious why that does no favors for SEO's reputation.
But as I said at the beginning, you're not wrong either. This stuff goes deep into 'small' questions like profit-motives and competition for limited resources. Obviously and trivially correct answers aren't here.
I really do appreciate that you're approaching this from the much better intentioned side of things. One of the few things that might be obviously correct is that your approach is better than the sleezy way.
But that doesn't make it all rainbows and sunshine either.
I think you can find exceptions in any discussion, but that's not exactly helpful.
Here's the thing: If I don't answer the user's query accurately, another website will. Then they will get all the traffic. So my long term incentive is to satisfy the query.
Within that there is a certain tolerance and understanding among search users that websites can't publish information for free. See Wikipedia constantly begging for money.
So companies (my clients) can justifiably get away with a small amount of self promotion. But if you take that too far you'll just get punished in the SERPs too.
I get this sentiment. "SEO" is basically an euphemism for spamming at this point. It's no longer just "Let's repeat this keyword 3 times" but "let's add 120 footer links and spam 35 blog comment sections per day"
I don’t think it’s how it works because I see that crap daily in footers, especially in the travel industry. Open any hotel listing on TripAdvisor and count the “[something] in [location]” links in the footer.
How do you think Google knows which pages are helpful when someone searches for “[something] in [location]”? Anchor text in links plays a key role, but with search listings on sites like TripAdvisor, it’s tricky to naturally insert those links into content. Footer links is one way to do it.
I’d agree With you if those links were misleading, but they usually aren’t. They really do point to pages with “[something] in [location]”. And they really are helpful for search engines looking to give useful results for that type of query.
They have am aggressive crawler that I block. Why would I let a company use my resources to gather my data just so they can sell it to others for a premium at my expense?
They use a specific UA which is in the spirit of robots.txt, you're able to identify and allow/disallow access.
Trying to masquerade as another agent would be considered bad form, but obviously happens a lot.
There's a similar bot, MJ12Bot that powers Majestic's index which is similar to ahrefs. IIRC they have a user agent but their crawling is distributed, it's impossible to verify whether someone with that UA is them or someone else masquerading.
Good practice by bot owners is having a UA and known IPs they crawl from which can be verified by DNS and reverse DNS lookups.
The massive amount of github/stackoverflow content clones sitting at the top of the results for programming related queries seem to disagree with this sentiment. Ok I guess they are "creating content people want to see", but if just copypasting everyting from another source is enough for that...
Also the large amount of bots creating fake accounts on forums (my experience is with Discourse) with profiles and spam posts to SEO boost some site indicate a whole shady background business going here.
Yeah, reminds me of anecdotes from the Dan Lyons book [0] about Hubspot. There was the shiny fun stuff happening "front of house", but back in the boiler-room it was ugly. And everyone gets shown only what they need to see to make them feel happy/sign/pay, and nothing else.
No-one actually wants to know how the sausage is made.
It really depends on who you ask. The most "successful" companies don't whitehat their way there. Just open any modern social media website and tell me if they are respectful of the users — and you can see that part. SEO is mostly invisible so they really don't have to care about you.
Content that people want to see ... plastered with affiliate links and/or biased to sell those people what you want them to buy. This is about extracting value and that value has to come from somewhere. Searchers are not benefiting here, they are the ones ultimately paying for it.
Is it just that it isn't literally desecrating a corpse?
Do you have an alternative example of "something which a shove is especially useful for and which which could be lucrative, but which is widely regarded as bad" ?
It is tainted because the entire premise of "web search" is flawed. IMHO. Sure, one can provide "free web search", and this existed before search engine advertising was a thing, but over time one can see that the concept is flawed. The database being searched is public information. Every web user should have access to it. However, access to a database of public www content is generally not free. An single exception is Common Crawl. In practice one is more likely to pay for access to a web index. I would bet even the alternative search engine mentioned is paying for access.
An analogous situation is DNS. It is public information. Generally some of the database is made available for free, e.g., https://czds.icann.org/home. In practice many people get the information piecemeal via lookups from a third party via a shared DNS cache, e.g., an ISP DNS cache. However there are other ways to get the information. It is public information. There should not be a restriction, real or imagined, that one has to go through a third party to get it.
A further analogous situation is scans of www ports that check for httpds. If one runs a server that listens for connections on a public network, one should expect that the public may try to connect to it. This again is public information. No one should have to pay for access to it. However, often we see that bulk access is restricted to "security researchers". For example, censys.io will allow the public to search for free, with limits.
The trend I am seeing is that the public is being trained to view the web as something opaque. People pay ISP's to access the web yet they are led to believe there is no way to view what it contains. At least, not for free. The public is led to believe the only way to see what the web contains is to do naive searches through third party that is Hoovering up the data and trying to sell advertising services.
Imagine person A has a list of public information. Person B asks, "May I see the list." Person A says, "Sorry, but I compiled the list and therefore access to it requires a subscription. However I will let you search the list for free." The problem is that Person B will never know what is on the list, even though it is public information. Person B cannot even copy a small portion of the ordered list. IMHO, coupled with the perverse motivation of Person A to collect data and sell advertising services, this makes the concept of "search" flawed. If Person A said, "Sure, have a look at the list. If you need help searching through it, I can provide that service for a fee," then I would not see search as flawed.
In other words, I want to see more work like Common Crawl to share public information. Seeing the list, so to speak, can influence how one chooses to search it, so I want to see the list first. One of the flaws of today's "web search" is that it hides list entries that are not "popular" because non-popular entries have no value to advertisers.
The web seems to be becoming more and more opaque to its users. Like a Yellow Pages where no one can see past the first page. The competition to reach the first page is controlled by a third party "search provider" who can manipulate it to collect data and sell advertising services. However I want to be able to see the first page, the last page and all the pages in between. I want to be able to copy portions of the full list. I want to search a subset of the list without the use of a third party.
It is a strange business to commercialise access to public information.
Not OP, but I loved Neeva’s results. That being said, it’s a bit too personalized to my liking. For example, it automatically detected my location to display the weather, and the app’s search history cannot be disabled without private mode. For a “private” search engine, it did way too many personalizations by default. These few things, and a somewhat confusing UI made me switch to Orion (Kagi).
I personally use neeva and like it so far. High quality results, some fun UI to interact with on cooking and sports topic. Also like the feature to create spaces for sites interested.
Let's see... Under their proposed profit share, if they achieved DDG levels of traffic (100M queries/day, assuming $0.01 per query in net ads profit, that is $900k/day distributed among conservatively speaking 10M creators/sites) distribution would be $0.09 /day to an average content creator owning an average site.
It does not sound much. If it was to magically fully replace Google tomorrow (10B queries/day) it would generate $300/month for 10M creators.
Assuming content creator needs $5k/month to make a living, Yep would need to achieve ~160B queries/day which is ~16x scale of Google (problem: not enough humans).
The idea sounds noble on paper but it seems that math does not check. Missing something?
Pages are not created equal and averaging numbers here seems to be of little use. In practice the popularity is likely closer to some exponential-like distribution. Similar to music streaming services. Lots of people wouldn't get anything but they probably don't expect anything either.
I was using averages because this model is supposed to help the "average creator", those left behind in the current Google-economy model.
"Whales" and "superwhales" (which is what as you correctly conclude this model would probably converge to too) already exist in Google-economy world so not really interesting to observe. They would just make the average creator make even less.
This person for example (from their article)
"Say, you love pancakes more than anything else in the world. Imagine getting paid to share creative recipes, take photos and instruct website visitors on how they too can make amazing pancakes."
Would make $3/month (+- order of magnitude) if Yep was ever to achieve DDG level of success.
You put "average creator" in quotes but from my reading of their site Yep has not claimed they will help the "average creator", just "creators".
If Yep shares a higher percentage of revenue with creators than Google, surely that is helping creators even if it doesn't mean you're achieving some sort of imaginary "no man left behind" standard for the average creator.
If so, this quickly leads to the end-game of a dedicated industry that juices the ~~Ahrefs~~ Yep algo with bot armies and shady SEO tactics to push sites up the curve to capture this non-linear payout.
> The idea sounds noble on paper but it seems that math does not check. Missing something?
The profit is too low. And they're not saying they're going to provide a living wage, just an extra revenue stream. Just because that revenue stream isn't enough by itself to live on doesn't take away that it's a revenue stream and people who are indie hacking need to generate multiple revenue streams to provide security for themselves.
Why would it be a search engine's role to provide the entirety of every creators' livable wage? That would obviously be impossible for many reasons, and I don't see Yep making any claims about being a primary (or even significant) revenue source. Instead, they're proposing that the value of search comes from the end content, not the results page. The creator revenue stemming from that is inherently the appropriate amount.
Inciting the notion that they're individually responsible for the whole of content creators' income only encourages more ads or even nefarious behavior -- which is completely antithetical to your own company's philosophy, so it's not clear why you're so publicly holding a competitor to a different standard.
It seems we read the same article and understood it differently. The announcement post said :
"Say Anne from Kentucky helps 4,000 people learn how to build container gardens via her Wordpress blog every month. She would receive $1,000 monthly from the search engine in return."
To me this sounds like the idea of this model is to allow an average person doing casual blogging to make significant income? Given this premise, can we make it work?
To be clear I absolutely love the idea, I think it sounds fair and in the spirit of the web that I would like to see. The web needs innovative ideas like this, unfortunately I do not see this concrete one being viable.
So every time someone clicks to that blog about container gardens, the writer/owner of the blog gets $ 0.25?
Who pays?
With money coming from where?
And how bad will SEO efforts become to get their hands on that kind of money, which is 10 times (not 1,000 times, my mistake) more than that same blog would make with advertising, if we considered a CPM rate of $ 25 ?
For the record, freediver deceptively misquoted the blog post, and Ahrefs never claimed that those would be the revenue amounts -- they weren't even talking about their own company.
Removing the context[1] from that quote feels disingenuous, at best, and purporting that they even made any claims about their product is approaching libel. The 2019 personal blog post you're referring to is a general (albeit flawed) analysis of how a Google profit share might work for the top content creators based on guesstimated information from YouTube revenues. They were very clear about all of this and plainly stated multiple times that they have no idea if their analyses are accurate, and that it's all effectively an experiment.
Here is the actual portion from that post:
"YouTube’s revenue [...] can approximate that top-earners receive $0.0015 for every one view. Considering how much more ad revenue Google SEPR[sic] clicks generate in comparison to YouTube views, a Google profit-share model could generate significantly higher returns for top content creators. Based on a quick analysis, serving one search could pay an average rate from $0.05 to $0.25 in the US. Say Anne from Kentucky helps 4,000 people learn how to build container gardens via her Wordpress blog every month. She would receive $1,000 monthly from the search engine in return."
Given the context, you're putting many words in their mouth. It wasn't a product announcement, the product itself didn't even exist yet, they didn't actually claim a $0.25/query profit or even make any claims at all, they never made any mention whatsoever about being a substantial revenue source for "average" creators (who don't make substantial revenue to begin with), and they weren't even talking about their own product.
So, again, you're holding them to an impossible standard of revenue to meet your arbitrary and unreasonable qualifications of being "impactful." It comes across as ethical posturing, and shaming someone for napkin math about a different company's offering in an old "I have an idea" post just doesn't seem appropriate from a competitor in the space. I can only presume that you wouldn't be a fan of people nitpicking quotes from your old posts, so I'm not sure why you're doing it to someone else.
Edit: In fact, the other person who replied to your comment just proved that what you're doing is completely irresponsible and libelous. You're sowing the idea that those two sentences were in reference to the company's actual offering, which is completely false, but you're actively convincing people otherwise.
Edit: Here's a screenshot[2] of your unedited comment, since you decided to remove false information and things that I referenced.
I was under the wrong impression that this [1] was the announcement blog post for Yep. As I read the comments here initially, many mentioned that blog post and I did not pay attention to the date on it and just took it for granted. [2]
It seems that the 'announcement' post I was referring to isn't fresh announcement for Yep but a 2019 blog post by the founder explaining the concept.
In any case my math was meant to be a critique of the proposed model, not any particular product, and I still stand by that analysis. There was no malice intended.
> In any case my math was meant to be a critique of the proposed model, not any particular product, and I still stand by that analysis.
Except that your conclusion was to declare that search ad revenue is "not much" money, which is both plainly false and extremely strange for you to say, given that the entire premise of your own search engine is to take ad revenue out of the equation because the company profits and personal costs of ads to the end user are too high. So, I'm really not sure why you would stand behind an analysis that hundreds of millions of dollars is an "unviable" and irrelevant amount of revenue.
When Kagi is bringing in $365M/year, feel free to make all the declarations you want about $328M being "not much." Until then, you're being hypercritical for no apparent reason -- with the results being identical to malice.
You are putting words in my mouth and starting to get personal at the same time.
My conclusion was that the idea of profit share sounds good on paper but the math does not check.
The reason I say that is because math does not care about my feelings, what you or I want or what the idea suggests. It has to work.
If Kagi had $300M /year extra, and shared that money with 10M websites, that would be $30 /year or less than $3/month to a website owner. In my opinion that is not how you make impact and the cost of distributing this money would be more than what is being distributed. It is a terrible idea.
Now what can make impact is you take $300M and invest in something that can scale.
Please quote me, because I never put any words in your mouth, and accusing me of doing so is brazenly false and escalatory.
> and starting to get personal at the same time.
Bringing attention to the hypocrisy of your baseless claims is not a personal attack, particularly when the only question raised was your own company's status and publicly stated philosophy about the industry being directly discussed. You are not a victim in this situation.
> My conclusion was that the idea of profit share sounds good on paper but the math does not check.
This is the exact claim that you just accused me of "putting in your mouth," so you're doubling down on your completely faulty analysis, while also claiming I've somehow antagonized you into doing so.
> If Kagi had $300M /year extra, and shared that money with 10M websites, that would be $30 /year or less than $3/month to a website owner. In my opinion that is not how you make impact
As folks in this thread have explained to you multiple times, there is no such thing as an even distribution of traffic. In reality, the content creators who currently get very little search traffic won't be affected by this additional revenue source, but they aren't expecting to and nobody is claiming that they will. Whereas a content creator who does receive a lot of search traffic would see an increase of revenue that is significant relative to their overall income.
You are still trying to push these fundamentally absurd notions that all content creators have the same sized audiences and that the search engine should be responsible for their baseline revenue -- and if the company doesn't meet those literally impossible requirements that no other business in the world would ever be held to, then the concept is somehow inherently flawed and won't have any relevant impact.
> and the cost of distributing this money would be more than what is being distributed.
You're inventing problems that don't exist. The advertising and content industries have utilized minimum payouts for decades. The claims process could be as simple as a site verification code, which has also been used for decades by thousands of companies.
> It is a terrible idea. Now what can make impact is you take $300M and invest in something that can scale.
This explains everything about your commentary, and I find it to be anti-social at its very core. You appear to be plainly declaring that money can only be impactful when invested in large businesses that "scale" -- scale at what, nobody knows, but apparently scaling is inherently beneficial to society in some way -- and that supporting small businesses is a "terrible idea." That is illogical, harmful and completely unjustifiable.
Can you please breaking the site guidelines like this? You've been doing it repeatedly, unfortunately. Regardless of how right you are or feel you are, it's not what this site is for, and it destroys what it is for. Therefore we end up having to ban accounts that keep doing this. I don't want to ban you, so if you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
This is exactly the situation I emailed you about, dang. Given that freediver actually broke twice as many guidelines -- his most recent comment alone broke five -- and used HN as a platform to repeatedly slander a competitor, it doesn't seem appropriate to be singling me out. You are giving the instigator(s) a free pass on everything, while treating those who respond to those instigations as "destroyers" of the site. Help me understand why this is the case?
Edit: Just got your email. I will continue the conversation there.
I made my other comment about the disparaging qualities of your commentary, but I just wanted to point out that the numbers you're discussing are actually good for content creators. Incredibly good, actually.
Presuming your DDG example of 100M/day @ $0.01, that would be $328M/year paid out to creators. Bandcamp currently pays out ~$75M annually, and nobody is declaring them "unviable." So, I can't imagine why you consider this amount to be "not much," particularly when it would be additional revenue for creators which is typically 100% absorbed by the search provider.
Not all content is equal. Good content require efforts, often efforts of a team.
Youtube is a good example of how revenue share will work in practice. Some creators are getting millions there, some maybe hundreds. But as a consumer you can find good content for almost any topic you can think of.
Youtube is slowly killing itself with abusive revenue practices but ignore THAT bombshell for a moment and focus on why you are wrong. Hint: it actually has almost nothing to do with the fact that a large portion of internet users don't like videos.
We live in a pyramid, so not everyone can live of art/sport. For instance there is 87 million tennis player in the world, but only 1000 or less players actually earn enough money to live comfortably. Wile most of the earnings is withing top 20.
Production level are not enough to give everyone a life they would like to have.
As a network operator that runs a number of websites I'm honestly one step away from null routing all of their traffic. I'm constantly seeing their bots and as far as I know it's not helping anything.
It seems the websites with the most poor-quality, ad-filled, possibly incorrect information are the ones trying to game the system the most. If you're someone who just has a blog or website and is focused on the content, this turns ahrefs into a 'cartel' you must pay into if you want your own SEO to be on the same par as those spending all their time on seo optimization.
I worked for a company in the SEO space a bit over a decade ago, initially to try and learn the things you call out above. My surprise at the time was to discover most of the snake oil being sold as “SEO” was nothing more than basic usability 101 stuff. Make sure titles are descriptive. Try to use terms that your readers would use and recognise to ensure the content resonates. Structure your content in a sensible way with clear headings. It’s almost like Google were trying to turn a human experience into an algorithm! ;) There was obviously a bunch of black hat stuff too but the efficacy of that was generally pretty short lived and the techniques tightly held. The whole SEO industry seemed to be in on the joke that you only spoke about those techniques or provided training on them after they stopped working.
Something seems to have changed since that time though. I don’t think it’s Ahrefs because there were similar services long before them. Did Google lose their way and stop trying to quantify a human experience? I had my concerns back then when they introduced the concept of a canonical URL for duplicate content, something that’s implemented in a way to only be useful for a bot and not a person. But I’ve been too distant to these things to know if it was an isolated example or the beginning of a trend. Did the cat and mouse game of improving usability come to a plateau and now we’re in the inevitable end game of exploiting edge cases and 2% improvements? Is this what should have been the completely predictable state we’d end up given the alignment of incentives? Search results that are _just_ good enough so that you don’t abandon Google, but not so good that you click a result instead of an ad.
I don’t know what the cause is, or if it’s even just one thing. But it definitely feels like it’s been getting progressively worse for a long time and that the rate of decline has been accelerating.
To that end I’ve personally switched to Kagi. It’s at least ticked the “just good enough” box for me for a few months, which is something that never stuck with DDG. And so I fronted the cash for an annual subscription incase the incentive system with Google was part of the problem. I figure me moaning about it while still active using it means I’m complicit in being part of the problem too. And so I should vote with both my feet and dollars in support of any team that’s actually trying to fix things.
I also worked at a SEO company but not with a goal, although I also left there disgusted with the practice. The issue however (IMO) is that there's SEO at the article level and SEO at a strategic level.
At the article level, it is indeed mostly harmless, except when they catch some Google bias such as removing dates and edit notices from blog posts (because evergreen articles supposedly rank better), force a certain word count, H1/H2 count and paragraph size (which is why you get a life story before a recipe), or when it teaches you to disguise keyword stuffing and you get articles with semi-nonsensical paragraphs that can rank high. This is mostly annoying, though.
But IMO the biggest problem is SEO at a strategic level, which is what's done with the help of tools like Moz and Ahrefs. Websites often hire content agencies to make a large number of low-quality articles targeting popular searches that don't show too many results yet. Basically "balancing" demand/supply. But this practice puts non-SEO-optimized results at the bottom and replaces them with low-quality articles written by freelancers. This forces non-commercial publishers into bigger websites (like StackOverflow, Quora or Reddit). It also causes a lot of duplicate content, since those content agencies uses existing content from results as the basis for their own articles, only SEO optimized. The articles are of course rewritten so it's not word-by-word plagiarism. They also help evading Google measures to fix results. The latest trend when I was still in the industry was having unrelated companies helping each other do inorganic link-building by putting random links in more popular websites also controlled by those agencies, but that's probably old stuff by now.
It's been about a decade since I did anything serious with SEO. I knew it was sleezy garbage then, but did it for the money. I would outright lie to non-profits to get links from their sites to my seemingly legit site. After a few weeks I'd 301 it to my main site. I really feel bad about this. I did a lot of other underhanded shit to rank highly and it all worked.
The biggest problem was feeling like a bad human and not wanting to tell my friends about how I made money on my side hustle. I am really glad I sold those sites and left SEO behind.
I disagree, and I'll also note that being out of a field for a decade in tech is a long time (late 2000's SEO was very different). To prove the contrary point is really the easiest thing in the world.
1. Think of some product niche: say, best dog toys.
2. Google it.
3. Click on the top 5 search results.
4. Now, judge for yourself: Are these good quality links which look like they were written by a smart human who simply bought a bunch of dog toys and tested them, and is speaking from their personal experience? OR, are the search results gamified to hell, and clearly not something any normal person would think to write by themselves?
I think it's clear in 2022: you have to write in an artificial style and make a lot of unintuitive choices to rank on Google, which is to say, to have anyone online be aware of and using your website. You also have to compete against giant incumbents now who will crush you if you don't come educated and ready to rank against them. So you no choice but to learn SEO techniques, really.
Note: I'm not an SEO guy, I make $0 off SEO. I'm an independent developer who's concluded, if I want to make any money off my own websites, I can learn SEO or I can be poor. And I don't enjoy being poor. So there's not really much of a choice imho, at this point in the timeline.
One of the big developments in the SEO space seems to be making every title too detailed and specific. If you are a plumbing company that serves the whole country, your seo involves auto generating pages that mention a specific suburb or city. So when someone searches something specific, it looks like your page is the most local and correct.
Another one for products is having the same category named with 2000 related titles so it shows up for the exact search terms someone is searching even if they aren’t quite accurate to the category.
No one is ever to blame for anything in economics. The ad industry isn’t a person or thing with any ethics bound to it. It’s just a creation of the market. You may as well try blaming the air for being responsible for tornados.
The only solution is to try to build a new system that can’t be abused.
From the AHREF founder's blog post[1], Yep is designed more as a PR and hiring stunt with the aim of getting Microsoft to revamp Bing with a publisher revenue share model.
"Considering the platform [Bing] only generates a fraction of the company’s [Microsoft] $120 billion revenue, the organization could easily revamp Bing under a profit-share model."
As a side bet, any takers on how these types of visionary projects / PR stunts will become the new norm for hiring in a post covid lockdown world? Especially since working from home is no longer a differentiator.
I was excited to see this, because I often wondered why Ahrefs didn't try making a search engine out of their index. Unfortunately, the results are really terrible, in specific ways:
* One of my websites is not showing up at all, even though it's a site I've added into Ahrefs and that they crawl every month to send me monthly reports. That site has an Ahrefs Domain Rating 20x higher than my other websites, so it isn't being omitted for reasons of obscurity or quality.
* Yep seems to be processing backlinks in the wrong direction. If I do an ego-search, instead of listing sites that mention my name, it lists sites that I have linked out from my own blog.
* Pure spam results, like random casino websites in the middle of search results. Or is that Yep's business model?
* Entirely random results. Not idea why the Fox homepage would show up on an ego-search for myself.
But Yep's results were better than what I got with Brave Search, so they might still be onto something.
As with most supposed Google killers, it suffers from the small issue of being terrible.
Right now I'm in Los Angeles and looking to buy a roll of film. When I search for "where to buy film in los angeles":
- Google gives me three top Maps results, all small photo stores relevant to my search, a Yelp results page, some relevant reddit discussions with recommendations, and then sites for some individual businesses.
- Yep gives TripAdvisor results for movie theaters, Fandango, ZipRecruiter results for film jobs, film studios, and some more theaters (and also an article about rent control for some reason).
>Right now I'm in Los Angeles and looking to buy a roll of film. When I search for "where to buy film in los angeles":
I remember I was in school they'd teach you how to formulate search queries. You wouldn't ask questions like "how to bake a cookie", you'd use keywords like "cookie recipe". Search engines have progressed much since those days, such that you can ask questions and it would mostly know what you were looking for, but obviously a less sophisticated search engine can't do that?
I’d encourage all those who believe this is just a keyword issue to try to find a search on Yep that produces film stores in LA. I just tried like 10 incantations to no avail
“Film store in los angeles” gives me DVD stores on yellowpages.com as the 3rd result…
To be honest I'm not even quite sure what you mean when you say "film stores". It could mean stores that sell photographic film, or stores that sell movies. Presumably you mean the former, but I certainly wouldn't call stores that sell film as "film stores". They'd most likely be photography stores or maybe at big box/department/pharmacies. In that respect, asking for "film stores" is like asking for "bacon stores".
I am not from the USA but I was going to respond to say I don't know if "film stores" have existed for decades, so searching for "photography store" or "camera store" would be far more appropriate... but sadly Yep fails at "photography store in los angeles" and for "camera store in los angeles" only gave one good result :(
Both pulled up full pages of results wkth just websites of stores that sell film.
Your search-fu is atrophied, if it ever was any good.
Non-ML heavy search engines search for identical text on the page. So if you want a store that sells film, you need a keyword combination that would appear on their website; not a vague question or request to a mystical search AI.
That's searching for specific types of shops, which is different from looking for film. (Your second search is also meh, including truly only 1 relevant result)
> if you want a store that sells film, you need a keyword combination that would appear on their website; not a vague question
Do you suspect "camera stores" in Los Angeles include neither the words "film" nor "Los Angeles" on their sites?
All my keywords are found in every first page Google result ("
Top 10 Best Film Camera Store in Los Angeles").
First the SEO industry owned google for these question searches, and then Google realized that just making them work was easier than trying to change human behavior.
Even duckduckgo does not seem to be able to do local search. Especially outside the US. Only Google seems to be able to to do it for some reason.
But in general terms both Google and Duckduckgo seem to be broken and they give almost the same results for most other queries. Which has become universally bad in a lot of cases. With either SEO spammed sites on top or just the wrong results shown, with the search engine ignoring words in the query. Hopefully these alternatives find a solution to that so that atleast non-local search improves.
The search space is really broken. So as long as a search engine can actually search even if it cannot do local to begin with, would be a great start.
Google is a large, horizontal platform. My intuition is the next Google killers will be specific to verticals, health comes up on HN a lot. All that to say, if I were yep I would focus on search that avoids big, corporate, SEO stuff and connects users with thoughtful, small creators. I would not focus on commerce eg film, as it’s a crowded vertical full of Google, Shopify, Etsy, and other large e-commerce players.
Yeah, my goal is to expand to add more verticals in the same rough neighborhood of exploration and discovery.
At least in the intermediate term, I don't think it's possible to create an "everything"-offering like Google's. I also don't know if that is desirable either, at least not if the goal is to encourage a sense of agency on the Internet.
> Yep gives TripAdvisor results for movie theaters, Fandango, ZipRecruiter results for film jobs, film studios, and some more theaters (and also an article about rent control for some reason).
So, the search engine from an SEO company is actually showing results from industries that are highly SEO optimized. (Movies, Jobs, Travel are all huge SEO consumers).
We start from the hard problem - search itself. When we are happy with a search results we will add ads and revenue share. There is no revenue to share at the moment.
This Yep thing isn’t a search engine. It’s a content-marketing campaign for Ahrefs. They don’t want to compete with Google. They want press for themselves.
That shows you have a distinct lack of understanding of what Ahref's core business model is built upon. It's not a PR stunt and they don't want to directly compete with Google either.
They want it for search data. They don't need to be anything near Google's size to make it worthwhile.
I'd like to throw mine into the mix (we're a non-profit that plans to donate 80% of our profits to other non-profit organizations and charities, but we need more traffic before Google will enable ads, so at the moment then we're ad-free and not generating any income to donate):
Better would be to show no/less ads and encourage users to donate to charities directly. Helping commercial interests exploit users so that you can donate some of the profit from that is really inefficient.
Sounds nice, I'm all for more search engine competition even regardless of the revenue share model. I'm curious how exactly they pay creators though? If I had a super-popular blog that got millions of hits a month through search, does that mean they'll send me a check or something? Also, what about sites like YouTube that mainly host content created freely by others?
This is how we see it. Your content is helpful to people, your content helps search engine to provide relevant results, relevant results help search engine to keep user base and get revenue from ads. Search engines could be supporting people creating helpful content. But they are not.
Google does it in Youtube. So if you create good video you can make living from it, but if you write a good blog you can't live from it. Unless you put affiliate links, ads or maybe paywall.
If HN existed when Google was founded, I'm sure we would have torn Larry and Sergey apart too.
People rarely comment online when they like something. People with opposing or differing opinions tend to be over represented since the same people are more likely to be vocal about their feelings.
I am pretty sure majority of the HN crowd was largely supportive of Google when it was founded.
But it was a different Google back then: pure no-ads search results (and remember, this was the time of banner ads and paid search placement). Heck, I wore a Google t-shirt I got from them proudly, and recommended all my non-techie friends and family switch to it from Yahoo and Altavista and whatnot.
It was only a few years down the line that, relying on word of mouth recommendations from geeks up to that point, they started considering ads as the business model.
Maybe redundant, but how do you actually join? On their web site I do not see any partner etc. page, so how do they actually share that profit?
I have so many questions:
- how do they share profit? by number of visits? then how to avoid fake sites and those that copycat other ppl content?
...
Why are all Google Search alternatives founded on gimmicks? Where is the startup with a straightforward pitch like "better results than Google, $5/mo"?
The closest I have seen to what you're describing is https://kagi.com, which seems to have just gone to open beta. A few commenters have mentioned it in this thread.
There was a time when for every new search engine that came out after about my first five searches, I was like:
Nah, that isn't gonna cut it.
Then DuckDuckGo came along and I immediately felt that it was going to be good enough for me - and it has been for many years and still is.
Now with Kagi and Yep it is the first time after DuckDuckGo that I feel the same. Playing around with both of them for a while, I think they'd be both good enough[1] for my purposes.
I'm happy with DuckDuckGo and it has almost[2] completely replaced Google for me, but I'm also relieved that now we seem to have viable alternatives for the first time now. I hope more are coming.
[1] When it comes to the quality of the results, both are on par in my opinion. What differentiates Yep from its competitors is that it is based on an independent index, which I think makes it interesting from a technological standpoint. I won't comment on their respective business models, because I have not thought much about that...
The expected result is Ahrefs about page but Yep is not even close and I agree with some HN comments Google results are worse today because of tools like Ahrefs, not sure I would trust their search engine. I would rather stay with kagi.com/google.com
yep.com is remarkably similar to fairsearch.com, another search engine built by ahrefs which advertises a revenue share. In fact, “Read the story behind Fairsearch” and “Story behind Yep" on their respective home pages both link to the same blog post. Also, both websites have identical 'bot' pages (https://fairsearch.com/fairbot and https://yep.com/yepbot).
Can we maybe not have daolf's editorial/opinion as the title? The page title is just "Yep", which granted, says nothing.. but so does linking to a search homepage. Maybe the last blog post is a better url/title [Investor money vs. public interest: did Google fail to build a non-evil platform?](https://medium.com/swlh/investor-money-vs-public-interest-di...)
The business model is not entirely unrelated to the quality of the search results. Anything ad-funded will always have incentives not to get you to the best non-paid result as fast as possible. And once the ad-funded search gets big enough they are likely also going to be doing the middle man part of the ads operation and selling that service to others - which then provides further incentives to not negatively rank ad-heavy sites.
Our system is big part custom OCaml code and also employs third-party technologies - Debian, ELK, Puppet, and anything else that will solve the task at hand. In this role, be prepared to deal with 25 petabytes storage cluster, 2,000 bare-metal servers, experimental large-scale deployments and all kinds of software bugs and hardware deviations on a daily basis.
Company is based in Singapore, but where are the servers? No way you're hosting 2k servers there, the costs are just too high.
In the DevOps/OnCall role you have to be prepared to deal with "all kinds of software bugs and hardware deviations on a daily basis" which doesn't sound like a barrel of laughs.
Wait, what? Ahrefs should know better than anyone that this is insane. Imagine explaining to the current SEO industry that Google would now be paying them based on how good they were at juicing the algo. Higher rankings, more clicks, *moar money*. It's bad enough already, this would be blood in the water for organic content.
This is early 2000s thinking where you believe people will read things. Audio, video, and images are what the masses consume in bulk, and to make money in advertising you need to use one of those formats.
You'd make far more converting your content to a podcast or YouTube channel and using affiliate product placement.
Anything else becomes a cat and mouse game with ad blockers.
The "Why is this good for search?" paragraph here https://yep.com/profit-share shows how a world could be better without Google being greedy. It just makes me sicking thinking of how cancerous Google is right now to the Internet.
Jesus has no one in this thread ever build a legitimate product online and tried to get noticed by legitimate customers that will get value from the tech you build?
SEO is something you just have to do, can't get around it for some products. Some of it you can think of like how you have to follow bullshit conventions to make a resume easy to understand for recruiters. And others is just plain getting your name out there.
The competition around black hat SEO and spam content is disgusting but what can you do, just gotta try to get more links. Here my analogy would be how we all complain about inflation and housing prices but then when you wanna move to SF or Seattle you will still participate in the madness and bit 100k over asking price.
Bummer it does not have image search. Still have to use Google because its image search - specially the file type filter, and letting search for things like SVGs.
Competing heads-on with Google makes no sense; They already nailed the product on desktop and they are decades ahead in R&D.
At most you can attack it from a niche like voice-activated search, video search, Smart TV searches or car-searches.
The real opportunity to disrupt them was mobile search but they also saw that and went aggressively after it. The next big mainstream opportunity probably is when brain-to-computer links such as neuralink become functional.
Someone once submitted a link here that used scribe.rip and I've had good luck replacing any medium-ish url (they're easy to spot via the "@" or the trailing hexadecimal hash in the URL) with scribe.rip: https://scribe.rip/swlh/investor-money-vs-public-interest-di...
I wonder if they are including in that the money they already invested in the Ahrefs SEO tools. It seems to me that their SEO tools were already crawling a lot of content so this may be an extension of that / a new way to monetize the data. I personally know a lot of people who pay for their SEO tools so I imagine they have a decent amount of revenue.
There is a lot of intersection between Ahrefs tools and search. We were collecting web data for Ahrefs tools for 12 years, now we repurpose these data for Yep.
downside: doesn't seem to care much for boolean ("-keyword") search terms, instead opting for the modern thing of "let's actually search for that keyword, too!" v______v
This raises an interesting question. Presumably you are referring to search results on a browser where JS/Ecmascript is disabled.
Another interesting place where Ecmascript comes into play is the pages that have been scraped. One difference between search engines is how they handle SPAs, and 100% Ecmascript rendered pages from frameworks like React.
I wonder if they execute JS in order to build good search results from pages that are React style SPAs.
I don't want a search engine that pays "creators" (a search engine finds things, if they want to monetize it it is after it is found). I want a search engine that runs exactly as many ads as are needed to pay for itself, nothing more. Preferably with an option to pay for myself as an option. I'm guessing that means https://kagi.com/ but I would like to see some more competition.