This is largely true, and you should read it. One caution: SEO is a little bit of marketing and a little bit of engineering. You know what both of these fields have in spades? Project risk. You can do competent execution of a good idea which "should" work, even an idea which does work simultaneously for someone else, and have it fizzle on you.

I am totally not saying this wearing my consultant who wants your money hat: you often will make repeated investments in getting this right, have no discernible success with some, and continue having to do so for forever as your business evolves. It isn't a matter of tweaking titles once and getting back to "real work."

I held the top spot for search terms relating to my service and site for more than a decade, until I shut it down. I never spent one second of my life worrying about SEO. I focused on usability, generating word of mouth (far more valuable - how do you think people found out about Facebook and Twitter?), and content.

The nature of a search engine is to derive the value of your content from the content. Therefore, focusing on your content is a far greater return in value than focusing on trying to draw the attention of a search engine to tags, meta data, or URL paths. Not saying that those things don't contribute, but I'm not certain that the ROI is worthwhile.

Think of it like a resume. If you have the talent and background to put on a resume, you'll get consideration. You don't need to spend hours upon hours refining it to add just the right trendy keywords or ordering things in the most appealing way to automated resume readers. On the other hand, if you have little to offer and you are concerned that you'll be weeded out quickly as a result, you can substitute your background and experience (content) with keywords and other tricks to try and get beyond the initial weeding out. Kind of a tactic of desperation and lack of confidence.

I do have a strong bias against the whole SEO thing. I won't pretend that I don't. I'm put off when I hear people focusing on it rather than content and it makes me think of greasy, scummy, snake-oil salesmen. Guys on late night infomercials trying to convince you that you can skip everything and get right to becoming rich and famous by thumbing through their special book and employing their special techniques for flipping property with only two hours of work per week.

This is probably a very unpopular opinion to share around a group like this, but what can I say?

We all have our biases, but not all biases are legitimate.

I'm biased as a developer who started a site that ranked top 3 for about 5K very competitive long tail phrases. So I've seen first hand the enormous benefit of allowing some time/effort/thought toward usability for bots.

It's not a given that Google will just "figure it out" and your reward will be in heaven. Anyone who thinks that is just hoping Google will do their job for them.

The upside for me is not having to work for the man thanks to selling the aforementioned site.

"It's not a given that Google will just "figure it out" and your reward will be in heaven. Anyone who thinks that is just hoping Google will do their job for them.'

i imagine in Google's perfect world there is little or no room for SEO. doesn't that mean it actually is Google's job?

I think the key phrase here is 'usability for bots'. If you recognize bots as an important customer (after all, they bring over most of your new traffic), then it makes sense to optimize things for them to find. That said, somewhere in between of not writing your whole site in Flash and creating a link farm, things start getting slimy and your arbitrary line will likely differ from, say, Google's.

I think the difference between the best SEO agencies and the majority is that the best continually run experiments to try and understand what Google is doing. This takes no little skill and a lot of time and effort.

The rest just read and regurgitate the results of those experiments when the experimenter decides to release those results (ie when it's not too much of a competitive advantage).

You mean like how stack overflow was having problems with Google not crawling their site, until they took a few minutes to set their site up to be Google bot friendly?

http://www.codinghorror.com/blog/2008/10/the-importance-of-s...

> generating word of mouth (far more valuable - how do you think people found out about Facebook and Twitter?), and content.

This is pretty much what SEO people mean by white hat SEO. Create interesting/valuable content that people will link to, and you will be rewarded in search engine rankings.

I was thinking about the same thing last week when I came up with my SEO strategy: Make something so great that people care about it.

And how do they know about it to care about it? Chicken/egg.

They know about it because other people care enough about it to talk about it. We're working on building that small critical mass through non-SEO means.

you've clearly never had any dealings with recruitment agents

Another reason to stay whitehat that Rand doesn't mention: it can screw your acquisition prospects.

Anyone in the business of buying SEO-driven sites (Demand Media, Internet Brands, Quinstreet, Specific Media, WhaleShark Media, etc.) looks VERY closely at the backlink profiles of acq candidates. They aren't going to do it unless they're clean, or if they sense there were blackhat links, they're going to discount the price massively.

I've watched several acquisitions go down the drain on account of perceived spam.

Having been part of one of the M&A teams mentioned, I'll second that. We used to talk a lot about "real" sites vs. those that were built around practices that made them rank quickly. I'll note that none of those teams are perfect (nor are any sites that I looked at), but black-hat tactics are a huge part of the deal analysis.

Totally agree with you for the most part, but there is one counterpoint: BeatThatQuote was bought by Google for $61million and had all sorts of blackhat stuff

http://www.theadmen.com/seo-info/beat-that-quote-google-hypo...

Loads of examples of companies that have apparently seen success with "white hat" SEO doesn't prove anything, without also showing what they have done to get that search ranking, and also showing that they haven't used "black hat" techniques.

Proof is a pretty high bar for any SEO assertion. You can (and should) apply the scientific method to SEO for years, and still never arrive at objective data because much like economics, experiments tend not to be reproducible to any great degree. The architects of the search engines are in an arms race against the black hats.

Which brings me to my next point: the gradient between black and white hat SEO is very wide and gray. The perception of hat color is largely dependent on the perceiver's opinion of the content in question. That's why Calacanis swears up and down Mahalo is not spam, and why no one here buys it.

Yeah, I wasn't complaining that conclusive evidence should have been included, more than, because of the lack of proof, a list of examples shouldn't be used to make the point at all.

It's a blog post, not a PhD thesis. If you read Rand's other stuff or watch his talks, you can find plenty of evidence.

What else is there though?

I've no idea, I have little/no experience with SEO. But someone showing a list of websites and saying "these did well with white hat SEO - see, it does work!" doesn't convince me to agree with either side of the argument.

Never going to happen. Welcome to a complex adaptive system that is SEO where you can never rewind history to pinpoint exactly what you did to get a result. At least not any more than an economist can tell you anything.

Rand applies the label "SEO" quite liberally, and I think it does everyone involved a disservice. If you look to see which sites linked to your last post and pinged them the next time you post to let them know there's new content, is that SEO? If you research the topics your target market are interested in and then address them, is that SEO? If you contact bloggers and media about your new product and encourage them to write about you and link to you, is that SEO?

Maybe you call it that. But I think it's about optimizing for searchers as much or more than it's about optimizing for search engines. Whatever it is, it's valuable, and "White hat SEO" (aka "marketing") has an important place in the weapons chest of startups.

How in the world is lib.nmsu.edu/rarecat a top search result? If I do the search and go to Google's cached version, it tells me "These terms only appear in links pointing to this page: buy propecia"

It's actually a really good collection of links for anyone interested in medieval manuscripts. Bookmarked.

Hacked sites suck, especially because the hackers can target so many pages at once. Please keep your web servers patched and up-to-date.

The page title shows up in the search as "buy propecia price - NMSU Library". My guess would be some sort of old HTML injection attack that's been cleaned out, but which hasn't fully flushed out of the index.

What's really odd is the two college sites below that, which don't have anything obvious about Propecia on them...

Probably had their CMS hacked or something. That sort of thing happens to a lot of sites, sadly. Google for "comment3" (make sure there's no space in there) and look through a couple pages of results and you'll find a lot of spam-filled sites. And that's after Google cleaned up a lot of it.

Maybe the page content is cloaked so that googlebot sees completely different text to human visitors.

this is most likely what is going on

Rand assumes that the top ranking sites today that produce great content got that way through white-hat techniques. I have no idea whether Zillow, Oyster, or Zappos never engaged in shady SEO practices and I doubt he does either.

Also, one minor addition...if your domain is naturally uninteresting, that should affect your competitors as well. One can argue that having an exciting domain that breeds lots of natural links is actually a bad thing for a startup because it's harder to rise over your entrenched competitors.

Rand does. He sells a web crawler service that does the same thing as Google's "link:" operator before Google closed it off.

Open site explorer is not a time machine. It's a snapshot of links TODAY. Your assertion is flatly false.

Google is a snapshot of links TODAY. Links that no longer almost certainly pass no value. Rand is wrong about a few of the domains on the list, as they have done shady SEO in the past. Also, OSE.org is not a fluid list - it only crawls a sliver of the web, and their index is not nearly as comprehensive as Google's. It was updated a week or so ago, and before that, had not been updated for a month and a half.

Was Google's link: operator caching old expired links?

I'm not a customer.

ps. Please be civil. See Site Guidelines.

The bit that confuses me is the list he included of what an SEO needs to do. It seems to me that as an SEO he's actually forgotten what his job is. There's a lot in that list that have absolutely nothing to do with SEO.

The clue is in the name SEO. Search Engine Optimization. Out of the entire list, only a third are SEO.

This is something I've seen SEO people and UX people do in so many blogs. They're not really sure what their job is so they pick a bunch of fun business stuff they'd actually like to be doing, that they have little or no experience in and call it SEO/UX. Then they can work on fun stuff which a business doesn't actually need, possibly already cover in house or should be paying a professional to do.

Actually SEO:

Keyword research + targeting - SEO

On-page optimization - SEO.

Making the site search-engine friendly - SEO.

XML Sitemaps - SEO.

Alternative search listings - SEO

Not SEO:

The business' overall product, marketing and sales strategy and where SEO makes the most sense. - This is marketing. Nothing to do with SEO at all. The last phrase does, but it's pretty obvious where. Hint, hint. On the website.

Funnel optimization - UX. While it might have an effect on SEO, it is not something an SEO should be touching.

Testing + optimizing content for users - UX. Nothing to do with SEO.

Content strategy - Marketing. Nothing to do with SEO.

Analytics - Not SEO. It can measure the effect of SEO, but isn't SEO.

Usability + user experience issues - UX. Definitely not SEO.

Reputation tracking + management - PR. Definitely not SEO.

Competitive research - Management consultancy, should be handled internally by small business at a director level. Definitely not SEO.

Social media marketing - PR/Marketing. Not SEO. Pretty big clue in the name there, with it ending with marketing.

Syndication, scraping, copyright and duplicate content issues - Legal. Not SEO. Unless it starts ranking higher than your sites.

I wouldn't mind having someone do the second list for me. But it's still not SEO.

You didn't mention where link building fits, but according to SEOMoz's big survey, it's 66% of what drives rankings.

I've never seen it successfully handed to Marketing or PR (well, except at Mint.com perhaps, but Noah Kagan is an SEO masquerading as a marketer), but I'm curious where you think it belongs?

Most of the items in your second list have a (potentially) very strong SEO component. For example, optimizing content for users involves making changes to the content that is the very basis of SEO marketing.

With the rather strict definition of SEO you seem to hold, and a very high-level definition of concepts, your above comments make sense. Yet, in practice, SEOers get involved in a larger number of areas than just figuring out how to optimise organic search, they get involved in the general role of getting more targetted visitors to the site, so pure technical search engine blackbox testing is one skill in their toolbox.

They get involved in the AdSense part of search engines, where changes in advertising wording affect the number of people clicking on that ad. A marketing department most likely wouldn't be able to come anywhere near an SEO who knows what audiences are searching for and the material on their site that matches that criteria. The flexibility and quick reaction times here, a decent SEO will outscore a "big campaign" focused marketing department.

So funnel optimisation then can be on the remit of an SEO - the funnel between the website, and the conversion goal (getting the right people from a search engine results page to the right piece of content. The route is an essential part of it - do people get there directly, or via an intermediate page - will marketers have sufficient technical knowledge and understanding of searching patterns to make a good decision?)

So in terms of funnel optimisation, of course, testing and optimising content that people see to increase conversion rates - it makes a great deal of sense to have an SEO involved.

Analytics - yes, it's the tool that measures the effect of SEO. But who is going to set that up accurately? Whose going to know what beacons/shims are needed where to be able to measure the effectiveness of their work? If anyone, SEO are quite the right group of people to ensure that beacons are in the right places, and remain in the right place, adjusting, correcting and improving as the site and it's traffic grows. I don't see a marketer, a web developer or an engineer having the inclination or dedication to keeping that running optimally.

Content strategy - do you really want product reference and howto guides in the hands of marketers? Seriously? Sure, for a brochureware site right before the signup, I guess marketers should be substantially involved here. But the support material? SEOers are well placed to connect the right support material with the people looking for it.

Reputation tracking - I'm not convinced this is entirely in the remit of PR - they are likely to ignore a handful of complaints in tweets in favour of the bigger picture. Best get a customer servicing person involved. And a couple of well thought out tweets and conversations from a knowledgeable SEO wouldn't go amiss either.

Competitive research - again, at such a high level fine, pay some consultant. At the "why is their website doing better than mine" an SEO will outdo a marketing consultant any day.

Social media marketing - again, you're focusing at a far to high level. PR/Marketing aren't the right people to tackle customer servicing issues that surface in social media conversations. And talking to the listening audience on Facebook and Twitter - the key is to do it with a human voice, not a PR/Marketing filtered voice.

Syndication - Legal? On a high-level approach on contracts/agreements between organisations for syndicating content (like AP news items), yes you should involve someone from legal. But tweaking your RSS feed?

duplicate content issues - I'd be very impressed if there was anyone working in a legal department on legal matters who could explain where there were duplicate content issues on their employer's site and recommend a good way of correcting / addressing that. My long experience of lawyers is that they believe deep linking is a breach of copyright.

You're focusing on a too high-level definition of these various aspects of running a website. An SEO is a very handy person to have around in the trenches. Whereas at the high level you are considering this, what PR, Marketing and Legal provide is close to useless on a day-to-day basis.

An SEO person is invaluable at the very granular level, the day-to-day issues that make a website work. PR, Marketing and Legal aren't good at working at such a fine-grained level.

This convinced me of the opposite actually. All his examples were something that people would love to blog about, except his hair medicine which he used as an example of something that is overrun by spam.

Get something to rank for Viagra or a penis pump using only white hat tactics, outline a good plan for doing so or conceded the point.

SEO (marketing) is critical to anybody out there who wants to find people.

Don't piss Google off. (Which means don't piss the searchers off)

I don't think there's much more to it than that. Maybe so, but from where I sit the problem is that "don't piss Google off" can mean pretty well damn much anything, so once you start picking it apart it's not logical or self-consistent. That probably means that continued analysis is a waste of time. After all, if you standardize how to market, you've destroyed the entire concept of marketing.

There are huge opposing forces at work here. I think the little guy does best by simply doing things that he would be happy to have made public -- and that's "public" as in normal people reading it, not "public" as in stuff that appeals to hackers. Different thing entirely.

It seemed like the point he was trying to make is that:

White Hat SEO, when defined as traffic generation and link building using forums, blogs, social, etc... "works."

Skip the text all the way to the last image. The point is made there.

If your niche was synchronous swimming albino eels, then you probably didn't need any SEO. IF your niche was Chicago real estate. You'd better get your SEO game on.

If people on HN care about cost effective ways to build traffic, there are few better options than SEO.

Have no money for marketing but have a great app, useful content, and a creative flair for promoting it? Then look at SEO.

And that starts by understanding the SEO means: a) how usable your site is for bots, and b) how you can attract links to it.

If SEO doesn't make sense for your business, ignore it the same way I ignore posts about Haskell and Lisp. I have nothing against them, that's just not my platform.

All sites rank "artificially", since search engines are incapable of measuring quality or relevance to a searcher's intent and instead rely on proxy signals like links and similarity to keywords on a page or in a link graph. Good SEO concentrates both on making sure that your content creation does not interact poorly with the 100%-guaranteed-to-be-artificial way that search engines perceive your website, and on strategic issues in shaping how you create and communicate value in such a way that it will actually be found by people. For example, if you routinely create long essays with metaphorical titles, you need to be told by your SEO guy that search engines don't do metaphors well and searchers don't typically look for them, so they need to be retitled to be more fact- or benefit-oriented. (This is a lesson that the traditional media learned, and it required absolute bloodletting in the newsrooms to get it across: "Studio Flutters As Swan Soars" used to be newspaper best practice, for web writing it is now "Strong Black Swan Opening Weekend Did Not Help Studio Stock Price.")

More broadly, this attitude is like the reflexive engineer distrust of marketing, believing that product quality excuses all sins. It doesn't. If your product sounds scary to a user, and she avoids trying it because of that, you fail to create value for her. You need to communicate that your product is not scary. If that requires a photo of a woman with a headset then it is your duty to your customer to put that same woman with a headset stock photo that everyone uses so that she feels comfortable enough to experience the life-changing benefits your product has to offer.

Not true, whitehat SEO is more about being seen and understood by a webcrawler. Honest SEO techniques include structuring your site in an appropriate way to allow optimal crawling, including appropriate content/meta data, making sure your http redirects are setup correctly. Stuff like that.