Search engines are very interesting, technologically. Optimizing a search engine would be extremely interesting. Optimizing websites to keep up with the changes of somebody else's business (the search engine) is not interesting.

You're playing catch-up. You essentially attempting to reverse engineer interesting problems, and applying methods to take advantage of the internals of those algorithms for your customers.

In 1999 you would have been wildly successful telling people they could put a ton of keywords on their website, using the same font color as their site background color to boost ranking without affecting user-visible content. That worked, for a while. Then it didn't. The technology to "optimize" content for search engines have changed, but the concepts are still the same. You're trying to game the system, and I personally don't find that interesting nor rewarding.

We're working to figure out what works and provide software to message that and help businesses optimize. It's not just for search engines (we try to help with, for example, how/why people retweet, link, share content, "like" on FB, etc), but the point is a reasonable one; optimizing against existing systems vs. creating something new may be part of the issue.

Thanks for sharing, Brian.

... and how to get a popular item on news.yc :) I guess it just goes to prove your point that you can can promote your brand without being spamy.

Use the Betty Ford Center, a residential drug and alcohol treatment center, as an example. You want to optimize for "private" and "recovery" rather than dilute your site with keywords like "clinic" which don't serve your target audience.* _This_ is what effective SEO is about.

It's only "gaming the system" in the same way that traditional advertising is--you want to do your best to get your message in front of the right eyes.

* "Clinic" is often associated with "cheap" or "free" by customers who are a better fit for the Betty Ford Center.

SEO=link farming, spamming blogs, generated blogs/sites, etc...

SEO=building accessible standards compliant markup, making it easy for people to find content they were actually looking for, etc...

My gut reaction to hearing "SEO Company" makes me think of #1, even though I know "good" SEO is really just good page development, good content, good use of keywords, etc... As an engineer I'd prefer to hear about standards compliant markup, accessible (508/WAI), canonical URLs, clear DOM structure, and developing content to better serve users: which is really what it's all about.

Catering to the user and offering content / services that people want should be the #1 priority, but if Google doesn't index your site properly then the people who want it will never see it.

Yes, it's a bit of evil mixed in with good - what marketing isn't, but but it can be extremely effective, especially if your business depends heavily on search engine ranking.

Convince the client you can bring them converted, spending customers at a CPA they'll like, without tarnishing their overall reputation, and you've got a winner.

I think it is a matter of continuing to demonstrate very basic things such as "SEO works", "SEO will make your business money", and "SEO is not black magic voodoo practiced by a bunch of ebook selling charlatans who will teaching you to Make Money Online".

If I were trying to get a bunch of savvy startups on board with SEO, I'd be banging the drum on how SEO is an absurdly effective force multiplier for startups, small businesses, and other resource-constrained entities who have agility, deep technical knowledge, personality, a story to tell, and all the other unfair advantages that warm me to my blackened SEO heart. If you're doing business on the Internet, you're almost certainly critically dependent on SEO these days. (Some businesses more critically than others: I could imagine B2B with horrifically long sales cycles that get very little accomplished online not worrying about SEO too much, and Facebook/iPhone apps get a pass in today's market. But for selling B2C or B2SmallBusiness web applications? Crikey, SEO is about as important as issues get.)

Oh, and educating people about what SEOs actually do for a living helps. (I mentioned at the time, too, but I really liked the presentation to YC about it.) Except, don't mention the roasted baby parties. I don't think they're ready for roasted baby. We'll start with kitty milkshakes and work our way up gradually.

The best developers take their head out of the code and realize that it's not a true product meritocracy-- in Google search results, in the App Store, or in any marketplace you care to name. AND they realize that really freakin' awesome things happen when you combine a great product with great marketing.

SEO matters, even in the complex business to business sale, <em>but only if your company and your offering is remarkable</em>. (In the Seth Godin sense - worthy of remark.) Look at how much SEO a company like salesforce.com does.

Fwiw I would still choose to be a hired assassin vs being an SEO consultant. ;-)

I would rather starve than work for companies like SEOMoz. If I am a typical good engineer (and I think I am a good engineer , not a great one - yet ;-)), then I guess that confirms the poster's anecdote of good engineers not choosing to work for SEO firms. Really why should they?

My focus is Machine Learning so if I wanted to work for someone else on interesting problems in ML, I'd work for NASA/Google/NSA whoever - NSA being as "slimy" as I'd like to get. I wouldn't choose to work for SEO companies/Spam Companies/Porn companies /criminal enterprises etc if they had problems ten times as interesting/sexy (and I am sure at least some of them do).

Why should a good engineer choose to work for a company in a shady industry, given she has any choice in the matter?

If you are running your own web app startup, knowing when and how (much) SEO works (and when and how it doesn't work) on the other hand, could be very useful (as patio11 correctly points out), depending on whether you think it is the best way to spend your time.

If I were running a startup (I am not, presently) and if Search Engine based marketing were an important part of my marketing strategy, sure I'd spend some time on it (again patio11 has written some incredible posts on how to do this, thanks much!). But I'd still see it as a necessary evil rather than something intrinsically good, and I'd never work on this stuff for other people.

Getting muddy on the way to some place important is one thing, making a career of jumping into slime pits for random strangers every day is another.

Sure, asshats can use tools for evil. People get killed by kitchen knives all the time-- but that doesn't make the tool intrinsically evil.

SEO is just one more tool in you toolbox. Easy to learn, hard to master, but massively effective.

And yes, there are lots of (very vocal) idiots out there. But this type of people is in every industry. So I don't get it why so many people see them as "the industry".

Wait, I wonder if the spammers think of themselves as good guys too...

But perhaps that message has been drowned out by the ongoing perception from those early years of SEO?

BTW - this might be good material to explain my point above - http://ycombinator.posterous.com/the-first-yc-conference

But this raises a valid point that we should address. If there are, as Matt Cutts points out, more than 100 attributes that can affect your search engine rankings, and you are purposefully attempting to manipulate a decent subset of them, then you have likely fallen from grace in that you are no longer just using 'creative marketing, accessibility and best practices'.

The SEO industry suffers because bad and/or over-zealous/self-promoting SEO gains more public attention than a strategy that is well thought-out, considerate and user-centric. Every time someone does a search and the results returned bring up sites a visitor positively does not want to see -- SEO foots the blame more often than not.

I'd suggest the SEO practitioners be seen branching-out and offer services in areas SEO overlaps with others: accessibility, user interaction design, clean copywriting, site monitoring, site metrics analysis, web development best practices, hands-on web site quality improvements, and web user psychology. Even though a lot of this is now fairly typical SEO work, I'd still suggest making it more obvious about the wide range of skills a top-notch SEO practitioner has to offer.

I've come to understand that SEO is no longer just Search Engine Optimisation, but a practice that when done ethically and considerately connects people with the information or service that best meets their requirements. And that involves quite a lot more than convincing a search engine to rank certain key pages for a certain set of keywords.

The common web developer perception of an SEO person is grim. SEO is seen as redundant to web best practice. For a while I considered SEO's main strength - this I gleaned from Danny Sullivan - as the goto person when a web developer fails to do his job properly. I think that's an over-simplification that doesn't help SEO, and the practice of SEO.

I also feel that the top-notch SEO people are too nice. Although the recognise there are elements within the SEO industry that give them all a bad name, they are reluctant to be direct and brutally honest when dealing with bad sections of their industry. In my view, I don't think the SEO industry does enough to throw it's bad people under buses. Bad practices and bad techniques are discussed, but organisations and groupings that are harming the perception of SEO aren't directly criticised. I'd suggest there'd be a cheer breaking out on the web if the SEO industry did a visible clean-up and called out bad practitioners directly (or fisking).

I feel in this regard SEO has earned a low reputation like the Internet Marketing industry. There are some people doing useful and constructive work, but a much more dominant/visible/significant portion just peddling yesterday's snakeoil.

I've been immensely lucky over the past three years to work with some wonderfully talented SEO people in the UK. I find it very interesting to compare and contrast deep insightful approaches to SEO with modern web development techniques and accessibility best practices.

It's not the techniques that have impressed me about professional SEO, but the considered, logical and insightful reasoning behind tailoring pages in certain ways for search engines.

I think that's the sign of a top notch SEO professional. Not just knowing techniques and tricks, but having the consideration of picking the right blend of methods for genuinely improving the site they are working on, their appearance in search engine listings, and bringing the right kind of people to their websites.

I think that's what's so easily missed by people disdaining SEO: SEO is about bringing the right type of people to the right type of sites from search engines. It's finding that balance where a site is listed appropriately to it's real quality to visitors.

A fair number of times SEO is about improving the quality of content on the existing site to justify it's generous rankings -- but we don't hear much of that.

I've watched how the self-identified Black Hat SEO crowd works over the past two years, and frankly I'm surprised how inaccurately they've been portrayed. Ask the typical internet-savvy person about the darker side of SEO and they talk about the dirty tricks of old - cloaked redirects, keyword stuffing, spam. And yet, Black Hat SEO is far more refined than that. It's not the choice of techniques that identifies someone as Black Hat, it's the rationale behind their choices.

To geekily draw a parallel with Star Wars, the Sith have a marked advantage over the Jedi in that they have the choice between using the light and dark sides of the force. The Jedi almost always have no choice but the light side (cue eternal flamewar about Vaapad...)

The strength of Black hat SEO is their ability to quickly and effectively automate techniques that work on increasing their rankings and search engine visibility. So the common routine for these people is to find a technique that improves ranking, and automate the hell out of it, and repeat over and over and over. (Also note, most Black Hat SEO people also keep a squeaky clean White Hat part independently of their darker parts - that's their solid financial base).

What happens is that Black Hat SEO guys right at the fine edge are highly skilled and motivated hacker/programmers - some are exceptional developers. They love solving hard technical problems, that are rewarded in a significant jump in income because of improved rankings. They love the challenge, and it drives them. The perception that Black Hat SEO is just lazy script-kiddies is misleading. I'd suggest that the leading Black Hat SEOers could code the pants off many people, including a lot of talented developers even here. And when Google finds and figures out how to bury their sites they just start again from fresh. They are determined, motivated and self-managed. And they love the challenge of it.

I have a love-hate relationship with SEO. Some stuff makes my blood boil, but every now and again there are moments when SEO practices add loads of value beyond self-promotion. We need to optimise the latter.

(Sorry for the long comment, this is something I've had considerably thought about over the past three or so years)

I'm writing this from a throwaway HN account. I work for a 4-year old company, where I was one of the first 3 employees. We have a very popular website that gets over 400 million page views per month. We get lots of traffic from Google. When I started, our priority was making a great, simple user interface. But we were also very aware of SEO, and paid much attention to self-links we put on the site, anchor text, URL parameters, all the usual smart SEO stuff. So far so good -- we did this opportunistically, and never at the expense of the user experience.

As we grew, we hired a person dedicated to SEO (a non-programmer). Then we hired another. These SEO people started having us add links and pages in strange places that made no sense to actual users. And adding links that sometimes made sense to users, but were unnecessary, and cluttered the user interface. The page footer grew and grew, eventually spanning 3 lines. We added funny redirect schemes that made the site slower for users. We were afraid of our links to other sites because we might leak valuable "Google juice". We added redundant tooltips that were useless to users. Many of these tactics were crap our SEO people read on some webmaster SEO forum, with no scientific basis. Sometimes our SEO people even had the gall to say "I have an idea I think will be better for the user experience", and go on to propose something that only benefited SEO, and made the user experience worse! In other words, they would focus on GETTING the user (via Google), but forgot about KEEPING the user (through good user experience). Thankfully, we never did the black-hat methods like cloaking, but some things very close were proposed and met with loud opposition from our developers.

For a long time I would fight against these changes, because they made the UI worse. I lost most of these battles because it was hard to convince the managers that it was hurting the site (but it never seemed to be necessary for the SEO people to prove it was actually helping SEO). I was eventually spending so much energy fighting these SEO proposals that I just gave up. User interface designers and developers spend a lot of effort to make a web site look good and run fast. Then the SEO people go and fuck it up. It's very frustrating for developers. I know SEO is important. All I'm saying is that SEO should never be at the expense of the user experience. I hate what it has done to our web site, and I know it's happened to other websites as well.

What I don't understand is how you would apply A/B testing to SEO. There's only one Google; you can't show half of Google one version and half another Google. How do you get data on which version is better for search engines without being able to compare them while holding other factors constant?

You can try to simulate a 'test' with two equal throwaway sites, but this will never give you the same insight as using it on a big site with real content and real visitors. But it can help you find problems in linking structures, tweak the (really small) benefits of using one or another html tag.

The success of SEO can't be isolated. But, increased crawling of the page and disproportionate visitor growth from Google are good signs.

Say that you're Yelp, and you make a change to the Boston section of your website. Unbeknownst to you, a competitor has sprung up for local Bostonian reviews, and is rapidly gaining market share. You see your Boston pages drop in rankings, but it may not be because of the change you made. It may be because a bunch of Google's other signals have picked up that this other competitor is suddenly much more popular, and are adjusting ranking accordingly.

Think about what the job of a SEO is: Make Google like the site more. How does it work? Make a better site, that gets more links. Okay, the "get more links" part is possible without any technical knowledge. But if the site is crap, this won't help. So how could anybody make a html site better without being a web programmer? Yeah, right....

Your ad could hold my attention longer by dropping the term "SEO", but I'd still decide you were probably just a bunch of spammers as soon as I saw the words "improve their rankings in search engines". I don't want sites to come up in search engines because someone paid a lot of money to put them there; I want sites to come up in search engines because they are a good match for what I'm trying to find. If you're gaming Google, you're making the Internet less useful, and there's no way I could work for you in good conscience.

Now, maybe what you're actually doing isn't so much gaming Google as it is teaching your customers how to build good web sites. Maybe you're teaching them to write good headlines, use quality hyperlinks, add a lot of useful, interesting content, get content out from behind paywalls and flash blobs, and so on; or maybe you've come up with some fascinating statistical analysis that lets people know how searchable their content is. If so, that's great: but why drag yourselves down with the poisoned label "SEO"? It's like calling yourselves "SpamWorks", or "BotNetMasters", or advertising your skills at obfuscating Cialis ads.

Web marketing attracts derision because web marketers make the Internet suck. Maybe you're a step above the spammers of the world, but you're not helping your case by marketing yourselves under their label.

> [...] but you've chosen a label for yourselves that was invented to describe the bad guys.

For me, probably randfish and lots of other people, some of them working in this industry, the word SEO just doesn't mean this.

If that is true, then great: they're marketing themselves effectively. If it's not true, they might not want to use terminology so directly associated with the asshole spammers of the world.

SEO allows small/smart companies to compete against established behemoths in search engine results.

Your statement is true of ALL marketing and ALL sales. The truth is that marketing and sales efforts are a success multiplier to the quality of your product. Yes, that means that good companies can get beaten by crappy companies who out-execute them in the arena of sales and marketing.

You can ignore sales and marketing in the hopes that your product is SO much better than the competition that it will win anyway... But that's almost never the case (note the $ that Apple spends on marketing, bizdev, and branding).

Many developers are developers precisely because they didn't want to get into these zero-sum games. Think of the other professions where smart, logical, creative people can end up. Finance, law, and advertising. Most of these pay better than software engineering does. If we wanted to go into them, we would've. But it's precisely the constructive, innovative part of software development that appeals to us.

The question was "Why do developers not want to go into SEO?", not "Why do salespeople and marketers not want to get into SEO?" It's because the question is already self-selecting for the people that don't want to do that. If we liked gaming the system for personal gain, we'd work for Goldman Sachs.

If you leave low-hanging opportunity on the table because your software business won't lower itself to things like sales, marketing, SEO, PR, etc-- you're doing a disservice to your investors and your co-founders.

My biggest gripe as a developer is having to deal with the perceptions of non-technical folks. I still frequently encounter folks who hope SEO can magically redeem their lousy-to-mediocre website. I always encourage them to first build content worth finding before going nuts worrying about how many people will find it.

This is sort of like saying "good programming is mostly easy, common-sense techniques that don't change very often", and since programming is a solved problem, anybody willing to pay a lot of money for programmers is clearly getting something hinky done.

Programming is not a solved problem. Neither is marketing. SEO exists at the intersection of programming and marketing. Every site and every business brings their own unique challenges to the mix, and they're not all addressable with "Slap some alt tags on those images and you're good to go", any more than you can solve any programming problem with "Rewrite it... with more AJAX!"

SEO for many startups will involve, among other considerations, "How do we convince our users to create? How do we convince our users to share?" Those are core product/marketing questions for many startups, but they're equally core to the SEO strategy, and they're tricky to get right. For that sort of startup, you could imagine someone looking over their widget's viral activity with the amount of intensity that Zynga spends on its viral channels, because sharing via the widget creates several types of value for the company (via direct traffic, branding, and ranking effects).

A lot of the time, especially with small common issues, the answer is standard and simple and you could have just looked it up yourself. A lot of sites actually just need to pick some decent keywords to target and write titles for the page.

Sometimes, you want a professional to tell you that even though it is mostly just standard stuff because otherwise you don't know if you could be saving tax by doing something else.

Sometimes things are complicated and the difference between a creative (which usually also mean experienced) consultant helps.

BTW, that doesn't mean rely on professionals and don't learn it yourself. Learning a bit of law or accounting might also be a good idea. Certain types of creative solutions are only really going to come from you. Some solutions are best engineered in from the beginning. EG, you might want a lawyer on the founding team of a music sharing startup.

I think the main issue is that most developers/business owners have seen SEO companies charge enormous sums for adding very little value, or using dubious ethical means such as link farming, keyword stuffing, invisible keywords and so on. It's going to take a long time for all of that to wear off.

Bad SEO is easy, good SEO is hard. Just like anything else in life; nothing worth having comes easy.

He wrote "good SEO" meaning ethical or non-sleazy and "bad SEO (gaming the system)" as sleazy SEO.

Not your re-defining them as "good" as "effective" and "bad" as "incompetent".

Thankfully, I haven't worked at a company that hired a "bad" SEO firm, yet.

a) the underlying need for SEO is equivalent to the underlying need for taxes. Or apartment brokers. Or other hated but essential intermediaries. SEO is a necessary lubricant, but in a perfect world, it doesn't exist.

b) the context SEO operates in is derivative, not fundamental. SEO may need to deal with 50b+ indexes, but the context is not google. The context is parasitic.

c) the business model and ROI calculations of many SEO companies are self-serving, magical, non-scientific, and non-provable. I can't tell you how many conversations I've had with SEO people where I suggest doing an experiment, and all I hear are crickets and fear.

I think SEO needs a new acronym to differentiate between spammy abuse SEO and site structure/good practices. Maybe Search Engine Readability? Search Engine Compatibility?

It shouldn't exist. There's "good" and "bad" SEO. "Bad" SEO is snake oil, it's cheating, it's gaming the system for commercial purposes. It's a race to the bottom to see who can screw up their UX and the Internet in general the most to get the most clicks from Google. It's evil.

"Good" SEO, though, is nothing but common sense. Search engines emphasise good structure, semantics, accessibility and practises, and this is what "good" SEO takes advantage of. But good developers know if you're doing your job in the first place, there's no need for "good" SEO. That's why we don't like SEO; it's a solution to a problem that doesn't exist. If you can somehow stealthily introduce good practises in the guise of SEO, then I guess that's good, but we cringe at the thought of having to label it like that to get acceptance.

Seriously, I am interested in hearing about your machine learning openings, and I can't even find the job description anywhere.

In regards to openings: http://seattle.craigslist.org/see/sof/1757631095.html is what's currently up.

The other common problem in SEO is ineptitude. Many people call themselves SEOs when really they're just opportunistic freelancers and entrepreneurs. Their SEO knowledge is poor and their business management skills lacking. This ultimately impacts EVERYONE connected with the job, even those only remotely connected (like the good SEOs out there).

If the SEO moniker is really really hurting your prospects, consider creating a shell company such as Initech (generic and tech related) when posting and wait till they're warmed up to the technology before you tell them it will be for SEO. In all honesty though i think you would be better off finding someone not only excited about the sexy problems, but someone who is interested in SEO...maybe harder and longer but would pay off in the long run...if you figure out a good answer to your own problem, then let us know!

At Pear Analytics, we haven't experienced that at all. In fact, most developers will tell you that they no nothing about SEO anyway, so I'm not sure where the bias would come from. Engineers are attracted to solving problems they've never solved before. We've had plenty of issues dealing with external API's, Google's ridiculousness, queuing systems, data storage and more - yet they continue to find ways to get around these issues.

You guys seem to be located where there is a wealth of talent, and with your success - I am surprised finding the engineers you need is difficult.

I would say keep the focus less on the SEO, and more on the solution/problem you are trying to solve, and making it sexy to the end user. Maybe that will get some attention.

Ryan Kelly

Focus your headline on the sexy work you're recruiting the engineers for, not the market your customers are in. Why mention SEO at all in your recruiting?

Oo turn it on it's head and address the fear in your recruiting efforts. Like, "how can a gig as awesome as this possibly be found in the slimy, shady underworld of search engine optimization? mwahaha."

Maybe I'm biased because I'm an SEOmoz client, but I'd imagine that once they get past the initial hurdle that your firm doesn't fit a preconception about SEO anyway. So get em over that initial bump.

One of your comments states that you want to make "SEO" a brand people respect. I don't think this is going to happen in the short term, when the experienced engineers you want to hire remember cursing the unethical behavior of other SEO practitioners.

As for recruitment, if your SEO solution is genuinely an arms race of outmanouvering search engines, as an engineer myself I'd have passed over the opportunity. Personally I relish solving problems but not reinventing the wheel or gaming search engines.

Part of it, as it seems to me, is that I simply don't understand what SEO research involves, and assume it means doing bad/lame stuff. adding transparency to the job/process would probably make it sexier, assuming that there's some real substance underneath...

Also - if you're interested and are in Seattle (or could be talked into moving here) please drop me a line - rand@seomoz.org

You are probably fighting an uphill battle. Last year I had to do a technical integration with another company and it turned out they were actively paying something like $30k per month to an SEO company to keep them in the top five results on the big three search engines. All of that money was going to black hat methods such as link farms, paid links, cloaking etc. These tactics were all working and for a very long time their sites were the number one and two links for dozens of relevant terms with the big search engines. As those tactics worked, my employer very much wanted us to start employing them, ignoring the objections of the design and engineering team. About six months later we were heavily involved in a project for another company that was supposedly run by an SEO expert. His expertise turned out to be having read a three year old E-book on the topic. While I've provided two anecdotal examples, from everyone I've talked to these types of examples seem to be the most prevalent. Or maybe people only talk about their negative SEO experiences. It's not even cut and dry for those companies that appear to be doing things right. Look at some of the backlash on reddit when people figured out that oatmeal used to work at seomoz.

I know you guys are well established, and I've read some of your stuff with interest, but that name is terrible.

You should try setting up a separate entity called "Search Analytics & Research" or something that does the actual engineering, and try A/B testing job ads. I bet you'd get a hugely different response.

Work alongside a good marketing team however, and it's night and day. I think that they have in many cases more interesting problems, especially in the context of growing a startup.

How can you tell a good marketing department? They're smart, able and willing to do math, are willing to be data and metrics driven and can't comprehend it could work any other way. If someone says to you "math has no place in marketing", you don't want to work with that marketing department.

Scientifically-minded marketing seems pretty rare. Or to put it another way: Quite often there are some scientifically-minded marketeers in the team, but no one tends to listen to them.

Good technology marketers generally are a) ex-engineers (this is a spay-shul breed of person), often those who went out in the field and sold when push came to shove or b) people that sincerely like technology and have lots of field experience.

A bad marketer says "Everything is a widget (or cpg.)" A good marketer says "Every buying process in every vertical is different and highly sensitive."

Generally, marketers arrive as communication-oriented (PR/Marcomm experience), technology-oriented (PMM/PM experience), or sales-oriented (former sales person, cares more about qualified leads than raw names and phone numbers.)

Read Steve Blank's book - most people point to him as one of the best marketers in Silicon Valley - or read about his adventures as head of the SuperMac marketing department - http://steveblank.com/category/supermac/ - That's what a GREAT marketer does.

But, yes, good tech marketers are rarer than hen's teeth. Most people in marketing are there because they lack the logical abilities to be an engineer and the courage to be a salesperson. (This is coming from a marketer.)

Generally, really good marketers know the hell out of their market, and know what metrics they're working towards. They should know every influencer in their field by name and viewpoint, if not personally. If anything, they really need to think of themselves as 'Market Response Engineers,' trying to find out what the market will do in response to the messages they communicate.

Additionally, avoid people who work/have worked at ad agencies. They know how to drink, not how to market things.