SEO (search engine optimisation) refers to the process of ‘optimising’ your website for search engines, i.e. to appear highly in the rankings for particular relevant phrases. It’s a massive industry for obvious reasons.
I’ve done a fair bit of research on this and I’ve come to the conclusion that nobody has a fucking clue. It is what the great Richard Feynmen would have referred to as ‘Cargo Cult Science’. The cargo cults are hilarious and bizarre; they popped up around WWII when Allied forces used remote islands as military bases (mostly US forces in the pacific). The indigenous population had never seen things like aeroplanes and believed that there was some paranormal force going on by which the planes brought supplies, and that the troops summoned the forces by engaging in flashy military exercises, like parades and marching. After the troops left, the cargo cults tried to recreate the supply drops by staging their own parades and building fake airports complete with fake radios which they spoke into.
SEO is a fairly similar business.
Nobody involved seems to have done anything that would qualify as “an investigation” into what really affects search engine rankings, let alone anything approaching scientific rigour on the subject. Occasionally you find people say “I did this and it didn’t work”, and I applaud those people for trying to measure the effect of an action, but these people are rare! Most people don’t document failures and I’d be surprised if most of them actually have anything in place to tell the difference between success and failure. SEO is a cargo cult in the sense that they repeat a few mantras and they have no idea if they work, and they certainly don’t know why they work, all they know is that a while ago it was decided that they do work so they’ll repeat them now.
Mostly if you search for derogatory phrases about SEO, you’ll find a bunch of SEO-ers writing blog articles treating such critique as religious heresy, YET NEVER CONVINCINGLY REBUTTING IT.
I do sympathise with them though because Google is a massive pain. The problem is that Google got big by being a really awesome search engine. Since then the internet has not got smaller and Google has not kept up with the increased demands. Its ranking is very strange and often nonsensical (it ranked me on page 1 for ‘cleverbot’ recently, I got 25k impressions in one day. I have no idea why it felt me to be a leading authority on the basis that I wrote a paragraph about Cleverbot when it supposedly passed the Turing test) and it hasn’t really managed to continue giving good relevant results. It has identified a few big sites which are often useful, and it always ranks those very highly. But if the information you’re looking for is not on those sites, you’re out of luck. But Google is still the only game in town, and so there’s no real competition for it. It just has to be better than Bing, which nobody uses anyway. Monopolies are bad, even when it’s Google.