Jennifer Pahlka argues that it can, and that it can be done quickly and cheaply.

I have worked in the SEO industry for a long time.  When I started SEO professionally, instead of as part of a web design hobby, the focus from my bosses at the company that soon became Bigmouthmedia was to encourage potential clients that they needed high rankings. Consequently, it was then to create the circumstances that they achieved these high rankings.  Frankly, the job was relatively easy, looking back. While there might have been a high quantity of competing websites, there were few well optimised ones.

Back in those days (cue wistful music and history montage), ranking was pretty easy to do, and rankings were generally the same whereever you were in the World, so long as you were looking at the same index. A number 1 in AltaVista was a number 1 in AltaVista everywhere. So if you were a small business selling those famous red widgets, if you could get to number one for red widgets in all the main search engines, you could clean up!  Back in those days, a lot of people did.

But then a few things happened that changed all this.

Firstly, Google grew from being an “emerging” search engine to being dominant.  It did this by focusing on the user’s search experience, not on monetising the search. This put pressure on other search engines, and ultimately cleared the field of all its main rivals in the UK and USA except MSN and Yahoo. Importantly, Google ranked sites based in part on its PageRank technology which, while still relatively new, mapped recommendations (links, essentially) into the ranking algorithm as well as content relevance.

Secondly, as the user-base for the Internet matured, and as stories of online fraud, identity theft and suchlike became more commonplace, users began to trust unknown websites less and less, especially if the user experience wasn’t great. The analogy would be, why would you buy something you can’t see from a scabby shop miles from home, if you can buy the same product from a chain store you’ve bought from before. Online shoppers began to wise up, and seek out more trustworthy shopping experiences.

Thirdly, and most recently, Google in particular began to vary its searches depending on the environment of the searcher. Not quite in a minority report kind of way, although that’s on its way I imagine, but in the sense that it began to promote sites based on things like time of day, time of year, and proximity.  For example, you might get different search results searching for “hotel” in Edinburgh than you would get in London.

For a lot of SEO’s, this fact was kind of brushed under the carpet a little. Heads were buried in sand. The objective remained to achieve a top ranking for a keyword even if it might not always be top in all circumstances.

Worse, though, is that SEO clients fall into two groups – those that want to capitalise on your expertise (lovely clients, lots of sense, often not a lot of budget yet!) and those that tell you what their objective is and how to achieve it and to Hell with you contradictory best practice (far too many, usually far too much money to burn, far too keen to hang you for not achieving what you said wouldn’t happen in the first place!).

This second group of clients, while my parentheses might be a little harsh on many of them, typically demand deliverable outcomes in terms of ranking.  Often they are at the opposite end of the country to where I do business. Dealing with such clients is hard work. My objective with such clients historically is to keep trying to nudge their thinking to “increase sales/conversion” and build strategies that  fit that, but clearly some are less flexible than others.

These days, I have a simple strategy to deal with such inflexible demanding requests.

Put yourself in this demanding client’s shoes for a moment, and I’ll give my strategy away for free (if you like, send PayPal donations to me, or something, you’ll feel better for it!).

The strategy is: tell the client to +1 all their own Google listings across the board. Get people to whom they are connected to do so as well. Pretty soon, the client sees top listings for ALL their main keywords. Job done. Send me a cheque now, please. Here’s my number when you realise your conversions haven’t gone up at all…

Okay, I haven’t actually used that strategy on a client yet, but I’ve been sorely tempted a few times!

If you’d prefer to fall into the category of a client that listens to best practice advice, then here’s my alternative advice:

1. Stop focusing on rankings – largely they’re vanity. With rankings, you win some, you lose some. Sometimes it won’t look to you like you are winning, but you are, and sometimes vice versa. Adjust your content aiming to make it more relevant for keywords, but don’t waste time worrying about any given search result.
2. Focus instead on providing value. Give users a reason to want to visit your site.  Make them want to share it with their friends or social connections. Create great useful content. Focus your value around what users want. Use what users search for to help determine what they want.
3. Promote your content. Use social media. Use other marketing channels. Spread the word. Encourage others to link to you. Get the word out.
4. Measure. Benchmark. Experiment. Learn. Adjust. Repeat. Learn to understand how your website users behave, what marketing brings traffic and sales. Learn what brings value to you. Grow those things.
5. Have bricks and mortar locations? Use Google Places listings online, and use QR-codes, and LBS services offline. Make you stores and website are one seamless entity, not competing deadly enemies (seriously, I’ve seen some ecommerce sites struggle partly because instore staff refuse to advise customers to check them if the store is out-of-stock). Have a common goal across the business!
6. Build community. It is cheaper to retain customers than attract new ones. Ensure customers come back.  Furthermore, building a community helps build brand evangelisers who will help bring more traffic by word-of-mouth.

Maybe that doesn’t sound like SEO, but that is the beast it is today. Google ranks sites based on utility, proximity, social recommendation, relevance, and trust. Optimise these things. Build it, and they will come.

And SEOs – stop whining about “not provided.” As Google personalises more and more, I heard they’re going to change that phrase to “who gives a rat’s ass!” Well, they are brash Americans, after all.

There are a number of ways a staging site can cause you problems with SEO.

For one thing, the search engines might find and index your staging site.  Worse, it might actually rank better than your main site – especially if you are road testing SEO-focused technical upgrades on the staging site first.  Worse still, the search engines might decide to drop the main site from the index showing preference to the staging site.

This is especially a problem for an ecommerce site.  Even more so if (like they should be) the admin systems for your staging and main websites are distinct – if a user places an order on the staging site, this might not get fulfilled, which ultimately damages your online reputation in the long term, even if you repair the technical issues to prevent it happening in the future.

You might try to prevent the staging site being indexed using a robots exclusion file (robots.txt).  This is quite common, but it carries its own risks.  Many times I have seen robots exclusion files from staging sites uploaded to main websites in the past.  The result is what stopped a staging site being indexed suddenly stops a main site from being indexed.  And until the sites are dropping from their normal ranking, it can be quite difficult to notice – do you go and check your robots.txt file every day to make sure it hasn’t changed?

To overcome these risks, there are two things you should do.

Firstly, you should implement the canonical tag as standard in your code.  This has to be done correctly, as a poor implementation is also risky.  However, the benefit of doing this right is that even if search engines do begin to index your staging site, the URLs they list should be those on the main website, so searchers should no inadvertently end up on the staging site in the first place.

The canonical tag should always use the domain of the main website, and the extended URI of the page which is being viewed.

For example, on http://staging.domain.com/page10.htm, the canonical tag should be

<link rel=”canonical” href=”http://www.domain.com/page10.htm” />

The second thing that you should do is to password protect your staging site.  This has two benefits.  The primary benefit is that the site cannot be indexed by search engines, and so should not appear in search results (remember to send the correct “Unauthorised” 401 header response, so the search engines understand it is protected content). The secondary benefit is that competitors cannot get an early heads up on new features of functionality that are being tested on the staging site.

As an aside, have you been able to find and explore your competitors’ staging sites at all?  If they have left them open to public interrogation by not doing the above, it gives you a chance to keep up with what they may be planning.  Thanks to Sam Crocker for that tip at last year’s ProSEO!

By implementing the staging site in the manner described above, you are able to maintain the same robots exclusion (robots.txt) file on both your main website and your staging site (written from the point of view of your main website), preventing ranking catastrophes should it be inadvertently uploaded to the main site from staging at any point.

Bear in mind though, that if you are using .htaccess on Apache to password protect your staging site, then there is a risk you might upload the .htaccess file that causes the password requirement – this is much easier to spot than an inadvertent robots.txt upload though, as all users will be asked for a password.  Make sure you have a backup of your normal root .htaccess file if your site uses one, so you can restore it if you need to.

You might (rightly) point out that if you carry out the password protection, the canonical tag is unnecessary.  This is true, but in my experience few developers will password protect their staging servers.  However, whether they do or not – if the password protection is ever absent (for example, if an .htaccess file is deleted by mistake), the canonical tag provides back-up protection.

More than this though, so long as it is done correctly, in my opinion, use of the canonical tag should just form part of your best practice, anyhow.

As part of her presentation, she discussed how she has raised ad revenues by including third-party advertising on Yahoo’s log-in page.

My impression at the time was that if Yahoo thinks that distracting its users with adverts while they log-in is a good idea, then they are doomed.

Now Yahoo have sacked Carol Bartz – is that an indication that the penny may have dropped?

Infographic by WordStream Internet Marketing

On 15^th April, 1989, following a breakdown in command by South Yorkshire police, and a catastrophic chain of planning mistakes and poor stadium design, 96 Liverpool fans from around the UK were crushed to death by sheer weight of numbers, in a disaster similar to those at the Roskilde music festival in June, 2000 (http://en.wikipedia.org/wiki/Roskilde_Festival) and the Love Parade in Germany in July 2010 (http://en.wikipedia.org/wiki/Love_Parade_disaster).

Why bring this up now, 21 years after the event? Well, feeling still runs high among Liverpool fans that there is no sense of justice for the families of the dead.  Despite a general acceptance among most people in the UK that the disaster was the result of a collection of mistakes by those in charge of the safety of fans on the day (http://www.bbc.co.uk/liverpool/content/articles/2006/12/07/local_history_hillsborough_feature.shtml), there is still resentment that some quarters of the press, through sensationalist journalism at the time, reported that fans were responsible for the disaster – claims subsequently retracted by most of the offending press.

The official enquiry into the disaster exonerated fans from blame, and highlighted several failure in command by the police, and issues with the design of stadiums and the general treatment of football fans by organising authorities.  It is because of this report that all UK top league stadia are all-seater now, for example.

Therefore, to read reporters in section of the press in the United States describe the Hillsborough disaster as a “riot” is not only disrespectful and ignorant, it is also distressing. Barely a family on Merseyside didn’t lose a brother or sister or friend at Hillsborough.  Describing Hillsborough as a riot would be like describing Hurricane Katrina as a riot – not only patently untrue, but indescribably upsetting to the families of the victims.

Therefore, I would suggest that Alex Beam of the Boston Globe does a little research into the subject matter of his articles before writing such ill-informed nonsense in the future. If he has the sense to do so, maybe he will consider retracting some of the nonsense he has written at http://www.boston.com/sports/soccer/articles/2010/12/07/alex_beam_hardball_in_liverpool/ and perhaps even consider a donation to the families support group.

To read more about the history of the Hillsborough Disaster, search Google, or try these handy links:

http://en.wikipedia.org/wiki/Hillsborough_disaster

http://www.bbc.co.uk/liverpool/content/articles/2006/12/07/local_history_hillsborough_feature.shtml

http://www.youtube.com/watch?v=W5PCrbRl3DI

http://news.bbc.co.uk/onthisday/hi/dates/stories/april/15/newsid_2491000/2491195.stm

http://www.mirrorfootball.co.uk/archive/The-Hillsborough-disaster-The-darkest-day-in-British-football-history-article392289.html

http://www.epltalk.com/the-truth-about-the-hillsborough-disaster/491

http://www.telegraph.co.uk/sport/picturegalleries/5145204/Hillsborough-disaster-Two-decades-of-hurt.html

http://www.liverpoolfc.tv/history/hillsborough

UPDATE: A few hours after the Boston Globe Article was brought to my attention, Globe editors appended the following to their article:

“Correction: Because of a reporting error, Alex Beam’s column on Tuesday in the “g” section mischaracterized the 1989 Hillsborough stadium disaster involving Liverpool soccer fans as a “riot.” The official investigation into the disaster, which cost 96 lives, placed the blame primarily on poor crowd control and inadequate stadium design.”

However, Latent Dirichlet allocation (LDA) is a fairly advanced statistical concept, using complex probability maths.  If you have a mathematically inclined brain, and you’re brave, you could check out the Wikipedia page entry for LDA here. Several people in the SEO community seem to confuse LDA with a concept many of us have known about in SEO for a few years – latent semantic indexing (LSI).

Latent semantic indexing is also a statistical concept using probability maths, but it is less complex than LDA. If you’re still feeling brave, the Wikipedia page describing the mathematical concept is here.

LDA and LSI both describe mathematical models that are designed to be used for information retrieval – i.e. returning search results.

In a nutshell, LSI examines the words used in a document and looks for their relationships with other words. Google issued a patent in 2005 that discuss looking at three types of relationship – lateral (where a word means the same or very similar to something else, e.g. car, automobile and auto), “kind of” (where a word is a kind of something else, e.g. car and vehicle), and “part of” (where a words is part of a larger concept, e.g. engine and car).

LSI allows a search engine to determine the kinds of words that a web page might be relevant for, even if they are not actually used on the web page itself.  Bu writing content that is packed full of words that have relationships with each other, you are strengthening the document for all of those words.

However, LSI has one major weakness – ambiguity.  How could a seach engine determine if you are talking about Microsoft office, or the office in which you work.  Especially if you excel in what you do, providing a fresh outlook in your job as a publisher of bibles (you want to give more people access to the word of God, see).  I can’t think of any OneNote or Powerpoint puns, but you get the idea.

LDA, on the other hand is a significant extension of LSI.  Words are grouped into topics.  They can exist in more than one topic, in fact most do.  LDA tackles ambiguity by comparing a document to two topics and determining which topic is closer to the document, across all combinations of topics which seem broadly relevant.  In doing so, LDA helps an information retrieval system (such as a search engine) to determine which documents are most relevant to which topics.

The boffins at SEOMoz (apologies for the Bigmouthmedia-style start to that sentence) have built an excellent tool to determine an LDA score for text documents against search terms, and have tests many hundreds of sets of Google search results against what might be expected by their LDA score.

Now clearly, the results did not exactly match the LDA-based predictions, but they came so very close that SEOMoz – famous for their belief that link profile far outweighed most other ranking factors in Google – now suggest that the text on your web pages has a much stronger influence than they used to believe.

Looking beyond that, clearly there are a large number of factors that Google uses to rank websites – it’s all a rich tapestry, so they say. However, LDA is possibly the most spam resistant way of determining what a web page is about.  LSI synonym spamming won’t necessarily help – especially if the synonyms are ambiguous, or worse change the context of the meaning of the original word. Google bombing links won’t help LDA scoring.  Admittedly, it doesn’t prevent some of the black-hat mainstays like hidden content and cloaking, but then there are human-applied ranking penalties and bans for that kind of behaviour.

I’m not suggesting LDA can’t be spammed (actually, clearly it can by some elements of keyword density spamming), but it does allow documents that aren’t spammed to compete well in topics with those that are.

So, do Google use LDA algorithms?  There don’t seem to be any patents specifically mentioning it yet, but the patents aren’t always published quickly.  Additionally, the patents (if they exist) will almost certainly be clouded in a little fog to prevent SEOs from backward engineering Google.  However, the correlation seems very strong, and of course LDA is a specific mathematical information retrieval model. My belief is that it is likely that they do apply it.

The final, albeit tenuous evidence is the way that Google talk about their search results, and about how you should write your content.

Matt Cutts is always saying things like “if you want to rank well, write great content.”  To many this seems like a cop out, but for an LDA-based information retrieval system, this is exactly what you should be doing.

Additionally, at the search event earlier this week, while Google Instant was being extolled to the masses of gathered journalists, Ben Gomes – one of the chief engineers at Google, and in many ways, the public face of Google Instant – referred over and over again to searching for topics, and refining your search topic.  Not keywords.  Not search terms. Topics.

Naysayers claim that search engine optimisation is irrelevant now, or that the long-tail is dead.  But I couldn’t disagree more.

First of all, search engine optimisation is not dead.  Google Instant does not change how Google crawls, indexes or ranks a website – the primary reasons for undertaking search engine optimisation in the first place.  Sites still need to compete against each other for popular keyword searches in order that they can acquire traffic from Google.
In fact, as it is now much easier and quicker for a searcher to refine a search and edit what they searched for, it is now more important than ever to appear in the first few positions for important keyword phrases than it ever has been.  The search engine optimisation industry should now be more vital than ever!

What will change is that users are able to refine and specify their search much more easily, and so users are much more likely to continue to refine their search until one of the first few results on a page is relevant to what they are searching for.  Far from being dead, the long-tail search is going to come into its own!

Google claims that Google Instant is going to save the world 11 billion hours a year that people will spend not searching.  I think it is more likely that users will learn to search in a different way.  They will save time, but they will search more and search around topics to find the information they want. People will be less willing to accept the first information they find about a subject, and will retrieve and digest information more than before.

Ecommerce sites will need to become more competitive.  Those with niche markets will be able to serve those niche markets, but it will become more difficult than ever to break into already busy markets, such as MP3 sales, groceries, perfumes, and gadgets.

Usability test at Google, including eye tracking, determined that user focused on the search box first until their suggested search was close to what they were really looking for, then their eye would scan the first result (or sometimes the first two or three results) of the organic search.  If the exact thing they were looking for was not in those first results, their eye would go back to the search box, and they’d refine their search and go through the same process again, until finally they would find an exact match for what they were searching.

On the whole, users’ eyes did not scan to sponsored results until the subject matter in the organic results was close to what was being searched.

Consequently, sites that run Adwords campaigns need to ensure they are managed properly.  Broad-brush, short tail search Adwords campaigns will become less effective and those which carefully manage long-tail will do better.

As in nature, the fittest will survive.  And, as in nature, that doesn’t necessarily mean the strongest, but actually those more adaptable to change.

Search marketing, both organic and paid is changing, and those who do nothing may be those who do die.

Here are some more of its points, in just a little more detail.

Using the Free Tools Effectively

Search engines give you a remarkable array of tools to help you understand how people use your website, and how well search engines can index it. These tools include:

• Google Webmaster Tools
• Yahoo Site Explorer
• Bing Webmaster Tools
• Google Analytics

Each of the above should be used in conjuction with the other three. The first three tell you a little about what the search engines know about your website, for example, how many pages they are aware of on your website, how many links they are aware of to your site, and so on.

Such information is extremely helpful in uncovering weaknesses in the SEO profile of your website.

Google Analytics is a different beast.  It tells you a wide range of metrics about the people that find your website, and how they behave within it.  It also indicates which of your marketing channels are successful, and which ones are not.  Used properly, it is extremely helpful in indicating how marketing can be improved.

Remember when using Google Analytics to set up goal recording and hence conversion correctly.  Funnels for goal achievement are also important.  Correctly set up, these help you identify what bottlenecks there are in the conversion process and therefore what you might need to improve to increase conversion.

Telling the Search Engines About Your Website

All the major search engines accept submission using the XML sitemap protocol.  This is a handy way of indicating to a search engine the URLs that make up your website, in order to ensure that the website URLs are crawled effectively.  It is important to understand that the sitemaps do not influence ranking, just whether a search engine knows of a URL or not.  They don’t even guarantee inclusion, because clearly you could include URLs of web pages that contravene the search engine guidelines.

Probably the one thing most neglected by websites that rely on XML sitemaps is that they cannot cause PageRank to flow to pages that cannot be reached by other navigation methods.  From a best practice SEO perspective, XML sitemaps DO NOT replace good, indexable navigation.

Today, I’m going to cover a few more of the issues Ian discussed in his post.  First of all, error pages.

Get Your Errors Right!

From time-to-time, every website has an error.  Whether you’ve got a typo in your code, or a user mistypes a URL, or maybe your database server decides that it’s out to lunch – every site has had to deal with such things.

From a user’s point of view, there’s little more irritating that getting an error page.  Even worse, getting an unhelpful error page with technobabble and no real help.

Therefore, it helps to create a useful, on-topic error page to handle different kinds of error.  If a 404 error is encountered, give a helpful, friendly error message inside a corporate themed web page.  If possible, try to link to what the user might have been looking for – if they seemed like they were looking for a product page, show them some product categories.  Show them a search box. In short, do what you can to help them find what they were actually looking for.

The more you help a user who encounters a problem, the less chance there is that they’ll turn round, head out, and buy their red widget from some other website.

Go For Diet HTML!

Yesterday, we mentioned page loading speed.  One of the factors of loading speed is the amount of bulk in the code on a page.  There are two competing factors that can affect this – DNS lookups (Ian doesn’t talk about this though) and JavaScript/CSS code.

Ideally, all JavaScript and CSS code should be in external files. That is, in your HTML code, you only reference the JavaScript or CSS as being in a separate file, you don’t include within the HTML itself.  This reduces the amount of code in the actual HTML page.

However, each time you request a new file from a server, the browser has to go and fetch it, which slows down the rendering of the page. Additionally, if the file is on a different domain, the browser also has to lookup the address of the server to find where it is before it can fetch it, which slows down the process further.

There is a balance to strike clearly. Look-ups and fetches slow the process of displaying a web-page down.  However, once you have downloaded a file, unless the server instructs the browser NOT to cache the file, it will be cached so that next time (within a reasonable time period) it doesn’t need to be downloaded again.  This means that when you view the next page of the same website, if it uses the same JavaScript files, and the same CSS files, they are already downloaded, and the page will render much quicker.

Ideally then, the balance must be to keep as much JavaScript or CSS as possible in just one or a few files that are shared across all pages of a website.  In this way, few look-ups and fetches are required, but the majority of the CSS and JavaScript is download on the first page and cached for later use – the total code downloaded over the space of a single multi-page visit can be dramatically reduced.