SEO indexing – The (very simple) Process

To get your new site indexed you’ll need to do a couple of things:

• Create a sitemap (or big list of your webpages) at XML Sitemaps, name it sitemap.xml and upload to your web server
• View the list of ping URLs – or list of website addresses you can visit directly to notify search engines about your website – see SEO Ping URLs E.g. for Google, visit http://www.google.com/webmasters/sitemaps/ping?sitemap=http:%3A//www.YOURWEBSITE.co.uk/sitemap.xml
• If you get time, create a Google Webmaster Account and add your site there too – not essential to get your indexed, but very useful in the long run

However, you can ignore the above and get your site indexed even easier. In short, get a backlink from a site already included within Google. So if you have a friend who has a website, ask them to link back to you. In all honestly, if you site is simple and is doesn’t have a huge complicated nested structure, you probably won’t need a sitemap at all -  a simple backlink will do. Or even easier, simply mention the site within twitter. I experimented the latter using a new ecommerce site with ~300 pages. The single mention of the web address in a tweet got the site fully indexed with 2 days. You can speed things up even further if you get mentioned on a very popular or authoritative site.

That’s it in a nutshell – some questionable companies are charging £500 for this! For an easy life, I’m not going to mention the company name here.

I’ll go through some of the items included on the list of “seo indexing services” that are typically offered:

• Yahoo.co.uk submission (1-2 minutes)
• Submit to Google.co.uk (1-2 minutes)
• Create robots.txt – you don’t need a robots.txt file to get your site indexed in Google or any other search engine
• Create sitemap (couple of minutes at most using XML Sitemaps
• Submission to bing (1-2 minutes, Bing is MIcrosoft’s own search engine)
• Upload sitemap to the latter 3 sites (bit of overlap here, but still, no more than 5 or 6 minutes for all 3)
• Verify sitemap in the latter 3 sites. This is a common phrase a lot of the sites I looked at made. I’m not sure if this means checking the correctness of the actual generated sitemap (which you won’t need to do if using that online generator) or checking a few days later if your site is included within say, Google (go to Google and search for site:www.YOURSITE.co.uk to check) – 5 minutes work at most.

Repeat Submissions – “The Ultimate Con”

There are companies out there who charge hundreds of pounds for this service AND charge for the service on a yearly basis. Yes – there is a further charge again the following year. You simply do not need this. If you site is relatively small you don’t need this at all – maybe get the odd backlink from twitter or facebook to keep Google interested. Or even easier, go to the official Google Add URL site – that allows you submit your site within in seconds. Additionally, after a year it’s a very plausable assumption to make that you will have naturally built up a few backlinks and Google will automatically revisiting your site.

An Example – web-design-talk.co.uk

To illustrate how easy it is to get new content included, yourself,  I’ll use this very blog as example. Whenever I publish a new entry on this site I do the following:

• Ping my XML Sitemap with Google (see above how to do this)
• Post a link to the post from Twitter
• Submit the post to digg
• Mention the post on my status updates profile on the web designer forum

This all takes roughly 5 minutes and gets my site included in Google within a couple of hours – sometimes faster.

Whenever you see an SEO or Web Company offering something as shady as “SEO Indexing” take a big step back and question it. As them why is it needed regularly, why does it cost so much and why a site will backlinks will not suffice – basically, ask questions about specifically thr steps they will be taking. It’s one of those grey areas of SEO that companies are only to happy to aggresively market. Worse still, it is aimed highly at people not in the know and therefore marketed to them. E.g. when the sales person asks if you would like your site to be included in Google the customer will instinctively say yes. It’s highly questionable in my opinion and should be questioned. If you’re ever unsure, ask on a forum.

The latter is not a new idea by any means, but is generally considored to be a positive practice to implement on any site.

SOC and Ecommerce

The effect is compounded on ecommerce sites – sites that usually have a heavy template footprint. For example, a typical ecommerce site may contain a category listing with child categories in the form of a “mega drop down menu”, as they seem to to be coined. This is fine, as such menus do contain a lot of good keyword rich material that will be relevant to your site.

However, I have recently been experimenting a little on an ecommerce site I regularly perform SEO on, that does have a fairly beefy drop down menu. I have been focusing on product pages – I like to think of these as your most important pages and pages you’d want people to land on, as it tends to help conversions. In my eyes, any white hat technique to give greater weight and meaning to such pages is a good thing.

For the site in question I edited the source to force the top menu to the very bottom of the page and used absolute positioniong to force the top menu to the top of the screen for the user, when the page is rendered. I performed a similar action for the product description. As a result the important content i.e. the product title and description are now very close to the top of the page within the HTML.

Further Product Page Optimisations for SEO

Additionally I made a couple of other minor optimisations that I’ve noticed some bigger sites are using. At the top of the site in question, there is a company logo – a link, styled with a background using css. For the product page I amended this to a h1 tag with the product title within the header tag. I then used text-indent to hide this. Further down the page, I replaced the old header one tag (was further down the page previously, next to the product description) with a h2 tag and placed the product strapline within a h3 tag. So, the HTML for my company logo, for a product page is as follows:

<h1>
<a href="/product-url.html">My Product Name Here</a>
</h1>

I’ve been runing the above for a month now and have noticed the following trends in Google Analytics – I can’t be 100% sure this is directly down to the above though:

• Increased traffic to product detail pages from Google
• 1.2% increase in sales conversions

Over the next month I’ll be experimenting with replacing the logo with the category name, on my category listing pages too. I’ll update this post with my findings.

Have you had any similar experiences using the above, or similar?

Let me elaborate.

As much as I love the web design forum, some of the advice given is frankly, awful, I can’t not post a reply. A lot of the advice given is a range of generic seo quotes that people have picked from the web. E.g. ‘use headings’, ‘have a keyword density of 4%’ and ‘get backlinks’. The majority of the advice centers around simply getting people on a website or appearing for a generic, competitive term. Compare the latter to SEO’s calling me up at work (and I quote) – “there are 9,000 people searching for xxxxx every month, imagine having 9,000 more people on your site every month”.

All this is fine in theory. However, say you do manage to appear for a generic competitive term or get 9,000 new people on your site – what then. There is a high likelyhood that your boucne rate (or percentage of people who leave your almost immediately) will increase a lot.

Enter Conversion Rate Optimization

So what exactly has happened in this instance? You now have a lot of traffic coming to your site with lots of people boucning off. For websites that require their visitors convert – basically any site that sells a service or product (read: any business website) this is bad news, not to mention a total waste of time.

As a website owner, if you found your site at the latter stage, you’d want to investigate something called conversion rate optimization (or CRO). This is a term given to the science of converting more of you visitors into actual customers – which is what any business requires from their website. Essentially, it the measure of success of a website. For clients who say they want a nicely designed site, with lots of graphics, or clients who want to appear number one for lots of keywords – they really mean they want a site that converts and makes them money.

This one factor is overlooked so much and seems to be a highly vicious circle. SEO and CRO go hand in hand, indeed they need to go hand in hand as they are tighly linked. Quality SEO practice is needed to get more people onto your site, while CRO is needed to convert these visitors into customers.

It amazes me that I still get marketing calls from professional SEOs who say they can get me onto page one for ‘web design’, without any mention at all of CRO – they simply say they’ll get lots of quality backlinks to my site. This is wrong for two reasons. Yes, my site will recieve a lot of traffic, but is it fully optimised to convert visitors into customers (currently it isn’t, currently undergoing a redesign). Additionally, it’sd a well known fact that short tail terms don’t convert as well as a long tail terms. I’m currently much happier being ranked highly for a longer tail version of ‘we design’ – ‘web design xxxx’, where xxxx is my county. Yes, it has less traffic than ‘web design’, but the traffic is more targetted and relevant. Additionally, I have a geo targetted page in the SERPs for this term, which helps too.

In relation to narrow views expressed on the web design forum, well, those people aren’t helping anyone and seem to measure the success of SEO in terms of ‘how many’ people land on their site. As I’ve explained above, this is only part of SEO. I’ll take a typical scenario. Someone asks how they can improve the SEO on their site. People instantly jump in with the generic seo quotes (which are a pet hate of mine). I’ll have a quick look att he OPs site – awful design, no useful content there at all, sometimes not even contact details are present. This is where people need to work backwards before quoting the seo quotes they love so much. Firstly, perform some CRO on the target website and ensure that if a visitor does land on your site that are more likely to convert.

To illustrate some examples of CRO, I’ll explain a few simple and small examples I’ve personally used on some sites I’ve been working on:

• Make call to action button stand out and adjust the button text – on a corporate site website, increased enquiries by 2.5% in the first month
• Adjust the way an existing ecommerce store displays related items – instead of selecting 10 random products on the basket page, select 10 products that people have ultimately ordered, based on the current shopping basket – increased sales by 2.5% in the first month
• Removed unecessary fields in our website enquiry form – cutting it down to 3 fields, from 6 – increase in enquiries
• Boldened and increased font size of first paragraph of text on inner, informational pages – resulted in people spending more time on the site and allowed the page to focus a clearer message to the visitor
• For one company, that had worked with several recognizable brands, I added their logos onto the site in prominant place to build instant trust and reputation – increase in conversions of 3.5% in first month

I’m by no means an internet marketer, SEO guru or leading web designers at all However, it doesn’t take any of those to realise that once you have attrcated people to a site you need to do everything humanly possible to get a conversion. On the flip side, you need SEO to get a decent level of targetted traffic on a website.

SEo and CRO, a match made in heaven

Take the fowllowing URL, this is the URL the developer intended – the news story is fetched from the ID in the querystring:

http://www.site.com/news/23/some-breaking-story.html

However, if the application is coded badly the following URLs may also exist in order to reach the same content:

http://www.site.com/news/23/ANYTEXTHERE-some-breaking-story.html

http://www.site.com/news/23/some-breaking-story.php

http://www.site.com/news/23

The latter can have potential issues for your SEO rankings. If search engines pickup on the fact that muliple URLs all goto the same duplicate content then your site send out bad signals to search engines – this should be avoided. There si recent talk that duplicate content is simply ignored by search engines and now a none issue. However, even ignoring direct penalties imposed on your site, having the same content divided over multiple URLs can reduce your rankings simply by diluting content.

The answer is URL correction and correct use of headers. A single URL for a single page is the goal. This is fairly easy toachieve by following the below methodology in your scripts:

• Make your mod_rewrite rules specific (E.g. don’t use lots of ([^/]*) flags as that will match anything!)
• Create a standard base URL for the content
• When the page loads, check that the base URL (or expected URL) matches the page requested by the visitor
• If there is a mismatch, set the correct HTTP headers and redirect to the correct, expected URL

Firstly, adjust your rewrite rule to be more specific:

RewriteRule ^news/([0-9]+)/(-a-zA-Z)\.html$ news.php?TITLE=$2&NEWSID=$1 [L]

This now looks for the following pattern: a number, letters and then a .html extension.

You can also adject your application logic to add in the following:

/* At this point, query your database to retreive information
The variables $title and $id would come from the database */
$actual_url = myClass:getCurrentRequestedURL();
$expected_url = LinkFactory::NewsItem($title, $id);

ob_start();

if ($expected_url != $actual_url) {
ob_clean();
header('HTTP/1.1 31 Moved Permanently'); //Send 301 status code
header('Location: ' . $expected_url); //redirect to expected url
ob_flush();
}

Depending on your url rewriting structure, you’ll notice that WordPress does a very similar thing.

The special care of multiple categories and SEO is that category pages contain a huge amount keyword rich anchor text. Yes, the majority of ecommerce software and system will allow filtering of results, but the canonical url tag is often used and results in messy links that are not ignored by search engines.

The major issue here is duplicate content – frowned upon by Google and can cause real issues for your site. Yes, a small numbers of pages is within acceptable limits, but when you have store that has hundreds of products duplicate content really can become an issue.

Using our tshirt example from above, say we place a product called ‘super baggy tshirt’ into the ‘red tshirts’ and ‘logo tshirts’ categories – the following two urls would be produced by our ecommerce software – the below structure is very common):

http://www.shop.com/tshirts/red-tshirts/super-baggy-tshirt/

http://www.shop.com/tshirts/logo-tshirts/super-baggy-tshirt/

At first glance, this all looks well: SEO friendly URLs, well structured, organised and keyword rich. All this correct, apart from the fact that both URLs represent the same product – here is our duplicate content issue. The duplicate content issue will get worse if the product is placed into more categories.  The easiest solution is to rewrite our product URL to something much simpler:

http://www.shop.com/super-baggy-tshirt/

This will allow us to place the product into as many categories as we need without creating any duplicate content at all – there will always be a single version of the product URL.

I physically cringe when I hear quotes such as ‘valid xhtml will help your users’. Valid xhtml will not help your users, to help your users a site needs to adhere to web coding standards – this is an entirely different beast. The main difference here is the practice of seperating content from presentation, thus giving the content increased meaning. For example, a page using tables to layout the whole web page would not adhere to web coding standards because using tables for layout is semantically incorrect and requires a lot more code. Tables should be used for tabular data, simple. Another example is the use of paragraph and header tags. Visually they are very similar but have a very very different meaning sementically. However, yet again, semantically incorrect pages will pass validation. The main Google webpage doesn’t even validate (interestingly, Google does’t even quote html attributes in order to save on page size). In my opinion, as long this is the case W3C validation will be a none issue, SEO wise.

Understanding which semantic elements add value to the document will affect the onsite of a website and is an SEO ranking factor.I have read several artuicles that describe W3C validation and SEO as a match made in heaven, this simply isn’t the case, although web semantics and SEO are.

There are many websites (40% is a figure thrown around a lot) that do not validate, yet perform quite well in search engines as they have a range of high quality content. Take a quick example. I searched for a very competitive term “houses”. The number one result was rightmove.co.uk. Rightmove even has an authorative listing for that term too – SEO wise there can’t be too many issues here. Running that site through the validator throws up 33 errors and 22 warnings. – see the result. These are mainly smaller syntax errors that quite rightly, the developers of that site have ignored. There are endless examples where sites a lot worse appear at the top of the SERPs, even though they fail to validate and sometimes, don’t follow web standards at all.

There is also another camp that stresses that W3C validation is important for page presentation, in turn page presentation affects your reputation and thus your SEO (the number of people willing to backlink to you). Well again, here I’d quote the subtle difference between validation and semantics. YOu can also have a perfectly presented and formatted page that fails to validate.

The addictive, green “congratulations” message that appears is far too often used to lure unknowledgeable clients into a false sense of trust and can go a very long way into building trust between the SEO and client. As Google uses over 200 ranking factors when indexing your website, I find it very hard to believe that minor syntx errors play much role at all. Whenver I see SEO companies that associate “100% valid xHTML code” and SEO I run a mile. It’s clear they have little knowledge of SEO from this one ststement.

Validator error severity

My next hate is the severity of these errors are often mis-quoted. Don’t get me wrong, validating your website is good practice, but not critical for SEO. A random unclosed paragraph tag, or site that uses ‘b’ tags instead of ‘strong’ tags is of little worry to me. I can spend my time of much more important, valuable areas – at the end of the day this won’t effect your SEO or the way the page displays. However, when there are more serious  (or hard) errors iccur these could very well stop the page from rendering all together. The latter is of course an erro you should take note of and fix quickly. However, minor syntax errors are of little concern. At the end of the day, pages that have an expected layout and content that the bots can consume easily is my major concern. However, pages that validate fully don’t give you SEO benenfit at all, however nice it is to have.

Full validation isn’t always possible

Another point is that full validation isn’t always possible. For instance, getting your site to display correctly across the plethora of browsers available can cause validation errors. This brings me to another issue. At lot of people seem to assume that a page that doesn’t pass the W3C validation is badly coded. I’ll use the example of the Google home page here. This page doesn’t validate due to a numbers of reason including the numbers of browsing devices it needs to cater for and page size (saving them a lot of money). By the latter logic, Google is a badly made webpage, I think not. I’m really not going into this anymore as it infuriates me so much. There is a trend where someone checks a page through the validator, if it fails they instanyly make a judgement that the site is badly made.

To conclude, at the minute there really is no evidence to suggest that Google factors in the validity of your code. In time search engines use it as a ranking factor, simialr to how Google began to use loading speed as a minor ranking factor. Validating or not is NOT the same thing as serious HTML errors that hinder crawling and indexing – this is one huge SEO scam in my opinion and is pushed upon unsuspecting an unknowledgable clients. Furthermore, I’m yet to see a site built so badly that it has failed to be included in search engines.For me, quality, quthorative content will always be at the heart of SEO, not coding syntax.

I’m also not saying that W3C validation should be ignored. Tomorrow, Google could well decide that W3C validation is a must and a suddenly a major ranking factor. However, for the time being, is a bit of a none issue for me.

Is W3C validation good practice for web developers, definately. Is W3C validation a must for SEO, not right now. Are SEO and W3C validation highly linked right now, very unlikely.

Personally, I wasn’t aware you could track specific links with Analytics and only ever considored this when a client asked ‘why doesn’t Google show me the numbers of times my marketing report (read: a PDF file) has been clicked?’ – a totally valid request that I wanted to investigate.

Use JQuery to improve Google Analytics and track downloads, RSS, Email & external links

First things first, make sure you have a google Analytics account, the latest version of JQuery and the latest version of the analytics code running on your website

As with the majority of the JQuery magic, everything happens within the doc ready event listener – this will used to capture various clicks to select elements.

Tracking Download Link Clicks (PDF, ZIPs etc.)

$(document).ready(function() {

	$("a[rel=download]").click( function() {
		var fileName = $(this).attr("href");
		pageTracker._trackPageview(fileName);
		return true;
	});

});

Then on every link you wish to track, simply add the rel attribute to your non HTML files as follows:

<a href="myData.zip" rel="download">Download My ZIP Data File</a>

Tracking Downloads of Specifc File Types (E.g. PDF files)

Using the dollar sign to match against links that end in .pdf (or any extension you wish to track).

$(document).ready(function() {

	$("a[href$=pdf]").click( function() {
		var myPDF = "/pdfDownloads/" . $(this).attr("href");
		pageTracker._trackPageview(myPDF);
		return true;
	});

});

The /pdfDownloads/ is used to identify and seperate report data within Google Analytics.

Tracking the click of a specific link such as an RSS feed

Simply add an identifier to your RSS feed link (in this example the link was given an id of ‘rssFeed’):

$(document).ready(function() {

	$("a#rssFeed").click( function() {
		pageTracker._trackEvent("RSS", "RSS Subscriber Link Clicked");
		return true;
	});

});

Tracking mailto: Link Clicks

$(document).ready(function() {

	$("a[href^=mailto:]").click( function() {
		pageTracker._trackEvent("Mail", "User clicked on mailto link");
		return true;
	});

});

Tidying up….

You should also disable the clicked element to prevent multiple event recording and provide feedback. To do this, simple add the following at the start of each piece of code – disbabling the element and changing the cursor to an egg timer (although you could display a small graphic to make things look prettier):

$(this).css("cursor", "wait");
$(this).attr("disabled", true);

I’m sure you use this fact throughout your website while performing onsite SEO. Any tutorial will rightly tell you that keyword relevancy is of upmost importance here. However, a lot of the time you end up with a ‘Home’ link on your menu, linking to your main page.

This is bad for SEO for a number reasons. Firstly, the anchor text ‘home’ is very poor choice of word to use as it’s meaning is highly diluted nowadays. Now unless you have a site relating to homes, the keyword isn’t very useful at all, as we don’t want to rank highly for the term ‘home’. However, at the same time users are familiar with such a link and it makes sense to name such a link to your main page. The situation is worsened if your site is large with a great number of internal links. Imagine having a 100 page site, all with the anchor text ‘home’ – this is a lot of inbound links telling search engines each page is related to ‘home’! Furthermore, the menu link’s are usually towards to the top of the page, giving them inscreased relevancy to search engines.

The solution is a compromise, use ‘home’ along with you main keyword(s) – making sure to avoid obvious stop words like ‘and’. For example ‘Graphic Design Home’. Even better use your main keyword directly in the anchor text i.e. ‘Graphic Design’. This is quite a powerful and simple SEO trick that is easy to implement into your site.

If you use Twitter as a marketing tool to drive traffic to your site then you should treat it in exactly the same way as you would a newsletter, a PPC advert or a banner and track each Tweet’s performance beyond simple click data. How many visits do you get, how long do they stay on your site, how deep do they go, what is the bounce rate like and how much revenue do they generate?

The benefit to ‘tagging’ this link is that Google Analytics will record more than use basic click data – you can record a whole host of advanced user data such as how they navigate your site and length of visit. By default Google will track such links, but traffic from services such as bit.ly will be dumped into the direct traffic area of Google Analytics. The steps to get the latter up and running are quite simple:

1: Go to Google’s URL builder to generate an url . Enter the following information:

Website URL: your website address

Campaign Source: enter a relevant source here to identify your campaign E.g. twitter

Campaign Name: enter a name used to identify the campaign, this is used to identify the campaign in Google Anlytics E.g. twittertracking

2: Click generate URL and something similar to the following will be created: http://www.web-design-talk.co.uk/?utm_source=twitter&utm_medium=social&utm_campaign=twittertrack

3: If posting to twitter you can paste this URL directly in the tweet box, as twitter will automatically shorten this url.

4: After approximately 24 hours data will appear in your analytics account. Simply navigate to Traffic Sources. If you’ve used the same terms to build the url as above you’ll see an entry called ‘twitter / social’. You can also view information by navigating to Traffic Sources > Campaigns where you can click the campiagn name (‘twittertrack’ was used in he example above).

Google Analytics Once Tracking is Installed

Another less known theory is that spiders get full when crawling a page, semantic coding practice will allow for cleaner and more lightweight code. For instance, when crawling a badly coded page with lots of line styles and JavaScript (E.g. content not useful to a spider) the spider may become full too quickly and leave – missing you important content contained further on within the page.

Validating your site to at least xHTML Transaitional 1.0 (the test strict version, compared to xHTMl 1.0 Strict) is highly encouraged and is an area often ignored by developers. Below, I’ll quickly outline some of the common validation errors and how to easily fix them:

cannot generate system identifier for general entity X - 99% of the time this relates to errors with entity references such as ampersands in URLs. E.g. having an url like product.php?id=2&mode=view would result in this error as the ‘&’ wasn;t used within the url.

required attribute “alt” not specified –  simply find the line number and add an alt tag for the image. The presence of an alt tag is required for both transitional and strict doc types.

XML Parsing Error: Opening and ending tag mismatch – Depending on how organised you are when coding this fix can take a matter of seconds or a lot longer. It relates to unclosed block level tags, such as a table or div. One plus point is that fixing such an error often results in several validation errors being fixed at once.

required attribute “TYPE” not specified – This relates to no type attribute being specified for things like script and style tags. Solution – use the type tag like this: <script type=”text/javascript” language=”javascript” src=”scripts.js”>

There is no attribute “HEIGHT” - xHTML does not allow for the presence of the height attributer. To resolve, add an inline style or better, specify the height within the css rule for that element.

NET-enabling start-tag requires SHORTTAG YES or document type does not allow element “META” here – caused by incorrect use of short tags for the page’s doctype. For example if you have specified an HTML doctype then you use use <br> instead of <br />.

ID ‘someDivname’ already defined – caused when you have used two dividers on the same page, with the same id. Solution, uses div class instead of div id if you need multiple dividers on one page.

Missing a required sub element of HEAD - almost definately caused by missing xHTML required tags from the head section. E.g. have you included a title tag (required in all HTML documents.