Making sure Search Engines can Recognize your Blog or Website

Search engines have the ability to drive a lot of traffic to your Blog or Website. Normally, if you’re going to get high levels of traffic it’s very important that the search engines can access all the information on them.

Do the search engines know about all of your pages?

You can find out which pages on your site the search engines know about by using a special search. If you search for ’site:’ and your URL or website address the search engine will tell you all of the pages on your Web site it knows about.  For example, search for: in Google. Yahoo or MSN and it will tell you how many pages they know about. If the search engines haven’t found some of the pages on your Web site, it is probably because they are having trouble spidering them. (’Spidering’ is when the search engine uses an automated robot to read your Web pages.)

Spiders work by starting off on a page which has been linked to by another Web site, or that has been submitted to the search engine. They then read and follow any links they find on the page, gradually working their way through your whole Web site. At least, that’s the theory. The problem is, it’s easy to confuse the spiders – especially as they are designed to be wary of following certain kinds of link.
Links which confuse spiders

If your links are within a large chunk of JavaScript code, the spider may not be able to find them, and will not be able to follow the links to your other pages.
This can happen if you have ‘rollovers’ as your navigation – for instance, pictures that change color or appearance when you hover your mouse pointer over them. The JavaScript code that makes this happen can be convoluted enough for the spiders to ignore it rather than try to find links inside. If you think your rollovers are blocking your site from being spidered, you will need to talk to your Web designers about changing the code in to a ‘clean link’ – or a standard HTML link, with no extra code.

Spiders will also ignore pages if they don’t like the URL (the address needed to find the page).
For example, a Web site that has URLs containing several variables can cause spiders to ignore the page content. You can spot pages like these as they have a ? in them, and &, for instance:


This URL has three variables, the parts with the = in them, between the ? and &s. We find that if a page has one variable, or even two, the most search engines will spider them without any problems. But if a URL has more than that, often the search engines will not spider them.
Spiders particularly avoid URLs that look like they have ’session IDs’ in them. They look something like this:


The set of numbers and letters do not make much sense to humans, but some Web sites use them to keep track of who you are, as you click through their Web site.  Spiders will generally avoid URLs with Session IDs in them, so if your Web site has them, you need to talk to the people who developed the site about re-writing it so they do not use these IDs, or at least that you can get around the Web site without them.

If you use clean, easy to follow links without several variables in them, your Web site should be spidered without problem. There are, of course, many other facets to successful Search Engine Optimization, but if the search engines can’t spider your content, your site will fall at the first hurdle.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl


  1. hi, i am a total newbie to internet marketing and wanted to know if you could offer me some advice/instructions on how i set up a successful video blog on to my website. thanks tony

  2. As a matter a fact I plan to add some video and audio to my blog. . I will keep you up to date. . I just ordered a video camera. .thanks Scott

  3. I have to say, that I can not agree with you in 100%, but it’s just my IMHO, which could be very wrong.
    p.s. You have a very good template . Where have you got it from?

  4. I am very glad to have to read this article. You have helped me again. There is a greate deal of knowledge to be gained from this article. I would love to read more of your aticles. Thank u so much for posting this article.

  5. I don’t usually reply to posts but I will in this case, great info…I will add a backlink and bookmark your site. Keep up the good work!

    I’m Out! 🙂

  6. A few weeks ago I stumbled your blog and have been reading along steadily. I felt I should give my opening comment. Im not sure exactly what to write except that I have loved reading. Interesting site. I shall carry on visiting this website in the future. I have also got the feed for any updates.

  7. theteentips says:

    Hi i’m a newbie too, i just made myself a wordpress blog. How do i make more people read my blog or enables search engine to regonize my blog.I could need a little more help with more details with this please…
    check out my blog

    thanks sir.

  8. theteentips says:

    one more thing…how to add tags like in your left side of your blog???

  9. I am very glad to have to read this article. You have helped me again. I like this template! you did a great job on this.

    Thanks I read more article

  10. Hi to everybody, I love your hark work in this community, you are helping web people with great writing! This site is much interesting!
    My name is Carla, from Sidney, and I will be a fan of this blog, My interests you may don´t want to know them but I say them off course I am very found of swimming as well as tv shows, and I also play a lot Shakira on my roadtrips, I´m without boyfriend now so boys watch out for me….Just flirting with you guys lol :)! I already tried online dating It did not worked out very well….
    I will also apoligize for my writting it was the only way I found to communicate with you….
    Bye to all of you, Goodbye


  1. […] Center, and Compaq Systems Research Center.  Basically, you have to understand how linking and search engine spiders work in order to produce the best quality linking for your […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: