iWeb hides from Google
iWeb hides from Google
Sunday, 25 March 2012
When I moved the blog over to newbery.gen.nz, of course I wanted to test that everything was working correctly. So I get up Google (and Bing) Webmaster tools and pointed them at the new site, and then I searched for some words on the new Blog entries. I was somewhat supposed to see that I wasn’t finding them, even after several days. I could see the old blog entires, still on MobileMe, and I could see the bits of the site not created by iWeb, but the Blog (and other bits) did not see to be showing up.
Eventually of course, I did the sensible thing and asked Google.
The top hit was this page, which explains what iWeb does and why it has problems.
Normally, for a site (or section of a site), like www.newbery.gen.nz/ or www.newbery.gen.nz/People/michael/ that doesn’t give an explicit file, the web server chooses a default page (usually index.html), so www.newbery.gen.nz/ becomes www.newbery.gen.nz/index.html and www.newbery.gen.nz/People/michael/index.html
When iWeb constructs a site, instead of making the default page (the home page) be called ‘index.html’, it keeps the name it would normally use (‘Welcome.html, About.html, Blog.html, etc.) and creates a special index.html page that looks like
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title></title><meta http-equiv="refresh" content="0;url= Welcome.html" /></head><body></body></html>
That is, it instructs the browser to immediately reload the page, redirecting to the ‘real’ page (see the underlined bit above). And it seems this causes search spiders such a Googlebot and Bingbot to think there is no interesting content and stop there—they don’t follow the redirect.
My solution was to point the link on www.newbery.gen.nz that used to point to www.newbery.gen.nz/People/michael/ to be instead www.newbery.gen.nz/People/michael/index.html.
At this point, you might wonder why I am using iWeb. So am I.