Ever wonder how Google sees your site? Ever wish you could click a button and correct all of your site’s search engine woes? As it turns out, you can, at least to a certain extent. Webmaster Tools is a free product from Google that allows you to interact with the world’s most (in)famous search engine spider and indicate how your site should be indexed.
Head over to Google Webmaster Tools (you’ll need a Google account to log in) and type your URL into the text input box. Make sure that you precede it with “www” if you intend to set that as your preference later on, because Google won’t allow you to upload a sitemap if your domain here doesn’t match your preferred domain later. Click the “Add Site” button once you’re sure.
That’s enough to get you in. Before you can really use the console, though, you’ll need to verify your site. This is a very simple process if you have FTP access. Click the “Verify your site” link. On the next page, select “Upload an HTML file” from the “Choose verification method…” drop-down (you can use a Meta tag if you want, but I find the file method to be less of a hassle). You’ll be given a file name like “google#.html”, where the hash mark is a string of alphanumeric characters. Just open your favorite text editor, save a blank file by that name, and upload it to your site’s base directory. Click the “Verify” button and you should be ready to dig in.
Once you’re in, you’ll immediately see when your site was last crawled and whether or not it is included in the index. Like a lot of Google’s Webmaster Tools, these are good indicators of possible problems. Obviously, a recent crawl date and proper inclusion are preferable. If you’re not included, Google will give you clues as to why.
Here you’ll see any problems Googlebot has had trying to crawl your site. This includes pages that couldn’t be accessed for one reason or another, why they were inaccessible, and what sort of crawl (web or mobile) was attempted. Ideally, you don’t want anything to show up here, so be sure to review and, if possible, resolve every error that you see.
Everything under Statistics is potentially interesting but not necessarily vital. You can see the top queries your site shows up for, which ones bring the clicks, what words are used in links to your site, what your PageRank distribution looks like, and how many subscribers you have. As Darren Rowse has pointed out, the subscription counts only include Google feed readers, and the numbers don’t often match up with FeedBurner, so be ready for some disparity. If you’re interested in more of this sort of data, I suggest checking out Google Analytics.
The Links tab shows you where your pages are linked, both externally and internally. It’s a far cry from Google’s “link:” operator, providing a breakdown of almost every page that links to your site. The important thing here is not to get overly excited about a large link count. For starters, link juice is about quality, not quantity. Secondly, Webmaster Tools doesn’t distinguish between followed and nofollowed links. So, by all means, scope out the pages that are linking to yours; just don’t expect all of them to be valuable.
Obviously, you won’t see much here until after you’ve added a sitemap. If you’ve got a WordPress blog, the sitemap generator plugin is a quick way to automate the process. Once you’ve added your sitemap, you can see when it was submitted, when it was downloaded, and whether or not there were any errors reading it. Don’t forget to add a sitemaps autodiscovery line to your robots.txt to make sure other search engines can find it also.
The Tools section deserves its own special attention because it provides most of the features you can use to communicate with Google.
Analyze robots.txt. This tool will show you the contents of your robots.txt file and allow you to make sure it isn’t blocking anything it shouldn’t.
Manage site verification. If you followed the steps above, you’ll never need to use this. Just be sure the file you uploaded stays right where it is to keep your site verified.
Set crawl rate. If Googlebot seems to visit too often for your liking, you can tell it not to come around as frequently. If your site is large and important enough, you may also be offered the option of faster crawling. Of course, a faster crawl rate doesn’t influence your rankings any; it just lets Google index your newest content that much faster.
Set preferred domain. I mentioned earlier that the domain you entered into the Google Webmaster Tools dashboard and your preferred domain had to match. This is where to make that selection. As above, be sure they’re the same or you won’t be able to submit a sitemap.
Enable enhanced image search. Opt in and Google will more thoroughly index the images on your site. Honestly, I can’t think of a good reason why anybody would say no to better search relevancy. Tick the check box, hit “OK”, and move on.
Remove URLs. Believe it or not, there are times when you want a page removed from the index. It’s rare, especially for bloggers, but it does happen. When the need arises, this is the tool you use to get the offending page delisted.
You may have thought that was all there was, and you’d be partly correct. That’s all Google gives you on a site-specific level. If you click back out to the dashboard, however, you’ll notice several other menu options to the right of your site list.
Message Center. This is a new feature Google added to contact webmasters directly. If they’ve got anything to say about sites you’ve verified, it will show up here.
Download data for all sites. Most of us only have a handful of sites, but some power users may manage dozens or even hundreds. This tool allows you to generate aggregate reports for all the websites linked to your account.
Report spam in our index. The infamous spam report lets anyone report spammy websites in the Google index. Note that you don’t need to be a Webmaster Tools user to report spam to Google. Exactly what effect it has on the reported website is anyone’s guess, but it might be useful if you’re dealing with a scraper.
Report paid links. I doubt that Google considers spam and paid links to be separate issues. However, they do offer separate forms for each. Considering how many bloggers use paid links as a source of revenue, using this tool may amount to bad netiquette.
Request reconsideration. If you’ve been bad and gotten your site thrown out of index, here’s where you beg and plead for Google to let you back in. Be sure to read the fine print and make sure your site adheres to Google’s webmaster guidelines, or else your chances of reinclusion are slim to nil.
More to Come…
Google’s way ahead of the game with Webmaster Tools, giving us the unprecedented ability to monitor how Googlebot interacts with our site. More importantly, they seem to come out with new Webmaster Tools on a regular basis. Learn it, love it, and check out the Google Webmaster Central Blog for regular updates.