If you have heard me speak about SEO at one of my many talks or one on one sessions, you have heard me mention the three major aspects of good SEO. Though they are all important, one of the three, that has been mostly a high-level factor for small businesses in the past, without having to dive too much into the nitty-gritty, is becoming more of an issue in 2017. Technical SEO, and more specifically, crawl efficiency, has become something you have to keep on your radar and worry about.
For review, below are the three main pieces of good SEO and a quick explanation.
3 Pieces of Good SEO
- On Page Optimization- Your Content: Making sure you have a page for every “concept” you want to rank for, and that you have good thorough content and proper tagging to “aim” that content at the right search queries.
- Off Page Optimization- Your links: The other sites that link to your site and your specific pages provide Google with signals of the page relevance and are the core force in how “strong” that page is and its ability to rank highly. The signals can show your local relevance or your industry relevance, or your general site strength, and ideally a mixture of all that is important to your topic.
- In Page Optimization- Technical Aspects: In general, for small businesses, this has included big ticket, and sometimes obvious issues like whether your site is mobile friendly or not, and how fast your pages load. These are clear to user experience. This year the issue of site crawl efficiency has become more important. An issue that only giant sites seemed to have to worry about in the past.
Crawl Efficiency and Why It Matters
For most small businesses, their site has less than 20 core pages and basically keeping it running and making sure all the links are in place are your biggest worry, once you have the site built as mobile friendly and you have decent page speed. With the push toward https, you may have had to update your site with an SSL certificate this year and the small tweaks that come with forwarding your old pages to the new versions.
However, with the growth of WordPress sites and regular blogging, whether unique, or paid services, your relationship with Google gets a little bit more technical. Controlling your crawl becomes much more important to making sure the content you want out there is out there, and the content that can get you in trouble is not.
What is Crawl?
When I refer to “crawl” in an SEO discussion, we are talking about when a search engine (in most cases we are most concerned with Google) indexes your site. That means that Google basically comes around and takes a “picture” of your site every once in a while. Your ranking is based on what Google has “on file” for your site.
When you make a change or update to your site, Google doesn’t know about it until it comes around and indexes your site again. It also doesn’t always index every page of your site. If you aren’t aware of how your pages are set up, it can also index pages you don’t want it to index.
In general, Google will only allow so much crawl budget to a site and adjust based on performance. If you have a lot of “garbage” pages on your site that Google hits when it is indexing your site, it is going to hurt your overall performance, because it may prevent Google from getting to more important pages in the time it dedicates to looking at your site. You can get a view of crawl rate and crawl stats in the Google Search Console, which everyone should have set up for their site. It basically tells you your sites relationship with the Google index. Google will take you through the step by step to get it set up for you.
Often with WordPress sites, you will find that segments of your site that you don’t necessarily want indexed are using up your “crawl budget” and either getting you in trouble because of unintended duplicate content or just lowering the priority of your important pages and possibly preventing Google from getting to them. Some common aspects of your site that may be getting indexed that you don’t want indexed are listed below (Keep in mind that some sites are structured where these are actually important pieces to be indexed, so make sure you are not noindexing important parts of your site)
- Archive Pages
- Image Pages
- Blog Postings from Blogs that are not Unique Content
- Blog Categories and Tags
- Back End Administrative Pages
NoIndexing
So what can you do to help sculpt your indexed pages a little better? You can noindex unimportant pages or entire segments of your site with your SEO plugin. If you use too many tags on your site and it leads to a lot of tag pages with only one post, that can lead to thin or duplicate content issues. Sometimes noindexing entire segments is handy to prevent these issues and make sure Google only indexes the more important pages you are actually creating for ranking.
The image to the right here shows some noindexing options from the All in One SEO Plugin, but Yoast also has similar settings. You can also noindex specific pages if you scroll down to the bottom and look at the plugin settings. Some clients will have basic blog posts that just announce events or are only a sentence or two and they don’t write them for ranking, whereas other posts are well thought out and meant to answer a search query. Individual post noindexing allows you control in that sense. There is an image below showing the All in One box you would check for that.
You may also have pages you have created for A/B testing or only for paid search landing pages and you don’t want them to use up any of your crawl. This is another time to click the noindex box.
This stuff starts to get a little complicated, but if you are having trouble getting important pages to show up or you suddenly saw many of your pages disappear from Google, it could be an indexing issue. Get a trusted SEO person to come take a look. We deal with this stuff every day and sometimes you can be fixed and back in good shape within a week or two. Give us a call.
Leave A Comment