If you work in SEO, it can be very useful to know how often the search engines are crawling your website and exactly which pages they’re crawling. This is important because if you’re suffering from crawling issues, then some of your...