Screaming Frog is becoming our most powerful SEO tool, especially for site launches and audits. It returns page metadata, errors, and more.
Screaming Frog has three different modes. Each serve a different purpose.
- Spider = Crawls the site just like a search engine spider. You input a URL and it runs through links it finds on the site until it reaches its URL limit or crawls everything. You can configure the spider to only focus on certain parts of the site (e.g. subdomain, blog directory, etc.).
- List = Crawls a list of URLs that you input or upload. This is great for quick checks on specific URLs. We just used this last week to test redirects that were implemented on IndSci.
- SERP = No crawling involved. This mode specifically shows you how Google will show your page titles and meta descriptions based on pixel width. You can pull all title tags and meta descriptions from the spider, edit them in Excel, then reupload into SERP mode to see how your edits will look.
Free and Paid version: Free version limits you to 500 URLs at a time. PIC has a paid subscription – if you use it, make sure you’re not kicking out someone who is also using it (similar to Adobe).
Screaming Frog User Guide =
https://www.screamingfrog.co.uk/seo-spider/user-guide/
