SEO
Screaming Frog
网站爬虫工具，扫描全站技术 SEO 问题：死链、重复内容、缺失 meta 标签、重定向链。
主要用途
Screaming Frog SEO Spider is a desktop crawler that walks through a website the way a search engine bot would, collecting page-level data into a sortable, filterable, exportable table. For every URL it records status codes, titles, meta descriptions, headings, canonicals, robots directives, word counts, internal links, and structured data, exposing problems that are invisible page by page: redirect chains, orphaned pages, duplicate titles, broken links, accidental noindex tags. It can render JavaScript, compare staging against production, extract custom data with XPath, and pull in Search Console and Analytics data per URL via API. The free version crawls 500 URLs; a paid license removes the cap and unlocks configuration, scheduling, and integrations.
所在链路
Screaming Frog handles the technical audit stage of SEO, generating the issue lists that developers fix and that tools like Search Console later confirm as resolved.
核心功能
- Full-site crawling with response codes, metadata, and directives
- JavaScript rendering for client-side content auditing
- Redirect chain, broken link, and canonical analysis
- Custom extraction with XPath and regex
- Search Console and Analytics API integration per URL
适合谁用
- Technical SEO specialists auditing site structure and indexability
- Teams validating migrations and redirect maps before launch
- Developers diagnosing template-level metadata problems at scale
新手提示
- The free 500-URL limit covers small sites and learning; you only need the license once you audit larger properties regularly.
- Crawl data tells you what is wrong, not what matters most; prioritize issues by the traffic and revenue of affected pages.
- Lower the crawl speed for small or shared hosting, because an aggressive crawl can noticeably slow a production site.