插件标签： crawler
-
FacetFence Product Filters
Protect WooCommerce filtered URLs from crawl floods with noindex/canonical controls, signed cookies, rate limits, logs, and rollback.
-
Valvai – AI Traffic Control Center
Block, log and monetise AI crawlers directly from your WordPress dashboard.
-
Rankal LLMs.txt – Get Cited by ChatGPT, Claude & Perplexity
Generate and serve a dynamic, cached /llms.txt AI context index. An add-on that works in partnership with Rankal SEO Optimizer.
-
AI Kansoku Lab Tracker
Track AI crawler visits such as GPTBot, ClaudeBot, and PerplexityBot server-side and visualize them in WordPress.
-
AI Bot Monitor
Tracks and logs AI, Search and Social bots including ChatGPT, Perplexity, Googlebot, Bing and Meta crawlers.
-
-
Spuncin
Blocks AI crawlers that collect your content to train models. Search engines and link previews are never touched.
-
Zicy AI Access Monitor
Monitor known AI crawlers and AI assistant fetches, with daily rollups, CSV exports, weekly summaries, and page briefs.
-
AI Visibility Control – Dynamic llms.txt & robots.txt Crawler Control
Manage AI crawler access. Block GPTBot, Claude & AI bots in robots.txt, generate dynamic llms.txt files, and track referral visits locally.
-
-
ADG Markdown for AI Crawlers
Serve AI crawlers clean Markdown of your content on the same URL, and auto-generate a virtual llms.txt / llms-full.txt. No external services.
-
Virtual LLMS.TXT Generator
Publish a safe, configurable llms.txt map from public WordPress content, with an opt-in bounded full-text export.
-
Promium AI GEO
Write a dedicated version of any post or page for AI crawlers such as GPTBot, ClaudeBot and PerplexityBot, and choose which bots may read it.
-
BotObservatory – Live Bot Tracker & Anti-Scraper
Observe human and machine traffic, identify suspicious automation, and optionally quarantine aggressive scrapers.
-
URLs Most Wanted for LiteSpeed LScache
A demand-driven sitemap for LiteSpeed Cache warmup. Record the URLs visitors actually use and warm only what people need.
-
Pixellize Site Issue Scanner
Powerful SEO auditor by Pixellize. Crawls your site to fix broken links, missing meta tags, and unoptimized images.
-
Imedes AI Bot Verify
Verify supported AI bot identities, allow legitimate agents, and block spoofed claims.
-
Whizz MD — Markdown Endpoints for AI Crawlers
Serve every WordPress post and page as a clean .md file with full SEO frontmatter — discoverable by AI crawlers and search engines on the fly.
