Google News Scraper
Extract news articles, headlines, and publisher data from Google News for media monitoring.
- Brand mention monitoring
- Industry news tracking
- Competitor PR analysis
Output Formats
Excel CSV JSON XML HTML RSS JSONL
🌐
General Web Scraping Scrapers
4 scrapers available
General-purpose web scrapers crawl any website and extract structured content — article text, metadata, links, images, and custom data fields. Use them to build datasets for AI training, content aggregation, SEO audits, site migrations, and research pipelines.
Google Search Scraper
Extract organic search results, ads, local pack, and 'People Also Ask' from Google Search for SEO analysis.
- Track daily keyword rankings (Rank Tracking) globally or locally
- Analyze competitor PPC ad copy and paid search strategies
- Discover new content ideas via 'People Also Ask' extraction
Output Formats
JSON CSV XML Excel JSONL HTML
Web Scraper
Crawl any website using a browser and extract structured data with custom JavaScript code.
- Custom data extraction from any website
- Recursive website crawling
- JavaScript-rendered page scraping
Output Formats
Excel CSV JSON XML HTML RSS JSONL
Website Content Crawler
Advanced website crawler extracting clean, structured content in Markdown, JSON, or plain text for AI and LLM applications.
- AI model training data
- RAG pipeline content
- Vector database ingestion
Output Formats
Excel CSV JSON XML HTML RSS JSONL
Frequently Asked Questions
What is the Website Content Crawler used for?
It crawls an entire website (or a list of URLs) and extracts the full text, headings, metadata, and links from each page. Common uses: AI training data collection, SEO audits, competitive content analysis, site migrations, and research datasets.
Can I scrape any website with the general scraper?
The general Web Scraper handles most public websites. Heavily protected sites (financial data, ticketing platforms, heavily bot-protected e-commerce) may need specialized scrapers from the actor store with built-in anti-bot bypass.
What output formats does the Web Scraper support?
All Apify scrapers export to 7 formats: Excel (.xlsx), CSV, JSON, XML, HTML Table, RSS, and JSONL. Data can also be delivered via API, webhook, or pushed directly to cloud storage.
Related Guides
tutorials
Apify MCP Server: Give Your AI Agent Access to 39,000+ Web Scrapers
How to connect Claude, GPT-4, and other AI agents to Apify's MCP server and give them access to 39,000+ real-time web scrapers — in under 10 minutes.
tutorialsThe Complete Guide to Web Scraping in 2026
Everything you need to know about web scraping in 2026: tools, techniques, legal considerations, anti-bot bypassing, and how to choose the right platform.
use-casesWeb Scraping for AI: How to Build Training Datasets
Learn how to collect web data for AI and machine learning. Build training datasets, create RAG knowledge bases, and power AI agents with scraped data.