Web Scraping
Web scraping is the automated extraction of structured data from websites.
A scraper sends HTTP requests to target URLs, parses the returned HTML (or JSON from APIs), and stores the extracted data in a structured format like CSV, JSON, or Excel. Modern scrapers use headless browsers to handle JavaScript-rendered content.
Related tools
Related Terms
Headless Browser
A headless browser is a web browser that runs without a graphical user interface.
Proxy Rotation
Proxy rotation is the practice of cycling through multiple IP addresses when making scraping requests, so the target server sees different IPs instead of repeated requests from the same source.
HTML Parser
An HTML parser reads raw HTML text and converts it into a structured tree of nodes (the Document Object Model, or DOM) that programs can traverse and query.
Rate Limiting
Rate limiting is a server-side defense that restricts the number of requests a client can make within a given time window.