User Agent
A user agent is the identifying string a browser sends with every request, telling the server which browser, version and operating system you are using.
Definition
A user agent is a string your browser includes in the headers of every HTTP request, describing the browser, its version, the rendering engine and your operating system. Servers use it to tailor responses and, importantly, to detect bots.
User agents and scraping
A missing, outdated or obviously automated user agent is a common giveaway for scrapers. Rotating realistic user agents — and keeping them consistent with the rest of your browser fingerprint — helps requests blend in. A mismatch, such as a desktop user agent paired with a mobile fingerprint, raises red flags.
Examples
Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/120.0.0.0 Safari/537.36
Setting a realistic User-Agent header in a Python scraper
Common Use Cases
Frequently Asked Questions
Keep Learning
All termsWeb Scraping
Web scraping is the automated extraction of data from websites — fetching pages programmatically and parsing their content into structured data.
Read definitionHeadless Browser
A headless browser is a real browser running without a visible window. It loads pages and runs JavaScript exactly like Chrome does, but it is driven by code instead of a mouse and keyboard.
Read definitionBrowser Fingerprinting
Browser fingerprinting identifies you by combining dozens of details your browser reveals — screen size, fonts, GPU, timezone — into a signature unique enough to track you without any cookies.
Read definition