GlossaryWeb ScrapingBeginner

User Agent

A user agent is the identifying string a browser sends with every request, telling the server which browser, version and operating system you are using.

Last updated May 28, 2026

Definition

A user agent is a string your browser includes in the headers of every HTTP request, describing the browser, its version, the rendering engine and your operating system. Servers use it to tailor responses and, importantly, to detect bots.

User agents and scraping

A missing, outdated or obviously automated user agent is a common giveaway for scrapers. Rotating realistic user agents — and keeping them consistent with the rest of your browser fingerprint — helps requests blend in. A mismatch, such as a desktop user agent paired with a mobile fingerprint, raises red flags.

Examples

1

Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/120.0.0.0 Safari/537.36

2

Setting a realistic User-Agent header in a Python scraper

Common Use Cases

Avoiding bot detection while scraping
Serving device-appropriate content
Browser analytics and statistics

Frequently Asked Questions

A single, default or bot-like user agent is easy to flag. Rotating realistic strings helps requests resemble genuine browsers.
Yes. A user agent that contradicts other fingerprint signals — like platform or screen size — is a strong bot indicator.

Keep Learning

All terms

Browser Fingerprinting

Browser fingerprinting identifies and tracks a device by combining dozens of browser and system attributes — like fonts, canvas rendering and user agent — into a near-unique signature.

Read definition

Web Scraping

Web scraping is the automated extraction of data from websites — fetching pages programmatically and parsing their content into structured data.

Read definition

Headless Browser

A headless browser is a real browser that runs without a visible interface, controlled by code — the workhorse for scraping JavaScript-heavy sites and automation.

Read definition
Back to Glossary