GOOPEN-SOURCEWEB

I Built a User-Agent Parser for Go (Because the Existing Ones Weren't Cutting It)

2026年4月4日·6 min read

For a while now, the community around nginx-clickhouse — a service that pipes Nginx logs into ClickHouse — had been asking for User-Agent parsing. People wanted to break down their traffic by browser, OS, device type, and especially bots, right inside ClickHouse. Fair ask. Having run a web analytics startup in a past life, this is something I've thought about more than most people probably should.

So the search began for a Go library that could handle it. The results were disappointing. Some were painfully slow. Others hadn't seen a commit in years. Most had zero awareness of modern AI crawlers like GPTBot or ClaudeBot — and if you're not tracking those in 2026, you're missing a growing chunk of your traffic. None of them handled Client Hints either, which is a problem because Chromium browsers have been phasing out useful information from the UA string for a while now.

Nothing out there fit the bill, so I built go-ua-parser.

What one parse call gives you

r := uax.Parse("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/123.0.6312.86 Safari/537.36")
 
r.Browser.Name    // "Chrome"
r.Browser.Version // "123.0.6312.86"
r.Browser.Family  // "Chromium"
r.Engine.Name     // "Blink"
r.OS.Name         // "Windows"
r.OS.Version      // "10"
r.DeviceClass()   // "desktop"
r.IsBot           // false

You get the full picture — browser (name, version, family, release channel), rendering engine, OS with proper Windows 10/11 detection, device type with helpers like IsPhone() and IsTV(), in-app browser detection for 50+ apps (Facebook, TikTok, WeChat, etc.), and bot classification with confidence scores.

The AI crawler problem

This was the itch that really pushed me to build this. GPTBot, ChatGPT-User, ClaudeBot, DeepSeekBot, Bytespider, PerplexityBot, xAI-Bot — these are showing up in production logs today, and most tooling just lumps them into "unknown." That's not good enough when you need to decide whether to block, throttle, or serve different content to them.

The library ships with 313 named bot rules across 7 classes:

ClassExamples
SearchGooglebot, Bingbot, YandexBot, Baiduspider, DuckDuckBot
SocialTwitterbot, facebookexternalhit, LinkedInBot, Slackbot, Discordbot
AIGPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, DeepSeekBot, Bytespider, xAI-Bot
SEOAhrefsBot, SemrushBot, MJ12bot, Screaming Frog
MonitorUptimeRobot, Pingdom, Datadog, NewRelic
Scrapercurl, wget, python-requests, Scrapy, HeadlessChrome, Puppeteer, Shodan
FeedFeedly, Feedbin, NewsBlur

There are also 14 heuristic patterns that catch unnamed bots — anything with "bot", "crawler", "spider" in its string gets flagged automatically. In practice, this covers the long tail surprisingly well.

bot := parser.DetectBot(ua)
if bot.IsBot && bot.Class == "ai" {
    fmt.Printf("AI crawler detected: %s by %s\n", bot.Name, bot.Vendor)
    // "AI crawler detected: GPTBot by OpenAI"
}

Making it fast enough to not think about

Since this runs on every request in nginx-clickhouse, performance was non-negotiable. I didn't want to be the bottleneck in a log processing pipeline that handles thousands of events per second.

The architecture ended up as a three-stage pipeline: a zero-alloc tokenizer splits the UA into product tokens, a hybrid trie + regex matcher runs 585 rules against them, and an assembler merges in Client Hints and computes derived fields. I went through a few iterations on the matching stage before landing on the trie approach — pure regex was readable but too slow, and a hand-rolled state machine was fast but impossible to maintain.

Benchmarks on an Apple M3 Pro:

Scenarions/opallocs
curl (short bot)2840
Googlebot1,1212
Chrome Desktop1,8525
Safari Mobile2,5297
LRU cache hit320
Sharded cache (contended)820

The cache is where it really shines. In production, most traffic comes from a handful of distinct UA strings, so your hit rate ends up being very high. A cached lookup takes 32ns with zero allocations — basically free. The sharded cache variant (16 shards) handles concurrent access at 82ns, which matters if you're running this in hot middleware.

For paths where even a single allocation per request is too much, there's ParseInto:

var result uax.Result
parser.ParseRequestInto(req, &result)
// Reuses your Result struct — no heap allocation

Client Hints support

This is something that caught me off guard when I first dug into it. Modern Chromium browsers are actively reducing what they include in the User-Agent string. Chrome already sends a "frozen" UA that makes it impossible to distinguish Windows 10 from Windows 11, or to get the actual browser version, without reading Sec-CH-UA-* headers.

Most parsers just ignore this. go-ua-parser reads both:

result := parser.ParseRequest(httpRequest)
 
result.OS.Version      // "11" — from Sec-CH-UA-Platform-Version, not the frozen UA
result.Browser.Version // actual version, not the reduced one

If the headers are there, they take priority. If not, you get the best result from the UA string alone. No config needed.

Batteries included

I wanted this to be useful out of the box, not just a parsing engine that requires a bunch of glue code. The repo includes:

parser, _ := uax.NewParser(
    uax.WithCustomBotRules([]uax.BotRule{
        {Token: "InternalBot/1.0", Name: "Internal Monitoring", Class: uax.BotMonitor, Vendor: "MyCompany", Match: "exact"},
    }),
)

All 585 builtin rules live in YAML files and get compiled to Go source at build time. Easy to read, easy to contribute to.

Try it

go get github.com/motiv8-team/go-ua-parser

Pure Go, zero external dependencies, MIT licensed. If you're dealing with UA parsing in Go and the existing options aren't doing it for you, give this a shot — it's what I wish existed when I went looking.

The bot landscape, browser list, and in-app WebView zoo are constantly evolving. All the detection rules live in plain YAML files, so adding a new bot or browser is just a PR away. If you're seeing traffic from something go-ua-parser doesn't recognize yet — open an issue or send a pull request. The more eyes on real-world traffic, the better the library gets for everyone.

github.com/motiv8-team/go-ua-parser