Metal Globe

Problem

Encyclopaedia Metallum has the data — nearly 200,000 bands — but its own site is a flat, unfilterable list. There is no way to see where a scene comes from, how a country's metal output compares to its neighbors, or how genres shift across regions without scrolling through hundreds of paginated tables.

Approach

A custom scraper over Encyclopaedia Metallum, plus a data pipeline and a framework-free visualization built to survive both a slow connection and a demanding 3D scene at once.

Metal Archives has no public API, and as of this build its pages sit behind a Cloudflare Turnstile challenge that blocks requests and headless browsers alike. The working path was Scrapling on top of Camoufox (a stealth Firefox build): a human solves the challenge once, and that same warmed-up browser session then sweeps every country page without being challenged again — a new browser launch gets a new fingerprint and gets blocked again, so the scrape runs as one continuous local session rather than a distributed job.

The scraped bands, genres and city-level metadata feed a Python pipeline that resolves genre taxonomy, geocodes countries, and assembles a single JSON the frontend consumes. The globe itself is hand-written WebGL through three.js: real TopoJSON borders projected onto a sphere, per-city pins, and level-of-detail that swaps in named countries and treemap panels as the camera zooms from planetary to continental to country view — no chart or globe library beyond three.js and ECharts for the side panels.

The original 4,408-line index.html shipped everything — markup, styles and app logic — in one file. It now splits cleanly into index.html, styles.css, data.js and app.js, with flags lazy-loaded per country instead of bundled up front.