hn.today

Hermes and REA and GLM 5.3 = Reverse engineering TikTok app for less than $20

scraping.club19 points15 comments
Screenshot of Hermes and REA and GLM 5.3 = Reverse engineering TikTok app for less than $20

A 5.6-billion-video TikTok dataset prompted an attempt to reproduce its metadata collection method without paying $1,699 for a detailed guide or relying on phones and emulators at scale. The work used GLM 5.3 through Hermes Agent, alongside Reverse Engineer Anything tools for examining Android code and native libraries, plus Frida and mitmproxy for runtime inspection. TikTok’s private API proved unusually restrictive: successful requests require a recognized device identity, a fresh signature over the exact request bytes, the correct regional endpoint, and a phone-like network fingerprint. Failures can still return HTTP 200, so response-body length - not status alone - serves as the practical test of success.

The investigation used Android decompilation to map the signing components, then found that request signing occurs entirely within native networking code, beyond the obvious Java hook point. Certificate pinning also blocked traffic inspection; the successful workaround preserved the certificate-checking process while changing its final rejection decision. Because signatures cannot be reused for altered requests, the team ran TikTok’s own native signing library outside the app with unidbg, producing fresh signatures without a phone. It also registered and validated new device identities, then combined a pool of identities, response checks, and rotating residential IPs. The result was a headless system that collected public video metadata at a steady rate. The repository contains supporting modules, but deliberately excludes code that directly contacts TikTok.

Read on scraping.club15 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.