hn.today

Meta's Muse is fantastic for web scraping

sigh.dev59 points74 comments
Screenshot of Meta's Muse is fantastic for web scraping

This describes using Meta’s Muse not as a personal assistant but as a brute-force web-scraping workhorse to find and vet concert videos for a side project called fullsets.fm, which aggregates professionally recorded performances and extracts tracklists. The scraper pipeline uses Luna and Gemini 3.8 (via Gemini Studio) to cheaply assess audio/video quality and content, but discovery across many small YouTube channels and Reddit required a different tool. Muse can be instructed to scan thousands of videos or iterate artist-by-artist, running many parallel browser processes on a managed VM with Chrome to hunt down rare uploads and produce candidate lists; it also helps with artist matching, reviewing Luna/Gemini outputs, and final accept/reject decisions.

Practical findings: a month of Muse cost $80 and the system consumed billions of tokens per week, often seeming not to count usage; Muse Spark 1.3 is mediocre at general writing and coding but handles this large-scale, Gemini-Flash-style workload well, effectively offering near-unlimited tokens and cloud CPU/bandwidth. Downsides include job freezes, clunky multi-job UI, occasional frustrating model communication, and an obvious abuse vector that has already led to blocks (Amazon), raising concerns about future site defenses. A referral code (69WSDQ) was offered for signup bonuses.

Read on sigh.dev74 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Security

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.