hn.today

Pepperwind, a fictional company dataset with 223 tables and 79M rows

news.ycombinator.com9 points0 comments
Screenshot of Pepperwind, a fictional company dataset with 223 tables and 79M rows

Pepperwind is a large synthetic dataset built around a fictional spice and tea trader based in Kochi, operating in India, Europe, and the United States. Across 223 tables, it contains 79 million rows, supplied as about 7 GB of Parquet files. Its systems cover retail, subscriptions, warehousing, multi-currency accounting, customer support, CRM, marketing analytics, HR, payroll, and company email. The project addresses a shortage of realistic test data for products that connect multiple business systems: records should refer consistently to the same customers and entities, foreign keys should work, and financial figures should reconcile. A browser preview lets users inspect part of the data and trace an individual customer across systems.

The dataset is designed to support testing and analysis, not merely provide realistic-looking records. The ledger balances, and 20 coordinated incidents are embedded across systems, including a Diwali courier failure, incorrectly charged Dutch VAT, referral fraud, and subscription churn following a price increase. It also includes monthly values for 92 KPIs and 420 questions users can ask of the data. Pepperwind was generated with Misata Studio’s backend; Misata is developing a from-scratch, multi-table synthetic-data generator, with an open-source Python library and a web tool for generating datasets from prompts. The full dataset is currently paid, while a preview is free; it is listed on Hugging Face under CC BY 4.0. Its creator is seeking feedback on potential applications and use cases.

Read on news.ycombinator.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Programming

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.