Pivot is a high-performance analytical engine that queries open data formats (Delta Lake, Apache Iceberg, Parquet) directly from object storage without copying data into proprietary systems. It exposes a Postgres-compatible server for BI tools and apps, plus a standalone CLI mode for ad hoc queries and AI agents, and handles ingestion, compaction, and statistics to keep tables ready for sub-second SQL. Because object storage holds the data and the service only stores minimal state, instances can scale up, down, or to zero almost instantly while multiple clients share the same source-of-truth files. Pivot emphasizes portability and openness: the same Parquet/Delta/Iceberg files remain queryable by Snowflake, Spark, DuckDB, Trino and others.
Benchmarks claim substantially faster performance than popular engines: across ClickBench (43 queries), TPC-H SF1000 and Star Schema SF1000 runs on AWS c8gd.metal-48xl, Pivot is used as the 1.0x baseline while DuckDB, ClickHouse, DataFusion and Spark show 2-44× slower results depending on workload (examples include ClickBench showing Pivot ~1.09× vs ClickHouse ~2.17× and Star Schema showing Pivot 1.00× vs DuckDB Parquet ~2.58×). The benchmark harness and queries are published for reproducibility. The product also offers a managed Delta Lake option and is open source, aiming to deliver low-latency, high-concurrency analytics on open formats without data duplication.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
OpenAI fired three safety researchers for allegedly mishandling sensitive information, but they dispute the claims and warn it creates a chilling effect on internal communication. The researchers argue that their dismissal signals a shift away from an open safety culture at the company. (techcrunch.com)
OpenTelemetry introduces native design principles that enable products to export telemetry data to any observability stack seamlessly. The update emphasizes compatibility, scalability, and new features across various SDKs and agents to improve observability infrastructure. (opentelemetry.io)
Users discuss the advantages of subscribing to hosted LLM services versus deploying open-source models locally. They share reasons related to cost, convenience, and customization options. (news.ycombinator.com)
Four coding agents were given a $100 budget to build a functional PDF editor, but their implementations had noticeable bugs and shortcomings. The experiment highlights that current AI agents struggle with goal pursuit and user interaction, leading to subpar software quality. (blog.nielstron.de)
A new self-pruning transformer model achieves 10 times KV-cache compression while maintaining or improving performance in language tasks. It uses a universal attention mechanism that adaptively prunes tokens, enabling better long-context processing with up to 25 times compression at 16k length. (arxiv.org)
OpenAI released a new AI model that has elicited strong reactions from mathematicians. The release has sparked both admiration and concern within the mathematical community. (nytimes.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.