StudentBench is a large-scale evaluation platform that measures whether large language models can produce real learning gains on GRE-style Quantitative and Verbal material. Researchers collected over 175,000 student-AI message exchanges and ran a randomized comparison with 2,383 participants assigned to AI tutoring, expert human tutoring, or no tutoring. Measured gains show AI tutoring is statistically equivalent to expert human tutoring overall (p = .015), and in five of seven GRE domains the best AI tutor outperformed the human tutor on average. One AI tutor in the study produced equivalent learning gains at p = .044 while costing roughly 918 times less per percentage-point gain (USD 0.0052 vs USD 4.81).
A second study had expert human tutors perform 2,028 pairwise rubric evaluations of LLM-generated lesson plans and practice problems, separating AI tutors across lesson planning, practice-problem creation, conversational pedagogy, cost, and engagement. For Quantitative sessions, faster AI reply times correlated with more student messages, more correct practice, and larger gains, linking conversational dynamics to outcomes. Code, data, and the StudentBench platform are publicly available, demonstrating that scalable LLM tutoring can match expert human instruction on standardized-test tasks while offering dramatic cost and throughput advantages.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
AI shopping agents can browse websites, compare products, and make purchases on behalf of users, promising convenience and accessibility. However, their trustworthiness raises concerns about security and privacy risks. (f-secure.com)
Allan Riordan Boll developed a Jev-like wrapper that enables querying both text and vision models by reading token probabilities, allowing flexible and quick assessments. The approach supports real-time webcam analysis by sending images as base64-encoded attachments and retrieving model responses, including visual scene understanding. (allanrbo.blogspot.com)
Large language models (LLMs) are transforming software development, prompting questions about the source of truth in coding. LLMs can work with both high-level frameworks like Rails and low-level code like C, raising issues of trust and editing scope. (intertwingly.net)
A new font design ensures that every token generated by large language models has the same visual width. This approach aims to improve readability and consistency in text display for AI-generated content. (ampdot.mesh.host)
Tesla workers are resisting training the Optimus humanoid robots as replacements, citing difficulties with complex robot hands and job security concerns. Tesla aims to produce 1,000 Optimus robots weekly by the end of 2026 despite these challenges. (arstechnica.com)
OpenAI is experiencing issues with Codex, with root cause identification underway. Most systems, including chat completions and fine-tuning, maintain near 100% uptime, but some components are affected. (status.openai.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.