LinkedIn·Wednesday, 5 August 2026·21d ago

Every marketing platform now offers to answer your questions in plain language. GA4, Looker, Power BI, your CDP. Ask, and receive. MIT put…

MultiMinds

2,503 followers

Every marketing platform now offers to answer your questions in plain language. GA4, Looker, Power BI, your CDP. Ask, and receive. MIT put that promise on a scale, and the result deserves more attention than it's getting. BEAVER is a benchmark built from three real private enterprise data warehouses. Not a demo schema: 9,128 questions taken from actual query logs, across databases averaging 101 tables and 869 columns. The agentic framework running GPT-5.2 answers 10.8% of them correctly. Then the researchers handed that same system the answers to the hard part. Which tables. Which join keys. Which columns map to which business concept. Which domain rules apply. Accuracy went to 30.1%. If you work with marketing data, that should feel familiar, because our data is the most semantically ambiguous in the business. "Session" means one thing in GA4 and another in your CRM. "Conversion" depends on an attribution window somebody picked in 2023. "Channel" depends on whether anyone ever governed UTMs. "Revenue" rarely matches finance. And the same customer sits as four unresolved records across web, email, app, and offline. Ask an assistant which campaigns drove revenue last quarter against that, and it will answer. Confidently. The query runs, the number looks plausible, and nobody can check it. The model you pick is the least important decision here, and it's the only one your vendor wants to discuss. What matters is resolved identities, collection you can trust, definitions that marketing and finance both signed, and documentation that survives the person who wrote it. And to be fair, even with every hint handed over, it still got seven out of ten questions wrong. Fixing the foundation doesn't make an AI analyst accurate. It makes it possible. You still need a human expert in the loop. Which metric does your organisation define three different ways? We'll start: "active customer."