The Customer: Who they are
A mid-size B2B software company wanting to embed a GenAI assistant into its core product to help customers query their own data in natural language.
The Challenge: Identifying the villain
Early prototypes hallucinated answers because they queried inconsistent, ungoverned customer data models — every customer’s schema was slightly different, and the LLM had no reliable ground truth to work from. “The demo looked magic until a real customer asked a real question,” said the VP of Product.
The Journey: The search for a solution
The company’s small internal data team tried prompt-engineering their way around the data quality problem, but no amount of prompt tuning fixed answers built on an unreliable schema.
The Discovery: Finding IDS
The company found IDS through the “GenAI is only as good as the data beneath it” positioning in an IDS webinar and reached out specifically for the data foundation work, not just the AI layer.
The Solution: The hero arrives
IDS built a governed semantic layer on Delta Lake that normalized customer data models into a consistent structure the GenAI assistant could query reliably, then supported the internal team in standing up its own data science capability to maintain and extend it.
The Implementation: The battle
The rollout started with a single customer segment to validate accuracy before general availability, with a human-in-the-loop review step for early responses. The obstacle: balancing response latency against accuracy — IDS tuned the retrieval layer to keep answers fast without sacrificing reliability.
The Results: The happy ending
- Hallucination rate: reduced from a double-digit rate in prototype to under 2%
- Feature adoption: 60%+ of active customers within the first quarter of GA
- Internal data science team stood up and self-sufficient post-engagement
- Time to add a new customer schema to the assistant: weeks → days
“We finally trust what it tells our customers.” — VP of Product
Technologies used: Python · Databricks · Delta Lake · Azure ML