AI-Assisted Semantic Model Generation for Prompt-to-SQL Applications

Abstract

A novel approach for automatically generating semantic models that enable large language models to accurately translate natural language queries into SQL statements. This approach addresses fundamental scalability limitations of manual semantic model and prompt creation by mining database query logs to extract usage patterns, analyzing table metadata to identify facts and dimensions, integrating data catalog information, and encoding all discovered relationships in a knowledge graph (e.g., in Neo4j). A custom processor queries the graph to automatically generate semantic model specifications that are eighty to ninety percent complete, with human review workflows providing final validation. The approach demonstrates clear differentiation through its integration of query pattern mining with knowledge graph reasoning, enabling automatic maintenance, SQL query scoring, and cross-database semantic reconciliation beyond basic view generation. Topics that this talk will cover:

1. What are semantic models and how they enable Prompt->SQL application in analytical databases and BI Tools
2. Scaling challenges with handcrafting semantic models
3. An approach to automate semantic model creation using metadata knowledge graph
4. Generating semantic models from the metadata knowledge graph
5. Additional application of the metadata knowledge graph

What will attendees learn:

1. Important for semantic models and how GenAI uses semantic models for translating natural language question to SQL
2. Understand the challenges with creating and maintaining semantic models
3. How to tailor metadata store to enable automatic creation and updating of semantic models
4. How to approach automatic creation of semantic models

Key audience: Business Users, Data Architects, Data Platform Engineers