Study evaluates Oracle SELECT AI: 95.5% NL2SQL accuracy, 3,906ms end-to-end latency

[TechRxiv] Sanjay Kumar Mishra has posted a preprint presenting an independent empirical evaluation of Oracle Autonomous Database 26AI's SELECT AI feature, which uses OpenAI GPT-4 to convert natural language into executable SQL. According to the paper, the author used a custom evaluation framework and 22 TPC-H benchmark queries spanning simple, medium and complex difficulty tiers to measure SQL generation accuracy and end-to-end latency. The results showed overall accuracy of 95.5%, including 90.9% exact SQL match, while mean end-to-end latency was 3,906ms, of which 94.5% was attributable to LLM generation overhead. The paper said the model demonstrated intelligent query optimization, selecting targeted columns over broad COUNT(*) patterns, a behavior not reflected in exact-match metrics. The author said the findings establish a reproducible baseline for enterprise NL2SQL deployment and identify LLM generation as the primary optimization target.