Building an AI data analyst
A chat-with-your-data tool lets a non-technical user upload a spreadsheet and ask questions in plain English — “which region grew fastest last quarter?” — and get back a real answer computed from the data. The key insight is that the LLM does not analyse the numbers itself; it writes code that a sandbox executes against the real dataframe. The model handles language and logic; the runtime handles arithmetic. This guide covers the pipeline, and the prototyper below turns a question plus a schema into the analysis prompt your backend would send.
How the pipeline works
There are four stages. Parse reads the upload into a dataframe and extracts a schema — column names, inferred types, and a few sample rows. Plan sends the user’s question and that schema (not the raw data) to the model, which returns pandas or SQL code. Execute runs the generated code in an isolated sandbox with a timeout, memory cap, and no network access. Explain feeds the real execution output back to the model so it can summarise the result in plain English and, if asked, render a chart.
Keeping the data out of the prompt is what makes this scale: the model needs only the shape of the data to write correct code, so files with millions of rows still produce a tiny, cheap prompt.
Tips and safety notes
Never execute model-written code in your main process — always sandbox it with resource limits and no secret access. Show the generated code next to the answer so users can verify the logic. Render the numbers the code actually produced; do not let the model freehand figures. Cap the schema sample to a handful of rows to protect privacy and save tokens. Finally, validate that the generated code only touches the uploaded dataframe before you run it.