MCP server for OpenAI
Connecting models to Federal Reserve data
Led a small team that built a Python MCP server for OpenAI in one month. It gives AI assistants access to Federal Reserve economic data, with hundreds of evaluation cases checking tool use, accuracy, and how the assistant handles unclear questions and errors.
Problem
OpenAI needed its models to find and analyze data from FRED, the Federal Reserve economic database. The assistant had to choose the right data, recognize incomplete results, and explain what the numbers mean.
My part
I led the team and worked on tool descriptions, input limits, and evaluations. We built tools to find economic data series, compare them, and examine revisions. Our tests inspect which tools the assistant calls and what it asks for. Another model grades its explanations. We also test unclear questions, requests the server cannot answer, and attempts to bypass instructions.
Result
We shipped in one month. The project has 389 evaluation cases, including 299 that test the assistant, plus logs and dashboards to investigate failures. The tests showed where the assistant struggled to decide whether to answer or ask a question.
