From Engineer to AI Orchestrator: Building Evaluation-Driven Agentic Systems — A CodinGame Case Study
How do you move from manually developing AI solutions to building systems that continuously improve them?
In this talk, I will share my transition from traditional software engineering and manual game bot development to orchestrating 40+ parallel, evaluation-driven AI pipelines — an approach that helped me reach #1 in the global CodinGame ranking.
Using CodinGame as a practical testing ground, I will explain how autonomous experimentation loops, objective evaluation, and multi-agent orchestration work together to improve solutions across very different problems.
I will cover the architecture of the system, the division of responsibilities between LLM agents and deterministic tools, and practical challenges involving reliability, CPU resources, token budgets, and human oversight.
The key takeaway is a broader engineering approach: moving beyond AI-assisted coding toward building scalable, measurable systems that can independently experiment, learn from feedback, and progressively improve their results.
Bio
https://www.linkedin.com/in/daniel-gorski