How I test new AI models
Three tools I built to test new AI models on the work I actually do.
When a new AI model comes out, I want to know if it can do my real work. So I built three tools to test them.
CircuitBench saves every chat turn so any turn can become a test case later. In one run, GLM-5.3-flash passed 83 percent of the tests and hermes-4-70b passed 17 percent.
Model-eval-kit runs coding tasks with hidden tests, long-context checks, and a speed audit. I used it on a free, unnamed preview model and DeepSeek. Both scored 128 out of 128 on coding, so that suite could not tell them apart. The preview model found every hidden note up to 372,608 tokens. My first speed numbers were wrong because of a timing bug, and the report says so.
Model-bench is a blind test of three low-cost models on the tasks my agent does, like web pages, 3D, ffmpeg, and writing. It is not finished yet.
- ToolsCircuitBench, model-eval-kit, model-bench
