7 AI Agents Tasked with Classifying and Correcting Bitcoin Flaws: What Did They Find?

• The models Qwen 3.8, GPT 5.6, and Fable failed: one got stuck and two refused to work.
• 4 out of the 7 tested models are open-source and can be used without filters.

Paul Miller, a developer who maintains cryptography libraries used in Bitcoin and Ethereum projects, tested 7 artificial intelligence (AI) agents to classify and correct 5 real security vulnerabilities reported by the Bitcoin Red Team. The results showed marked differences between the models, both in the dollar cost incurred to run each one and in the accuracy of their diagnoses.

Of the 7 agents evaluated, 3 did not complete the task. Qwen 3.8 from Alibaba got stuck during the process, with no reason stated by Miller. GPT-5.6 Sol from OpenAI and Fable from Anthropic outright refused to work on the code, according to Miller himself, despite the task being to identify and fix flaws, not to exploit them.

The 4 agents that did complete the work, Grok 4.6 (xAI), Kimi K3 (Moonshot AI), DeepSeek V4 Pro, and DeepSeek V4 Flash (DeepSeek), had very different dollar costs for token consumption (user instructions) when executed:

• Grok 4.6 cost $4.70 in computing consumption.
• DeepSeek V4 Pro cost $0.30 for the same task, a difference of 15.7 times (almost 1500%) between the most expensive and the cheapest model to run.