OpenAI has put numbers on a long-standing complaint about AI coding tools: they are much weaker in old or niche languages than in Python or JavaScript. In a post dated October 19, 2026, the company introduced two evaluations built around languages such as COBOL, Fortran, OCaml and Visual Basic, released one of them as an open benchmark on GitHub, and reported that its current models score far higher on both than the GPT-5.6 Sol generation did.
The first test, MultiLangBench, asks a model to write a database engine compatible with SQLite in a chosen language and then grades it with tests derived from SQLite's own suite. Because every language has to produce the same artifact, a modified database file, the same checks can be applied whether the engine was written in Rust or COBOL. According to the repository's README, the target is SQLite 3.40.1, the model receives the official documentation inside an offline container from which every copy of SQLite has been removed, and about 24,000 of roughly 51,000 TCL tests could be converted into this format. Scores are reported as a pair: API cost and test pass rate. To keep runs affordable, the README tells evaluators to use a fixed subset of 272 tests unless they state otherwise, and it says a solution written in the wrong language counts as zero.
On that benchmark, OpenAI says GPT-6 Astra at maximum reasoning effort averages an 87.4% pass rate across 20 languages, compared with 26.8% for GPT-5.6 Sol, and clears 83% in 19 of them. COBOL is the exception at 72.1%, although that is up from 10.9%. Fortran rises from 21.4% to 83.5% and OCaml from 25.6% to 85.8%. The cheaper GPT-6 Luna averages 40.3%, a smaller jump, but OpenAI estimates its cost at $0.77 per sample against $3.48 for GPT-5.6 Sol, roughly 78% less.
The second evaluation is closer to what banks and insurers actually pay for. Models must translate COBOL programs into C++, Java, Kotlin, Python or Rust, and the result passes only if hidden tests show identical observable output, byte for byte. OpenAI gives an example of why that bar matters: COBOL's fixed-point arithmetic drops surplus decimal digits unless rounding is requested, so a port that quietly rounds can run cleanly and still change financial results. Here the company reports that GPT-6.1 Sol reaches 86.6% at low reasoning effort, where GPT-5.6 Sol managed 5.4%, at about 61% of the older model's API cost. With more reasoning, accuracy climbs to 97.9%, which OpenAI describes as close to saturating the test.
The motivation is commercial as much as technical. OpenAI cites estimates that COBOL underpins more than 40% of online banking systems and most ATM transactions, and notes that Fortran still runs in weather forecasting and reactor safety analysis. Much of that code is poorly documented and maintained by shrinking teams, which is why migration projects are slow and risky. The company says customer feedback about models struggling in these languages shaped the work.
There are clear limits to what the post shows. Both the benchmarks and the results come from OpenAI, and no independent replication was available at publication. The two evaluations compare different models at different reasoning settings, so the figures cannot be lined up against each other. The COBOL migration suite was not published alongside MultiLangBench, and at the time of checking the public repository held the test setup and three commits but no per-language result files, even though the post says results are being released. A self-contained database engine is also a narrower task than maintaining a sprawling enterprise codebase with external dependencies.
OpenAI itself cautions that production migrations still need expert review and testing against each system's business rules. The open MultiLangBench suite at least gives other labs and customers a way to check the claims on their own models.