GPT-6 Astra Hits 72% on COBOL Benchmark [2026]

OpenAI said on October 19, 2026 that its newest flagship model, GPT-6 Astra, had sharply closed a gap that has quietly embarrassed the entire AI coding industry: writing and translating code in older, unglamorous languages like COBOL and Fortran. In a post titled “Advancing coding in low-resource programming languages,” OpenAI reported that Astra reached an average 87.4 percent pass rate on a new 20-language benchmark, up from 26.8 percent for the prior GPT-5.6 Sol model. On COBOL specifically, the score jumped from 10.9 percent to 72.1 percent.

The announcement lands at a moment when AI coding tools are shipping new releases almost weekly, from OpenAI’s own Codex CLI to Anthropic’s Claude Code. But most of that race has focused on mainstream languages: Python, JavaScript, TypeScript, Go. COBOL and Fortran rarely show up in demo videos, even though they still run much of the world’s banking, insurance, and government infrastructure. OpenAI’s new numbers suggest the company is betting that modernizing decades-old code, not just writing new code faster, is where the next big enterprise budget line sits.

Google · Preferred Sources

Don't miss new tech stories on Google

Add TrendinTech once in the Google app and our stories appear in your news suggestions.

Add Now

What OpenAI Actually Announced on October 19

OpenAI’s announcement bundled two separate pieces of work. The first is MultiLangBench, a new open evaluation that asks a model to build a SQLite-compatible database engine entirely from scratch, once in each of 20 programming languages, including COBOL, Fortran, OCaml, and Visual Basic. The model’s output is then graded against tests derived from SQLite’s own test suite, so a passing score means the generated code actually behaves like a working database engine, not just code that looks plausible. OpenAI said it was releasing the evaluation setup, the test suite, and the full per-language results on GitHub.

The second piece is a separate, narrower evaluation focused specifically on legacy-code migration. Rather than building something new, this test asks a model to take an existing COBOL program and translate it into another language, C++, Java, Kotlin, Python, or Rust, while preserving the program’s observable behavior. That distinction matters: a bank cannot simply rewrite a 40-year-old payroll system and hope the new version behaves the same way. It needs proof that outputs match, line for line, transaction for transaction.

Coverage of the announcement from outlets including BleepingComputer and Thurrott framed the release as OpenAI directly targeting the mainframe-modernization market, an area long dominated by consulting firms and specialist vendors rather than AI labs.

Why Legacy Languages Suddenly Matter to AI Labs

For most of the current AI coding boom, benchmark bragging rights have centered on languages that dominate GitHub: Python, JavaScript, TypeScript, Go, and Rust. That makes sense commercially, since most new software is written in those languages. But it also means large language models have historically seen far less COBOL, Fortran, or Visual Basic in their training data, simply because far less of it exists in public repositories. OpenAI’s own figures illustrate the gap starkly: GPT-5.6 Sol, a capable model by most measures, managed only 10.9 percent on COBOL and 21.4 percent on Fortran, even as it handled far better on more commonly represented languages.

COBOL dates back to 1959 and Fortran to 1957, making them among the oldest programming languages still in active commercial use. Decades of patchwork maintenance, departing specialists, and undocumented business logic have turned COBOL systems in particular into a well-known liability for banks, insurers, and government agencies that still depend on them but increasingly struggle to find engineers willing or able to maintain them. That is the backdrop against which OpenAI chose to frame Astra’s gains, not as a benchmark curiosity, but as a direct answer to a staffing and modernization problem that has been building for years.

Inside MultiLangBench: Building a Database From Scratch

MultiLangBench’s design choice is deliberately hard to game. Instead of asking a model to complete a short function or fix a known bug, as many coding benchmarks do, it requires constructing an entire SQLite-compatible database engine, a project that in human terms would normally take an experienced engineer weeks. Doing that in COBOL or Fortran, languages with limited modern tooling, no mainstream package ecosystems, and unusual memory and data-handling models compared with languages like Python, requires genuine language fluency rather than pattern-matching against common snippets.

Because OpenAI released the evaluation setup and test suite publicly on GitHub, outside researchers and rival labs can run their own models against the same 20-language suite and compare results directly, rather than relying on self-reported numbers. That transparency is notable in a field where benchmark claims are frequently disputed. It also raises the bar for competitors: if MultiLangBench becomes a standard reference point, any lab claiming strong “general coding” performance will likely be asked how it performs on the low-resource end of that same test.

GPT-6 Astra’s Score Card, Language by Language

The headline 87.4 percent average score hides considerable variation between languages. OpenAI’s own published breakdown shows that Astra exceeded an 83 percent pass rate in 19 of the 20 tested languages, with COBOL the lone outlier still trailing the rest of the pack even after its large improvement. The table below lays out the four specific language results OpenAI highlighted alongside the overall 20-language average.

Language or metric	GPT-6 Astra pass rate	GPT-5.6 Sol pass rate	Improvement
COBOL	72.1%	10.9%	+61.2 points
Fortran	83.5%	21.4%	+62.1 points
OCaml	85.8%	25.6%	+60.2 points
Visual Basic	89.4%	27.5%	+61.9 points
Average across 20 languages	87.4%	26.8%	+60.6 points

The consistency of the gains, all four highlighted languages improved by roughly 60 to 62 percentage points, suggests the improvement is coming from something systemic in how Astra was trained or fine-tuned for code, rather than from narrow, language-specific tuning. OpenAI did not detail the exact technique behind the jump in its public post, so the mechanism itself remains the company’s own description rather than an independently verified architectural claim.

The Hardest Language: Why COBOL Still Lags Behind

Even with its dramatic jump, COBOL remained the single hardest language in OpenAI’s 20-language test, the only one where Astra fell short of the 83 percent threshold the other 19 languages cleared. That is a meaningful detail for anyone evaluating whether this announcement means COBOL modernization is now a solved problem. It is not. A 72.1 percent pass rate on a from-scratch database-building exercise means roughly one in four attempts still fails to produce fully correct, working code, a gap that matters enormously when the system in question processes real financial transactions.

Part of the difficulty likely comes from COBOL’s verbose, fixed-format syntax and its unusual data-division structure, which differs sharply from the C-family syntax that dominates most training data available to large language models. Fortran, by contrast, shares more superficial structural similarities with newer procedural languages, which may partly explain why it scored noticeably higher than COBOL despite being roughly as old.

A Second Test: Can AI Migrate, Not Just Write, Legacy Code?

Writing new COBOL code from scratch is one challenge. Translating an existing, decades-old COBOL program into a modern language while keeping its behavior identical is a different and arguably more commercially relevant one, since most enterprises do not want new COBOL, they want out of COBOL entirely. OpenAI’s separate migration evaluation targets exactly that scenario, checking whether a translated program produces the same observable outputs as the original. Additional reporting on the release noted that OpenAI frames this migration capability as central to the practical use case it expects enterprises to pursue, rather than treating COBOL fluency as an end in itself.

The table below breaks down the five target languages OpenAI’s migration evaluation translates COBOL into, and why each one tends to appeal to a different type of enterprise buyer.

Migration target	Language category	Typical appeal for COBOL shops
C++	Compiled, high performance	Preserves mainframe-level speed for heavy batch processing
Java	Enterprise-standard, JVM-based	Matches skills already common on existing enterprise IT teams
Kotlin	Modern JVM language	Interoperates directly with existing Java-based systems
Python	Interpreted, widely taught	Eases hiring in a market short on COBOL specialists
Rust	Memory-safe systems language	Reduces memory-safety bugs introduced during modernization

Notably, OpenAI’s own material is careful to frame this as an evaluation result, not a finished migration product. There is no indication, in OpenAI’s announcement or in the coverage that followed, of a dedicated commercial COBOL-to-modern-language migration service with its own pricing or support contract. What exists today is a benchmark showing the underlying model has gotten substantially better at the task, which is a meaningfully different thing from a turnkey enterprise offering.

GPT-6 Astra vs. GPT-5.6 Sol: What the Comparison Tells Us

GPT-5.6 Sol, the model OpenAI used as its baseline for these comparisons, was itself a capable, previously released system. The scale of the jump to GPT-6 Astra, more than 60 percentage points on several individual languages, is large even by the standards of a field accustomed to rapid year-over-year model improvements. OpenAI positions Astra as its most capable current model for demanding reasoning and end-to-end professional work, available through ChatGPT, the OpenAI API, and inside Codex for software-engineering workflows.

It is worth separating two different kinds of claims here. The MultiLangBench scores themselves come directly from OpenAI’s own published post and are corroborated across multiple outlets that reviewed the same release. Other figures that have circulated around the same announcement, including specific migration-evaluation percentages tied to the related GPT-6.1 Sol model, have appeared mainly in secondary reporting rather than OpenAI’s primary announcement, and should be treated with more caution until OpenAI’s own documentation confirms them in full.

Where Astra Fits in OpenAI’s Broader GPT-6 Lineup

Astra sits at the top of a GPT-6 family that OpenAI has been rolling out in stages through October 2026. Alongside Astra, the company has introduced lighter, lower-cost variants intended for higher-volume or budget-sensitive workloads, positioned as faster and cheaper alternatives that still carry forward some of the same underlying coding improvements. OpenAI has described these lighter models as using techniques related to Astra’s training, though at a fraction of Astra’s processing cost, a split that mirrors how most frontier labs now ship a flagship model alongside cheaper siblings rather than a single undifferentiated product.

For enterprises evaluating whether to run large migration jobs through Astra specifically, that tiered lineup matters. A from-scratch, whole-codebase migration project involving millions of lines of COBOL is a fundamentally different cost proposition than a short code-completion task, and the choice between a flagship model and a cheaper sibling will likely come down to how much accuracy a given migration project can afford to sacrifice for lower per-token cost.

The Competitive Field: AI Coding Tools Are Multiplying

OpenAI’s legacy-language push lands in the middle of an unusually crowded period for AI coding tools generally. GitHub Copilot and Cursor have both been shipping frequent updates aimed at agentic, multi-file coding workflows, while Anthropic’s Claude Code has pushed out its own rapid release cadence. None of those competing tools have made comparable low-resource-language claims public to the same degree that OpenAI did on October 19, which gives OpenAI a distinct, if narrow, marketing angle: rather than competing purely on general coding speed or chat-style helpfulness, it is staking a claim on a specific, underserved technical niche that most rivals have largely ignored.

That said, niche leadership in COBOL and Fortran performance does not automatically translate into broader market share. Enterprise buyers evaluating AI coding tools weigh many factors beyond raw benchmark scores, including existing platform relationships, data-governance requirements, and integration with tools they already use. OpenAI’s existing enterprise partnerships, including an expanded relationship with Atlassian announced earlier in October 2026 that gives Atlassian broader access to GPT-6 Astra and GPT-5.6 models, suggest the company is trying to extend its enterprise footprint on multiple fronts at once, not relying on the legacy-language story alone.

Historical Context: From Autocomplete to Legacy-Code Translators

The trajectory of AI coding tools over the past few years has moved in fairly clear stages: first simple autocomplete suggestions, then full-function generation, then multi-file agentic editing capable of planning changes and running commands on its own. OpenAI’s low-resource-language push represents something closer to a fourth stage, aiming at code that most mainstream tooling has simply never targeted well, because the available training data was thin and the commercial upside looked small next to Python or JavaScript.

That shift has not been universally smooth. A widely discussed METR study found that experienced developers were actually about 19 percent slower when using AI coding agents on certain real-world tasks, a reminder that benchmark gains do not automatically translate into measured productivity gains in practice. Separately, surveys have found that AI now writes roughly half the code for a meaningful share of working developers, highlighting how central these tools have already become even before legacy-language performance catches up to mainstream-language performance.

What This Means for Enterprises Still Running COBOL

For IT leaders at banks, insurers, and government agencies still running COBOL systems, the practical takeaway from OpenAI’s announcement is cautious optimism rather than an immediate green light. A model that performs well on a from-scratch benchmark task is not the same thing as a model that has been proven safe to run against a production payroll system or a core banking ledger. The gap between “builds a working database engine in COBOL” and “safely migrates our 1.2-million-line claims-processing system” is still substantial, and OpenAI’s own framing acknowledges that the available evidence demonstrates improved AI coding capability rather than a fully automated, production-ready migration workflow.

What the announcement does change is the baseline expectation. Six months ago, a model scoring barely above 10 percent on COBOL tasks was not a realistic candidate for any serious modernization project. A model scoring north of 70 percent, even with real limitations, starts to look like a tool worth piloting on narrower, lower-risk portions of a legacy codebase, with the kind of behavior-preservation checks OpenAI’s own migration evaluation is designed to encourage.

Market and Business Impact: Chasing the Modernization Budget

Legacy-system modernization has historically been the domain of large consulting firms and specialist mainframe vendors, a slow and expensive category of IT spending that rarely makes technology headlines. OpenAI’s decision to publish benchmark results aimed squarely at that market signals an attempt to reposition AI coding tools, including Codex, which OpenAI has continued to expand through acquisitions and partnerships, as a credible alternative or complement to that traditional modernization industry.

If AI-assisted migration tools prove reliable enough for even a subset of real production workloads, the addressable market is large: COBOL alone still underpins a substantial share of core banking and government transaction processing worldwide. Even modest success at chipping away at that backlog, rather than replacing it wholesale, would represent meaningful new enterprise revenue for whichever AI vendor can demonstrate the most trustworthy results, which helps explain why OpenAI chose to make this specific announcement now rather than quietly improving the capability without publicity.

Risks and Limitations: Benchmarks Are Not Production Migrations

It is worth repeating the central caveat clearly, since it is easy to lose in headline benchmark numbers. MultiLangBench measures whether a model can build a new, self-contained SQLite-compatible database engine from scratch. That is strong evidence of general coding fluency in a given language, but it is not, by itself, proof that a model can safely migrate a specific, idiosyncratic, poorly documented legacy system that has been patched by dozens of different engineers over several decades. Real-world COBOL systems often contain undocumented edge cases, business logic embedded in comments or tribal knowledge, and dependencies on quirks of specific mainframe environments that a general benchmark simply cannot capture.

There is also a verification gap worth flagging for readers evaluating any vendor’s legacy-migration claims, OpenAI’s included: independent, third-party audits of production-scale migration accuracy are not yet common practice in this industry. Enterprises considering AI-assisted migration should expect to run extensive parallel testing against existing systems before trusting any AI-generated replacement with live financial or operational data.

Predictions: Where AI and Legacy Code Go From Here

Several trends seem likely to follow from this announcement over the coming months. First, expect rival AI labs to publish their own low-resource-language benchmark results in response, since OpenAI has effectively set a new comparison point that competitors will feel pressure to address rather than ignore. Second, expect more emphasis on behavior-preservation testing as a product feature in its own right, not just a research evaluation, as enterprises demand proof that migrated code matches original system behavior before they will deploy it.

Third, consulting firms and systems integrators that currently handle mainframe modernization work are likely to begin incorporating AI coding models like Astra into their own service offerings, rather than being displaced by them outright, since enterprise buyers still want an accountable vendor relationship around any migration of critical systems. Fourth, pricing is likely to keep splitting between flagship, high-accuracy models and cheaper, faster siblings aimed at lower-stakes migration tasks, mirroring the tiered approach OpenAI has already taken across its broader GPT-6 lineup. Finally, expect scrutiny of these benchmark claims to increase, with independent researchers and rival vendors testing MultiLangBench themselves now that OpenAI has published it openly, which could either reinforce or complicate the headline numbers over the next several months.

Frequently Asked Questions

What is GPT-6 Astra?
GPT-6 Astra is OpenAI’s newest flagship model, positioned by the company as its most capable system for demanding reasoning, professional work, and software engineering tasks, including inside Codex.

What is MultiLangBench?
MultiLangBench is a coding evaluation OpenAI released on October 19, 2026 that tests whether a model can build a SQLite-compatible database engine from scratch in each of 20 programming languages, including COBOL, Fortran, OCaml, and Visual Basic, with results checked against SQLite-derived functional tests.

How much did GPT-6 Astra improve on COBOL specifically?
Astra scored 72.1 percent on the COBOL portion of MultiLangBench, up from 10.9 percent for the previous GPT-5.6 Sol model, according to OpenAI’s own published results.

Does this mean AI can now fully migrate COBOL systems to modern languages?
No. MultiLangBench tests building new code from scratch, not migrating a specific real-world legacy system. OpenAI’s separate migration evaluation checks behavior-preserving translation into languages such as Java, Python, and Rust, but the available evidence shows improved capability, not a fully automated production migration workflow.

Why was COBOL still the hardest language for GPT-6 Astra?
Even after its large improvement, COBOL was the only one of the 20 tested languages where Astra scored below 83 percent, likely reflecting COBOL’s verbose, fixed-format syntax and limited representation in the kind of code typically available to train large language models.

Which languages does OpenAI’s COBOL migration evaluation target?
The evaluation tests translation of existing COBOL programs into C++, Java, Kotlin, Python, or Rust, while checking that the translated program preserves the original’s observable behavior.

Is MultiLangBench open to other companies to test their own models?
Yes. OpenAI said it released the evaluation setup, test suite, and per-language results publicly, which allows outside researchers and competing labs to run the same benchmark against their own models.

How does this compare to what Anthropic and GitHub are doing with their coding tools?
Anthropic’s Claude Code and GitHub Copilot have both focused primarily on general-purpose, multi-file agentic coding workflows rather than publishing comparable low-resource-language benchmark results, giving OpenAI a distinct angle on this specific technical niche as of this announcement.

GPT-6 Astra Jumps COBOL Coding Score to 72% [2026]