Astra improves coding efficiency without delivering a uniform capability gain. Artificial Analysis scores it 67 on its Coding Agent Index, but 61 on its Intelligence Index with a 75% higher task bill than GPT-5.6 Sol at max effort; Epoch separately records a new ECI high of 169, reflecting different workloads that cannot be treated as interchangeable.
GPT-6 Astra makes significant gains in the Artificial Analysis Coding Agent Index, scoring equal to Fable 5 at lower cost. In the Intelligence Index, it uses fewer tokens than GPT-5.6 Sol for similar performance, but this is outweighed by higher prices
Pricing is 2.5x GPT-5.6…
GPT-6 Astra makes significant gains in the Artificial Analysis Coding Agent Index, scoring equal to Fable 5 at lower cost. In the Intelligence Index, it uses fewer tokens than GPT-5.6 Sol for similar performance, but this is outweighed by higher prices
Pricing is 2.5x GPT-5.6 Sol’s current prices across the board, up from $4/$20 to $10/$50 per million input/output tokens, with the same 90% discount for cache reads and 25% premium for cache writes.
We see distinct stories across our two flagship Indices. In the Artificial Analysis Coding Agent Index, GPT-6 Astra equals Fable 5 at less than half the cost, driven by significant token efficiency gains. In the Artificial Analysis Intelligence Index, GPT-6 Astra is more token efficient than its predecessor for similar performance, but this is offset by the price increase.
Artificial Analysis Coding Agent Index - key takeaways:
➤ Rivals top models: In Codex, GPT-6 Astra scores 67 in the Index - approximately equal to Claude Opus 5 and Fable 5 in Claude Code, and Muse Spark 1.3 in Muse Code. Fable 5.1 in Claude Code leads the Index with a score of 70.
➤ 70% more token efficient than GPT-5.6 Sol: GPT-6 Astra sees a substantial improvement in token efficiency, using one third of the tokens compared to GPT-5.6 Sol (max) in the Codex harness, and one fifth of the tokens of Claude Opus 5 (xhigh). Various effort levels of the model occupy the Pareto frontier of token efficiency.
➤ Leads Coding Agent Index cost efficiency frontier: At max effort, GPT-6 Astra costs about the same as GPT-5.6 Sol (max) while scoring 2 points higher on the Index. Per task, the model is less than half the cost of Claude Fable 5, for the same score.
Artificial Analysis Intelligence Index - key takeaways:
➤ Sits beside GPT-5.6 Sol in Intelligence: GPT-6 Astra scores equal to GPT-5.6 Sol in the Index at 61. This is 5 points lower than Claude Fable 5.1 (max with fallback). The model also trails Meta’s newly released Muse Spark 1.3 (max).
➤ ~10% fewer output tokens, offset by price increase: GPT-6 Astra defines a new Pareto frontier for Intelligence Index vs Output Tokens per Task - with a ~10% reduction in token use at max effort compared to GPT-5.6 Sol. However, due to the 2.5x increase in price, the model is 75% more expensive per task than its predecessor at max effort.
➤ Hallucinates half as much as GPT-5.6 Sol: GPT-6 Astra sees a large jump in AA-Omniscience, our knowledge and hallucination benchmark. This is driven by a significant decrease in hallucination rate from 92% to 51% at max effort. Unlike some models, this improvement does not come at the cost of accuracy - Astra increased accuracy by 4 points at the same time.
➤ ~80 point gain in AA-Briefcase Elo: GPT-6 Astra improves ~80 points in AA-Briefcase, our frontier long-horizon knowledge work evaluation. Models are tested on multi-week projects, with many linked tasks and thousands of source files. Astra sees a significant increase in both rubric scores and Analytical Quality Elo in AA-Briefcase compared to its predecessor. In the other direction, we observe a reduction in Presentation Quality Elo, where GPT-5.6 Sol (max) still leads all models.
➤ Mixed progress on other evaluations: The model sees a 6 point gain in Humanity’s Last Exam, a long-standing evaluation with emphasis on mathematics, science, and humanities. This is offset by a drop of ~80 Elo points in GDPval-AA v2 - a benchmark we adapted from OpenAI’s dataset measuring economically valuable tasks across 44 occupations. We also observe 2-3 point regressions on other evaluations across a mix of capabilities, including reductions in τ³-Banking (customer support), SciCode (Python problems in a scientific domain), and AA-LCR (long context reasoning over large documents).
Congratulations @OpenAI and @sama on the launch!
GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our…
GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our long-horizon coding benchmark, MirrorCode, Astra ranks between Opus 4.7 and Fable 5.
OpenAI gave us pre-release access to test Astra. Charts and more details for Astra’s individual benchmark results in the thread.
Astra has also set the top score on our newest math benchmark, FrontierMath Erdős, which consists of 68 unsolved and especially interesting Erdős problems curated by @thomasfbloom. AI systems are tasked with writing solutions in Lean, a special-purpose programming language for automatically verifying mathematical proofs.
No prior model solved any of the problems, but Astra solved 2/68, scoring 3%. See our post introducing the benchmark for more details, including our open-source scaffold.
https://t.co/iJ7qrrg3UF
Astra achieved a raw score of 46.7% on MirrorCode, our long-horizon coding benchmark landing squarely between Opus 4.7 and Fable 5.
We ran some internal tests indicating that Astra might score higher on MirrorCode with greater reasoning effort. We only report the High-effort score here, as it's what we report for every other model, and we haven't run other models at Max yet.
https://t.co/BVozmfyCDq
A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".
It's always interesting to read about new or different approaches (including…
A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".
It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.
About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."
Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.
In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.
In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.
Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)
While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.
In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.
Also, the statement "the new technique works in a way that obscures some or all of the AI's reasoning, otherwise known as 'chain-of-thought'" is not necessarily true with respect to the looped transformer method. It's possible that The Information journalist refers to some other technique or misunderstood the looped transformer method.
Reusing layers does not by itself suppress visible chain of thought. It adds computation in hidden states before the next token is emitted, just as ordinary transformer layers do.
But based on the information we have, the only plausible interpretation here is that if a model uses more of these recurrent passes, it may need to generate fewer intermediate reasoning tokens. So then more of its computation happens in latent activations that cannot be read as text. But we would get the same effect if we were scaling up the model size, like GPT 5.6 Luna -> GPT 5.6 Sol.
source image · full size ↗
Raschka separates a real architecture trade-off from speculation about Astra. His worked Nanbeige example applies the same 22-layer stack twice, increasing effective depth without duplicating its weights while spending roughly twice the transformer-block compute. Read the explanation of adaptive recursion and visible reasoning: repeated layers alone do not establish that a model hides its chain of thought, and the open Nanbeige report does not disclose Astra’s architecture.
Google’s September 3 engineering account explains how live satellite mosaics and sparse weather-station observations feed forecasts refreshed hourly, with key surface variables resolved at 5 kilometres. Read the distinctions among 5-, 10- and 25-kilometre outputs, precipitation training targets and turbine-height winds: finer grids alone do not establish greater accuracy for every variable. The reported gains are specific to the named evaluation datasets.
We have updated the Artificial Analysis Image Editing Arena to expand the range of editing tasks we test for, from Enhancement & Restoration to Identity-Preserving Edits, and from Marketing & Advertising to UI/UX Design. Our updated evaluation measures model performance on…
We have updated the Artificial Analysis Image Editing Arena to expand the range of editing tasks we test for, from Enhancement & Restoration to Identity-Preserving Edits, and from Marketing & Advertising to UI/UX Design. Our updated evaluation measures model performance on complex edit tasks, and assesses not just which model is best overall, but which model is best for specific editing needs. The new leaderboard is live and voting is open.
Image Editing models are advancing fast. AI now fits into different stages of image creation workflows, from generation through to post-production, and editing is no longer a side feature. We treat Image Editing and Reference to Image as separate benchmarks because they sit at different points in that workflow: Image Editing covers post-production, changing an image you already have; while our upcoming Reference to Image benchmark covers generating novel images from reference images.
For Image Editing, we test a model's ability to make specific changes to an image while keeping everything else unchanged. This includes complex edit instructions that chain multiple different asks, as frontier models have largely saturated single-instruction edits.
Different edit requests call for different models. Relighting a cinematic scene is a different problem from reworking the design of a marketing asset. We rank models on human preference across 7 editing actions, such as Object-Level Edit, Identity-Preserving Edit, and Enhancement & Restoration, and 10 real-world use cases, such as Marketing & Advertising, UI/UX Design, and Live-Action Film. The overall benchmark samples evenly across both.
Initial insights from an in-depth analysis of the 10 highest ranking models on the Artificial Analysis Image Editing Leaderboard:
➤ MAI-Image-2.6-Preview leads the overall leaderboard and 4 of the 7 editing action boards: Scene & Style Edit, Text or Symbol Edits, Reasoning-Based Edit, and Enhancement & Restoration, where it is tied #1 with MAI-Image-2.5. It excels at restyling, relighting, and retouching images.
➤ GPT Image 2 (high) ranks #2 overall but #1 on Object-Level Edit and Composition & Framing. It is the strongest at precise local edits and spatial reframing. It is weaker at whole-image transformations that must keep the image's content intact, ranking #7 on Scene & Style Edit and #6 on Enhancement & Restoration.
➤ Seedream 5.0 Pro is the character and identity specialist, ranking #1 on Identity-Preserving Edit.
➤ MAI-Image-2.5-Flash is the value pick of the top 10, at $20 per 1,000 images against $211 for GPT Image 2 (high) and $90 for Seedream 5.0 Pro.
See below for the editing action and use case breakdowns 🧵
Example Use Case: UI/UX editing includes changing UI component sizes, colors, styles, and in-UI text for app, web, in-car, and spatial UI mockups and flows, dashboards, screens, icons, and design system assets.
MAI-Image-2.6-Preview leads by 26 Elo, with MAI-Image-2.5 behind it. GPT Image 2 and Nano Banana 2 follow at #3 and #5.
UI edits are dense with in-image text, and the Text or Symbol Edits board tells the same story: MAI-Image-2.6-Preview leads that editing action by 30 Elo. MAI-Image-2.5 ranks #3 at $48.1 per 1,000 images.
Example Use Case: Productivity & Knowledge Work covers edits for charts, diagrams, infographics, data visualizations, slides, and document graphics.
Productivity & Knowledge Work is a three-way tie at the top: GPT Image 2, Nano Banana Pro and MAI-Image-2.6-Preview sit within the margin of error of each other.
Productivity edits combine two editing actions: Object-Level Edits, moving a legend or changing one series, and Reasoning-Based Edits, since the edit must respect the information the chart or diagram conveys. GPT Image 2 is the only model in the top 10 ranking top on both, #1 on Object-Level Edit and #2 on Reasoning-Based Edit.
Artificial Analysis explains why its redesigned editing evaluation separates seven editing actions and ten use cases instead of relying on one overall rank. Its results put different models first for whole-scene changes, object edits and identity preservation. The full breakdown is useful for building a routing test set: edit preservation, reference-based generation and overall visual preference are different objectives.
K2 Horizon 375B A23B, a new open weights model from UAE's MBZUAI, scores 47 on the Artificial Analysis Intelligence Index, with relatively strong agentic performance and a 30 point jump over its predecessor
K2 Horizon 375B A23B is an open weights Mixture-of-Experts model with…
K2 Horizon 375B A23B, a new open weights model from UAE's MBZUAI, scores 47 on the Artificial Analysis Intelligence Index, with relatively strong agentic performance and a 30 point jump over its predecessor
K2 Horizon 375B A23B is an open weights Mixture-of-Experts model with 375B total and 23B active parameters from @IFM_MBZUAI, MBZUAI's Institute of Foundation Models. It scores 47 on the Intelligence Index, alongside models such as MiniMax-M3 (45, also a MoE with 23B active parameters), and a large upgrade from its predecessor K2 Think V2 (17, 70B dense model). It leads nearby open weights models on agentic evals and has a low hallucination rate, but trails on knowledge and the hardest reasoning evals. K2 Think V2 ranks among the most open models on our Openness Index; MBZUAI is updating the supporting documentation and code for K2 Horizon and we expect to add it to the Openness Index soon.
Key takeaways:
➤ Strong on agentic tasks, weaker on knowledge and deep reasoning. MiniMax-M3, a recent model that is close to it on the Intelligence Index, makes the cleanest comparison: K2 Horizon 375B A23B leads on GDPval-AA, our real-world knowledge work benchmark (Elo 1430 vs 1380), and on τ³-Banking (34.2% vs 15.3%), but trails on GPQA Diamond (87.3% vs 92.9%) and Humanity's Last Exam (32.0% vs 39.0%)
➤ Low hallucination rate, driven by abstention rather than knowledge. K2 Horizon 375B A23B attempts only 40% of AA-Omniscience questions, declining the remaining 60% rather than guessing. The result is a 26% hallucination rate, among the lower rates we have measured, while accuracy is 18%, essentially unchanged from K2 Think V2
➤ A new architecture over its predecessor. K2 Horizon 375B A23B is a 375B parameter Mixture-of-Experts model with 23B active, succeeding the 70B dense K2 Think V2, and extends context from 262K to 512K tokens. Its 23B active parameters match MiniMax-M3 (428B total, 23B active)
Key model details:
➤ Architecture: Mixture-of-Experts, 375B total parameters, 23B active
➤ Context window: 512K tokens
➤ Multimodality: Text input and output only
➤ Pricing and availability: Yet to be announced
➤ Licensing: Open weights (license details to be announced)
The new 375B-parameter, 23B-active model scores 47 on Artificial Analysis’s Intelligence Index and leads nearby models on several agentic tests. Its 26% hallucination rate accompanies attempts on just 40% of AA-Omniscience questions, while accuracy remains 18%. A system that declines more questions needs a fallback route; lower hallucination alone does not establish broader knowledge.
Microsoft AI has released MAI-Transcribe-2, achieving 2.0% AA-WER (#2) at ~411x real time (#2) for $1.67 per 1,000 minutes, placing it on the accuracy-speed Pareto frontier at one of the lowest prices among high-accuracy models
MAI-Transcribe-2 is @MicrosoftAI’s latest speech…
Microsoft AI has released MAI-Transcribe-2, achieving 2.0% AA-WER (#2) at ~411x real time (#2) for $1.67 per 1,000 minutes, placing it on the accuracy-speed Pareto frontier at one of the lowest prices among high-accuracy models
MAI-Transcribe-2 is @MicrosoftAI’s latest speech transcription model, ranking 2nd overall on the Artificial Analysis Word Error Rate (AA-WER) leaderboard, behind Alibaba’s Fun-Realtime-ASR-preview (1.7% WER) and ahead of ElevenLabs Scribe v2 (2.2% WER) and Smallest AI Pulse Pro (2.4% WER). It is one of the fastest models benchmarked, processing audio at 411x real time, around 1.5x the speed of Smallest AI Pulse Pro and more than 7x the speed of Scribe v2.
At $1.67 per 1,000 minutes, MAI-Transcribe-2 costs less than half as much as high-accuracy peers including Scribe v2 ($3.67) and Smallest AI Pulse Pro ($4). The new model expands language support from 43 to 60 languages and adds speaker diarization, word-level timestamps, as well as clean and verbatim transcription styles.
See more details below ⬇️
Artificial Analysis measures 2.0% word error rate and roughly 411 times real-time throughput at $1.67 per 1,000 audio minutes. Microsoft’s new model adds diarization, word timestamps and 60-language support. The benchmark makes it worth testing for high-volume transcription; aggregate error rates do not establish accuracy for a particular language, accent or recording environment.
Meta's Muse Spark 1.3 (max), which is in limited preview for Meta's partners, scores 68 on the Artificial Analysis Coding Agent Index in the Muse Code harness, #2 behind only Claude Opus 5 (xhigh) in Claude Code. The variant available now, Muse Spark 1.3 (xhigh), scores 64 and…
Meta's Muse Spark 1.3 (max), which is in limited preview for Meta's partners, scores 68 on the Artificial Analysis Coding Agent Index in the Muse Code harness, #2 behind only Claude Opus 5 (xhigh) in Claude Code. The variant available now, Muse Spark 1.3 (xhigh), scores 64 and costs the least per task of any agent above a 60 index score
Muse Spark 1.3 (xhigh) enters the Artificial Analysis Coding Agent Index at 64 in Muse Code, up 2 points from Muse Spark 1.2 (62, August). It enters level with Grok 4.5 (high) in Grok Build (64) and behind GPT-5.6 Sol (max) in Codex (65). At $1.72 per task, it costs the least of any agent above a 60 index score, around a fifth of the cost of Claude Opus 5 (xhigh) in Claude Code ($8.17)
Muse Spark 1.3 (max), which is in a limited preview stage, lands at 68 in Muse Code. It enters behind only Claude Opus 5 (xhigh) in Claude Code (68), and ahead of Claude Fable 5 (max) in Claude Code (67) and GPT-5.6 Sol (max) in Codex (65). Muse Spark 1.3 (max) is excluded from cost comparisons as Meta has not announced pricing for the limited release
Claude Fable 5.1 results are in progress and will be added when complete.
Congratulations @AIatMeta, @finkd, and @alexandr_wang on this result!
Muse Spark 1.3 (xhigh) in Muse Code sits on the Pareto frontier for Coding Agent Index vs. cost per task at $1.72. Muse Spark 1.3 (max) is excluded from cost comparisons as Meta has not announced pricing for the limited release, but on tokens it averages ~14M total tokens per task, around a third fewer than Claude Opus 5 (xhigh) in Claude Code (~22M) at the same score.
The new Coding Agent Index measures Muse Spark 1.3 xhigh in Muse Code at 64 and $1.72 per task, the lowest reported cost among agents scoring above 60 in that evaluation. That adds a separate workflow measurement to yesterday’s general-model result; the two indexes and task prices are not interchangeable. The max variant scores 68 in limited preview, with pricing still undisclosed.
Wan 3.0 debuts at #1 on the Artificial Analysis Video Editing Leaderboard, and is a close #2 in Text to Video with Audio
Wan 3.0 is Alibaba's new all-in-one video generation and editing model, positioned as a single system for turning multimodal creative direction into video.…
Wan 3.0 debuts at #1 on the Artificial Analysis Video Editing Leaderboard, and is a close #2 in Text to Video with Audio
Wan 3.0 is Alibaba's new all-in-one video generation and editing model, positioned as a single system for turning multimodal creative direction into video. It generates up to 30 seconds at 1080p with native audio and accepts text, images, video, audio, documents, and web pages as creative references. The same model supports Text to Video, Image to Video, reference-based generation, and instruction-led editing, including changes to visuals, plot, dialogue, and sound.
In the Artificial Analysis Video Arena, Wan 3.0 ranks #1 in Video Editing with Audio, #2 in Text to Video with Audio, and #5 in Image to Video with Audio.
Wan 3.0 marks a large generational improvement: against the most recent Wan 2.7 version on each leaderboard, it rises from #5 to #1 in Video Editing with Audio, #6 to #2 in Text to Video with Audio, and #12 to #5 in Image to Video with Audio.
Wan 3.0 is available now in public preview through Alibaba Cloud Model Studio. Pricing starts at $0.05 per second for 480p, increasing to $0.10 for 720p and $0.20 for 1080p.
Congratulations to @Alibaba_Wan and @alibaba_cloud on the release!
See below for comparisons between Wan 3.0 and other leading models in the Artificial Analysis Video Arena 🧵
Artificial Analysis ranks Wan 3.0 first in video editing with audio, second in text-to-video with audio and fifth in image-to-video with audio. The public preview supports up to 30 seconds at 1080p, priced at $0.20 per second at that resolution. These are task-specific human-preference rankings; test continuity and editing fidelity on the intended workflow before treating the model as the strongest option across video tasks.
Asimov 1 is shipping now and you can order it.
The bill of materials is open if you want to source it yourself. Self-sourcing runs about $31k in parts, plus logistics on top.
Ordering through https://t.co/qp1Etq9rzs is $20k with logistics included.
Asimov now says its robot is shipping and prices an assembled order at $20,000 with logistics included, versus about $31,000 in parts plus logistics for self-sourcing. That adds a procurement decision to yesterday’s CAD release: the open bill of materials supports inspection, but the maker’s own estimate favors buying the unit. Shipping and parts costs are first-party claims, without independent delivery verification.
This new Richmond Fed working paper models weekly reserve redistribution using 2010–24 data. Reserves move slowly across bank groups when aggregate supply is abundant, but redistribution increases at lower, still ample levels. Read the counterfactuals separating Treasury-account flows from asset-policy flows: identical changes in total reserves can leave different banks holding the liquidity. Weekly balance-sheet snapshots cannot resolve intraday funding stress.
The call adds FY2027 and FY2028 AI-revenue targets of about $115B and $230B to the quarterly figures reported yesterday. Management says supply is secured for that outlook, while land, power and buildings constrain when customers can deploy it. Read the customer roadmap and financing questions: chip shipments, installed gigawatts and the timing of customer revenue are separate assumptions, and the multi-year figures remain management forecasts.
BIS researchers demonstrate a way to attach publisher identity and tamper evidence to statistical releases, batching data fingerprints on the XRP Ledger. The full paper’s useful distinction is between an authentic release and a correct one: revisions require new anchors and discoverable revocation records. Its prototype verifies in one to two seconds under controlled conditions. Read the workflow, batching trade-off and key-management limits before treating it as production infrastructure.
July’s goods-and-services deficit widened to $88.6B from a revised $71.2B in June. Imports rose $10.8B while exports fell $6.6B; computer and computer-accessory imports increased $6.9B and $6.6B respectively. The September 3 release reverses the earlier monthly narrowing, but its seasonally adjusted nominal figures do not isolate AI spending or translate directly into GDP’s net-export contribution.
BLS revised Q2 manufacturing productivity growth up to 2.4% from 1.9%, and manufacturing unit labor costs down to a 0.3% decline. Nonfarm-business productivity remains 1.4%, with unit labor costs revised to 1.2% from 1.3%. These September 3 estimates are seasonally adjusted annualized quarterly rates; they strengthen the manufacturing cost picture without identifying AI as the cause.
The August survey, released September 1, puts the LMI at 66.6 versus 68.9 in July, while transportation prices rise to 90.0 and capacity remains in contraction at 40.0. Inventory levels sit near neutral at 52.8. The divergence adds a survey-based warning to the earlier fuel-cost evidence: slower stock accumulation has not removed freight pricing pressure. These are diffusion indexes, not percentage changes in shipping prices.
Astra has also set the top score on our newest math benchmark, FrontierMath Erdős, which consists of 68 unsolved and especially interesting Erdős problems curated by @thomasfbloom. AI systems are tasked with writing solutions in Lean, a special-purpose programming language for automatically verifying mathematical proofs. No prior model solved any of the problems, but Astra solved 2/68, scoring 3%. See our post introducing the benchmark for more details, including our open-source scaffold. https://t.co/iJ7qrrg3UF
Astra achieved a raw score of 46.7% on MirrorCode, our long-horizon coding benchmark landing squarely between Opus 4.7 and Fable 5. We ran some internal tests indicating that Astra might score higher on MirrorCode with greater reasoning effort. We only report the High-effort score here, as it's what we report for every other model, and we haven't run other models at Max yet. https://t.co/BVozmfyCDq