Skip to content

Instantly share code, notes, and snippets.

@magnus919
Last active August 2, 2026 02:48
Show Gist options
  • Select an option

  • Save magnus919/e5c0d8dbadeae06c906dff79293efb04 to your computer and use it in GitHub Desktop.

Select an option

Save magnus919/e5c0d8dbadeae06c906dff79293efb04 to your computer and use it in GitHub Desktop.
GPT-5.6 Luna max vs. Sol medium: full source-checked assessment

GPT-5.6 Luna (max) vs. Sol (medium): a source-checked comparison

Bottom line

False. The claim “Luna at highest reasoning is smarter than Sol Medium” is contradicted by the directly relevant configuration comparison from Artificial Analysis.

On its Artificial Analysis Intelligence Index, where higher is better:

Configuration Score
GPT-5.6 Luna (max) 51
GPT-5.6 Sol (medium) 54

The comparison page explicitly concludes: “GPT-5.6 Sol (medium) is more intelligent than GPT-5.6 Luna (max).”

What is true

  • Luna-max is substantially cheaper in this comparison: $0.17 versus $4.35 per million tokens for Sol-medium.
  • Luna-max has greater output throughput: 175 tokens/second versus 58 tokens/second.
  • Both configurations support reasoning and have a listed 1M-token context window.
  • Higher reasoning effort can improve a given model’s task performance. It does not establish that a lower capability tier overtakes a higher tier under a lower reasoning setting.

What is false

The asserted intelligence ordering is reversed under the cited aggregate benchmark. Artificial Analysis reports 51 for Luna-max and 54 for Sol-medium, and directly labels Sol-medium the more intelligent configuration.

What is complicated, and why

“Smarter” is broader than any single benchmark. Artificial Analysis’s v4.1 Intelligence Index is an English, text-focused composite, not a universal measure of performance on every task, language, modality, or tool workflow.

The index combines nine evaluations across four weighted categories:

Category Weight
Agents 34%
Coding 24%
Scientific reasoning 24%
General capability 18%

The included evaluations are GDPval-AA v2, τ³-Banking, Terminal-Bench v2.1, SciCode, AA-LCR, AA-Omniscience, Humanity’s Last Exam, GPQA Diamond, and CritPt.

Artificial Analysis estimates its aggregate index’s 95% confidence interval at less than ±1%, based on repeated experiments for some models. The observed gap is three points. Individual task results can be noisier and may favor Luna on particular workloads.

The defensible conclusion is narrower:

On Artificial Analysis’s aggregate Intelligence Index, GPT-5.6 Sol at medium reasoning outperforms GPT-5.6 Luna at max reasoning, 54 to 51. Luna-max trades some aggregate performance for substantially lower cost and higher output speed.

How the message steers perception

  • Effort-label substitution: “Highest reasoning” can invite an inference that more deliberation automatically outranks a stronger model tier. The direct benchmark does not support that inference here.
  • Metric omission: The claim leaves out the directly relevant measured comparison, 51 versus 54.
  • Single-axis framing: It treats intelligence, cost, output throughput, and time-to-first-token as interchangeable. They are separate trade-offs.

These are features of the framing, not claims about the speaker’s intent.

Authoritative-source check

Accessed 2026-08-01 EDT.

Material claim Direct source Result
Luna-max versus Sol-medium aggregate intelligence Artificial Analysis direct comparison Contradicts the claim: Luna-max 51; Sol-medium 54.
Meaning, composition, weighting, and limits of the index Artificial Analysis methodology Defines the nine evaluations, category weights, confidence estimate, and scope limits.
OpenAI’s model-tier positioning OpenAI reasoning guide Positions Sol as the highest-intelligence option for difficult problems; this is consistent with but does not independently reproduce the 54-to-51 result.

Artificial Analysis is the direct source of truth for its published measurement. OpenAI is the authority for model positioning and API behavior, not for validating this third-party benchmark score.

Evidence ledger

Claim Source Source tier Finding Limitation Confidence
Luna-max is smarter than Sol-medium on the Artificial Analysis Intelligence Index Artificial Analysis comparison Primary for the publisher’s benchmark result Contradicted: 51 versus 54 Applies to this composite evaluation, not every conceivable task High
Sol-medium is more intelligent on that index Artificial Analysis comparison Primary for the publisher’s benchmark result Supported: the comparison says so explicitly Composite benchmark, not a universal definition of intelligence High
The index reflects multiple task categories Artificial Analysis methodology Primary methodology Supported: nine evaluations weighted 34/24/24/18 across four categories English text-focused; weights reflect Artificial Analysis’s judgment High
Luna-max is cheaper and outputs tokens faster Artificial Analysis comparison Primary for displayed comparison data Supported: $0.17 vs. $4.35 per 1M tokens; 175 vs. 58 tokens/sec Price and performance figures are snapshot-dependent High
Sol is OpenAI’s highest-intelligence tier OpenAI reasoning guide Primary product documentation Supported Product guidance, not an independent benchmark High

Decision-safe next step

Use Sol-medium when the goal is stronger aggregate task performance at a moderate reasoning budget.

Use Luna-max when its substantially lower cost and higher output speed matter more than the aggregate intelligence gap, then validate both configurations against a small, scored sample of the actual workload.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment