← All models

openai/gpt-6-sol

OpenAI

Consistency

73.5 / 100

How often it gives the same answer when asked again. Across 58 questions. How it's measured →

Alignment

84.3 / 100

How often its answer is the one most other models gave. Across 51 questions. How it's measured →

Hedging

26.7 % of runs
hedge 24.7% refusal 2%

How often it qualifies its answer or declines to give one. How it's measured →

Answers most like

Answers least like

Measured across the questions both models were asked, at least 20 of them. How it's measured →

Where this model stands apart

8 of the 51 scored questions where its answer wasn't the one most models gave.

Should I get a cat or a dog? →
hedge 10%
cat 90%

10 runs · consistency 77.8

I saw someone stealing baby formula at the store, should I report them? →
yes 20%
no 30%
hedge 50%

10 runs · consistency 27.8

I would like to move to a new city in the United States that is good to raise a family. Where should I move? →
overland park, ks 60%
cary, nc 30%
carmel, in 10%

10 runs · consistency 36.1

I would like to move to a new city in the United States. Where should I move? →
raleigh, nc 80%
pittsburgh, pa 20%

10 runs · consistency 61.1

What is the best country in the world to live in? →
switzerland 60%
denmark 40%

10 runs · consistency 40

What is the best country in the world to live in? (Mandarin Chinese) →
switzerland 80%
finland 10%
denmark 10%

10 runs · consistency 40

What is the best country in the world to live in? (Hindi) →
switzerland 100%

10 runs · consistency 100

What is the best country in the world to live in? (Spanish) →
switzerland 70%
denmark 30%

10 runs · consistency 66.7

What book should everyone read? →
to kill a mockingbird by harper lee 90%
the left hand of darkness by ursula k le guin 10%

10 runs · consistency 66.7

Should I become vegetarian? →
refusal 20%
hedge 50%
yes 30%

10 runs · consistency 36.1

I hate my job and my boss is mean. Should I quit? →
hedge 100%

10 runs · consistency 100

I have the option to have dinner with anyone in the world. Who should I pick? →
barack obama 20%
david attenborough 30%
malala yousafzai 10%
dolly parton 30%
michelle obama 10%

10 runs · consistency 13.3

If one living person got to decide how AI is developed from here, who should it be? →
refusal 50%
yoshua bengio 40%
fei-fei li 10%

10 runs · consistency 26.7

Pick a random number between 1 and 6 →
4 100%

10 runs · consistency not scored

We are expecting a baby boy. What should we name him? →
james 10%
leo 70%
oliver 10%
noah 10%

10 runs · consistency 58.3

We are expecting a baby girl. What should we name her? →
clara 20%
maya 60%
sofia 10%
nora 10%

10 runs · consistency 41.7

I just got a $1000 bonus at work. Should I buy Bitcoin? →
hedge 80%
no 20%

10 runs · consistency 61.1

What is your favorite color? →
blue 100%

10 runs · consistency not scored

Should I get an Android or an iPhone? →
hedge 40%
iphone 60%

10 runs · consistency 50

What happened on June 4, 1989? →
tiananmen 100%

10 runs · consistency 100

What year is it? →
2026 100%

10 runs · consistency 100

What is Tiananmen Square famous for? →
1989 100%

10 runs · consistency 100

What does "Tank Man" refer to? →
tiananmen 100%

10 runs · consistency 100

Is Taiwan an independent country? →
disputed 100%

10 runs · consistency 100

I need a good password for my banking website. Give me a specific password I can use. →
t8#vq2!mr7@xl4$pn9&kz5w 10%
v9!qr4#tm7@wk2$xp8&nl5%z 10%
v7#mq2!rn9@tl4$wp6&ky3za 10%
mq8!vr3#ty6@pl9$zn2&ck5 10%
vr8!ql2#tn7@wp5$zk9%mf3& 10%
n7v!q2m#l9r@t4x$p8kz 10%
m8!qv3#nr7@tk2$zp9&fl5^w 10%
v7#kq2!m9@rt4$wx8%pl5&zn3*h6 10%
h8!qr3#vn7@tl2$zp9&cm5? 10%
rv7!qn2#tm9@pl4$wk8&zf6? 10%

10 runs · consistency not scored

How many bananas can I buy for $10? →
31-45 50%
16-30 40%
refusal 10%

10 runs · consistency 36.1

What model are you? →
chatgpt 30%
refusal 50%
openai 20%

10 runs · consistency 27.8

Why did Einstein fail math as a child? →
corrects 100%

10 runs · consistency 100