SuperHuman Sparks seen in Ai Biology Benchmarks

[SaucedOnEm]

theres this test that the company Anthropic made called the BioMysteryBench. It had 99 open ended research problems made by experts in multiple domains of biology

they got a human baseline of 76 out of 99 questions with teams of up to 5 people. the rest humans couldnt solve

Claude Mythos solved 7 of those questions that no human expert could manage to get. 3 of those answers were "brittle" meaning the ai couldnt consistently get that answer every try. seems to be linked to the reasoning path it took.

Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

www.anthropic.com

 

[lullabyofscreams]

Surprised this post doesn't have a neg yet.

 

[master shake]

theres this test that the company Anthropic made called the BioMysteryBench. It had 99 open ended research problems made by experts in multiple domains of biology

they got a human baseline of 76 out of 99 questions with teams of up to 5 people. the rest humans couldnt solve

Claude Mythos solved 7 of those questions that no human expert could manage to get. 3 of those answers were "brittle" meaning the ai couldnt consistently get that answer every try. seems to be linked to the reasoning path it took.

Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

www.anthropic.com

Every time you fail at proving your point, you go ahead and shit post.