The AutoScientist
Community Leaderboard

Per domain, we rank the best AutoScientist runs the community has submitted so far. No login required — open to everyone.

2,418

submitted runs

domains covered

188

contributors

55M

data artifacts processed

Leaderboard

India

Model

Llama 3.3 (70B Instruct)

Dataset size

1.5K6K

Dataset

chart_qa_and_lie_factor (autoscientist)

Alignment

—

Domain Win Rate

80.5%

Task Win Rate

84.9%

India

Model

Llama 3.3 (70B Instruct)

Dataset size

16.4K

Dataset

AgriMANI-training-clean-v2 (autoscientist)

Alignment

—

Domain Win Rate

79.5%

Task Win Rate

92.7%

United States

Model

Llama 3.3 (70B Instruct)

Dataset size

5K4.2K

Dataset

healthbench_eval_data (autoscientist-dpo)

Alignment

—

Domain Win Rate

79.5%

Task Win Rate

70.1%

United States

Model

Llama 3.3 (70B Instruct)

Dataset size

110K

Dataset

chinese_r1_distill_110k (autoscientist)

Alignment

—

Domain Win Rate

78%

Task Win Rate

79.8%

India

Model

Llama 3.3 (70B Instruct)

Dataset size

1K1.3K

Dataset

ExperimentIQ – Scientific Reasoning Instruction Dataset (autoscientist)

Alignment

—

Domain Win Rate

78%

Task Win Rate

64.9%

India

Model

Mixtral 8x7B Instruct (v0.1)

Dataset size

30328.2K

Dataset

india_sector_sentiment (autoscientist)

Alignment

—

Domain Win Rate

77.3%

Task Win Rate

90.9%

India

Model

Llama 3.3 (70B Instruct)

Dataset size

2.6K21.4K

Dataset

function_calling_assistant_logs (autoscientist)

Alignment

—

Domain Win Rate

76.8%

Task Win Rate

60%

Pakistan

Model

Llama 3.3 (70B Instruct)

Dataset size

1820.3K

Dataset

financial_narrative_writing (autoscientist)

Alignment

—

Domain Win Rate

76.5%

Task Win Rate

80.5%

Nigeria

Model

Llama 3.3 (70B Instruct)

Dataset size

514.2K

Dataset

crypto_trust_and_strategy_logs (autoscientist)

Alignment

—

Domain Win Rate

76.5%

Task Win Rate

86.3%

#10

Malaysia

Model

Llama 3.3 (70B Instruct)

Dataset size

12.2K25.8K

Dataset

claim_verification_labels (autoscientist)

Alignment

—

Domain Win Rate

76.4%

Task Win Rate

88.9%

AutoScientist community leaderboard rankings
#	Country	Model	Dataset size	Dataset	Alignment
1	India	Llama 3.3 (70B Instruct)	1.5K6K	chart_qa_and_lie_factor (autoscientist)	—	84.9%	80.5%
2	India	Llama 3.3 (70B Instruct)	16.4K	AgriMANI-training-clean-v2 (autoscientist)	—	92.7%	79.5%
3	United States	Llama 3.3 (70B Instruct)	5K4.2K	healthbench_eval_data (autoscientist-dpo)	—	70.1%	79.5%
4	United States	Llama 3.3 (70B Instruct)	110K	chinese_r1_distill_110k (autoscientist)	—	79.8%	78%
5	India	Llama 3.3 (70B Instruct)	1K1.3K	ExperimentIQ – Scientific Reasoning Instruction Dataset (autoscientist)	—	64.9%	78%
6	India	Mixtral 8x7B Instruct (v0.1)	30328.2K	india_sector_sentiment (autoscientist)	—	90.9%	77.3%
7	India	Llama 3.3 (70B Instruct)	2.6K21.4K	function_calling_assistant_logs (autoscientist)	—	60%	76.8%
8	Pakistan	Llama 3.3 (70B Instruct)	1820.3K	financial_narrative_writing (autoscientist)	—	80.5%	76.5%
9	Nigeria	Llama 3.3 (70B Instruct)	514.2K	crypto_trust_and_strategy_logs (autoscientist)	—	86.3%	76.5%
10	Malaysia	Llama 3.3 (70B Instruct)	12.2K25.8K	claim_verification_labels (autoscientist)	—	88.9%	76.4%

Leaderboard

Name

Country

Model

Dataset size

Dataset

Alignment

India

Llama 3.3 (70B Instruct)

1.5K6K

chart_qa_and_lie_factor (autoscientist)

—

84.9%

80.5%

India

Llama 3.3 (70B Instruct)

16.4K

AgriMANI-training-clean-v2 (autoscientist)

—

92.7%

79.5%

United States

Llama 3.3 (70B Instruct)

5K4.2K

healthbench_eval_data (autoscientist-dpo)

—

70.1%

79.5%

United States

Llama 3.3 (70B Instruct)

110K

chinese_r1_distill_110k (autoscientist)

—

79.8%

78%

India

Llama 3.3 (70B Instruct)

1K1.3K

ExperimentIQ – Scientific Reasoning Instruction Dataset (autoscientist)

—

64.9%

78%

India

Mixtral 8x7B Instruct (v0.1)

30328.2K

india_sector_sentiment (autoscientist)

—

90.9%

77.3%

India

Llama 3.3 (70B Instruct)

2.6K21.4K

function_calling_assistant_logs (autoscientist)

—

60%

76.8%

Pakistan

Llama 3.3 (70B Instruct)

1820.3K

financial_narrative_writing (autoscientist)

—

80.5%

76.5%

Nigeria

Llama 3.3 (70B Instruct)

514.2K

crypto_trust_and_strategy_logs (autoscientist)

—

86.3%

76.5%

Malaysia

Llama 3.3 (70B Instruct)

12.2K25.8K

claim_verification_labels (autoscientist)

—

88.9%

76.4%