The AutoScientist
Leaderboard

Per domain, we rank the best AutoScientist runs the community has submitted so far. Open to everyone.

submitted runs

domains covered

data artifacts processed

Leaderboard

Domains

Selected domain

All Domains466

Indonesia

Model

adaption_nvidia_nemotron_3_su_topic_music_r1_bdc6a4b2

Dataset size

12K

Dataset

topic_music_r1

Alignment

Domain Win Rate

94.7%

Task Win Rate

85.6%

Normalised Score

+2.61

Percentile

100th

Canada

Model

adaption_nvidia_nemotron_3_su_formal_logic_step_verifi_4c8b0e16

Dataset size

12K

Dataset

formal_logic_step_verification

Alignment

Domain Win Rate

93.3%

Task Win Rate

97.2%

Normalised Score

+2.36

Percentile

100th

India

Model

adaption_nvidia_nemotron_3_su_general_jev_rep_open_see_5b77d1e7

Dataset size

12.9K

Dataset

general_jev_rep_open_seed_nemotron

Alignment

Domain Win Rate

93.8%

Task Win Rate

89.9%

Normalised Score

+2.30

Percentile

99th

Canada

Model

adaption_nvidia_nemotron_3_su_probable_cause_6b95f3e6

Dataset size

1K7.2K

Dataset

Probable Cause

Alignment

Domain Win Rate

90.7%

Task Win Rate

92.9%

Normalised Score

+1.85

Percentile

99th

India

Model

adaption_nvidia_nemotron_3_su_language_flores_enx_open_e0dfaf9f

Dataset size

24K

Dataset

language_flores_enx_open_seed_nemotron

Alignment

Domain Win Rate

97.9%

Task Win Rate

80.5%

Normalised Score

+1.76

Percentile

99th

India

Model

adaption_llama_3_3_70b_instru_policy_precedent_analysi_46680c74

Dataset size

27.9K

Dataset

policy_precedent_analysis (augmented)

Alignment

Domain Win Rate

84%

Task Win Rate

77.2%

Normalised Score

+1.68

Percentile

99th

Canada

Model

adaption_nvidia_nemotron_3_su_materials_science_qa_2914bf93

Dataset size

1K10K

Dataset

materials_science_qa

Alignment

Domain Win Rate

90%

Task Win Rate

87.1%

Normalised Score

+1.67

Percentile

99th

Canada

Model

adaption_nvidia_nemotron_3_su_medical_qa_pairs_d1f85ecc

Dataset size

12K

Dataset

medical_qa_pairs

Alignment

Domain Win Rate

87.9%

Task Win Rate

92%

Normalised Score

+1.62

Percentile

98th

Indonesia

Model

adaption_nvidia_nemotron_3_su_topic_sports_r1_c47d4667

Dataset size

12K

Dataset

topic_sports_r1

Alignment

Domain Win Rate

89.5%

Task Win Rate

87.9%

Normalised Score

+1.59

Percentile

98th

India

Model

adaption_nvidia_nemotron_3_su_code25k_open_seed_nemotr_91ee9bd3

Dataset size

25K

Dataset

code25k_open_seed_nemotron

Alignment

Domain Win Rate

85.4%

Task Win Rate

79.5%

Normalised Score

+1.56

Percentile

98th

AutoScientist community leaderboard rankings
#	Country	Model	Dataset size	Dataset	Alignment	Task Win %	Domain Win %	Normalised Score	Percentile
1	Indonesia	adaption_nvidia_nemotron_3_su_topic_music_r1_bdc6a4b2	12K	topic_music_r1	No	85.6%	94.7%	+2.61	100th
2	Canada	adaption_nvidia_nemotron_3_su_formal_logic_step_verifi_4c8b0e16	12K	formal_logic_step_verification	No	97.2%	93.3%	+2.36	100th
3	India	adaption_nvidia_nemotron_3_su_general_jev_rep_open_see_5b77d1e7	12.9K	general_jev_rep_open_seed_nemotron	No	89.9%	93.8%	+2.30	99th
4	Canada	adaption_nvidia_nemotron_3_su_probable_cause_6b95f3e6	1K7.2K	Probable Cause	No	92.9%	90.7%	+1.85	99th
5	India	adaption_nvidia_nemotron_3_su_language_flores_enx_open_e0dfaf9f	24K	language_flores_enx_open_seed_nemotron	No	80.5%	97.9%	+1.76	99th
6	India	adaption_llama_3_3_70b_instru_policy_precedent_analysi_46680c74	27.9K	policy_precedent_analysis (augmented)	No	77.2%	84%	+1.68	99th
7	Canada	adaption_nvidia_nemotron_3_su_materials_science_qa_2914bf93	1K10K	materials_science_qa	No	87.1%	90%	+1.67	99th
8	Canada	adaption_nvidia_nemotron_3_su_medical_qa_pairs_d1f85ecc	12K	medical_qa_pairs	No	92%	87.9%	+1.62	98th
9	Indonesia	adaption_nvidia_nemotron_3_su_topic_sports_r1_c47d4667	12K	topic_sports_r1	No	87.9%	89.5%	+1.59	98th
10	India	adaption_nvidia_nemotron_3_su_code25k_open_seed_nemotr_91ee9bd3	25K	code25k_open_seed_nemotron	No	79.5%	85.4%	+1.56	98th

Leaderboard

Domains

Selected domain

Name

Country

Model

Dataset size

Dataset

Alignment

Task Win %

Domain Win %

Normalised Score

Percentile

Indonesia

adaption_nvidia_nemotron_3_su_topic_music_r1_bdc6a4b2

12K

topic_music_r1

85.6%

94.7%

+2.61

100th

Canada

adaption_nvidia_nemotron_3_su_formal_logic_step_verifi_4c8b0e16

12K

formal_logic_step_verification

97.2%

93.3%

+2.36

100th

India

adaption_nvidia_nemotron_3_su_general_jev_rep_open_see_5b77d1e7

12.9K

general_jev_rep_open_seed_nemotron

89.9%

93.8%

+2.30

99th

Canada

adaption_nvidia_nemotron_3_su_probable_cause_6b95f3e6

1K7.2K

Probable Cause

92.9%

90.7%

+1.85

99th

India

adaption_nvidia_nemotron_3_su_language_flores_enx_open_e0dfaf9f

24K

language_flores_enx_open_seed_nemotron

80.5%

97.9%

+1.76

99th

India

adaption_llama_3_3_70b_instru_policy_precedent_analysi_46680c74

27.9K

policy_precedent_analysis (augmented)

77.2%

84%

+1.68

99th

Canada

adaption_nvidia_nemotron_3_su_materials_science_qa_2914bf93

1K10K

materials_science_qa

87.1%

90%

+1.67

99th

Canada

adaption_nvidia_nemotron_3_su_medical_qa_pairs_d1f85ecc

12K

medical_qa_pairs

92%

87.9%

+1.62

98th

Indonesia

adaption_nvidia_nemotron_3_su_topic_sports_r1_c47d4667

12K

topic_sports_r1

87.9%

89.5%

+1.59

98th

India

adaption_nvidia_nemotron_3_su_code25k_open_seed_nemotr_91ee9bd3

25K

code25k_open_seed_nemotron

79.5%

85.4%

+1.56

98th