Post
520
Wiki-SmartBotLM Benchmarked. Here are the results:
Benchmark WikiSB | Gemma3_270M
BoolQ 37.8 | 61.4
ARC Challenge 24.2 | 28.2
HellaSwag 26.3 | 37.7
PIQA 53.5 | 66.2
Winogrande 50.2 | 52.3
Summary:
Closest benchmark: Winogrande (2.1-point gap)
Strong performance: ARC Challenge (4.0-point gap)
Moderate gap: HellaSwag and PIQA
Largest gap: BoolQ (Reading Comprehension)
It shows despite the training difference between Gemma3 270M and WikiSmartBotLM, It does come close to Gemma 3 in some benchmarks.
pranavupadhyaya52/Wiki-SmartBotLM-Instruct
Benchmark WikiSB | Gemma3_270M
BoolQ 37.8 | 61.4
ARC Challenge 24.2 | 28.2
HellaSwag 26.3 | 37.7
PIQA 53.5 | 66.2
Winogrande 50.2 | 52.3
Summary:
Closest benchmark: Winogrande (2.1-point gap)
Strong performance: ARC Challenge (4.0-point gap)
Moderate gap: HellaSwag and PIQA
Largest gap: BoolQ (Reading Comprehension)
It shows despite the training difference between Gemma3 270M and WikiSmartBotLM, It does come close to Gemma 3 in some benchmarks.
pranavupadhyaya52/Wiki-SmartBotLM-Instruct
