You need to enable JavaScript to run this app.
DevQualityEval v1.0
https://github.com/symflower/eval-dev-quality
Overall chattiness
0
150
300
450
600
Characters per point scored (lower is better)
DeepSeek: R1 Distill Qwen 1.5B
Mistral: Mixtral 8x7B (Base) (v0.1)
Meta: Llama 3.2 1B (Instruct)
Microsoft: WizardLM-2 7B
Cohere: Command
Microsoft: Phi-3 Medium (Instruct) (128K)
Meta: Llama 3.2 3B (Instruct)
Liquid: LFM 3B
Microsoft: Phi-3 Mini (Instruct) (128K)
NousResearch: Hermes 13B
XWin-LM: Xwin 70B
DeepSeek: DeepSeek R1 Distill Llama 70B
Microsoft: Phi-3.5 Mini (Instruct) (128K)
Mistral: Ministral 8B
Liquid: LFM 7B
Perplexity: Llama 3.1 Sonar 8B
Mistral: Mistral Tiny (v0.3)
NousResearch: Hermes 2 Mixtral 8x7B (DPO)
Mistral: Mistral 7B (Instruct)
Mistral: Mistral 7B (Instruct) (v0.3)
DeepSeek: DeepSeek R1 Distill Qwen 14B
Cognitive Computations: Dolphin 2.6 Mixtral 8x7B
Qwen: QwQ 32B
Meta: Llama 3.1 8B (Instruct)
Liquid: LFM MoE 40B
Mistral: Ministral 3B
Cohere: Command R (03-2024)
OpenChat: OpenChat 3.5 7B
Jon Durbin: Airoboros 70B
Meta: Llama 3 8B (Instruct)
Qwen: Qwen 2 7B (Instruct)
Cohere: Command R7B (12-2024)
Google: Gemini Pro 1.5
Microsoft: WizardLM-2 8x22B
Teknium: OpenHermes 2.5 Mistral 7B
NousResearch: Hermes 2 Pro - Llama-3 8B
Cohere: Command R (08-2024)
Amazon: Nova Micro 1.0
Mistral: Mistral NeMo (v24.07)
DeepSeek: DeepSeek R1 Distill Qwen 32B
NousResearch: Hermes 3 70B (Instruct)
Google: Gemini Flash 1.5 8B
Cohere: Command R+ (04-2024)
AI21: Jamba 1.5 Large
Mistral: Mixtral 8x7B (Instruct) (v0.1)
Mistral: Mistral Medium
Cognitive Computations: Dolphin 2.9.2 Mixtral
8x22B
NVIDIA: Llama 3.1 Nemotron 70B (Instruct)
AionLabs: Aion-1.0-Mini
Cohere: Command R+ (08-2024)
Mistral: Codestral Mamba
Mistral: Mixtral 8x22B (Instruct) (v0.1)
NousResearch: Hermes 3 405B (Instruct)
Google: Gemma 2 27B
Microsoft: Phi 4
Amazon: Nova Lite 1.0
MiniMax: MiniMax-01
Meta: Llama 3.3 70B (Instruct)
Mistral: Mistral Small 3
Qwen: Qwen2.5 7B (Instruct)
Mistral: Pixtral 12B (v2409)
Google: Gemma 2 9B
Perplexity: Llama 3.1 Sonar 70B
Mistral: Mistral Small (v24.02)
Databricks: DBRX 132B (Instruct)
Anthropic: Claude 3 Sonnet
Perplexity: Llama 3 Sonar 70B (Online)
Google: Gemini Flash 2.0
Meta: Llama 3.1 70B (Instruct)
Meta: Llama 3.1 405B (Instruct)
Meta: Llama 3 70B (Instruct)
AionLabs: Aion-1.0
Amazon: Nova Pro 1.0
Qwen: Qwen-Turbo (2024-11-01)
OpenAI: o3-mini (2025-01-31)
(reasoning_effort=medium)
OpenAI: o3-mini (2025-01-31)
(reasoning_effort=low)
AI21: Jamba-Instruct
OpenAI: o3-mini (2025-01-31)
(reasoning_effort=high)
Qwen: Qwen 2 72B (Instruct)
01.AI: Yi Large
Google: Gemini Flash 1.5
AI21: Jamba 1.5 Mini
OpenAI: o1-preview (2024-09-12)
Anthropic: Claude 3 Haiku
DeepSeek: DeepSeek V2.5
Anthropic: Claude 3.5 Haiku (2024-10-22)
OpenAI: GPT-4o-mini (2024-07-18)
Mistral: Pixtral Large (2411)
Anthropic: Claude 3.7 Sonnet (Thinking)
Qwen: Qwen-Plus
Mistral: Codestral (2501)
Mistral: Mistral Large 2 (2411)
Anthropic: Claude 3.7 Sonnet (2025-02-19)
Google: Gemini 2.0 Flash Lite
Anthropic: Claude 3.5 Sonnet (2024-06-20)
Qwen: Qwen-Max
Qwen: Qwen2.5 72B (Instruct)
Qwen: Qwen2.5 Coder 32B (Instruct)
OpenAI: o1-mini (2024-09-12)
xAI: Grok-2 (1212)
Mistral: Mistral Large 2 (2407)
DeepSeek: DeepSeek V3
Qwen: Qwen2.5 32B Instruct
DeepSeek: DeepSeek R1
Anthropic: Claude 3 Opus
OpenAI: GPT-4o (2024-11-20)
Anthropic: Claude 3.5 Sonnet (2024-10-22)
598.10
363.89
126.77
115.76
106.32
96.56
84.83
84.40
79.08
75.10
72.43
70.67
67.49
59.46
56.38
52.96
52.47
51.35
51.22
50.72
48.67
48.45
48.23
43.17
41.25
39.66
38.98
38.58
38.46
37.40
35.97
35.04
34.01
30.50
28.87
28.10
27.57
27.45
26.31
24.75
24.73
24.66
24.40
24.18
24.06
24.00
23.85
23.17
23.04
22.49
22.28
22.22
21.99
20.70
20.43
20.29
19.57
18.99
18.93
18.05
17.83
17.81
17.36
17.27
17.22
17.20
17.15
16.83
16.56
16.46
16.22
15.79
15.65
15.24
15.22
15.20
15.13
14.93
14.67
14.49
14.43
14.36
14.14
14.08
14.04
13.98
13.70
13.60
13.58
13.58
13.52
13.46
13.41
13.34
13.24
13.16
13.14
13.07
12.81
12.67
12.61
12.46
12.30
12.06
12.03
11.51
10.33
Download SVG
0