deepseek

deepseek-v4-pro

Pass rate: 39%23/59 passedAvg: 2m 45sTokens: 6.4MCost: $1.98

Per-task results

Task	Suite	Status	Duration	Tokens	Cost	Turns	Patch
Add capitalize util	fast	pass	48.0s	43.5k	$0.013	—	yes
A test passes for the wrong reason — find the real fix	hard-targeting	pass	6m 17s	125.5k	$0.053	—	yes
Migrate from fs.readFile(callback) to fs.promises.readFile()	hard-targeting	pass	57.8s	115.1k	$0.033	—	yes
Find and fix an event listener leak	hard-targeting	done	1m 53s	159.9k	$0.050	—	yes
Rename a public function across the codebase	hard-targeting	pass	1m 15s	106.3k	$0.033	—	yes
Fix concurrent counter race condition	hard-targeting	pass	2m 47s	72.4k	$0.023	—	yes
Restore linear performance in a hot loop	hard-targeting	done	5m 17s	110.6k	$0.034	—	yes
Add an options parameter to format() without breaking callers	hard-targeting	pass	2m 39s	160.7k	$0.050	—	yes
Read SPEC.md and implement the right behavior	hard-targeting	done	1m 21s	86.9k	$0.027	—	yes
Two parsers disagree on the same config file	hard-targeting	pass	3m 10s	124.0k	$0.039	—	yes
Migrate from fs.readFile(callback) to fs.promises.readFile()	hard-targeting	pass	7m 43s	174.7k	$0.054	—	yes
Fix implicit string→number coercion in comparison	hard-targeting	done	3m 21s	100.2k	$0.031	—	yes
A test passes for the wrong reason — find the real fix	hard-targeting	pass	5m 11s	100.4k	$0.031	—	yes
Find and fix an event listener leak	hard-targeting	pass	53.5s	83.4k	$0.025	—	yes
Rename a public function across the codebase	hard-targeting	pass	34.4s	92.6k	$0.027	—	yes
Fix concurrent counter race condition	hard-targeting	pass	27.8s	64.7k	$0.019	—	yes
Restore linear performance in a hot loop	hard-targeting	pass	33.4s	84.8k	$0.025	—	yes
Add an options parameter to format() without breaking callers	hard-targeting	pass	50.3s	133.4k	$0.038	—	yes
Read SPEC.md and implement the right behavior	hard-targeting	pass	39.9s	88.1k	$0.026	—	yes
Two parsers disagree on the same config file	hard-targeting	pass	56.9s	110.9k	$0.033	—	yes
Migrate from fs.readFile(callback) to fs.promises.readFile()	hard-targeting	pass	49.1s	85.0k	$0.026	—	yes
Fix implicit string→number coercion in comparison	hard-targeting	pass	1m 10s	162.9k	$0.047	—	yes
A test passes for the wrong reason — find the real fix	hard-targeting	pass	2m 44s	126.4k	$0.046	—	yes
Debug an off-by-one in binary search	harder	done	5m 44s	85.9k	$0.027	—	yes
Fix off-by-one in interval overlap detector	harder	done	50.8s	56.7k	$0.018	—	yes
Fix the async cache stampede	harder	done	1m 7s	73.3k	$0.023	—	yes
Rename `process` → `handle` across 4 files	harder	done	1m 54s	82.1k	$0.026	—	yes
Extract `validateAmount` from `submit`	harder	done	1m 1s	78.9k	$0.025	—	yes
Split src/store.js into store + selectors	harder	done	1m 44s	99.2k	$0.031	—	yes
Build a token-bucket limiter with exact semantics	harder	pass	8m 49s	145.8k	$0.045	—	yes
Implement formatDuration(ms) with strict spec	harder	done	1m 56s	93.4k	$0.029	—	yes
Implement deepEqual with cycle detection	harder	done	2m 7s	88.5k	$0.028	—	yes
Test the groupBy implementation	harder	done	3m 44s	235.3k	$0.073	—	yes
Test the flatten implementation	harder	done	1m 49s	138.7k	$0.043	—	yes
Test the debounce implementation	harder	done	4m 46s	131.0k	$0.041	—	yes
Debug an off-by-one in binary search	harder	timeout	10m 41s	65.9k	$0.021	—	yes
Fix off-by-one in interval overlap detector	harder	done	54.9s	80.3k	$0.025	—	yes
Fix the async cache stampede	harder	done	1m 5s	64.0k	$0.020	—	yes
Rename `process` → `handle` across 4 files	harder	done	1m 6s	100.6k	$0.031	—	yes
Extract `validateAmount` from `submit`	harder	done	1m 12s	93.6k	$0.029	—	yes
Split src/store.js into store + selectors	harder	done	45.9s	87.4k	$0.027	—	yes
Build a token-bucket limiter with exact semantics	harder	done	13m 4s	117.4k	$0.037	—	yes
Implement formatDuration(ms) with strict spec	harder	done	1m 3s	78.4k	$0.024	—	yes
Implement deepEqual with cycle detection	harder	done	1m 57s	110.2k	$0.034	—	yes
Test the groupBy implementation	harder	done	2m 40s	206.5k	$0.064	—	yes
Test the flatten implementation	harder	done	1m 59s	138.2k	$0.043	—	yes
Test the debounce implementation	harder	done	3m 6s	231.5k	$0.072	—	yes
Debug an off-by-one in binary search	harder	done	5m 46s	56.4k	$0.018	—	yes
Fix off-by-one in interval overlap detector	harder	done	1m 1s	91.4k	$0.028	—	yes
Fix the async cache stampede	harder	done	57.3s	56.3k	$0.018	—	yes
Rename `process` → `handle` across 4 files	harder	done	1m 50s	132.5k	$0.041	—	yes
Extract `validateAmount` from `submit`	harder	done	1m 6s	106.4k	$0.033	—	yes
Split src/store.js into store + selectors	harder	pass	1m 4s	57.9k	$0.018	—	yes
Build a token-bucket limiter with exact semantics	harder	pass	13m 4s	157.8k	$0.049	—	yes
Implement formatDuration(ms) with strict spec	harder	pass	1m 28s	69.2k	$0.022	—	yes
Implement deepEqual with cycle detection	harder	done	2m 26s	80.9k	$0.025	—	yes
Test the groupBy implementation	harder	failed	2m 54s	120.6k	$0.038	—	yes
Test the flatten implementation	harder	done	1m 56s	143.6k	$0.045	—	yes
Test the debounce implementation	harder	done	1m 28s	96.5k	$0.030	—	yes

All models