【Maka agent】跑起K3 单挑 KimiCode

本帖使用社区开源推广，符合推广要求。我申明并遵循社区要求的以下内容：

- 我的帖子已经打上 开源推广 标签： 是

- 我的开源项目完整开源，无未开源部分： 是

- 我的开源项目已链接认可 LINUX DO 社区： 是

- 我帖子内的项目介绍，AI生成、润色内容部分已截图发出： 是

- 以上选择我承诺是永久有效的，接受社区和佬友监督： 是

以下为项目介绍正文内容，AI生成、润色内容已使用截图方式发出

同一个模型，配不同的 harness，差距能有多大？

github.com

GitHub - maka-agent/maka-agent: Maka — local-first AI desktop assistant

Maka — local-first AI desktop assistant

我们拿 Kimi K3 跑了一遍 Terminal-Bench 2.1，对比官方 Kimi Code CLI 和 Maka：整体通过率 Kimi Code 59.6%，Maka 69.7%，差 10 %。困难题子集差距更大：43.3% 对 63.3%，差 20 %。

这里面有 1/3 的任务是因为 Kimi 推理服务超时失败的，跟 harness 本身无关。如果只看按时跑完的任务：Kimi Code 85.7%，接近官方给 K3 的基准分 88.3%；Maka 95.1%，逼近 TB 2.1 目前最高分 89.5%（gpt-5.6-sol）。也就是说，把推理速度这个变量剔除掉之后，差距其实更悬殊。

差距从哪来，拆开看很朴素：

◉ 第一，context-budget tool-result prune。89 个任务全开，省了约 187 万 token。这不是 K3 专属功能，是我们对所有模型都通用的机制——6 月底做过 121 组任务的 A/B 测试，验证过它性能不降反升（+2.48pp），token 消耗降 41.7%，成本降 31.6%。

◉第二，工具面和 system prompt。Maka 用的是精简工具面 + system prompt，Kimi Code 官方是 20KB 产品 prompt + 全量工具面。prompt 越厚、工具越多，模型要在噪音里找信号的成本就越高。另外 Maka 还根据 K3 的特点进行了一些小优化和bugfix。

◉第二，各种工程实践的过程中，有很多细小的优化点。经过了大量的AB test积攒出来的小的优势，最终累积成了一个大的优势。

不是说 Kimi 团队不用心——Kimi 在国产模型厂里已经算是对 harness 最上心的一家了。

差距主要是来自我们从一开始就把"跑分-看 trace-针对性改进-重跑"当成日常迭代循环，每次改动都要确认 terminal benchmark 分数不掉才合并。这个习惯攒了两个多月，才攒出这个差距。

完整报告和 harness 已经开源，欢迎复现：

github.com/maka-agent/maka-agent

docs: Terminal-Bench 2.1 Maka vs Kimi Code v11 results (#1217)

main ← docs/tb21-maka-vs-kimi-code-v11-report

已合并 03:36AM - 19 Jul 26 UTC

Astro-Han

+269
-0

Summary

Docs-only: English write-up of Terminal-Bench 2.1 harness A/B run k…3-maka-vs-kimi-code-tbench-2.1-full-v11 (Maka vs Kimi Code, both k3 / max thinking), plus a redaction-minimal accepted-outcome CSV.

Full report: 0

Results

Official reference: 88.3% for Kimi K3 + KimiCode harness (tech blog, accessed 2026-07-19).

Kimi Code's accepted 53/89 is operator-adjusted (mechanical first verifier = 50/89). Three score-changing recoveries: prove-plus-comm, hf-model-inference, pypi-server. Run admissions: 203 physical → 178 accepted (25 recovery). Single-repetition; descriptive, not statistical.

Absolute scores are throughput-limited

Decode was ~34–37 output tok/s with ~14 s median first-token latency against task-native agent budgets. About a third of tasks per arm hit the deadline mid-generation and rarely passed (14–15%).

Finished-in-time pass rates (95.1% / 85.7%; 95.9% / 89.8% on the 49 tasks both arms finished) sit around the official 88.3%. Under healthy throughput, Maka would plausibly reach or exceed that reference and Kimi Code land near it — subsets are self-selected, so a re-run is decisive.

Official frontier is already compressed (~88.8% / 88.3%). Unconstrained subsets pass at 85.7–95.9%. TB 2.1 is close to saturated at the frontier; harder benchmarks will carry more signal.

Relative gap is harness asymmetry

Thinking effort, thinking retention, and wall-clock are aligned (same degraded network path). The +10.1 pp (Maka) persists among finished tasks (95.1% vs 85.7%). Differences below are listed, not ranked as score causes.

Background Bash is not the exclusive-win story (2/12 Maka-only tasks used run_in_background on Kimi). Heavy-task self-check is in the build but was off (zero tool/gate events). Recorded cost is $0 (account-plan); list-price uses official K3 API rates.

Verification

• Load-bearing numbers recomputed from the frozen run: controller/results.jsonl{,.attempts.jsonl}, trial result.json, Kimi provider-request-telemetry.json, Maka maka-cell-output.json / runtime streams, harness-ab-report.json, task task.toml budgets
• Official 88.3% cross-checked via secondary sources quoting the tech blog table
• CSV 89/89 match to accepted outcomes; SHA-256 integrity table in the doc
• External review (Codex + Claude Fable); P0–P2 findings resolved
• Docs-only; no code, tests, or raw eval artifacts; git diff --check clean