Mervin Praison

Shubham Saboo’s workflow puts Codex on build, Claude Code on review, and Hermes Agent on verification—so no worker can claim “tests passed” without the shell proving it. The shared primitive is /goal: you define what done means once; the agent loops until a judge agrees—or budget runs out.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  USER[You define done criteria] --> GOAL[/goal standing objective]
  GOAL --> BUILD[Codex builds]
  BUILD --> REVIEW[Claude Code reviews]
  REVIEW --> VERIFY[Hermes runs shell checks]
  VERIFY --> JUDGE{Judge: done or continue?}
  JUDGE -->|continue| GOAL
  JUDGE -->|done| SHIP[Merge / ship]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class BUILD agent
  class REVIEW agent
  class VERIFY hook
  class JUDGE decision

Codex implements, Claude Code reviews, Hermes re-runs checks in the terminal.

Why agents lie about “done”

A normal prompt optimises for the next reply. You read it, steer, repeat. Agents routinely report success without evidence: builds that never ran, tests that were written but not executed, green checkmarks in prose only. Saboo’s fix is structural—measurable end states plus a verifier that does not trust self-report. On a Mac Mini orchestrator, Hermes re-runs npm test, cargo build, or whatever your goal specifies before accepting completion.

The /goal primitive defines done once; a judge decides continue or complete each turn.

Prompt vs /goal

The pattern shipped in OpenAI Codex CLI 0.128.0 (Eric Traut; see Follow a goal) and was adapted independently in Hermes Agent—same Ralph-loop idea, different persistence and gateway plumbing.

Anatomy of a good goal (four parts)

Saboo’s cheat sheet adds a verifier checklist: if you cannot reproduce PASS from a terminal command, treat the agent’s narrative as unverified. That turns /goal from a longer prompt into a contract.

Three tools, one primitive

Codex — builder

Use /goal for long-horizon implementation: migrations, multi-file refactors, eval loops. Enable features.goals = true in Codex config.toml if the slash command is missing. Codex injects continuation and budget prompts each turn (goals/continuation.md, goals/budget_limit.md per release notes). Pair with codex-plugin-cc inside Claude Code for /codex:review and /codex:rescue without leaving the session.

Claude Code — reviewer

Run /goal on review-shaped work: “Refactor module X; measurable end state = tests pass + no new lint errors + ADR updated.” Use Skills to inject CLAUDE.md, pre-approve Bash(npm test), and fork Plan/Explore subagents for planning before execution. Official plugin: /codex:review for read-only Codex audit; /codex:setup --enable-review-gate can block Claude from finishing until Codex reviews.

Hermes Agent — orchestrator + verifier

Hermes persists goals in SessionDB.state_meta, survives /resume, and runs a separate goal_judge model each turn (~4 KB of the last response → JSON {"done": bool, "reason": "..."}). Default 20 continuation turns (goals.max_turns); /goal resume resets the counter. Subgoals tighten criteria mid-loop: /subgoal add regression test for bug Y.

Saboo’s Kanban pattern (community, also documented on goal-feature guides): cards like CODEX GOAL: BUILD …, CLAUDE CODE GOAL: REVIEW …, HERMES GOAL: VERIFY … on a board at 127.0.0.1:9118/kanban—each card is its own /goal, agents keep looping until judges confirm.

Hermes /goal commands

/goal Fix every failing test in tests/auth/ and confirm scripts/run_tests.sh passes

/goal status
/goal pause
/goal resume    # resets turn counter
/goal clear

/subgoal add a regression test for the JWT refresh bug
/subgoal        # list subgoals

Cheap judge routing (optional) in ~/.hermes/config.yaml:

goals:
  max_turns: 20

auxiliary:
  goal_judge:
    provider: openrouter
    model: google/gemini-3-flash-preview

Anti-patterns (and fixes)

Verifier checklist (shell-first)

• Did the agent paste stdout/stderr from the real command, or only claim success?
• Re-run npm test / pytest / cargo test yourself—or let Hermes run it before marking done
• Check git status and diff scope match constraints
• Reject goals completed on a dirty tree without an explicit branch strategy
• Read the judge reason on ↻ Continuing / ✓ Goal achieved lines when verdicts look wrong

Example goal (copy-paste template)

/goal Implement JWT refresh tokens for the auth module.

Measurable end state:
- pytest tests/auth/ -q exits 0 with ≥90% coverage on app/auth/
- bandit -r app/auth/ reports no HIGH issues
- docs/API.md lists /auth/refresh with request/response schema
- git status --porcelain is empty

Constraints:
- Only edit app/auth/ and tests/auth/
- No changes to billing or admin packages
- Conventional Commits; one commit per logical step

Stop: after 20 turns write BLOCKERS.md and pause.

Performance and cost snapshot

Saboo’s line—“Workers change. The primitive stays the same.”—is the takeaway: whether Codex ships the feature, Claude reviews the diff, or Hermes orchestrates from a Mac Mini, success depends on defining done in the shell and verifying before you believe. Start with a 10-minute goal (four files, one test command), watch the judge loop once, then promote the same template to your real refactor.

CodeGraph pre-indexes your repository into a local semantic knowledge graph so coding agents (Claude Code, Cursor, Codex, OpenCode, and others) spend fewer tokens on grep-and-read exploration. Dr. Alvaro Cintas’s LinkedIn post highlights up to 94% fewer tool calls and 77% faster codebase exploration; the project’s own multi-repo benchmarks report medians closer to ~71% fewer calls and ~46% faster on average—with the largest wins on big TypeScript and Rust trees.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  REPO[Source files] --> INDEX[tree-sitter + SQLite graph]
  INDEX --> MCP[CodeGraph MCP server]
  MCP --> AGENT[Claude Code / Cursor / Codex]
  AGENT --> OUT[Answer with fewer Read/Grep loops]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff

  class AGENT agent
  class INDEX hook
  class MCP hook

Without an index, agents re-scan the repo; CodeGraph answers from a local map built once.

The problem: exploration burns tokens

When an agent lacks structural context, it often spawns Explore sub-agents that chain grep, glob, and Read across thousands of files—paying model tokens for every hop. Architecture questions on repos like VS Code or Excalidraw can balloon to dozens of tool calls and millions of tokens before the model reads the right module.

SQLite-backed graph keeps source intelligence on-device for Claude Code, Cursor, and Codex.

What CodeGraph does

CodeGraph (MIT, by Colby McHenry) builds a pre-indexed graph on your machine: symbols, call relationships, full-text search (SQLite FTS5), framework routes, and optional cross-language bridges (Swift↔ObjC, React Native, Expo). Agents reach it through an MCP server (codegraph serve --mcp)—no source upload, no API keys for indexing.

• Smart context: tools like codegraph_context return entry points, related symbols, and snippets in one shot
• Traversal: explore callers, callees, and impact radius before refactors
• Routes: 14+ web frameworks (Django, FastAPI, Express, NestJS, Rails, Spring, Gin, Axum, etc.) link URL patterns to handlers
• Fresh index: native file watchers (FSEvents / inotify / ReadDirectoryChangesW) with debounced re-sync; staleness banners during pending updates
• 20+ languages: TypeScript, Python, Rust, Go, Java, Swift, Kotlin, C#, PHP, and more

Benchmarks (with vs without CodeGraph)

Official methodology (re-validated on v0.9.4, May 2026): headless Claude Code with Opus 4.7, one architecture question per repo, 4 runs per arm, median reported. WITH = CodeGraph MCP enabled; WITHOUT = empty MCP config but built-in Read/Grep/Bash still available.

Example medians on VS Code (“How does the extension host communicate with the main process?”): 8 tool calls with CodeGraph vs 55 without; ~601k vs ~2.8M tokens. Cintas’s 94% / 77% figures align with the best large-repo cells (e.g. Excalidraw 96% fewer calls, 73% faster)—not every project sees that peak; small repos like Gin show narrower margins because naive search is already cheap.

Supported agents

Interactive installer wires MCP config for Claude Code, Cursor, Codex CLI, OpenCode, Hermes Agent, Gemini CLI, Antigravity, and Kiro. The LinkedIn post matches the README’s core quartet plus OpenCode.

Install and index a project

# macOS / Linux (bundled runtime — no Node required)
curl -fsSL https://raw.githubusercontent.com/colbymchenry/codegraph/main/install.sh | sh

# Or npm / npx
npm i -g @colbymchenry/codegraph

# Register MCP with your agent(s)
codegraph install

# Index the repo (interactive init)
cd your-project
codegraph init -i

# Optional: serve MCP manually
codegraph serve --mcp

Indexes live under .codegraph/ per project. Remove agent integration with codegraph uninstall; drop project data with codegraph uninit.

Why it wins (and when it does not)

Maintainers note CodeGraph only helps when the primary agent queries the graph directly; otherwise Explore sub-agents may still burn tokens on raw file reads. Project instructions steer agents toward codegraph_context first, then targeted exploration—mirroring the “don’t burn tokens exploring” message in Cintas’s post.

Performance snapshot

For teams running agents on big codebases daily, CodeGraph is a practical layer between “raw repository” and “model context”: pay indexing cost once, then replace repetitive discovery loops with graph queries. Start with codegraph init -i on your main app, confirm MCP is active in your agent, and compare tool-call counts on the same architecture prompt—with and without the index.

Voxtral TTS is Mistral’s new open-weight text-to-speech model—a 4B-parameter stack aimed at voice agents that can be self-hosted or called via API. Viral posts claim Mistral “made ElevenLabs open source”; in practice Mistral shipped a competing TTS layer with public weights on Hugging Face, not ElevenLabs’ proprietary models.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  TEXT[Text + 3s voice reference] --> VOX[Voxtral TTS 4B]
  VOX --> SEM[Semantic tokens AR]
  SEM --> FLOW[Flow-matching acoustic]
  FLOW --> CODEC[Voxtral codec 12.5 Hz]
  CODEC --> AUDIO[24 kHz speech stream]
  AUDIO --> AGENT[Voice agent / support / dubbing]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff

  class VOX agent
  class AUDIO agent
  class SEM hook
  class FLOW hook

Voxtral ships downloadable weights; proprietary voice platforms keep models behind APIs.

What actually launched

On 23 March 2026, Mistral announced Voxtral TTS: its first production TTS model for enterprise voice workflows. Weights and preset reference voices live on Hugging Face under CC BY-NC 4.0 (research and non-commercial use of those weights; commercial deployment typically goes through Mistral’s paid API). ElevenLabs remains a separate, closed platform—Mistral’s pitch is quality and control without renting every audio frame from a single vendor.

Three-second cloning and emotion steering target real-time agent workflows.

Headline specs

Architecture (how it works)

Voxtral TTS is a hybrid generative stack, not a single end-to-end waveform net:

• 3.4B transformer decoder (autoregressive semantic speech tokens)
• 390M flow-matching acoustic transformer (16 NFEs per frame)
• 300M in-house Voxtral codec (semantic VQ + acoustic FSQ at 12.5 Hz)
• Inputs: text + voice prompt (roughly 5–25 s in the technical blog; cloning demos use 3 s minimum)
• Output: 24 kHz audio (WAV, PCM, FLAC, MP3, AAC, Opus via API)

Zero-shot cross-lingual adaptation is a differentiator: e.g. English text with a French reference clip can yield French-accented English without explicit cross-lingual training—useful for dubbing and cascaded speech-to-speech pipelines alongside Voxtral Transcribe.

Benchmarks and caveats

Mistral and the arXiv report emphasise native-speaker listening tests, not word-error rate alone. Reported highlights:

• 68.4% win rate vs ElevenLabs Flash v2.5 on multilingual zero-shot custom voices
• Competitive with strong proprietary systems on flagship preset voices (smaller margin than cloning setup)
• Automatic metrics: strong on SEED-TTS / MiniMax-TTS; speaker-similarity claims vs ElevenLabs v3 in paper tables
• vLLM-Omni on one H200: ~70 ms latency at concurrency 1; RTF ≈0.10 (standard convention: lower is faster)

Read claims carefully: evaluations were run by Mistral; Flash is the speed tier, while ElevenLabs v3 is the expressive flagship—Mistral argues parity on emotion, not a clean “beats everything” sweep. CC BY-NC is not the same as Apache-style commercial open source: product teams needing unrestricted commercial use of weights should confirm license terms or use the API.

Run it yourself (vLLM-Omni)

# Install (see HF model card for pinned versions)
uv pip install -U vllm
uv pip install vllm-omni --upgrade  # >= 0.18.0
python3 -c "import mistral_common; print(mistral_common.__version__)"  # >= 1.10.0

# Serve on a GPU with >= 16 GB VRAM
vllm serve mistralai/Voxtral-4B-TTS-2603 --omni

# Client (OpenAI-style audio/speech endpoint)
import io, httpx, soundfile as sf
BASE_URL = "http://localhost:8000/v1"
payload = {
    "input": "Paris is a beautiful city!",
    "model": "mistralai/Voxtral-4B-TTS-2603",
    "response_format": "wav",
    "voice": "casual_male",
}
r = httpx.post(f"{BASE_URL}/audio/speech", json=payload, timeout=120.0)
r.raise_for_status()
audio, sr = sf.read(io.BytesIO(r.content), dtype="float32")

Try presets in Mistral Studio or Le Chat; record a custom reference for cloning. Gradio demo ships with vllm-omni examples; a Hugging Face Space is linked from the model card.

When to pick which stack

Enterprise use cases (from Mistral)

• Customer support and contact-centre voice bots
• Banking / KYC voice agents (demo narratives in launch materials)
• In-vehicle and industrial hands-free UX
• Real-time translation and dubbing with cross-lingual voice carry-over
• Sales, marketing, and compliance read-outs paired with Voxtral speech-to-text

Performance snapshot

The LinkedIn framing—“Mistral made ElevenLabs open source”—captures the shift (frontier TTS weights you can run yourself) more than the literal fact pattern. For builders, the actionable story is simpler: Voxtral TTS is a credible open-weight speech layer for agents, with measured wins on cloning latency and multilingual naturalness, while proprietary incumbents still win on ecosystem breadth until you need on-prem control.

Multi-token prediction (MTP) in llama.cpp speeds up local Qwen 3.6 generation by building speculative decoding into the model itself—Hugging Face CTO Julien Chaumond’s quickstart shows you only need a recent build, an MTP GGUF from ggml-org, and two flags on llama-server.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  CLI[llama-server + MTP GGUF] --> FLAGS["--spec-type draft-mtp"]
  FLAGS --> DENSE[Dense 27B MTP]
  FLAGS --> MOE[MoE 35B-A3B MTP]
  DENSE --> OUT[Faster token stream]
  MOE --> OUT

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff

  class CLI agent
  class OUT agent
  class FLAGS hook

Multi-token prediction bundles draft guesses inside the same model file so decode steps emit more accepted text.

What MTP changes

MTP is a draft head trained with the base model, not a separate small “speculator” you download and wire up by hand. At decode time the head proposes several candidate next tokens; the main model verifies them in one pass. When draft tokens are accepted, you emit more text per forward step—Chaumond and the merged llama.cpp MTP PR (#22673) describe roughly ~2× generation throughput in favourable setups, though real gains depend on hardware, quantisation, and how many draft tokens you allow.

The MTP weights ship in the same GGUF as the main checkpoint; llama.cpp loads a lightweight MTP context (extra KV cache, typically under ~10% memory versus the full model). You opt in with flags—MTP does not run unless you ask for it.

Both checkpoints use the same MTP flags; pick the variant that matches your RAM and speed goals.

Prerequisites

Commands (copy-paste)

Install or refresh llama.cpp, then start the server with MTP enabled. Chaumond’s post uses --spec-draft-n-max 2 on dense and 3 on MoE; community benchmarks on the MoE often favour n-max 2 when acceptance rate drops at wider draft windows.

# Refresh llama.cpp (macOS example)
brew upgrade llama.cpp
# Or until stable packages catch up:
# brew install llama.cpp --HEAD

# Dense 27B — balanced quality (~30 tok/s on author’s box)
llama-server -hf ggml-org/Qwen3.6-27B-MTP-GGUF \
  --spec-type draft-mtp --spec-draft-n-max 2

# MoE 35B-A3B — much faster when it fits (~100 tok/s in the post)
llama-server -hf ggml-org/Qwen3.6-35B-A3B-MTP-GGUF \
  --spec-type draft-mtp --spec-draft-n-max 3

Optional: add --no-mmproj if you do not need vision—saves memory. Advanced users can combine MTP with ngram drafting on supported builds; treat that as experimental.

Dense vs MoE: which to pick

How to read speed-up claims

• Decode vs prefill: MTP mainly helps token generation; prompt processing can be slower because of extra embedding transfers (noted in the PR).
• Acceptance rate: Wider --spec-draft-n-max drafts more tokens per step but wastes work when guesses are wrong—measure predicted_per_second and draft acceptance, not prompt-processing rate.
• Quality: PR authors ran AIME-style evals; scores stayed in line with Qwen’s published benchmarks when MTP is enabled.
• Hardware spread: Strix Halo, RTX 4090/5090, and laptop 6 GB+RAM reports range from modest (~1.2×) to near ~2× depending on quant and n-max.

Common confusion (answered)

Performance snapshot

MTP turns Qwen 3.6 local runs from “one token per heavy step” into “verify a short bundle of guesses”—with a single Hub pull and two CLI flags once llama.cpp is current. Start with the dense GGUF if memory is tight; reach for the MoE MTP pack when you have headroom and care about tokens per second for long coding or agent loops.

HF Viewer (hfviewer.com) is a free browser tool from Embedl that turns any public Hugging Face model into an interactive architecture graph—paste a repo URL, swap huggingface.co for hfviewer.com, or embed the graph in your model card without installing PyTorch locally.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  HF[Hugging Face model page] --> URL[hfviewer.com/owner/model]
  URL --> GRAPH[Interactive architecture graph]
  GRAPH --> ZOOM[Granularity: overview to blocks]
  GRAPH --> EMBED[Optional README embed]
  GRAPH --> EXT[Chrome extension on HF]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff

  class GRAPH agent
  class HF hook
  class EMBED hook

The fastest way to open a graph is to change the domain in any public model link.

What HF Viewer does

Model cards explain what a checkpoint is for; they rarely give you a fast map of how it is wired. HF Viewer fills that gap: open a graph of layers, attention blocks, MoE routes, vision encoders, and merges directly in the browser. Embedl describes it as a “first architectural pass” before you read configs, trace code, or plan deployment and latency.

Use granularity levels to move from system shape down to specific traced paths.

Three ways to open a graph

Example: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro becomes https://hfviewer.com/deepseek-ai/DeepSeek-V4-Pro.

Granularity and exploration

The viewer exposes granularity levels: start at the high-level system shape (encoder–decoder, decoder-only, dual-tower CLIP, sparse MoE), then drill into traced sub-blocks and data paths. That slider is useful when you care whether a vision tower feeds a merger, how many decoder layers repeat, or where experts route.

Popular entry points on the site include gpt2 (classic decoder), t5-small (deeper encoder–decoder), openai/clip-vit-base-patch32 (dual encoder), google/vit-base-patch16-224, Qwen/Qwen3.5-4B, deepseek-ai/DeepSeek-V4-Pro (sparse MoE), and nvidia/parakeet-tdt-0.6b-v3 (Conformer speech).

Gemma 4 family compare

hfviewer.com/family/gemma-4 lines up the Gemma 4 lineup with synchronised pan, zoom, and granularity so you can compare variants side by side—useful when size classes differ but the narrative in a blog post refers to a specific block (Embedl links prose sections to graph regions for a text↔graph reading loop).

Embed graphs in Hugging Face READMEs

The model-card embed builder generates HTML in roughly ten seconds: paste owner/model, pick card style (standard summary or block granularity), copy HTML into README.md. Community models already showcase embedded cards (custom GPT-X2 stacks, MEGA-based small LMs, emotion classifiers, Pegasus-X summarisation, Gemma 4 fine-tunes, and others).

If a visualization is not ready yet, the embed page offers email notification when generation completes—then you copy the final widget HTML.

How graphs are built (high level)

HF Viewer derives structure from Hugging Face model metadata and PyTorch module layout. Embedl staff on Hacker News noted multiple passes over the HF config, sometimes including torch.export and recombination steps to make repeated layer classes readable in the graph—hybrid architectures (Mamba + attention, MoE) remain harder and community feedback has flagged occasional mis-labelling on complex stacks.

It visualises the implemented architecture, not every hyperparameter from the card (hidden size, layer count, tokenizer details may appear inconsistently). It does not replace reading the paper or source for training and numerics.

Who it is for

• Developers comparing candidate open models before fine-tuning or quantisation
• Authors who want an architecture graphic on the model card
• Technical writers linking blog sections to live graph nodes
• Teams evaluating Embedl’s edge deployment products after inspecting structure

Limitations

• Public Hugging Face models only—private or local checkpoints are out of scope
• Browser-side—very large or exotic graphs may be slow or ambiguous
• Not a substitute for config files, weights inspection, or benchmark numbers
• Complex hybrids may need manual verification (community reports on some Nemotron-style layouts)

Embedl context

Embedl (edge AI optimisation, quantisation, MLOps) positions HF Viewer as a community gift to Hugging Face users; the homepage cross-links embedl deploy, embedl hub, and optimised GenAI models for teams moving from exploration to edge deployment.

At a glance

DeepSWE, released by Datacurve on 26 May 2026, is a long-horizon agentic coding benchmark built to show where frontier models actually diverge when public leaderboards make them look neck-and-neck—113 original tasks across 91 open-source repositories and five languages, with hand-written behavioural verifiers and no solutions lifted from public pull requests.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  P[Short behaviour-focused prompt] --> A[Coding agent in isolated repo]
  A --> PATCH[Multi-file patch]
  PATCH --> V[Hand-written verifier]
  V -->|pass| OK[Task solved]
  V -->|fail| NO[Regression or wrong behaviour]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class A agent
  class V hook
  class OK agent

DeepSWE is meant to mirror day-to-day agent gaps that saturated leaderboards hide.

What Serena Ge announced

Datacurve CEO Serena Ge (@serenaa_ge) posted that DeepSWE is a new standard for agentic coding benchmarks: on many public leaderboards, top models cluster in a narrow band, but DeepSWE is designed to reflect how developers experience agents in day-to-day work—with a much wider spread between best and worst performers.

Primary materials: deepswe.datacurve.ai, the methodology blog, and the open benchmark repo datacurve-ai/deep-swe. Runs use Pier with mini-swe-agent on Modal sandboxes.

Each task is an original change in a real repository, graded on observable behaviour not patch shape.

Four design bets vs older benchmarks

Leaderboard snapshot (mini-swe-agent harness)

All listed scores use the same agent harness so rankings reflect model differences, not Codex vs Claude Code scaffolding. Datacurve reports confidence intervals on pass rates; figures below are point estimates from the public leaderboard.

On these models, Datacurve notes DeepSWE pass rates span roughly 70 percentage points from worst to best versus about 30 points on publicly reported SWE-bench Pro scores—matching the tweet’s claim that leaderboards can hide real-world gaps.

Efficiency: score is not the whole story

Datacurve’s analysis stresses that more tokens, longer runs, or higher dollar cost do not reliably mean more passes—teams choosing an agent should weigh accuracy, latency, and price together, not assume the loudest/longest run wins.

Task format and how to run it

Tasks follow the Harbor layout: task.toml, instruction.md, Docker environment, tests/ verifier, and a held-out solution/ for human review only. Example task themes on the site include PromQL label sorting, Yjs map conflict policies, Wasm trap coredumps, and XML diff/merge in Go.

git clone https://github.com/datacurve-ai/deep-swe
uv tool install datacurve-pier

export OPENAI_API_KEY=...
pier run -p deep-swe/tasks --agent mini-swe-agent --model openai/gpt-5.5

# Random 10-task subset
pier run -p deep-swe/tasks --agent mini-swe-agent --n-tasks 10 --sample-seed 0

Why SWE-bench Pro rankings can mislead

Datacurve’s qualitative audit highlights structural issues on PR-derived benchmarks—notably gold commits visible in .git history (Claude Opus sometimes recovers fixes via git show), tests that import private helpers the prompt never names, and prompts that tell agents not to write tests—which suppresses self-verification behaviour strong models use on DeepSWE. DeepSWE shallow-clones the base commit so there is no merged fix hash to read.

Reported verifier disagreement rates (LLM judge vs automated grader, sampled rollouts): SWE-bench Pro ~32% disagreement overall; DeepSWE ~1.4%. False negative rates were ~24% vs ~1.1% respectively in their audit—wide error bars on older benchmarks make small leaderboard deltas hard to trust.

Failure modes developers should know

• Claude families — often miss one branch of multi-part prompts (“sync and async”, “line and block comments”).
• GPT-5.x — Datacurve finds lower MISSED_REQUIREMENT rates; tends to implement prompts literally.
• Cheating on Pro — Opus passes via reading gold history; GPT-5.x showed none in their sample.
• Weaker models — may skip running existing tests entirely on hard tasks.

Limitations (from Datacurve)

• Fixed mini-swe-agent harness—not native Claude Code / Codex CLI / Cursor workflows.
• Open-source repos with ≥500 stars only—may not reflect private or long-tail codebases.
• Five languages; C++, Java, and heavy refactor/localisation tasks under-represented.
• Qualitative tags use an LLM analyzer—some verdicts will be wrong.

Who should care

• Engineering leaders picking coding agents for production—not just benchmark leaderboard rank.
• Model labs needing contamination-resistant, long-horizon evals.
• Datacurve customers — the company sells curated coding data for frontier training; DeepSWE doubles as research marketing.

At a glance

Lamina Labs builds Simi, an AI explainer studio that turns a text prompt or uploaded document into a whiteboard-style video in seconds—aimed at students, course creators, customer training, and EdTech products that need concepts explained visually, not as walls of text.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  IN[Prompt or PPT/PDF/Word/TXT/MD] --> SIMI[Simi generation]
  SIMI --> ANIM[Step-by-step whiteboard animation]
  ANIM --> MP4[Explainer MP4]
  MP4 --> USE[Students / L&D / EdTech apps]
  SDK[lamina-sdk] --> API[api.laminalabs.ai]
  API --> SIMI

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff

  class SIMI agent
  class ANIM agent
  class SDK hook
  class API hook

Simi accepts uploads or a short description and outputs a drawn explainer video instead of static slides.

What Lamina Labs is building

At laminalabs.ai, Lamina positions itself as the visualisation layer for AI-native EdTech: infrastructure that helps intelligent systems draw, explain, and teach. The consumer-facing product is Simi (“AI explainer studio”), marketed as the world’s fastest explainer video tool—drop a document or type an idea, get a clear whiteboard walkthrough.

The company is a Y Combinator Spring 2026 batch startup (YC profile), founded in 2025 and based in San Francisco with a two-person founding team: Kartikesh Mishra (MIT EECS BS ’24, MEng ’25) and Sudip Rokaya (MIT CS & Math, on leave). Founders offer “Talk to Founder” booking via the site and host the live app at app.laminalabs.ai/simi.

Naming note: laminalabs.ai (Simi / EdTech explainers) is unrelated to Lamini (LLM tuning at lamini.ai) and unrelated to uselamina.ai (e-commerce creative generation). This article covers Lamina Labs only.

Lamina bets sequential drawing and pauses teach hard concepts better than glossy generative video.

How Simi is meant to feel

Lamina’s copy stresses pacing over production value: a rough line drawn in the right order should teach more than a glossy cinematic clip. Simi is described as drawing like a patient teacher—slow enough to follow, fast enough to stay engaged—with pauses as part of the pedagogy. Each stroke is framed as part of an argument (“because of this, therefore that”) rather than a finished illustration dropped on screen.

Example topics showcased on the homepage include recursion explained to a child, Netflix customer-support day-one training, and quantum tunnelling—signals that the product targets explanation-heavy STEM and onboarding content, not short-form social ads.

Inputs and outputs

The on-site workflow is deliberately simple: describe what to explain → Simi generates the animation → watch in seconds. Lamina argues a one-minute explainer is easier to share and rewatch than a five-page PDF, with less room for misreading.

Developer API: lamina-sdk

Integrators use the async-first Python package lamina-sdk (MIT licence, Python ≥3.11). The client defaults to https://api.laminalabs.ai; authenticate with LAMINA_API_KEY or pass api_key to simi().

from lamina import simi

async with simi(api_key="lamina_live_your_key") as client:
    video = await client.generate(
        "Explain derivatives with a simple graph",
        duration=20,
    )
    await video.save("lesson.mp4")

Additional patterns from the PyPI readme:

• submit_async + stream_events for progress streaming
• Callback style: onstream / oncompletion on jobs
• Sync helpers: submit, generate, save
• Dependencies: httpx, Pillow, websockets

Co-founder Sudip Rokaya’s public demos describe wiring Simi into agent stacks (for example Hermes Agent via Slack) so a single API call produces multi-minute whiteboard explainers without a video editor—positioning Simi as video generation infrastructure for EdTech platforms generating curriculum at scale, not only a web UI.

EdTech positioning vs other video AI

YC’s one-liner—“accurate visual explanations in seconds”—aligns with Lamina’s emphasis on correct explanatory visuals for learning, as opposed to templated or physically inconsistent generative video. Third-party databases sometimes reference an earlier internal name “Pictor”; the public product brand is Simi.

Who it is for

• Students and self-learners turning lecture confusion into a rewatchable minute-long explainer
• Course creators scaling lessons without hiring animators per concept
• Customer education / L&D (onboarding flows like support training)
• EdTech and agent builders embedding lamina-sdk so tutors, copilots, or curriculum bots emit video explanations automatically

Getting started

At a glance

Secure MCP Tunnel lets teams keep Model Context Protocol (MCP) servers on private networks while ChatGPT, Codex, and the Responses API reach them through outbound-only HTTPS—no inbound firewall ports and no public MCP endpoint.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  P[ChatGPT / Codex / Responses API] --> E[OpenAI-hosted MCP tunnel endpoint]
  E --> CP[Control plane api.openai.com]
  CP --> TC[tunnel-client inside your network]
  TC --> MCP[Private MCP server]
  MCP --> DATA[Internal tools and data]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class P agent
  class E hook
  class TC hook
  class MCP agent

Secure MCP Tunnel avoids public MCP endpoints and inbound firewall rules by pulling work from inside your network.

What OpenAI Developers announced

On 27 May 2026, @OpenAIDevs posted that private MCP servers can stay inside your network while OpenAI products connect through outbound-only HTTPS, linking to the official Secure MCP Tunnel guide. Greg Brockman quoted the post as “bring-your-own MCP servers”; developers including Steven Heidel highlighted using the same path to connect the Responses API to local MCP servers.

Automated fetch of status 2059703536825565499 returned 403 in some environments; claims below align with that post (via syndication) and OpenAI’s published documentation and tunnel-client repository.

The same tunnel-backed MCP server can power ChatGPT connectors, Codex sessions, and Responses API tool calls.

The problem it solves

Remote MCP usually means a public server_url that OpenAI’s platform can call over the internet. That is a poor fit when the MCP server lives on a laptop, in a VPC, or behind corporate firewalls. Opening inbound ports or publishing an internal tool stack is often blocked by security review.

Secure MCP Tunnel flips the direction: a customer-run agent, tunnel-client, inside your network initiates outbound HTTPS to OpenAI’s control plane, pulls queued MCP work, forwards JSON-RPC to the private server (stdio or HTTP), and posts responses back. The MCP server never needs a public listener.

Supported surfaces

Network and control-plane flow

The client long-polls GET /v1/tunnel/{tunnel_id}/poll and returns work via POST /v1/tunnel/{tunnel_id}/response. On startup it may fetch tunnel metadata from GET /v1/tunnels/{tunnel_id} for operator visibility. Optional mTLS uses --control-plane.client-cert / --control-plane.client-key (or env vars); with the default API host, control-plane traffic automatically targets mtls.api.openai.com.

When to use it

• MCP server is on-premises, on a developer machine, or in a private VPC.
• Security will not approve inbound internet access to the MCP process.
• Outbound HTTPS to OpenAI (api.openai.com:443, or mTLS host) is allowed from the tunnel host.
• You need ChatGPT, Codex, or API agents to call the same internal tools without exposing them publicly.

Quickstart (binary path)

OpenAI documents a binary-first path: download tunnel-client from Platform → Tunnels, create a tunnel (UI or tunnel-client admin tunnels create with an admin key), then run a profile against your local MCP server.

tunnel-client help quickstart

tunnel-client init \
  --sample sample_mcp_stdio_local \
  --profile local-stdio \
  --tunnel-id tunnel_0123456789abcdef0123456789abcdef \
  --mcp-command "python /path/to/server.py"

tunnel-client doctor --profile local-stdio --explain
tunnel-client run --profile local-stdio

For an HTTP MCP server inside the network, use an HTTP-oriented sample profile instead of stdio. Keep the daemon running while ChatGPT discovers the connector or while API/Codex sessions issue MCP calls. Health endpoints: /healthz, /readyz, /metrics, plus a local admin UI at /ui.

Keys, permissions, and workspace scope

Runtime principals need Tunnels Read + Use; managers who create tunnels need Manage as well. If a tunnel does not appear in ChatGPT, docs call out checking workspace association and the connector operator’s Tunnels permissions.

Harpoon: scoped private HTTP (not a full proxy)

The tunnel client embeds Harpoon, an MCP server that exposes allowlisted HTTP targets by label so agent flows can call a small set of private REST endpoints through the tunnel. OpenAI stresses this is not a general-purpose proxy—callers cannot pick arbitrary hosts; methods and targets are customer-configured with bounded request/response limits.

Security and trust

Outbound-only networking reduces exposure, but you must trust the MCP server you attach. OpenAI’s MCP guidance warns that malicious remote servers can exfiltrate anything that enters the model context. Prefer official servers operated by the service provider; for private tunnels, treat tunnel-client hosts like production infrastructure: patch the binary, rotate runtime keys, scope tunnels to the right workspace, and review tools exposed by your MCP implementation.

Public MCP vs Secure MCP Tunnel

At a glance

SOUL.md is a compact markdown “constitution” for local AI agents: roughly 30–80 lines that define role, voice, values, and boundaries before tools, memory, or skills load—so every run starts from a stable identity instead of a generic “be helpful” default.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  S[Session start] --> SOUL[SOUL.md identity]
  SOUL --> M[MEMORY.md + USER.md]
  M --> SK[Skills catalog]
  SK --> T[Tools + MCP]
  T --> DB[Session DB search]
  DB --> RUN[Agent run]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class SOUL agent
  class M agent
  class SK hook
  class T hook
  class DB hook

Identity and guardrails are injected before durable memory or tool definitions so every session starts from the same character.

Why a SOUL file matters

Most agents ship with a vague system prompt. A SOUL.md forces you to decide—up front—who the agent is, how it speaks, what it will not do, and how it should behave when facts are missing. That file is typically injected as slot #1 in the system prompt on every run (the pattern used by Hermes Agent and echoed in community frameworks such as Soul Agent Framework and soul-spec).

A widely shared LinkedIn breakdown (Charly Wargnier, May 2026) popularised a visual “anatomy” of SOUL.md: keep the file short, prioritise specificity over coverage, and define identity before memory or tools. The infographic itself is third-party art—we recreated the ideas below as original explainers rather than republishing that image.

A strong SOUL file stays short: five sections, specific rules, and no giant instruction dumps.

What belongs inside SOUL.md

Length and style rules

SOUL.md is not memory

Hermes Agent’s three-layer memory model separates concerns cleanly:

Good memory entries are declarative facts (“deploy via GitHub, not direct VPS shell publish”). Bad memory is a task log (“fixed bug X today”). Procedures with commands and verification steps belong in skills, not SOUL or MEMORY.

Skills and self-improvement

Hermes-style agents expose a skills catalog first, then load full SKILL.md content only when relevant—keeping the base prompt small. Agents can propose new skills or refine existing ones (for example via skill_manage and optional offline evolution such as GEPA), which is the “self-improving skills” angle in Akshay Pachaar’s Hermes masterclass coverage. That is orthogonal to SOUL: skills say how to do work; SOUL says who is doing it and what is off-limits.

Ecosystem: same pattern, different layouts

Starter SOUL.md skeleton

# Soul

## Role
You are a [role] helping [user] with [outcomes].

## Communication
- Tone: concise, plain English
- Structure: lead with the answer, then detail

## Values
- State uncertainty explicitly
- Never invent commands or file paths

## Boundaries
- Ask before destructive shell or network actions
- Refuse requests that violate policy X

## Continuity
- On session start, read MEMORY.md and USER.md
- Promote only durable facts to memory; keep SOUL stable

Practical checklist

• Write SOUL.md first; add MEMORY and skills second.
• Cap SOUL at ~80 lines; move procedures to skills.
• Review monthly: remove stale paths from memory, not from SOUL unless principles change.
• Use separate profiles if one install serves work, personal, and public bots.
• Never store secrets in SOUL or MEMORY—treat them like config under version control.

At a glance

ElevenLabs Music v2 is a studio-grade text-to-music upgrade that can shift genres inside one track, build songs intro-by-intro, and regenerate individual sections—trained on licensed material and cleared for broad commercial use on ElevenMusic and ElevenCreative, with API rollout following.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  P[Prompt or composition plan] --> M[Music v2 model]
  M --> S1[Section intro]
  M --> S2[Section verse]
  M --> S3[Section chorus]
  S1 --> ST[Stitched full track]
  S2 --> ST
  S3 --> ST
  ST --> OUT[MP3 export]
  OUT --> EM[ElevenMusic creators]
  OUT --> EC[ElevenCreative brands]
  OUT --> API[ElevenAPI products]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class M agent
  class P hook
  class ST agent
  class API hook

Music v2 lets you stitch a full track from parts and re-prompt a single section without redoing the whole song.

What ElevenLabs posted on X

On 26–27 May 2026, @ElevenLabs announced Music v2—described in coverage as a model that can switch genres mid-track (for example opera to heavy metal and back), keep fast rap coherent, add non-musical sound effects, and let creators rebuild only part of a song while leaving the rest untouched. For music, ElevenLabs routes creators to ElevenMusic and brand teams to ElevenCreative.

Automated fetch of status 2059312414198235642 returned 403 here; feature claims below align with ElevenLabs’ Music v2 announcement, TechCrunch, and the Eleven Music documentation.

Creators remix on ElevenMusic; developers embed via API; brand teams license through ElevenCreative.

What Music v2 adds over v1

ElevenLabs positions Music v2 as roughly ten months after its first music model—entering a crowded field alongside Google Flow Music, Stability AI, Suno, and others, but emphasising licensing where some rivals faced label lawsuits.

Three platforms, one model

Availability at launch: Music v2 on ElevenMusic and ElevenCreative immediately; ElevenAPI documented as rolling out (announcement: “coming soon,” with sales contact for early access). The public compose API reference currently lists music_v1 as the selectable model ID—expect music_v2 to appear as the API catches up.

Pricing changes announced with v2

ElevenLabs’ launch post states concurrent price cuts: up to 50% for Music v1/v2 on ElevenAPI, and up to 40% for self-serve ElevenCreative customers. Self-serve API tiers on the Music API page advertise pay-as-you-go access with generation limits up to 4,800 minutes/month on the top self-serve tier; Enterprise adds inpainting, expanded media rights, and higher concurrency.

Using the Music API today

Music API access requires a paid ElevenLabs plan. Quick path: create an API key, install the SDK, call music.compose with a text prompt or a structured composition plan.

from elevenlabs.client import ElevenLabs
import os

client = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])

track = client.music.compose(
    prompt=(
        "Upbeat pop verse with warm guitars, then switch to driving "
        "electronic chorus with layered vocals"
    ),
    music_length_ms=60_000,
)

with open("track.mp3", "wb") as f:
    for chunk in track:
        f.write(chunk)

For precise structure, generate a composition plan first—sections carry positive_local_styles, negative_local_styles, duration_ms (3s–2min per section), and optional lines lyrics (max 200 characters per line). Total song length via prompt: 3 seconds to 10 minutes.

Enterprise inpainting (section surgery)

Developers on Enterprise can store tracks with store_for_inpainting=True, then reference unchanged audio via source_from while regenerating other sections—this is how API-level “change only the chorus” works. negative_ranges can replace a few seconds inside an otherwise preserved slice. Upload path: music.upload with optional composition-plan extraction.

Guardrails and limits

• Copyright prompts blocked — naming artists or copying known songs returns bad_prompt / bad_composition_plan with safer suggestions
• Not a legal guarantee — commercial rights vary by subscription; film/TV/large-studio games often need Enterprise terms
• Inpainting is Enterprise-only on the API today; consumer UI may expose section editing without the same API surface
• Quality vs strict timing — respect_sections_durations=false can flex per-section lengths while keeping total duration

Music Finetunes and the wider stack

Music v2 sits beside ElevenLabs’ voice products (TTS, conversational agents, Scribe transcription). Optional Music Finetunes let you train on your own non-copyrighted audio for a consistent sonic identity inside ElevenCreative (docs: roughly 5–10 minutes after upload screening).

At a glance