Mervin Praison

Google’s Gemini Interactions API reached general availability on 22 June 2026 and is now the primary interface for Gemini models and agents — replacing generateContent as the default in Google AI Studio, official docs, and new frontier agent features.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
    A[Developer] --> B[Interactions API]
    B --> C{model or agent?}
    C -->|model ID| D[Gemini inference]
    C -->|agent ID| E[Managed Agent sandbox]
    D --> F[Typed steps timeline]
    E --> F
    B --> G[background=true]
    G --> H[Async Deep Research / agents]
    F --> I[previous_interaction_id]
    I --> J[Server-side state 55-day retention]

    classDef agent fill:#8B0000,color:#fff
    classDef hook fill:#189AB4,color:#fff
    classDef decision fill:#444,color:#fff

    class B hook
    class E agent
    class D agent
    class H agent
    class C decision

What GA means for Gemini developers

The Interactions API launched in public beta in December 2025. GA brings a stable schema, Managed Agents with remote Linux sandboxes, background execution, typed execution steps, Flex/Priority service tiers, and documentation that defaults to Interactions everywhere. generateContent remains supported for mainline models, but long-running agent capabilities will increasingly ship exclusively on Interactions.

One endpoint: model inference or managed agent

The core design: pass a model ID for inference or an agent ID for autonomous multi-step work. Same SDK call shape, same endpoint (POST /v1beta/interactions).

from google import genai

client = genai.Client()

# Talk to a model
interaction = client.interactions.create(
    model="gemini-3.5-flash",
    input="Explain quantum entanglement simply.",
)

# Run a managed agent
interaction = client.interactions.create(
    agent="antigravity-preview-05-2026",
    input="Plot solar energy growth globally and make HTML slides.",
    environment="remote",
)

Key GA capabilities since December beta

Managed Agents and the Antigravity sandbox

antigravity-preview-05-2026 is the default managed agent. A single interactions.create() with environment="remote" provisions a sandbox where the agent reasons, runs code, browses the web, and manages files. You can also define custom agents with instructions, skills, and data sources. Other built-in agents include Deep Research variants (deep-research-preview-04-2026, deep-research-max-preview-04-2026) and Computer Use (gemini-2.5-computer-use-preview).

Background execution for long-running work

Set background=true on any interaction. The server processes asynchronously — ideal for Deep Research, Deep Think, and multi-step agent tasks. Poll or stream status; retrieve the completed interaction when done. Cancel in-flight background jobs with the cancel endpoint.

interaction = client.interactions.create(
    agent="deep-research-max-preview-04-2026",
    input="Research the state of solid-state batteries in 2026.",
    background=True,
)

# Poll until status is complete, then read interaction.steps

From roles to typed steps

The biggest schema change: instead of role: user / role: model message blobs, every action is a typed step — user_input, thought, function_call, model_output, and more. Streaming emits step.start, step.delta, step.stop events. This makes agent UIs, debugging, and intermediate rendering (search widgets, thoughts) straightforward. SDK convenience: interaction.output_text extracts final text; complex multimodal responses require iterating steps.

Server-side state and multi-turn chat

Interactions store server-side by default (store=true). Continue conversations with previous_interaction_id instead of resending full history. Opt into stateless mode with store=false when you manage history client-side. Paid tier retains interactions for 55 days.

# Continue a conversation
follow_up = client.interactions.create(
    model="gemini-3.5-flash",
    input="Now explain it for a 10-year-old.",
    previous_interaction_id=interaction.id,
)

Flex vs Priority service tiers

Per-interaction tier selection: Flex cuts cost by roughly 50% for batch and non-latency-sensitive workloads. Priority optimises for lower latency on interactive applications. Errors now pinpoint the exact invalid field in requests.

Migrating from generateContent

Google published a field-by-field migration guide. Docs include a toggle to switch code snippets back to legacy format. Automate migration with the gemini-interactions-api skill:

npx skills add google-gemini/gemini-skills --skill gemini-interactions-api

# Then in Gemini CLI or Jules:
# /gemini-interactions-api migrate my app to the interactions api

Ecosystem and getting started

Available via Python and JavaScript SDKs, REST, and partner integrations (LiteLLM, Eigent, Agno). Grab an API key from Google AI Studio. Full reference: Interactions API reference and quickstart.

Summary

AI loops replace the one-prompt-at-a-time habit with a goal the model keeps working toward — planning, executing, verifying against objective criteria, and iterating until done or a hard stop limit is hit. Anatoli Kopadze’s X Article breaks down how Claude Code /goal and /loop, ChatGPT self-check prompts, and Telegram Mira Skills each implement the same pattern at different complexity levels.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
    A[Set goal + verify gate] --> B[DISCOVER]
    B --> C[PLAN]
    C --> D[EXECUTE]
    D --> E{VERIFY}
    E -->|pass| F[DONE]
    E -->|fail| G[ITERATE with state]
    G --> B
    H[Stop condition] --> E
    I[State / memory] --> G

    classDef agent fill:#8B0000,color:#fff
    classDef hook fill:#189AB4,color:#fff
    classDef decision fill:#444,color:#fff

    class A agent
    class D agent
    class E decision
    class F agent
    class G hook
    class H decision

The slow way most people still use AI

The default workflow: type a request, wait, fix the output, ask again — every step routed through you. The AI never moves unless you push it. That works for one-off tasks but hits a ceiling fast: you are the engine, and the model is only the tool in your hand.

The alternative: give the goal once and let the system run the cycle itself — plan, execute, verify, fix weaknesses, repeat until the bar is met. Engineer Geoffrey Huntley frames it plainly: “You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.”

What a loop actually is

A prompt is a single instruction. A loop is a recursive goal the AI keeps working toward until completion. The canonical cycle from the article:

DISCOVER  →  work out what needs doing
PLAN      →  decide how to do it
EXECUTE   →  do the work
VERIFY    →  check it against the goal
ITERATE   →  not there yet? feed the result back in and repeat

Three parts people get wrong

The four-box test: do you even need a loop?

A loop is worth building only when all four conditions hold. Miss one and a single good prompt is cheaper:

• The task repeats at least weekly — setup cost must pay back
• Something can automatically reject bad output (tests, linter, type checker, hard rule)
• The agent can finish end-to-end without handing half back to you
• “Done” is objective — not a matter of taste where humans still win

The coding loop: five building blocks

Loops took off in software first because code is trivially verifiable — tests pass or fail. Claude Code and OpenAI Codex now ship primitives for all five blocks:

▸ LOOP SPEC
GOAL: every test in /tests/auth passes, lint is clean, no type errors.

EACH ITERATION:
  1. run the test suite and read every failure
  2. pick the single highest-impact failure
  3. write the smallest change that fixes it
  4. re-run tests, lint, and type checker

VERIFY: green tests + zero lint warnings + zero type errors
STOP WHEN: verify passes, OR 8 iterations reached
ON STOP: summarize what changed and what still fails

The cost nobody mentions

Loops bill in tokens, and cost compounds. Every iteration re-sends the full context — goal, code, last result, failures — and that pile grows each pass. Ten iterations is not ten equal prompts; each prompt gets bigger. Maker-and-checker doubles the bill because two models read the work.

Without a hard gate, loops fail quietly — the “Ralph Wiggum loop” where the agent declares victory early and keeps billing while producing nothing. Production teams add iteration caps, token budgets, cheap models on boring steps, and monitoring.

Build order that survives in production

1. Get ONE manual run reliable first.
2. Turn that into a skill (save the instructions).
3. Wrap the skill in a loop (add the gate + stop condition).
4. THEN put it on a schedule.

Scheduling something you have not proven by hand is how loops blow up while you sleep.

Build a basic loop in any LLM (no code)

Paste this into Claude or ChatGPT to feel the loop without tooling. It forces PLAN → DO → VERIFY → DECIDE until every criterion scores 8+:

▸ SELF-CHECKING LOOP  (paste into Claude or ChatGPT)
You will work in a loop until the task meets the bar.

TASK:
[describe exactly what you want produced]

SUCCESS CRITERIA (be strict, no soft passes):
- [criterion 1]
- [criterion 2]
- [criterion 3]

LOOP PROTOCOL, repeat every turn:
1. PLAN   - state the single next step.
2. DO     - produce or improve the work.
3. VERIFY - score the result 1-10 on each criterion.
            Be brutally honest. List exactly what is still weak.
4. DECIDE - if every criterion is 8+, print "FINAL" and stop.
            Otherwise print "ITERATING" and go again.

RULES:
- Never call it done until every criterion is 8 or higher.
- Each pass must fix the weakest score from the last VERIFY.
- Do not ask me questions. Make a sensible assumption and keep going.

Begin. Run the loop until FINAL.

What is still missing: you are the trigger. Close the tab and the loop vanishes. No schedule, no event trigger, no background execution.

Claude Code: /goal vs /loop vs /schedule

The expensive mistake: using /loop on work with a finish line (re-runs blindly after done) or /goal to poll something external (condition cannot become true no matter how hard Claude works). Official docs: /goal, /loop and scheduled tasks.

GPT and Codex loops

ChatGPT supports the same self-checking prompt pattern manually. OpenAI’s agent improvement flywheel (traces → human/model feedback → evals → harness changes via Codex) is the production version: run agent, score output, diagnose failures, persist lessons, repeat. Codex automations and scheduled exec runs mirror Claude’s /schedule for cloud-side recurring jobs. The underlying pattern is identical — only the verify gate and persistence layer differ.

Mira: life loops inside Telegram

The article’s Mira refers to @mira on Telegram — a consumer agent with ~1M monthly users, not a coding framework. The distinction from ChatGPT: ChatGPT answers; Mira acts. Skills are natural-language loops with triggers, multi-step actions across connected apps (500+ via Composio — Gmail, Calendar, Linear, GitHub, Notion), and persistent memory across sessions and group chats.

▸ SKILL
"Every weekday at 7am, check my Gmail and Google Calendar.
Send me a short brief: my 3 most important meetings, anything
urgent in the inbox, and one thing I said I'd follow up on but
haven't. Keep it under 120 words."

That is a real loop: time trigger + multi-app action + autonomous delivery. No code, hosting, or API keys — described in one message. Example Skills from the article span work (meeting prep, Linear tickets, weekly digests), creators (voice note → multi-platform posts, image/video generation), voice (transcription, TTS, group chat summaries), and life (habit streaks, journaling, flight price watches, news digests).

Quick start commands

@mira, plan my week
@mira, summarize this chat
@mira, remind me to review PRs every Monday at 9am
@mira, write a post about [topic] for X and Instagram

When loops are a trap

• One-off tasks where setup never pays back
• Subjective quality with no objective verify gate
• Agent cannot complete the work without constant human handoff
• No iteration cap — silent billing on half-finished jobs
• Scheduling before one reliable manual run exists

Summary

Agent-Native is Builder.io’s open-source framework for building apps where agents and UI are equal citizens — one defineAction() powers clicks, chat tools, HTTP, MCP, A2A, CLI, and scheduled jobs over shared SQL state, so agents act inside real products instead of beside them.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
    A[defineAction] --> B[React UI hooks]
    A --> C[In-app agent tools]
    A --> D[HTTP API]
    A --> E[MCP server]
    A --> F[A2A agents]
    A --> G[CLI]
    A --> H[Scheduled jobs]
    I[(SQL database)] --> B
    I --> C
    C --> J[Agent runtime]
    J --> K[Skills + memory]
    J --> L[Observability]

    classDef agent fill:#8B0000,color:#fff
    classDef hook fill:#189AB4,color:#fff
    classDef decision fill:#444,color:#fff

    class A agent
    class C agent
    class J agent
    class B hook
    class E hook
    class F hook
    class I agent

The problem: chat beside the app

Most “AI features” bolt a chat sidebar onto an existing SaaS shell. The agent cannot reliably mutate app state, share auth with the UI, or expose the same operations a button click would trigger. Raw coding agents (Claude Code, Codex) are powerful but do not ship multi-tenant dashboards, live sync, or product-grade persistence. Agent-Native targets rung 3 on the framework’s ladder: agent and application as partners over one database, one action layer, and one runtime.

One action, seven surfaces

The central primitive is defineAction() in @agent-native/core. You declare a Zod schema, a run() function, and optional metadata; the framework auto-discovers files under actions/ and exposes them everywhere the product needs them. The ActionRunContext.caller field tells your business logic whether the invocation came from the UI, agent tool loop, HTTP, MCP, A2A, or CLI.

import { defineAction } from "@agent-native/core";
import { z } from "zod";

export default defineAction({
  description: "Create a reply to an email",
  schema: z.object({
    emailId: z.string(),
    body: z.string(),
  }),
  http: { method: "POST" },
  publicAgent: { expose: true, readOnly: false },
  run: async ({ emailId, body }, ctx) => {
    await db.insert(replies).values({ emailId, body });
  },
});

What ships in the framework

Agents and UI, fully connected

Official docs describe six architecture rules: data lives in SQL; all AI routes through the agent; operations go through actions; UI and agent stay live-synced; the agent can modify app code over time; ephemeral UI state persists in SQL (application_state). Humans and agents co-edit documents via Yjs/TipTap collaboration. Select text and hit Cmd+I for context-aware commands. Tag another agent from any app and they coordinate over A2A on a shared workspace deploy.

Production templates (not scaffolds)

The monorepo ships 15+ cloneable open-source SaaS apps under templates/ — complete products you fork and customise, not empty starters. A workspace deploy mounts multiple apps under one origin with shared auth and zero-config cross-app A2A.

Skills without scaffolding a full app

The lowest-friction entry is installing app-backed skills into Claude Code, Codex, Cursor, Pi, or GitHub Copilot:

npx @agent-native/core@latest skills add visual-plan

/visual-plan opens a structured, reviewable plan with diagrams, wireframes, file-by-file implementation maps, and commentable annotations before code lands. /visual-recap turns a PR or git diff into a high-altitude visual recap with a shareable review link. Hosted Plan app: plan.agent-native.com; MCP endpoint at /_agent-native/mcp.

Protocols and interoperability

Tech stack

Quick start

# Interactive — pick full template, chat shell, or headless
npx @agent-native/core@latest create my-app

# Minimal chat UI
npx @agent-native/core@latest create my-app --template chat

# Action-first, no UI shell
npx @agent-native/core@latest create my-agent --headless

cd my-app
pnpm install
pnpm action hello --name World
pnpm agent "Call hello for World"
pnpm dev

Agent-Native vs alternatives

Summary

Movez’s self-improving loop pairs Kimi K2.6’s 300-agent swarm — up to 4,000 coordinated steps per run — with an Opus 4.8 verification gate, so each completed task leaves behind reusable skills and constraints instead of resetting to zero on the next prompt.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
    A[Write spec] --> B[Review decomposition plan]
    B --> C[Kimi K2.6 swarm]
    C --> D[Structured file outputs]
    D --> E{Opus 4.8 verify gate}
    E -->|pass| F[Save Skill]
    E -->|fail| G[Patch + CONSTRAINTS.md]
    G --> C
    F --> H[Replay on new inputs]
    H --> I[Background agent]

    classDef agent fill:#8B0000,color:#fff
    classDef hook fill:#189AB4,color:#fff
    classDef decision fill:#444,color:#fff

    class A agent
    class C agent
    class E decision
    class F hook
    class G hook
    class I agent

What the X article argues

Most Kimi users treat it as a chatbox: one question, one answer, close the tab. Movez’s playbook treats Kimi K2.6’s Agent Swarm as an execution engine that can run hundreds of parallel sub-agents, emit real deliverables (PDFs, spreadsheets, code, decks), and compound across runs by saving verified workflows as Skills. The twist is architectural: Kimi does volume; Claude Opus 4.8 sits at a single verify gate whose only job is to stop flawed output from being promoted into permanent skills.

Kimi K2.6 swarm specs (verified numbers)

The 4,000-step figure is a total coordinated budget across the swarm, not 4,000 steps per agent. At a 300-agent ceiling that implies roughly a dozen tool steps each — specialised subtasks with bounded context windows, with only structured outputs flowing back to the coordinator. That is the structural reason long-horizon research does not collapse into lossy summarisation the way a single-thread agent does.

The compounding loop in one diagram

Why two models instead of one

Movez’s pattern is not “pick the benchmark winner.” Kimi K2.6 is optimised for cheap parallel execution at open-weight pricing; Opus 4.8 is positioned for judgment — planning nuance, catching its own mistakes, and refusing to rubber-stamp flawed results. The swarm’s known failure mode without verification is confident, under-cited claims and contradictory sub-agent outputs. Premium tokens at the verify gate are cheaper than permanently saving garbage into a Skill library.

The 10-step playbook (condensed)

Spec template (step 01)

# PROJECT: [name]
GOAL: [one sentence — the deliverable, not the topic]
SCOPE: [what's in, what's explicitly out]
RULES: [validation — what counts as a verified row/finding]
SOURCES: [official posts, papers, primary only — no aggregators]
OUTPUT: [file type / count / naming / format details]
ON CONFLICT: flag the row, never resolve silently
STOP CONDITION: [when to halt and report instead of guessing]

Enable swarm via API

from openai import OpenAI

client = OpenAI(
    api_key="MOONSHOT_API_KEY",
    base_url="https://api.moonshot.ai/v1",
)

response = client.chat.completions.create(
    model="kimi-k2.6",
    messages=[{"role": "user", "content": "YOUR_SPEC_HERE"}],
    extra_body={
        "agent": {
            "type": "swarm",
            "max_agents": 300,
            "max_steps": 4000,
        }
    },
)

Moonshot’s API is OpenAI-compatible. Production guidance from third-party write-ups: cap max_agents to 10–30 for predictable tasks even though the ceiling is 300; stream responses to kill bad runs early; log token usage on every response.

What “self-improving” actually means

The model is not retraining weights between your runs. The system around it compounds: Skills capture proven workflows, document-to-skill captures domain voice, and CONSTRAINTS.md turns verify failures into hard rules. A competitor cannot copy that library in a week — it is built from months of verified runs on your data. That is the honest version of self-learning Movez describes, distinct from static LangGraph DAGs that behave identically on run #1 and run #50.

Summary

Open Knowledge Format (OKF) v0.1 is Google’s vendor-neutral open standard that turns the LLM-wiki pattern into portable markdown bundles — one concept per file, YAML frontmatter for queryable fields, and cross-links that form a knowledge graph agents can traverse without an SDK or proprietary catalog.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
    A[Source systems] --> B[Producer]
    B --> C[OKF bundle]
    C --> D{Consumers}
    D --> E[AI agent]
    D --> F[HTML visualizer]
    D --> G[Knowledge Catalog]
    C --> H[index.md]
    C --> I[concept .md files]
    I --> J[YAML frontmatter]
    I --> K[Markdown body]
    I --> L[Cross-links graph]

    classDef agent fill:#8B0000,color:#fff
    classDef hook fill:#189AB4,color:#fff
    classDef decision fill:#444,color:#fff

    class A agent
    class B hook
    class C agent
    class E agent
    class F hook
    class G hook
    class D decision

Why context fragmentation blocks agents

Foundation models improve every quarter, but agent accuracy still hinges on internal context: table schemas, metric definitions, join paths, runbooks, deprecation notices, and tribal knowledge locked in senior engineers’ heads. Today that knowledge is scattered across metadata catalogs, wikis, shared drives, code comments, and notebook cells — each with its own API, schema, and export format.

Knowledge as a living wiki

Andrej Karpathy’s LLM Wiki pattern argues that agents do not get bored updating cross-references and can touch many files in one pass — the bookkeeping humans abandon in personal wikis. Similar patterns appear as Obsidian vaults wired to coding agents, metadata-as-code repos inside data teams, and repos full of index.md / log.md artefacts. OKF formalises the interoperability surface so Karpathy-style wikis, team wikis, and catalog exports can cooperate without translation.

Bundle anatomy: one concept, one file

An OKF knowledge bundle is a directory tree of UTF-8 markdown files. Each concept — a table, metric, API, playbook, or abstract idea — maps to exactly one .md file. The file path is the concept ID (path without .md). No runtime, database, or account is required: ship as a git repo, tarball, or mounted subdirectory.

sales/
├── index.md
├── datasets/
│   ├── index.md
│   └── orders_db.md
├── tables/
│   ├── index.md
│   ├── orders.md
│   └── customers.md
└── metrics/
    ├── index.md
    └── weekly_active_users.md

Concept document shape

---
type: BigQuery Table
title: Orders
description: One row per completed customer order.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
tags: [sales, revenue]
timestamp: 2026-05-28T14:30:00Z
---

# Schema

| Column        | Type      | Description                              |
|---------------|-----------|------------------------------------------|
| `order_id`    | STRING    | Globally unique order identifier.        |
| `customer_id` | STRING    | FK to [customers](/tables/customers.md). |

# Joins

Joined with [customers](/tables/customers.md) on `customer_id`.

Frontmatter specification (v0.1)

Reserved files and progressive disclosure

All other .md files are concept documents. Bundles may declare okf_version: "0.1" in root index.md frontmatter — the only place frontmatter is permitted on an index file.

Cross-links build a richer graph than folders

Concepts link with standard markdown. Bundle-relative paths starting with / (e.g. /tables/customers.md) are preferred because they stay stable when files move within subdirectories. Link semantics are conveyed by surrounding prose — joins-with, depends-on, references — not by link type. Consumers must tolerate broken links (targets may be not-yet-written knowledge).

Three design principles

• Minimally opinionated — only type is required; taxonomy, body sections, and tooling are producer choices.
• Producer/consumer independence — human-authored bundles, LLM-generated exports, and catalog pipelines all speak the same contract.
• Format, not platform — no proprietary SDK, cloud lock-in, or account needed to read or write OKF.

Conformance and permissive consumption

A bundle is conformant when every non-reserved .md file has parseable YAML frontmatter with a non-empty type, and reserved files follow index/log structure when present. Consumers must not reject bundles for missing optional fields, unknown types, extra frontmatter keys, broken links, or absent index.md files — the spec is deliberately permissive so partially generated agent output remains useful.

Reference implementations Google ships

Repository: GoogleCloudPlatform/knowledge-catalog/okf. Full spec: okf/SPEC.md. Community guide with validator and templates: openknowledgeformat.com.

How OKF compares to adjacent patterns

Getting started: minimal conformant concept

# Create bundle root
mkdir -p okf/tables && cd okf

# Root index for progressive disclosure
cat > index.md <<'EOF'
# Sales knowledge

* [Orders table](/tables/orders.md) - One row per completed customer order.
EOF

# Concept file — only `type` is strictly required
cat > tables/orders.md <<'EOF'
---
type: BigQuery Table
title: Orders
description: One row per completed customer order.
tags: [sales]
---

# Schema

| Column | Type | Description |
|--------|------|-------------|
| order_id | STRING | Unique order identifier. |
EOF

# Ship as git repo or tarball — any agent can cat the files

Summary

DeepEval is an Apache-2.0 Python framework that brings pytest-style unit tests to LLM apps—scoring agents, RAG pipelines, and chatbots with 50+ research-backed metrics, optional trace-level evals, and a v4 “vibe coding” loop where your IDE agent runs deepeval test run and patches failures.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  A[Dataset / goldens] --> B[Your LLM app]
  B --> C[LLMTestCase + traces]
  C --> D[Metrics layer]
  D --> E{Score vs threshold}
  E -->|Pass| F[CI green]
  E -->|Fail| G[Metric reason + span]
  G --> H[Coding agent patch]
  H --> B
  C --> I[Confident AI optional]
  I --> J[Reports + production monitor]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class B,F agent
  class D,I,H hook
  class E decision

What DeepEval is

Each test is an LLMTestCase (single-turn) or ConversationalTestCase (multi-turn) with input, actual_output, optional expected_output, and retrieval_context. Metrics return a 0–1 score, pass/fail against a threshold, and a natural-language reason—the signal coding agents use in the v4 iteration loop.

Metric families

Two evaluation shapes

Framework integrations (preferred over hand-rolled @observe) cover OpenAI Agents, LangChain/LangGraph, Anthropic, CrewAI, Pydantic AI, LlamaIndex, Google ADK, AWS AgentCore, and more—auto-instrumenting LLM and tool spans for scored traces.

CI/CD and local runs

pip install -U deepeval

# test_chatbot.py — use assert_test, not evaluate(), inside pytest files
deepeval test run test_chatbot.py -n 4   # parallel workers

deepeval login    # optional: push reports to Confident AI
deepeval view     # open latest test run in browser

Test files must be named test_*.py. In CI, always use assert_test() inside test functions—evaluate() is for notebooks. Flags like --identifier, --num-processes, and --ignore-errors support repeatable agent-driven iteration rounds.

Vibe-coding loop (v4)

DeepEval v4 targets IDE agents (Cursor, Claude Code, Codex, Windsurf). Install the agent skill with npx skills add confident-ai/deepeval --skill deepeval, commit suites under tests/evals/, then let the agent:

• Generate goldens with deepeval generate if no dataset exists
• Run deepeval test run and read failing metric reason strings
• Patch the smallest app change (prompt, retriever, tool schema)—not thresholds
• Re-run until scores improve without regressions

Docs ship llms.txt and per-page .md URLs so agents can ingest the metrics catalog without scraping HTML.

Confident AI (optional platform)

DeepEval is local-first. Confident AI (same team) adds shared datasets, testing reports, production trace monitoring, and an MCP server so agents can pull datasets and inspect traces from the IDE. It is optional—offline evals work without an account.

Summary

Context engineering is how you stop AI agents from going sloppy around step 15—by curating what the model sees (system prompt, tools, history, RAG, memory) so the finite context window stays high-signal instead of filling with stale tool dumps.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  A[Agent turn starts] --> B[Collect sources]
  B --> C[Select for this step]
  C --> D[Compress to budget]
  D --> E[Arrange stable prefix first]
  E --> F[LLM call]
  F --> G{Step quality OK?}
  G -->|Yes| H[Next step]
  G -->|No| I[Diagnose failure mode]
  I --> J[Write / Isolate / prune]
  J --> B
  H --> B

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class A,F,H agent
  class B,C,D,E,J hook
  class G decision

Why agents fail after ~15 steps

Rahul’s playbook opens with a pattern most builders recognise: the first ten agent steps look sharp, then wrong tool calls, forgotten instructions, and shallow outputs appear—not because the model changed, but because context rot set in. Anthropic frames context engineering as optimising token utility across multi-turn loops; LangChain’s OS analogy treats the context window as RAM that degrades when crowded, often well before the hard limit (Chroma’s study showed decline across 18 frontier models as length grew).

Seven things fighting for the same window

The four strategies: Write, Select, Compress, Isolate

Four failure modes (and fixes)

Production workflow: phased compaction

Dex Horthy’s frequent intentional compaction pattern (cited in the playbook) structures long coding runs into phases, each ending in a compact artifact and a fresh context:

• Research — sub-agents explore; parent gets research.md (isolate + write + compress)
• Plan — new window with research + problem only; human review gate
• Implement — new window with plan + progress.md tracker

Staying under 40–60% of context capacity per phase avoids the “sloppy step 20” cliff. Claude Code’s auto-compaction at ~95% and Manus’s KV-cache-aware prefix ordering follow the same principle: stable instructions and tool defs at the top (cacheable), dynamic tail at the bottom.

The universal turn pipeline

Summary

Vercel eve is an open-source, filesystem-first TypeScript framework that treats an AI agent as a folder of files—instructions, tools, skills, channels—and compiles it into a production agent with durable sessions, sandboxes, approvals, and evals already wired in.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  A[agent/ directory] --> B[eve compiler]
  B --> C[Durable session]
  B --> D[Sandbox]
  B --> E[Tools + skills]
  B --> F[Channels]
  C --> G[AI Gateway model calls]
  D --> H[bash / code / files]
  E --> I[MCP + APIs via Connect]
  F --> J[Slack / Discord / HTTP / cron]
  B --> K[OpenTelemetry traces + evals]
  B --> L[vercel deploy]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class A,G agent
  class B,D,E,I,K hook
  class C decision

An agent is a directory

The pitch mirrors early Next.js: stop hand-rolling the same agent plumbing on every project. Vercel says hundreds of internal agents exposed the same shape—model loop, state, sandbox, credentials, channels—so eve codifies that shape and discovers files at build time, similar to how Next turns folders into routes.

Production primitives included

Minimal setup

npx eve@latest init my-agent
# installs deps, inits git, starts dev TUI

# agent/agent.ts
import { defineAgent } from "eve";
export default defineAgent({ model: "anthropic/claude-opus-4.8" });

# agent/tools/run_sql.ts — filename = tool name, Zod schema + execute
# eve dev — terminal UI + HTTP API on same structured events

HTTP clients create a session with POST /eve/v1/session, stream NDJSON from GET /eve/v1/session/:id/stream, and continue with the returned continuationToken. Models route through AI Gateway with provider fallbacks; on Vercel you authenticate with OIDC instead of copying API keys.

Ship and operate like normal software

An eve project is a standard Vercel app: vercel deploy ships the same directory that ran locally. Preview deployments carry channels too—your team can test the next Slack bot before it replaces production. Sessions started before a deploy finish on the version they began on.

Channels are one CLI command each (eve channels add slack writes channels/slack.ts). Launch integrations include Slack, Discord, Teams, Telegram, Twilio, GitHub, and Linear, plus custom defineChannel adapters. Schedules deploy as Vercel Cron Jobs.

How Vercel uses eve internally

Vercel reports agent-triggered deployments rose from under 3% a year ago to about 29% today, with half of deployments expected to come from agents soon. Those agents now live in one monorepo with shared conventions instead of separate stacks per team.

Summary

GLM-5.2 is Z.ai’s new flagship coding model: a 753B-parameter MoE stack with a usable 1M-token context, high/max reasoning effort, and MIT weights on Hugging Face—aimed at repo-scale agent work rather than single-file autocomplete.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  A[Developer goal] --> B[Coding agent]
  B --> C{Task horizon}
  C -->|Single file| D[GLM-5.2 high]
  C -->|Repo / multi-hour| E[GLM-5.2 max + 1M context]
  D --> F[Z.ai coding API]
  E --> F
  F --> G[Plans + tools + patches]
  G --> H[Verified repo output]
  F --> I[Self-host weights]
  I --> J[vLLM / SGLang / Transformers]

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class A,B,H agent
  class F,I,J hook
  class C decision

What Z.ai announced

Benchmarks that matter for builders

The jump from GLM-5.1 is largest on long-horizon suites—FrontierSWE, PostTrainBench, and SWE-Marathon—where 1M context and agentic RL training pay off. On standard SWE-bench Pro the gain is smaller (+3.7 points) but still tops other open models in Z.ai’s table.

Architecture: making 1M context affordable

Drop it into your coding agent

Z.ai ships an Anthropic-compatible coding endpoint at https://api.z.ai/api/coding/paas/v4. In Claude Code, point Sonnet/Opus slots to glm-5.2[1m], set CLAUDE_CODE_AUTO_COMPACT_WINDOW to 1000000, and use /effort—low/medium/high map to GLM high; xhigh/max map to max.

{
  "env": {
    "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "1000000",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "glm-5.2[1m]",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "glm-5.2[1m]"
  }
}

OpenClaw, Cline, and ZCode follow the same pattern: custom base URL, model id glm-5.2, 1M context window, reasoning enabled. GLM Coding Plan subscribers get bundled access; peak-hour prompts cost 3× quota, off-peak 2× (promo: 1× off-peak through September).

Self-hosting

Weights are 753B parameters (BF16/F32) on Hugging Face. Supported runners include vLLM 0.23+, SGLang 0.5.13+, Transformers, and KTransformers; Ascend NPU builds exist via vLLM-Ascend and xLLM. This is VPS/cluster territory—not laptop inference—unless you quantise heavily.

Summary

Qwen Robot Suite is Alibaba Tongyi Lab’s three-model stack for embodied AI—RobotNav for where to go, RobotManip for how to act, and RobotWorld for what happens next—so general Qwen agents can treat the physical world as callable tools instead of a separate silo.

%%{init: {"theme": "base", "themeVariables": {"background": "transparent", "lineColor": "#000000"}}}%%
graph TD
  A[User goal in language] --> B[General Qwen agent]
  B --> C{Physical task type}
  C -->|Navigate| D[RobotNav]
  C -->|Manipulate| E[RobotManip]
  C -->|Imagine / plan| F[RobotWorld]
  D --> G[Waypoints + EQA]
  E --> H[Camera-frame delta poses]
  F --> I[Future video + language actions]
  G --> J[Real robot or sim]
  H --> J
  I --> K[Planner / verifier loop]
  K --> E
  K --> D

  classDef agent fill:#8B0000,color:#fff
  classDef hook fill:#189AB4,color:#fff
  classDef decision fill:#444,color:#fff

  class A,B agent
  class D,E,F hook
  class C decision

What the suite covers

Qwen-RobotNav: see, move, and answer in the world

RobotNav targets vision-and-language navigation (VLN) and embodied question answering (EQA). Instead of dumping every frame into context, it uses a controllable observation protocol: you can tune token budget, temporal decay, and per-camera weights so the model spends capacity on what matters for the current sub-task.

Outputs are structured as eight waypoints with planar coordinates and heading—compact enough for real-time control on platforms like Unitree Go2. On hard indoor benchmarks it reports 76.5% success rate on VLN-CE RxR and 91.4 PDMS on NAVSIM v1/v2, with strong zero-shot transfer across simulators and real quadrupeds.

For agentic use, a higher-level planner (e.g. Qwen3.7-Plus) can call RobotNav as a tool: the LinkedIn launch demo shows a robot answering “where is the green umbrella at Cotti Coffee?” by navigating and grounding the answer in live observations.

Qwen-RobotManip: one action space across arms and hands

RobotManip pairs a Qwen3.5-4B vision–language backbone with a flow-matching diffusion transformer (DiT) action head. The design bet is a single 80-dimensional canonical state–action vector and end-effector deltas expressed in the camera frame, so the same policy head can transfer across UR5 arms, humanoids, and dexterous hands without retraining the whole stack per robot.

Long-horizon demos compose a VLM planner with RobotManip as executor—e.g. tidy a desk by decomposing language goals into grasps, places, and recovery steps. Public codenames in coverage include Lira and Atlas for manipulation variants.

Qwen-RobotWorld: language-conditioned futures (with limits)

RobotWorld is a dual-stream MMDiT video world model (~60 layers, ~20B parameters) with a frozen Qwen2.5-VL encoder. It was trained on 8.6M video–text pairs and 200M+ frames spanning 20+ embodiments and 500+ action types. You steer it with natural language as the unified action interface—“push the mug”, “open the drawer”—and it rolls forward plausible future video.

It tops reported EWMBench and DreamGen style benchmarks for embodied world modelling. That makes it strong for imagination, data augmentation, and plan sketching before spending real robot time.

Practitioners should treat it as a plausibility engine, not a contact-accurate physics simulator: community feedback on the launch thread notes that slip, grasp, and collision-rich moments remain risky if you close the loop on predicted pixels alone. Best pattern—use RobotWorld to rank coarse plans, then verify with RobotManip/RobotNav on hardware or sim.

Benchmark snapshot

Agents + robots: composition patterns

Alibaba Cloud is running pilot programmes with enterprise customers alongside open blogs and model cards. RobotManip and RobotNav have public GitHub releases in third-party coverage; RobotWorld is primarily paper- and blog-backed at launch.

Summary