Reference

Run test conversations

Runs up to 5 test conversations (up to 4 messages each) against the agent as it is now, through the real chat path in test mode, and checks expectations on the last reply: contains, not_contains, grounded (answered from its knowledge) and components (cards such as links, booking, contact).

REST	`POST /api/v1/agents/{agent_id}/tests`
MCP tool	`run_agent_test`
Classification	Write
Scope	`agents:test`
Minimum role on the agent	editor
Confirmation	Not needed
Retry safety	Send an `Idempotency-Key` header (REST) or `idempotency_key` argument (MCP) to retry safely
Success status	201

What it does

Side effects

Runs test conversations through the real chat path. They use the AI reply allowance but never appear in analytics, People or Conversations.

Inputs

Name	Type	Required	In	Description
`agent_id`	string	yes	path	The agent id (agt_...), from list_agents.
`cases`	array of object	yes	body

Example

REST

curl -X POST "https://covo.lanaai.io/api/v1/agents/agt_4edc89bb2964/tests" \
  -H "Authorization: Bearer $COVO_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"cases":[{"name":"Explains the studio","messages":["What does Northwind Labs do?"],"expect":{"contains":["private AI"],"grounded":true}}]}'

MCP (POST /mcp)

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "run_agent_test",
    "arguments": {
      "agent_id": "agt_4edc89bb2964",
      "cases": [
        {
          "name": "Explains the studio",
          "messages": [
            "What does Northwind Labs do?"
          ],
          "expect": {
            "contains": [
              "private AI"
            ],
            "grounded": true
          }
        }
      ]
    }
  }
}

Response

Returns an object with id, revision, passed, failed, cases, client and created_at. This is a real response, shortened to two items per list.

201 response

{
  "id": "68c583c0-3bb6-42d3-85b4-193ea6eeca79",
  "revision": 7,
  "passed": 1,
  "failed": 0,
  "cases": [
    {
      "name": "Explains the studio",
      "turns": [
        {
          "reply": "Northwind Labs\nNorthwind Labs builds private AI systems for small law firms.",
          "message": "What does Northwind Labs do?",
          "grounding": "known",
          "components": [
            "suggested_questions"
          ],
          "latency_ms": 40,
          "handed_to_person": false
        }
      ],
      "passed": true,
      "failures": [],
      "conversation_id": "b2412a0b-9da5-4df5-adc9-031895e6a98c"
    }
  ],
  "client": {
    "name": "docs-examples",
    "interface": "rest"
  },
  "created_at": "2026-10-07T20:14:41.959Z"
}

Over MCP the same object comes back as the tool result, in structuredContent and as JSON text.

Errors

Status	Code	Meaning
401	`unauthorized`	No token, or it is unknown, revoked or expired. OAuth apps refresh or reconnect.
403	`account_suspended`	The account that owns the token is paused.
403	`insufficient_scope`	The token lacks `agents:test`.
404	`agent_not_found`	Unknown agent, or not reachable with this token.
400	`bad_request`	An input is invalid; `field` names it.
409, 422	`idempotency_in_progress`, `idempotency_key_reused`	See Idempotency on the REST page.
429	`rate_limited`	Wait `Retry-After` seconds.