RelayPlane Logo

Learning Ledger

The immutable event store that captures every AI agent operation with full context.

Overview

The Learning Ledger is the foundation of RelayPlane Agent Ops. Every request made through the proxy is recorded as a run with complete context including:

  • Auth context — Who made the request (API key vs consumer)
  • Execution mode — Interactive, background, or scheduled
  • Policy decisions — Which policies were evaluated and their outcomes
  • Routing decisions — Why a specific model/provider was selected
  • Performance metrics — Latency, tokens, cost

Run Entry Structure

1interface RunEntry {
2  // Identification
3  run_id: string;              // Unique run identifier
4  workspace_id: string;        // Workspace context
5  agent_id: string;            // Agent making the request
6  
7  // Auth context
8  auth_type: 'api' | 'consumer';
9  execution_mode: 'interactive' | 'background' | 'scheduled';
10  auth_risk: boolean;          // Consumer auth in non-interactive mode
11  compliance_mode: 'recommended' | 'permissive';
12  
13  // Request details
14  provider: string;
15  model: string;
16  
17  // Metrics
18  input_tokens: number;
19  output_tokens: number;
20  total_tokens: number;
21  cost_usd: number;
22  latency_ms: number;
23  ttft_ms: number;             // Time to first token (streaming)
24  
25  // Decisions (summarized)
26  policy_decision: PolicyDecisionSummary;
27  routing_decision: RoutingDecisionSummary;
28  
29  // Status
30  status: 'pending' | 'running' | 'completed' | 'failed' | 'blocked';
31  error?: RunError;
32  
33  // Timestamps
34  created_at: Date;
35  completed_at?: Date;
36}

Auth Types

Two authentication modes with different security implications:

Auth TypeUse CaseSafe Modes
apiYour own agents, internal toolsAll modes
consumerEnd-user facing agents, chatbotsInteractive only (recommended)
Consumer auth in background/scheduled mode sets auth_risk: true. This is a security signal — consumer tokens should not run automated tasks.

Execution Modes

  • interactive — User-initiated, synchronous. User is waiting for the response.
  • background — System-initiated, async processing. No user waiting. Runs with API auth.
  • scheduled — Cron or timer-triggered. No user context. Runs with API auth.

Querying Runs

1# List recent runs
2curl "http://localhost:3001/v1/runs?workspace_id=ws_123&limit=50"
3
4# Filter by auth type
5curl "http://localhost:3001/v1/runs?workspace_id=ws_123&auth_type=consumer"
6
7# Filter by status
8curl "http://localhost:3001/v1/runs?workspace_id=ws_123&status=failed"
9
10# Time range
11curl "http://localhost:3001/v1/runs?workspace_id=ws_123&since=2026-02-06T00:00:00Z"

Event Stream

Each run has a detailed event stream for debugging:

1curl http://localhost:3001/v1/runs/run_123/events
2
3{
4  "events": [
5    {
6      "event_id": "evt_1",
7      "event_type": "run.started",
8      "timestamp": "2026-02-06T12:00:00.000Z",
9      "data": { "model": "claude-3-5-sonnet" }
10    },
11    {
12      "event_id": "evt_2",
13      "event_type": "auth.verified",
14      "timestamp": "2026-02-06T12:00:00.005Z",
15      "data": { "auth_type": "api" }
16    },
17    {
18      "event_id": "evt_3",
19      "event_type": "policy.evaluated",
20      "timestamp": "2026-02-06T12:00:00.010Z",
21      "data": { "policies_checked": 3, "all_passed": true }
22    },
23    {
24      "event_id": "evt_4",
25      "event_type": "routing.decided",
26      "timestamp": "2026-02-06T12:00:00.015Z",
27      "data": { "selected": "anthropic:claude-3-5-sonnet", "reason": "primary" }
28    },
29    {
30      "event_id": "evt_5",
31      "event_type": "provider.request_sent",
32      "timestamp": "2026-02-06T12:00:00.020Z",
33      "data": { "provider": "anthropic" }
34    },
35    {
36      "event_id": "evt_6",
37      "event_type": "run.completed",
38      "timestamp": "2026-02-06T12:00:02.500Z",
39      "data": { "tokens": 150, "cost_usd": 0.002 }
40    }
41  ]
42}

Analytics from the Ledger

Aggregate metrics from the ledger:

1# Usage by model
2curl "http://localhost:3001/v1/analytics/usage?workspace_id=ws_123&group_by=model"
3
4{
5  "total_runs": 15420,
6  "total_tokens": 2500000,
7  "total_cost_usd": 125.50,
8  "breakdown": [
9    { "key": "claude-3-5-sonnet", "runs": 8000, "tokens": 1500000, "cost_usd": 75.00 },
10    { "key": "gpt-4o", "runs": 5000, "tokens": 800000, "cost_usd": 40.00 },
11    { "key": "gpt-4o-mini", "runs": 2420, "tokens": 200000, "cost_usd": 10.50 }
12  ]
13}

Storage Options

  • SQLite (default) — Local file storage. Great for development and single-instance deployments.
  • PostgreSQL — Production-ready. Required for multi-instance deployments.

Next Steps