Telemetry

Track token usage, latency, costs, and workflow performance with built-in observability.

Telemetry requires Cloud features to be enabled.

Overview

Telemetry automatically collects metrics about your workflow executions. Track how much you're spending on AI APIs, identify slow steps, and monitor error rates—all without writing any code.

Enabling Telemetry

Enable telemetry in your SDK configuration:

1import { relay } from "@relayplane/sdk";
2
3relay.configure({
4  cloud: {
5    enabled: true,
6    accessToken: process.env.RELAYPLANE_ACCESS_TOKEN,
7    telemetry: {
8      enabled: true,
9      // Optional: batch settings
10      flushInterval: 5000,  // Flush every 5 seconds
11      maxBatchSize: 100     // Max events per batch
12    }
13  }
14});

Telemetry is async and non-blocking. It won't slow down your workflows.

Tracked Metrics

The telemetry system automatically tracks:

Token Usage

•Input tokens per step
•Output tokens per step
•Total tokens per workflow
•Tokens by provider (OpenAI, Anthropic, etc.)

Latency

•Step execution time
•Total workflow duration
•Time to first token (TTFT)
•Provider response times

Costs

•Estimated cost per step (based on current pricing)
•Total workflow cost
•Cost breakdown by provider
•Daily/weekly/monthly aggregates

Errors & Retries

•Error rates by step
•Retry counts and success rates
•Timeout occurrences
•Error types and messages

Dashboard

View your telemetry data in the RelayPlane dashboard:

1# Open the dashboard
2npx relay dashboard
3
4# Or visit directly
5https://app.relayplane.com/dashboard

The dashboard provides:

•Real-time workflow execution monitoring
•Historical usage charts and trends
•Cost forecasting based on current usage
•Workflow-level and step-level drill-downs
•Alert configuration for anomalies

Programmatic Access

Access telemetry data programmatically:

1import { telemetry } from "@relayplane/sdk";
2
3// Get usage summary for a time range
4const summary = await telemetry.getSummary({
5  startDate: "2024-01-01",
6  endDate: "2024-01-31",
7  groupBy: "workflow"
8});
9
10console.log(summary);
11// {
12//   workflows: [
13//     {
14//       name: "invoice-processor",
15//       runs: 1523,
16//       totalTokens: 2500000,
17//       estimatedCost: 45.50,
18//       avgDuration: 3200,
19//       errorRate: 0.02
20//     },
21//     ...
22//   ]
23// }
24
25// Get detailed run history
26const runs = await telemetry.getRuns({
27  workflow: "invoice-processor",
28  limit: 100,
29  status: "failed"  // Filter by status
30});

Custom Metadata

Add custom metadata for filtering and analysis:

1const result = await relay
2  .workflow("customer-support")
3  .step("classify")
4  .with("openai:gpt-4o")
5  .run(input, {
6    metadata: {
7      customerId: "cust_123",
8      environment: "production",
9      version: "2.1.0",
10      region: "us-east-1"
11    }
12  });
13
14// Query by metadata in dashboard or API
15const runs = await telemetry.getRuns({
16  metadata: {
17    customerId: "cust_123"
18  }
19});

Alerts

Configure alerts for important thresholds:

1// Configure via SDK
2await telemetry.createAlert({
3  name: "High Error Rate",
4  workflow: "invoice-processor",
5  condition: "errorRate > 0.05",  // 5% error rate
6  window: "1h",
7  channel: "slack",
8  destination: "#alerts"
9});
10
11await telemetry.createAlert({
12  name: "Cost Spike",
13  condition: "dailyCost > 100",  // $100/day
14  channel: "email",
15  destination: "team@company.com"
16});

Data Retention

Data Type	Retention
Run summaries	90 days
Step-level details	30 days
Aggregated metrics	1 year
Cost reports	1 year