Invoice Processor

Extract structured data from invoices using vision + language models in a local-first workflow.

Quick Example

1import { relay } from "@relayplane/workflows";
2
3const result = await relay
4 .workflow("invoice-processor")
5 .step("extract").with("openai:gpt-4o-vision")
6 .step("validate").with("anthropic:claude-3.5-sonnet").depends("extract")
7 .run({ fileUrl: "https://example.com/invoice.pdf" });

Full Production Implementation

This example demonstrates a complete invoice processing pipeline with schema validation, error handling, and executive summary generation.

1import { relay } from "@relayplane/workflows";
2import { z } from "zod";
3
4// Define structured invoice schema
5const InvoiceSchema = z.object({
6 invoiceNumber: z.string().describe("Invoice number or ID"),
7 date: z.string().describe("Invoice date in ISO format"),
8 dueDate: z.string().optional().describe("Payment due date"),
9 vendor: z.object({
10 name: z.string(),
11 address: z.string().optional(),
12 taxId: z.string().optional(),
13 }),
14 billTo: z.object({
15 name: z.string(),
16 address: z.string().optional(),
17 }),
18 items: z.array(z.object({
19 description: z.string(),
20 quantity: z.number(),
21 unitPrice: z.number(),
22 amount: z.number(),
23 })),
24 subtotal: z.number(),
25 tax: z.number().optional(),
26 totalAmount: z.number(),
27 currency: z.string().default("USD"),
28});
29
30type Invoice = z.infer;
31
32// Build multi-step workflow
33async function processInvoice(fileUrl: string) {
34 const result = await relay
35 .workflow("invoice-processor")
36
37 // Step 1: Extract structured data from invoice image/PDF
38 .step("extract", {
39 schema: InvoiceSchema,
40 systemPrompt: `Extract all invoice fields as structured JSON.
41 - Parse line items carefully with quantities and prices
42 - Calculate subtotals and verify against invoice total
43 - Extract vendor and customer information
44 - Preserve currency information`,
45 })
46 .with("openai:gpt-4o-vision")
47
48 // Step 2: Validate extracted data
49 .step("validate", {
50 systemPrompt: `Verify the extracted invoice data:
51 1. Check that item amounts = quantity * unitPrice
52 2. Verify subtotal matches sum of all items
53 3. Check that total = subtotal + tax
54 4. Flag any discrepancies or missing critical fields
55 Return: { valid: boolean, issues: string[], confidence: number }`,
56 })
57 .with("anthropic:claude-3.5-sonnet")
58 .depends("extract")
59
60 // Step 3: Generate executive summary
61 .step("summarize", {
62 systemPrompt: `Create a brief executive summary for finance approval:
63 - Key vendor and amount
64 - Payment due date
65 - Any validation issues or red flags
66 - Recommendation (approve/review/reject)
67 Keep it under 3 sentences.`,
68 })
69 .with("openai:gpt-4o-mini")
70 .depends("validate")
71
72 .run({ fileUrl });
73
74 // Return structured results
75 return {
76 invoice: result.steps.extract as Invoice,
77 validation: result.steps.validate,
78 summary: result.steps.summarize,
79 };
80}
81
82// Usage example
83const invoice = await processInvoice("https://example.com/invoice-2024-001.pdf");
84
85console.log("Invoice Number:", invoice.invoice.invoiceNumber);
86console.log("Total Amount:", invoice.invoice.totalAmount, invoice.invoice.currency);
87console.log("Summary:", invoice.summary);

How It Works

The invoice processor uses a three-step pipeline:

  1. Extract - GPT-4 Vision analyzes the invoice image/PDF and extracts structured data matching the Zod schema. The schema provides field descriptions that guide extraction.
  2. Validate - Claude 3.5 Sonnet verifies calculations, checks for missing fields, and flags potential issues. This step depends on the extract step.
  3. Summarize - GPT-4 Mini generates a concise executive summary for approval workflows. Uses the cheaper model since summarization is less critical.

Use Cases

  • Accounts Payable Automation - Auto-extract invoice data into ERP systems
  • Expense Report Processing - Convert receipt photos to structured expense entries
  • Contract Data Extraction - Extract key terms from vendor contracts
  • Purchase Order Matching - Compare invoices against POs automatically
  • Audit Trail Generation - Create structured records for compliance

Adding Webhooks

To trigger this workflow via HTTP (requires RelayPlane Pro):

1# Install CLI and authenticate
2npx relay login
3
4# This workflow can now be triggered via webhook
5curl -X POST https://api.relayplane.com/webhooks/trigger \
6 -H "Authorization: Bearer YOUR_API_KEY" \
7 -H "Content-Type: application/json" \
8 -d '{
9 "workflowName": "invoice-processor",
10 "input": {
11 "fileUrl": "https://example.com/invoice.pdf"
12 }
13 }'

Adding Schedules

Process invoices from a folder on a schedule (requires RelayPlane Pro):

1// Scheduled batch processing
2const result = await relay
3 .workflow("invoice-batch-processor")
4 .schedule("0 9 * * 1-5") // Weekdays at 9am
5
6 .step("list-files", {
7 systemPrompt: "List all unprocessed PDFs in S3 bucket",
8 })
9 .with("openai:gpt-4o-mini")
10
11 .step("process-all", {
12 systemPrompt: "Process each invoice using the invoice-processor workflow",
13 })
14 .with("openai:gpt-4o-vision")
15 .depends("list-files")
16
17 .run({ bucketName: "invoices-inbox" });

Best Practices

  • Use vision models for extraction - GPT-4 Vision handles PDFs and images better than OCR + LLM
  • Always validate calculations - Use a second model to verify critical financial data
  • Add schema descriptions - Help extraction by describing each field's purpose
  • Use cheaper models for summaries - GPT-4 Mini works great for non-critical text generation
  • Store original files - Keep PDFs for audit trails even after extraction
  • Handle multi-page invoices - Pass all pages to the vision model at once

Error Handling

1try {
2 const invoice = await processInvoice(fileUrl);
3
4 // Check validation results
5 if (!invoice.validation.valid) {
6 console.warn("Validation issues:", invoice.validation.issues);
7 // Route to manual review queue
8 } else {
9 // Auto-approve or route based on amount threshold
10 if (invoice.invoice.totalAmount > 10000) {
11 await routeForApproval(invoice);
12 } else {
13 await autoApprove(invoice);
14 }
15 }
16} catch (error) {
17 console.error("Invoice processing failed:", error);
18 // Route to error queue for manual processing
19 await routeToErrorQueue(fileUrl, error);
20}

Cost Optimization

This workflow costs ~$0.05 per invoice.

  • Extract (GPT-4 Vision): ~$0.03
  • Validate (Claude 3.5 Sonnet): ~$0.015
  • Summarize (GPT-4 Mini): ~$0.002

For 1000 invoices/month, total AI cost is ~$50. Compare to manual data entry at $5-10 per invoice.

Common Mistakes

  • Not validating totals - Always verify that subtotals and totals match line items
  • Missing currency conversion - Handle multi-currency invoices explicitly
  • Ignoring validation failures - Route failed validations to manual review
  • Using wrong model for extraction - Vision models (GPT-4V) work better than OCR + text models

See Also