Invoice Processor
Extract structured data from invoices using vision + language models in a local-first workflow.
Quick Example
1import { relay } from "@relayplane/workflows";23const result = await relay4 .workflow("invoice-processor")5 .step("extract").with("openai:gpt-4o-vision")6 .step("validate").with("anthropic:claude-3.5-sonnet").depends("extract")7 .run({ fileUrl: "https://example.com/invoice.pdf" });Full Production Implementation
This example demonstrates a complete invoice processing pipeline with schema validation, error handling, and executive summary generation.
1import { relay } from "@relayplane/workflows";2import { z } from "zod";34// Define structured invoice schema5const InvoiceSchema = z.object({6 invoiceNumber: z.string().describe("Invoice number or ID"),7 date: z.string().describe("Invoice date in ISO format"),8 dueDate: z.string().optional().describe("Payment due date"),9 vendor: z.object({10 name: z.string(),11 address: z.string().optional(),12 taxId: z.string().optional(),13 }),14 billTo: z.object({15 name: z.string(),16 address: z.string().optional(),17 }),18 items: z.array(z.object({19 description: z.string(),20 quantity: z.number(),21 unitPrice: z.number(),22 amount: z.number(),23 })),24 subtotal: z.number(),25 tax: z.number().optional(),26 totalAmount: z.number(),27 currency: z.string().default("USD"),28});2930type Invoice = z.infer; 3132// Build multi-step workflow33async function processInvoice(fileUrl: string) {34 const result = await relay35 .workflow("invoice-processor")3637 // Step 1: Extract structured data from invoice image/PDF38 .step("extract", {39 schema: InvoiceSchema,40 systemPrompt: `Extract all invoice fields as structured JSON.41 - Parse line items carefully with quantities and prices42 - Calculate subtotals and verify against invoice total43 - Extract vendor and customer information44 - Preserve currency information`,45 })46 .with("openai:gpt-4o-vision")4748 // Step 2: Validate extracted data49 .step("validate", {50 systemPrompt: `Verify the extracted invoice data:51 1. Check that item amounts = quantity * unitPrice52 2. Verify subtotal matches sum of all items53 3. Check that total = subtotal + tax54 4. Flag any discrepancies or missing critical fields55 Return: { valid: boolean, issues: string[], confidence: number }`,56 })57 .with("anthropic:claude-3.5-sonnet")58 .depends("extract")5960 // Step 3: Generate executive summary61 .step("summarize", {62 systemPrompt: `Create a brief executive summary for finance approval:63 - Key vendor and amount64 - Payment due date65 - Any validation issues or red flags66 - Recommendation (approve/review/reject)67 Keep it under 3 sentences.`,68 })69 .with("openai:gpt-4o-mini")70 .depends("validate")7172 .run({ fileUrl });7374 // Return structured results75 return {76 invoice: result.steps.extract as Invoice,77 validation: result.steps.validate,78 summary: result.steps.summarize,79 };80}8182// Usage example83const invoice = await processInvoice("https://example.com/invoice-2024-001.pdf");8485console.log("Invoice Number:", invoice.invoice.invoiceNumber);86console.log("Total Amount:", invoice.invoice.totalAmount, invoice.invoice.currency);87console.log("Summary:", invoice.summary);How It Works
The invoice processor uses a three-step pipeline:
- Extract - GPT-4 Vision analyzes the invoice image/PDF and extracts structured data matching the Zod schema. The schema provides field descriptions that guide extraction.
- Validate - Claude 3.5 Sonnet verifies calculations, checks for missing fields, and flags potential issues. This step depends on the extract step.
- Summarize - GPT-4 Mini generates a concise executive summary for approval workflows. Uses the cheaper model since summarization is less critical.
Use Cases
- •Accounts Payable Automation - Auto-extract invoice data into ERP systems
- •Expense Report Processing - Convert receipt photos to structured expense entries
- •Contract Data Extraction - Extract key terms from vendor contracts
- •Purchase Order Matching - Compare invoices against POs automatically
- •Audit Trail Generation - Create structured records for compliance
Adding Webhooks
To trigger this workflow via HTTP (requires RelayPlane Pro):
1# Install CLI and authenticate2npx relay login34# This workflow can now be triggered via webhook5curl -X POST https://api.relayplane.com/webhooks/trigger \6 -H "Authorization: Bearer YOUR_API_KEY" \7 -H "Content-Type: application/json" \8 -d '{9 "workflowName": "invoice-processor",10 "input": {11 "fileUrl": "https://example.com/invoice.pdf"12 }13 }'Adding Schedules
Process invoices from a folder on a schedule (requires RelayPlane Pro):
1// Scheduled batch processing2const result = await relay3 .workflow("invoice-batch-processor")4 .schedule("0 9 * * 1-5") // Weekdays at 9am56 .step("list-files", {7 systemPrompt: "List all unprocessed PDFs in S3 bucket",8 })9 .with("openai:gpt-4o-mini")1011 .step("process-all", {12 systemPrompt: "Process each invoice using the invoice-processor workflow",13 })14 .with("openai:gpt-4o-vision")15 .depends("list-files")1617 .run({ bucketName: "invoices-inbox" });Best Practices
- •Use vision models for extraction - GPT-4 Vision handles PDFs and images better than OCR + LLM
- •Always validate calculations - Use a second model to verify critical financial data
- •Add schema descriptions - Help extraction by describing each field's purpose
- •Use cheaper models for summaries - GPT-4 Mini works great for non-critical text generation
- •Store original files - Keep PDFs for audit trails even after extraction
- •Handle multi-page invoices - Pass all pages to the vision model at once
Error Handling
1try {2 const invoice = await processInvoice(fileUrl);34 // Check validation results5 if (!invoice.validation.valid) {6 console.warn("Validation issues:", invoice.validation.issues);7 // Route to manual review queue8 } else {9 // Auto-approve or route based on amount threshold10 if (invoice.invoice.totalAmount > 10000) {11 await routeForApproval(invoice);12 } else {13 await autoApprove(invoice);14 }15 }16} catch (error) {17 console.error("Invoice processing failed:", error);18 // Route to error queue for manual processing19 await routeToErrorQueue(fileUrl, error);20}Cost Optimization
This workflow costs ~$0.05 per invoice.
- Extract (GPT-4 Vision): ~$0.03
- Validate (Claude 3.5 Sonnet): ~$0.015
- Summarize (GPT-4 Mini): ~$0.002
For 1000 invoices/month, total AI cost is ~$50. Compare to manual data entry at $5-10 per invoice.
Common Mistakes
- Not validating totals - Always verify that subtotals and totals match line items
- Missing currency conversion - Handle multi-currency invoices explicitly
- Ignoring validation failures - Route failed validations to manual review
- Using wrong model for extraction - Vision models (GPT-4V) work better than OCR + text models