Best Practices

Recommended setup

Use Successful Jobs instead of Jobs when accuracy matters

The Successful Jobs entity pre-filters out failed and in-progress runs, so your reports reflect only clean, completed data. This saves you from building manual filters in your destination.

Join Jobs with Job Results

Job Results alone won't tell you which workflow produced them or when. Join on Job ID to attach execution metadata — timestamps, workflow name, record count — to each result row for richer analysis.

Separate entities into different data flow sources

If you need both workflow-level and result-level data, add them as separate sources within the same data flow and use a Join transformation. This keeps your sync logic clean and easier to debug.

Data refresh and scheduling

Align your sync frequency with your workflow run cadence

If your Captain Data workflows run once a day, a daily Coupler.io schedule is enough. Syncing more frequently than your jobs actually run just adds API calls without new data.

Always do a successful manual run first

Before setting a schedule, confirm your data flow returns the expected records manually. This catches permission issues or empty entities before they silently fail on a schedule.

Performance optimization

Pull Job Results per workflow, not globally

If you have many workflows, pulling all job results at once can return a very large, mixed-schema dataset. Structure separate data flows per workflow type to keep result fields consistent and easier to use in your destination.

Send results to BigQuery for large datasets

If your workflows process thousands of records per job, Google Sheets will hit row limits quickly. BigQuery or another warehouse destination handles high volumes without breaking.

Common pitfalls

Job Results have no fixed schema — the fields vary by workflow. If you append results from multiple workflow types into the same sheet or table, you'll end up with misaligned columns and a lot of empty cells. Keep result sets from different workflow types in separate destinations or use an Aggregate transformation to normalize before loading.

Do

• Use the Successful Jobs entity as your primary jobs reference

• Join Job Results to Jobs on Job ID before sending to a destination

• Verify your API key scope matches the workspace you're trying to access

• Use BigQuery or a warehouse for high-volume job result data

Don't

• Mix job results from workflows with different output schemas in one table

• Sync more frequently than your Captain Data workflows actually run

• Rely on record counts from the UI to validate Coupler.io output — use Job ID for reconciliation