# Best Practices

## Recommended setup

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Use Successful Jobs instead of Jobs when accuracy matters</strong></td><td>The Successful Jobs entity pre-filters out failed and in-progress runs, so your reports reflect only clean, completed data. This saves you from building manual filters in your destination.</td></tr><tr><td><strong>Join Jobs with Job Results</strong></td><td>Job Results alone won't tell you which workflow produced them or when. Join on Job ID to attach execution metadata — timestamps, workflow name, record count — to each result row for richer analysis.</td></tr><tr><td><strong>Separate entities into different data flow sources</strong></td><td>If you need both workflow-level and result-level data, add them as separate sources within the same data flow and use a Join transformation. This keeps your sync logic clean and easier to debug.</td></tr></tbody></table>

## Data refresh and scheduling

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Align your sync frequency with your workflow run cadence</strong></td><td>If your Captain Data workflows run once a day, a daily Coupler.io schedule is enough. Syncing more frequently than your jobs actually run just adds API calls without new data.</td></tr><tr><td><strong>Always do a successful manual run first</strong></td><td>Before setting a schedule, confirm your data flow returns the expected records manually. This catches permission issues or empty entities before they silently fail on a schedule.</td></tr></tbody></table>

## Performance optimization

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Pull Job Results per workflow, not globally</strong></td><td>If you have many workflows, pulling all job results at once can return a very large, mixed-schema dataset. Structure separate data flows per workflow type to keep result fields consistent and easier to use in your destination.</td></tr><tr><td><strong>Send results to BigQuery for large datasets</strong></td><td>If your workflows process thousands of records per job, Google Sheets will hit row limits quickly. BigQuery or another warehouse destination handles high volumes without breaking.</td></tr></tbody></table>

## Common pitfalls

{% hint style="danger" %}
Job Results have no fixed schema — the fields vary by workflow. If you append results from multiple workflow types into the same sheet or table, you'll end up with misaligned columns and a lot of empty cells. Keep result sets from different workflow types in separate destinations or use an Aggregate transformation to normalize before loading.
{% endhint %}

{% columns %}
{% column %}
**Do**

* Use the Successful Jobs entity as your primary jobs reference
* Join Job Results to Jobs on Job ID before sending to a destination
* Verify your API key scope matches the workspace you're trying to access
* Use BigQuery or a warehouse for high-volume job result data
  {% endcolumn %}

{% column %}
**Don't**

* Mix job results from workflows with different output schemas in one table
* Sync more frequently than your Captain Data workflows actually run
* Rely on record counts from the UI to validate Coupler.io output — use Job ID for reconciliation
  {% endcolumn %}
  {% endcolumns %}