> For the complete documentation index, see [llms.txt](https://docs.coupler.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.coupler.io/sources/category/other/captain-data/best-practices.md).

# Best Practices

## Recommended setup

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Use Successful Jobs instead of Jobs when accuracy matters</strong></td><td>The Successful Jobs entity pre-filters out failed and in-progress runs, so your reports reflect only clean, completed data. This saves you from building manual filters in your destination.</td></tr><tr><td><strong>Join Jobs with Job Results</strong></td><td>Job Results alone won't tell you which workflow produced them or when. Join on Job ID to attach execution metadata — timestamps, workflow name, record count — to each result row for richer analysis.</td></tr><tr><td><strong>Separate entities into different data flow sources</strong></td><td>If you need both workflow-level and result-level data, add them as separate sources within the same data flow and use a Join transformation. This keeps your sync logic clean and easier to debug.</td></tr></tbody></table>

## Data refresh and scheduling

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Align your sync frequency with your workflow run cadence</strong></td><td>If your Captain Data workflows run once a day, a daily Coupler.io schedule is enough. Syncing more frequently than your jobs actually run just adds API calls without new data.</td></tr><tr><td><strong>Always do a successful manual run first</strong></td><td>Before setting a schedule, confirm your data flow returns the expected records manually. This catches permission issues or empty entities before they silently fail on a schedule.</td></tr></tbody></table>

## Performance optimization

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Pull Job Results per workflow, not globally</strong></td><td>If you have many workflows, pulling all job results at once can return a very large, mixed-schema dataset. Structure separate data flows per workflow type to keep result fields consistent and easier to use in your destination.</td></tr><tr><td><strong>Send results to BigQuery for large datasets</strong></td><td>If your workflows process thousands of records per job, Google Sheets will hit row limits quickly. BigQuery or another warehouse destination handles high volumes without breaking.</td></tr></tbody></table>

## Common pitfalls

{% hint style="danger" %}
Job Results have no fixed schema — the fields vary by workflow. If you append results from multiple workflow types into the same sheet or table, you'll end up with misaligned columns and a lot of empty cells. Keep result sets from different workflow types in separate destinations or use an Aggregate transformation to normalize before loading.
{% endhint %}

{% columns %}
{% column %}
**Do**

* Use the Successful Jobs entity as your primary jobs reference
* Join Job Results to Jobs on Job ID before sending to a destination
* Verify your API key scope matches the workspace you're trying to access
* Use BigQuery or a warehouse for high-volume job result data
  {% endcolumn %}

{% column %}
**Don't**

* Mix job results from workflows with different output schemas in one table
* Sync more frequently than your Captain Data workflows actually run
* Rely on record counts from the UI to validate Coupler.io output — use Job ID for reconciliation
  {% endcolumn %}
  {% endcolumns %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.coupler.io/sources/category/other/captain-data/best-practices.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.