Best Practices

Recommended setup

Start with Pipelines + Insights metrics

These two entities give you the most useful CI/CD picture immediately. Use Join to link them on project slug for a combined view of volume and performance.

Use a dedicated API token

Create a CircleCI personal API token specifically for Coupler.io rather than reusing a shared or personal token. This makes it easy to revoke access without breaking other integrations.

Set a realistic start date

For the initial sync, use a start date 30–90 days back rather than your full history. CircleCI's API pagination means large historical loads are slow, and you can always extend the window later.

Combine multiple projects with Append

If you manage several CircleCI projects, add multiple sources to a single data flow and use the Append transformation to unify pipeline or job data across all of them in one destination table.

Data refresh and scheduling

Schedule based on your pipeline cadence

If your team pushes code multiple times a day, an hourly or every-few-hours refresh keeps your dashboards current. For slower release cycles, daily is usually enough.

Run Insights separately from pipeline data

Insights metrics aggregate data on CircleCI's side and update less frequently than raw pipeline records. You can schedule Insights entities less often (e.g., daily) to avoid unnecessary API calls.

Performance optimization

Scope to specific projects and workflows

Use the Project ID and Workflow ID parameters to narrow your data pull. Fetching org-wide data across all workflows is much slower and returns more noise than you usually need.

Use Aggregate for duration analysis

Rather than exporting every raw job record, use Coupler.io's Aggregate transformation to pre-calculate average and p95 job durations grouped by workflow name or branch. This keeps your destination tables lean.

Common pitfalls

Don't confuse Project ID with project slug. Some entities (like Specific projects) need the UUID-format Project ID from your project settings URL, while others accept the slug (e.g., gh/myorg/myrepo). Using the wrong format will return an error or empty data.

Do

• Specify a project ID or workflow ID when querying Jobs or Workflow jobs

• Check that your API token belongs to a user with access to the target project

• Use the Workflow jobs entity to auto-populate job numbers before pulling the Jobs entity

Don't

• Leave the start date blank on large orgs — it will try to pull all history

• Share your personal API token across team members or store it in a shared doc

• Query Contexts data unless your token belongs to an org admin account