Migrating to Opik 2.0

Opik 2.0 organizes everything around projects. Data created after upgrading is automatically project-scoped. Data created on Opik 1.x — datasets, prompts, experiments, optimizations, automation rules, and alerts — requires six background jobs to assign each item to the right project. This page walks you through running those jobs.

Before you start

1. Update to the latest Opik image. All six migration jobs ship with the 2.0 release. Make sure your opik-backend is running the 2.0 image before enabling any flag.
2. Take a database snapshot. Each job writes to MySQL or ClickHouse. Back up both before starting — see Advanced ClickHouse backup for ClickHouse, and use mysqldump or your RDS snapshot tooling for MySQL.
3. Confirm your deployment is healthy. The opik-backend container should be running with no restart loops, and MySQL and ClickHouse should be at baseline load.

How the jobs work

Each job runs inside opik-backend on a 30-second cycle. They are:

• Safe to run alongside live traffic — no user-visible locks; users can keep working throughout.
• Resumable — stopping a job mid-run and re-enabling it picks up where it left off.
• Stoppable, not undoable — already-committed writes stay; there is no rollback mode.

When an item’s associated data spans multiple projects, it is assigned to the dominant project — the one most referenced by its runs. When no project can be determined at all (no associated runs, or the original project was deleted), it falls back to the workspace’s Default Project, which is created on demand if missing.

Order of execution

Step 1 — Experiments

Enable

1
services:
2
  backend:
3
    environment:
4
      - EXPERIMENT_PROJECT_MIGRATION_ENABLED=true

$
docker compose -f deployment/docker-compose/docker-compose.yml up -d backend

Watch for completion

$
docker compose -f deployment/docker-compose/docker-compose.yml logs -f backend \
>
  | grep -i "experiment.*migration"

The job is done when this line appears consistently for several cycles:

No workspaces with eligible experiments found, consider disabling the job

Disable

Set EXPERIMENT_PROJECT_MIGRATION_ENABLED=false using the same method above and apply. Wait for the completion signal to be stable before moving to Step 2.

Step 2 — Datasets and prompts

Enable both once Step 1 is complete. They are independent of each other and can run in parallel.

1
services:
2
  backend:
3
    environment:
4
      - DATASET_PROJECT_MIGRATION_ENABLED=true
5
      - PROMPT_PROJECT_MIGRATION_ENABLED=true

$
docker compose -f deployment/docker-compose/docker-compose.yml up -d backend

Completion signals:

No workspaces with eligible datasets found, consider disabling the job
No workspaces with orphan prompts found, consider disabling the job

Disable each flag individually once its completion signal is stable before moving to Step 3.

Step 3 — Optimizations

Enable only after both Step 1 and Step 2 are complete. Optimizations infer their project from both experiments and datasets.

1
services:
2
  backend:
3
    environment:
4
      - OPTIMIZATION_PROJECT_MIGRATION_ENABLED=true

$
docker compose -f deployment/docker-compose/docker-compose.yml up -d backend

Completion signal:

No workspaces with eligible optimizations found, consider disabling the job

Disable the flag once the signal is stable.

Step 4 — Automation rules and alerts

These jobs inspect project references already stored in their own rows and have no dependency on the other jobs. Run them at any point — before, during, or after Steps 1–3.

1
services:
2
  backend:
3
    environment:
4
      - AUTOMATION_RULE_PROJECT_MIGRATION_ENABLED=true
5
      - ALERT_PROJECT_MIGRATION_ENABLED=true

$
docker compose -f deployment/docker-compose/docker-compose.yml up -d backend

Completion signals:

No workspaces with multi-project automation rules found, consider disabling the job
No workspaces with orphan alerts found, consider disabling the job

Disable each flag once done.

Verifying the migration

Once all six jobs have reported their completion signals and been disabled, run these queries to confirm the result.

ClickHouse — orphan experiments (should be 0):

1
SELECT countIf(project_id = '') AS orphan_experiments
2
FROM experiments FINAL;

MySQL — orphan datasets (should be 0):

1
SELECT COUNT(*) FROM datasets WHERE project_id IS NULL OR project_id = '';

MySQL — orphan prompts (should be 0):

1
SELECT COUNT(*) FROM prompts WHERE project_id IS NULL;

ClickHouse — orphan optimizations (should be 0):

1
SELECT countIf(project_id = '') AS orphan_optimizations
2
FROM optimizations FINAL;

MySQL — multi-project automation rules (should be 0):

1
SELECT COUNT(*) FROM (
2
  SELECT rule_id FROM automation_rule_projects
3
  GROUP BY workspace_id, rule_id HAVING COUNT(project_id) > 1
4
) t;

MySQL — orphan alerts (should be 0):

1
SELECT COUNT(*) FROM alerts WHERE project_id IS NULL;

Troubleshooting

Next steps

Once migration is complete, update your SDK and pass project_name on your API calls. See Upgrading to Opik 2.0 for the full code reference and examples.