Import/Export by command line | Opik Documentation

The export/import command-line functions enable you to:

Export: Export specific traces, spans, datasets, prompts, and experiments from a project to local JSON or CSV files
Import: Import data from local JSON files into a project
Migrate: Move data between projects or environments, including experiments and prompts
Backup: Create local backups of specific project data

`opik export WORKSPACE TYPE NAME`

Exports specific data types from the specified workspace to local files.

Arguments:

WORKSPACE: The workspace name to export from
TYPE: The type of data to export (all, dataset, project, experiment, or prompt)
NAME: Exact name of the item to export (not used with all)

Options:

--path, -p: Directory to save exported data (default: opik_exports)
--max-results: Maximum number of items to export per data type (default: 1000)
--filter: OQL filter string applied to traces — works with project, experiment, and all
--force: Re-download items even if they already exist locally
--format: Format for exporting data (json or csv, default: json)
--debug: Enable debug output to show detailed information about the export process
--no-attachments: Skip downloading attachment files.
--page-size INTEGER: Number of traces to fetch per API request when exporting projects (1–1000, default: 500). Applies to project and all. Increase for fewer round-trips; decrease if you hit persistent rate-limit errors.

all-specific options:

--include: Comma-separated list of data types to include (datasets, prompts, projects, experiments). Defaults to all four.

experiment-specific options:

--dataset NAME: Filter experiments by dataset name (only experiments using this dataset will be exported)
--max-traces INTEGER: Maximum number of traces to export (limits total traces downloaded)

Examples:

Use all when you want to export every data type in one shot — useful for full workspace backups or bulk migrations:

$ # Export every data type (datasets, prompts, projects, experiments)
$ opik export my-workspace all
$ 
$ # Export only datasets and prompts — skip projects and experiments
$ opik export my-workspace all --include datasets,prompts
$ 
$ # Export all traces logged after a given date across every project
$ opik export my-workspace all --filter 'created_at >= "2024-01-01T00:00:00Z"'

Use dataset, project, experiment, or prompt when you need to export a single named item:

$ # Export specific dataset by exact name
$ opik export my-workspace dataset "my-test-dataset"
$ 
$ # Export specific project by exact name
$ opik export my-workspace project "my-project"
$ 
$ # Export specific experiment by exact name (--force re-downloads even if local files exist)
$ opik export my-workspace experiment "my-experiment" --force
$ 
$ # Export specific prompt by exact name
$ opik export my-workspace prompt "my-template"

Use --filter to narrow exports to a time window or any OQL predicate — avoids downloading data you don’t need:

$ # Export project traces logged after a given date
$ opik export my-workspace project "my-project" --filter 'created_at >= "2024-01-01T00:00:00Z"'
$ 
$ # Export project traces in a date range
$ opik export my-workspace project "my-project" --filter 'created_at >= "2024-01-01T00:00:00Z" AND created_at < "2024-02-01T00:00:00Z"'
$ 
$ # Export experiment traces logged after a given date
$ opik export my-workspace experiment "my-experiment" --filter 'created_at >= "2024-06-01T00:00:00Z"'

Use experiment-specific options to scope large experiment exports:

$ # Export only traces from experiments that used a specific dataset
$ opik export my-workspace experiment "my-experiment" --dataset "my-dataset"
$ 
$ # Cap total traces downloaded (useful for very large experiments)
$ opik export my-workspace experiment "my-experiment" --max-traces 100

Use --format csv when the target is a spreadsheet or analysis tool rather than a re-import:

$ # Export project traces as CSV for analysis in Excel, Google Sheets, or pandas
$ opik export my-workspace project "my-project" --format csv --path ./csv_data

Use --no-attachments to skip downloading attachment files (faster exports when you only need trace/span data):

$ # Export project without downloading attachments
$ opik export my-workspace project "my-project" --no-attachments
$ 
$ # Export all data without attachments
$ opik export my-workspace all --no-attachments

Miscellaneous options:

$ # Save to a custom directory
$ opik export my-workspace dataset "my-dataset" --path ./backup_data
$ 
$ # Combine filter, result cap, and custom path
$ opik export my-workspace project "my-project" --filter 'start_time >= "2024-01-01T00:00:00Z"' --max-results 100
$ 
$ # Show detailed API call and file I/O information during export
$ opik export my-workspace dataset "my-dataset" --debug --force
$ 
$ # Reduce page size if you are hitting persistent rate-limit errors on large projects
$ opik export my-workspace project "my-project" --page-size 100

`opik import WORKSPACE TYPE NAME`

Imports specific data types from local files to the specified workspace.

Arguments:

WORKSPACE: The workspace name to import to
TYPE: The type of data to import (dataset, project, experiment, or prompt)
NAME: Name pattern to match items (case-insensitive substring matching)

Options:

--path, -p: Directory containing exported data (default: opik_exports)
--dry-run: Show what would be imported without actually importing
--force: Discard the migration manifest and re-import everything from scratch
--no-attachments: Skip uploading attachment files.
--debug: Enable debug output to show detailed information about the import process

Note: Experiment imports automatically recreate experiments where possible. No additional flags are needed.

Resumable imports

Every import automatically maintains a migration_manifest.db SQLite database in the --path directory. It tracks which files have been imported and the ID mappings needed to link experiments back to their traces.

If an import is interrupted (e.g. network error, Ctrl-C), simply re-run the same command — it will resume from where it left off without creating duplicates:

Resuming interrupted import: 142/500 file(s) already completed

When a previously completed import is re-run:

Import already completed. Use --force to re-import.

Use --force to discard the manifest and start fresh.

Examples:

Basic imports — reads from the default opik_exports directory created by opik export:

$ # Import a dataset by exact name
$ opik import my-workspace dataset "my-dataset"
$ 
$ # Import a project by exact name
$ opik import my-workspace project "my-project"
$ 
$ # Import an experiment (automatically recreates the experiment record)
$ opik import my-workspace experiment "my-experiment"
$ 
$ # Import a prompt by exact name
$ opik import my-workspace prompt "my-prompt"

Use a name pattern when you want to import multiple items whose names share a substring:

$ # Import every dataset whose name contains "test"
$ opik import my-workspace dataset "test"

Use --path when your exported data is not in the default opik_exports directory:

$ opik import my-workspace dataset "my-dataset" --path ./custom-exports

Use --dry-run to preview what would be imported before committing — helpful to verify names and counts:

$ opik import my-workspace project "my-project" --dry-run

Resuming and forcing — the manifest in --path handles resumption automatically; use --force only to start over:

$ # Interrupted import: just re-run the same command — progress is resumed from the manifest
$ opik import my-workspace project "my-project" --path ./migration_data
$ 
$ # Discard the manifest and re-import everything from scratch
$ opik import my-workspace project "my-project" --path ./migration_data --force

Use --no-attachments to skip uploading attachment files (faster imports when you only need trace/span data):

$ # Import project without uploading attachments
$ opik import my-workspace project "my-project" --no-attachments
$ 
$ # Import all data without attachments
$ opik import my-workspace all --no-attachments

Use --debug to see per-file status and API call details:

$ opik import my-workspace experiment "my-experiment" --debug

File Format

JSON Format (Default)

The exported data is stored in JSON files with the following structure:

OUTPUT_DIR/
├── migration_manifest.db          # Created automatically during import to track progress
└── WORKSPACE/
    ├── datasets/
    │   ├── dataset_DATASET_NAME_1.json
    │   └── dataset_DATASET_NAME_2.json
    ├── projects/
    │   └── PROJECT_NAME/
    │       ├── trace_TRACE_ID_1.json
    │       ├── trace_TRACE_ID_2.json
    │       └── attachments/
    │           ├── trace/
    │           │   └── TRACE_ID/
    │           │       └── image.png
    │           └── span/
    │               └── SPAN_ID/
    │                   └── document.pdf
    ├── experiments/
    │   ├── experiment_EXPERIMENT_NAME_1.json
    │   └── experiment_EXPERIMENT_NAME_2.json
    └── prompts/
        ├── prompt_PROMPT_NAME_1.json
        └── prompt_PROMPT_NAME_2.json

Each trace file contains:

1 {
2   "trace": {
3     "id": "trace-uuid",
4     "name": "trace-name",
5     "start_time": "2024-01-01T00:00:00Z",
6     "end_time": "2024-01-01T00:01:00Z",
7     "input": {...},
8     "output": {...},
9     "metadata": {...},
10     "tags": [...],
11     "thread_id": "thread-uuid"
12   },
13   "spans": [
14     {
15       "id": "span-uuid",
16       "name": "span-name",
17       "start_time": "2024-01-01T00:00:00Z",
18       "end_time": "2024-01-01T00:01:00Z",
19       "input": {...},
20       "output": {...},
21       "metadata": {...},
22       "type": "general",
23       "model": "gpt-4",
24       "provider": "openai"
25     }
26   ],
27   "attachments": [
28     {
29       "entity_type": "trace",
30       "entity_id": "trace-uuid",
31       "file_name": "image.png"
32     },
33     {
34       "entity_type": "span",
35       "entity_id": "span-uuid",
36       "file_name": "document.pdf"
37     }
38   ],
39   "downloaded_at": "2024-01-01T00:00:00Z",
40   "project_name": "source-project"
41 }

Each evaluation rule file contains:

1 {
2   "id": "rule-uuid",
3   "name": "rule-name",
4   "project_id": "project-uuid",
5   "project_name": "project-name",
6   "sampling_rate": 1.0,
7   "enabled": true,
8   "filters": [...],
9   "action": "evaluator",
10   "type": "llm_as_judge",
11   "created_at": "2024-01-01T00:00:00Z",
12   "created_by": "user-id",
13   "last_updated_at": "2024-01-01T00:00:00Z",
14   "last_updated_by": "user-id",
15   "evaluator_data": {
16     "llm_as_judge_code": {
17       "prompt": "Evaluate the response...",
18       "model": "gpt-4",
19       "temperature": 0.0
20     }
21   },
22   "downloaded_at": "2024-01-01T00:00:00Z"
23 }

Each experiment file contains:

1 {
2   "experiment": {
3     "id": "experiment-uuid",
4     "name": "experiment-name",
5     "dataset_name": "dataset-name",
6     "type": "regular",
7     "metadata": {...},
8     "created_at": "2024-01-01T00:00:00Z"
9   },
10   "items": [
11     {
12       "trace_id": "trace-uuid",
13       "dataset_item_id": "dataset-item-uuid",
14       "dataset_item_data": {...},
15       "feedback_scores": [...],
16       "trace_reference": {
17         "trace_id": "trace-uuid",
18         "note": "Full trace data not included to avoid duplication"
19       }
20     }
21   ],
22   "downloaded_at": "2024-01-01T00:00:00Z"
23 }

Each prompt file contains:

1 {
2   "name": "prompt-name",
3   "current_version": {
4     "prompt": "Your prompt template here...",
5     "metadata": {...},
6     "type": "MUSTACHE",
7     "commit": "commit-hash"
8   },
9   "history": [
10     {
11       "prompt": "Previous version of the prompt...",
12       "metadata": {...},
13       "type": "MUSTACHE",
14       "commit": "previous-commit-hash"
15     }
16   ],
17   "downloaded_at": "2024-01-01T00:00:00Z"
18 }

CSV Format

When using --format csv, data is exported as CSV files with flattened data structure. This format is ideal for:

Data Analysis: Easy to import into Excel, Google Sheets, or data analysis tools
Large Datasets: More efficient storage for large numbers of traces
Spreadsheet Integration: Direct compatibility with business intelligence tools

CSV File Structure:

OUTPUT_DIR/
└── WORKSPACE/
    ├── datasets/
    │   └── datasets_DATASET_NAME.csv      # Dataset data in CSV format
    ├── projects/
    │   └── PROJECT_NAME/
    │       └── traces_PROJECT_NAME.csv    # All traces in a single CSV file
    ├── experiments/
    │   └── experiments_EXPERIMENT_NAME.csv # Experiment data in CSV format
    └── prompts/
        └── prompts_PROMPT_NAME.csv        # Prompt data in CSV format

CSV Format Benefits:

Single File: All data combined into one CSV file per data type
Flattened Structure: Nested JSON data is flattened with dot notation
Column Headers: Clear column names for easy analysis
Compatible: Works with Excel, Google Sheets, pandas, etc.
Universal Format: All data types (datasets, projects, experiments, prompts) support CSV export

Example CSV Structure:

1 trace_id,trace_name,start_time,end_time,thread_id,span_id,span_name,span_type,span_model,span_provider,input,output,metadata
2 trace-123,my-trace,2024-01-01T00:00:00Z,2024-01-01T00:01:00Z,thread-456,span-789,llm-call,llm,gpt-4,openai,"{""prompt"":""Hello""}","{""response"":""Hi""}","{""tokens"":10}"

Use Cases

1. Migrating between Opik installations

Migration is a two-step process because the source and destination typically have different API keys. Export with your source credentials, then import with your destination credentials. The migration_manifest.db created during import travels with the exported data and makes the import safe to interrupt and resume.

$ # Step 1: Export from source (run with source API key / Opik URL)
$ OPIK_API_KEY=<source_key> OPIK_URL_OVERRIDE=https://source.opik.example.com \
>   opik export my-workspace project "my-project" --path ./migration_data
$ 
$ # Step 2: Import to destination (run with destination API key / Opik URL)
$ # If this is interrupted, just re-run the same command — it resumes automatically.
$ OPIK_API_KEY=<dest_key> OPIK_URL_OVERRIDE=https://dest.opik.example.com \
>   opik import my-workspace project "my-project" --path ./migration_data

To migrate all data types at once:

$ # Export everything (source credentials)
$ OPIK_API_KEY=<source_key> opik export my-workspace dataset "my-dataset" --path ./migration_data
$ OPIK_API_KEY=<source_key> opik export my-workspace prompt "my-prompt" --path ./migration_data
$ OPIK_API_KEY=<source_key> opik export my-workspace project "my-project" --path ./migration_data
$ OPIK_API_KEY=<source_key> opik export my-workspace experiment "my-experiment" --path ./migration_data
$ 
$ # Import everything (destination credentials)
$ OPIK_API_KEY=<dest_key> opik import my-workspace dataset "my-dataset" --path ./migration_data
$ OPIK_API_KEY=<dest_key> opik import my-workspace prompt "my-prompt" --path ./migration_data
$ OPIK_API_KEY=<dest_key> opik import my-workspace project "my-project" --path ./migration_data
$ OPIK_API_KEY=<dest_key> opik import my-workspace experiment "my-experiment" --path ./migration_data

2. Data Backup

$ # Create backup of specific data (requires knowing exact names)
$ opik export my-workspace dataset "my-dataset" --path ./backup_$(date +%Y%m%d)
$ opik export my-workspace project "my-project" --path ./backup_$(date +%Y%m%d)
$ opik export my-workspace experiment "my-experiment" --path ./backup_$(date +%Y%m%d)

3. Environment Sync

$ # Sync from staging to production
$ opik export my-workspace project "staging-project" --filter 'tags contains "ready-for-prod"'
$ opik import my-workspace project "staging-project"

4. Data Analysis

$ # Export traces for a specific time window
$ opik export my-workspace project "my-project" \
>   --filter 'created_at >= "2024-01-01T00:00:00Z" AND created_at < "2024-04-01T00:00:00Z"'
$ 
$ # Export all traces logged before a cutoff date across every project
$ opik export my-workspace all --filter 'created_at < "2024-01-01T00:00:00Z"'
$ 
$ # Export recent experiment traces only
$ opik export my-workspace experiment "my-experiment" --filter 'created_at >= "2024-06-01T00:00:00Z"'

5. Dataset Management

$ # Export specific dataset from a workspace
$ opik export my-workspace dataset "my-dataset"
$ 
$ # Import datasets to another workspace (uses default opik_exports directory)
$ opik import my-workspace dataset "my-dataset"

6. Data Analysis with CSV

$ # Export traces in CSV format for analysis
$ opik export my-workspace project "my-project" --format csv --path ./analysis_data
$ 
$ # Export datasets in CSV format for analysis
$ opik export my-workspace dataset "my-dataset" --format csv --path ./analysis_data
$ 
$ # Export experiments in CSV format for analysis
$ opik export my-workspace experiment "my-experiment" --format csv --path ./analysis_data
$ 
$ # Export prompts in CSV format for analysis
$ opik export my-workspace prompt "my-template" --format csv --path ./analysis_data
$ 
$ # Open in Excel or Google Sheets for analysis
$ # Or use with pandas in Python:
$ # import pandas as pd
$ # df = pd.read_csv('./analysis_data/my-workspace/projects/my-project/traces_my-project.csv')

7. Prompt Management

$ # Export specific prompt from a workspace
$ opik export my-workspace prompt "my-template" --path ./prompt_backup
$ 
$ # Export another prompt template
$ opik export my-workspace prompt "system-prompt" --path ./templates
$ 
$ # Import prompts to another workspace
$ opik import my-workspace prompt "my-template" --path ./prompt_backup
$ 
$ # Import with name pattern matching
$ opik import my-workspace prompt "production" --path ./prompt_backup

8. Experiment Migration

$ # Export specific experiment from source workspace
$ opik export my-workspace experiment "my-experiment" --path ./experiment_data
$ 
$ # Export experiment with dataset filtering
$ opik export my-workspace experiment "evaluation-exp" --dataset "test-dataset"
$ 
$ # Export only traces logged after a given date
$ opik export my-workspace experiment "large-experiment" --filter 'created_at >= "2024-06-01T00:00:00Z"'
$ 
$ # Export experiment with trace limit (useful for large experiments)
$ opik export my-workspace experiment "large-experiment" --max-traces 50
$ 
$ # Import experiments (automatically recreates experiments)
$ opik import my-workspace experiment "my-experiment" --path ./experiment_data
$ 
$ # Import experiments matching name pattern
$ opik import my-workspace experiment "evaluation" --path ./experiment_data

Troubleshooting

Common Issues

Import was interrupted and re-running creates duplicates
- This should not happen — the import automatically detects and resumes from the migration_manifest.db in your --path directory
- If you do not see the “Resuming interrupted import” message, check that you are pointing to the same --path as the original run
- If the manifest database is unreadable, delete migration_manifest.db and use --force to start fresh (this will re-import everything)
“Import already completed. Use —force to re-import.”
- The manifest records that a previous run finished successfully
- If you genuinely need to re-import (e.g. the destination was wiped), use --force
Experiment items are missing after import
- Experiments are linked to traces by ID; if the trace import was interrupted and no manifest was present, some mappings may be missing
- Re-run the project import first (it populates the trace ID map in the manifest), then re-run the experiment import
- Use --debug to see how many trace IDs were matched
“No traces found”
- Check if the project name is correct
- Verify you have access to the project
- Try without filters first
“Project directory not found”
- Make sure you’ve exported data first
- Check the input directory path
- Verify the project name matches
“Opik SDK not available”
- Ensure Opik is properly installed
- Check your Python environment
- Verify the installation with opik healthcheck
“Dataset/Project/Experiment/Prompt not found”
- Check that the exact name is correct
- Verify you have access to the item
- Use --debug for more detailed error information
“No datasets/projects found”
- The system will show available items to help you choose the right name
- Check spelling and case sensitivity
- Ensure the item exists in the workspace
“Dataset not found”
- The system will show datasets used by matching experiments
- Verify the dataset name is correct
- Use --debug to see detailed search information
Export is slow or keeps hitting rate limits
- The exporter automatically retries 429 responses, honouring Retry-After headers where present
- Reduce --page-size (e.g. --page-size 100) to lower the per-request payload and decrease the chance of triggering rate limits
- The exporter pauses between pages and uses at most 3 concurrent workers to stay within server-side limits; no extra configuration is needed

Getting Help

$ # Get help for export command
$ opik export --help
$ 
$ # Get help for import command
$ opik import --help
$ 
$ # Check system health
$ opik healthcheck

Example Workflow

Here’s a complete example of exporting and importing trace data:

JSON Format Workflow

$ # 1. Export specific data from source workspace (JSON format)
$ opik export my-workspace dataset "my-dataset" --path ./temp_data
$ opik export my-workspace project "my-project" --path ./temp_data
$ opik export my-workspace experiment "my-experiment" --path ./temp_data
$ opik export my-workspace prompt "my-template" --path ./temp_data
$ 
$ # Alternative: Export experiment with specific dataset filtering
$ opik export my-workspace experiment "evaluation-exp" --dataset "test-dataset" --max-traces 100 --path ./temp_data
$ 
$ # 2. Inspect the exported data
$ ls ./temp_data/my-workspace/datasets/
$ ls ./temp_data/my-workspace/projects/
$ ls ./temp_data/my-workspace/experiments/
$ ls ./temp_data/my-workspace/prompts/
$ cat ./temp_data/my-workspace/projects/my-project/trace_*.json | head -20
$ 
$ # 3. Dry run import to see what would be imported
$ opik import my-workspace dataset "my-dataset" --path ./temp_data --dry-run
$ opik import my-workspace project "my-project" --path ./temp_data --dry-run
$ opik import my-workspace experiment "my-experiment" --path ./temp_data --dry-run
$ opik import my-workspace prompt "my-template" --path ./temp_data --dry-run
$ 
$ # 4. Import all data — if interrupted, just re-run the same commands to resume
$ opik import my-workspace dataset "my-dataset" --path ./temp_data
$ opik import my-workspace project "my-project" --path ./temp_data
$ opik import my-workspace experiment "my-experiment" --path ./temp_data
$ opik import my-workspace prompt "my-template" --path ./temp_data
$ 
$ # 5. Clean up temporary data
$ rm -rf ./temp_data

CSV Format Workflow

$ # 1. Export data in CSV format for analysis
$ opik export my-workspace project "my-source-project" --format csv --path ./csv_data
$ opik export my-workspace dataset "my-dataset" --format csv --path ./csv_data
$ opik export my-workspace experiment "my-experiment" --format csv --path ./csv_data
$ opik export my-workspace prompt "my-template" --format csv --path ./csv_data
$ 
$ # 2. Inspect the CSV files
$ ls ./csv_data/my-workspace/projects/my-source-project/
$ ls ./csv_data/my-workspace/datasets/
$ ls ./csv_data/my-workspace/experiments/
$ ls ./csv_data/my-workspace/prompts/
$ 
$ # Each trace is exported to its own CSV file
$ head -5 ./csv_data/my-workspace/projects/my-source-project/trace_*.csv | head -20
$ head -5 ./csv_data/my-workspace/datasets/datasets_test-dataset.csv
$ 
$ # 3. Analyze with pandas (optional)
$ python -c "
> import pandas as pd
> import glob
> 
> # Read all trace CSV files and combine them
> trace_files = glob.glob('./csv_data/my-workspace/projects/my-source-project/trace_*.csv')
> df_traces = pd.concat([pd.read_csv(f) for f in trace_files], ignore_index=True) if trace_files else pd.DataFrame()
> df_datasets = pd.read_csv('./csv_data/my-workspace/datasets/datasets_test-dataset.csv')
> print(f'Exported {len(df_traces)} trace records from {len(trace_files)} files')
> print(f'Exported {len(df_datasets)} dataset records')
> print('Trace columns:', df_traces.columns.tolist() if not df_traces.empty else 'No traces')
> print('Dataset columns:', df_datasets.columns.tolist())
> "
$ 
$ # 4. For import, you would need to convert back to JSON format
$ # (CSV format is primarily for analysis, not import)

This workflow ensures you can safely migrate all data including experiments and prompts between workspaces while maintaining data integrity and providing visibility into the process. The CSV format is particularly useful for data analysis and reporting, while the JSON format preserves the complete structure needed for experiment and prompt recreation. The new command structure provides better organization with separate commands for datasets, projects, experiments, and prompts, making it easier to manage specific data types.