Observability for GitHub Copilot with Opik

View as Markdown

GitHub Copilot can export traces, metrics and events over OpenTelemetry (OTel) from both the Copilot Chat extension in VS Code and the Copilot CLI. The spans follow the OTel GenAI semantic conventions, which Opik ingests natively, so every chat turn, model call and tool execution lands in Opik as a trace you can review, evaluate and share.

When this guide applies

Use this guide if you want to log Copilot conversations to Opik: the prompts your developers send, the models Copilot calls, the tools it runs, and the tokens each turn consumed.

This guide covers telemetry flowing from Copilot to Opik. For the other direction, letting Copilot read your traces, score outputs and run evaluations from chat, register the Opik MCP server with it. One command, uvx opik mcp configure, installs the server and the Opik skills. The two are independent, and you can use either or both.

Looking to reduce coding-agent spend rather than review sessions?

Opik shows you what your agents did. If the question is where the tokens went and how to spend fewer of them, that is Cost Intelligence: exact per-call token attribution across system prompt, tools, MCP servers, skills and memory, per-user and per-repository, with policy controls that typically cut spend by 15% to 30%. Cost Intelligence currently supports Claude Code, Codex and Cursor; talk to your Comet account team about Copilot coverage.

Quick start

Copilot Chat in VS Code and Copilot CLI both honor the standard OpenTelemetry environment variables. For a single developer on Opik Cloud, this is the whole setup:

$export COPILOT_OTEL_ENABLED=true
$export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
$export OTEL_EXPORTER_OTLP_ENDPOINT=https://www.comet.com/opik/api/v1/private/otel
$export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=<your-workspace>,projectName=<your-project-name>'
$export COPILOT_OTEL_CAPTURE_CONTENT=true # omit to log structure and tokens only, no prompt text

Then launch VS Code from that shell with code . (or run copilot for the CLI) and start a chat. The trace appears in the Opik project named in projectName. The rest of this page covers the other configuration paths, self-hosted endpoints, what the spans contain and how to validate.

Prerequisites

  • An Opik API key and workspace name. Opik Cloud: comet.com/api/my/settings.
  • GitHub Copilot Chat extension in VS Code, or GitHub Copilot CLI, signed in.
  • Permission from your organization to export prompt content, if you plan to enable content capture.

What Opik receives

Copilot emits four span kinds, all with gen_ai.* attributes:

SpanWhat it representsKey attributes Opik maps
invoke_agentOne chat turn (root span)gen_ai.agent.name, gen_ai.conversation.id
chatA model requestgen_ai.request.model, gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, cache read/creation tokens
execute_toolA tool callgen_ai.tool.name, gen_ai.tool.type, gen_ai.tool.call.id
execute_hookA hook runhook metadata

Opik maps gen_ai.input.messages and gen_ai.output.messages to the trace and span input and output, and uses gen_ai.usage.* for token counts. gen_ai.conversation.id groups turns into a thread in the Opik Threads tab.

Content is off by default. Copilot does not export prompt text, responses or tool arguments unless you turn on content capture (github.copilot.chat.otel.captureContent or COPILOT_OTEL_CAPTURE_CONTENT). Without it your Opik traces show structure, timing and token counts, but empty input and output. Check your organization’s policy before enabling it.

Opik OTLP endpoint modes

The examples on this page use the Opik Cloud endpoint. Swap in the endpoint for your deployment from the tabs here; everything else stays the same. Copilot appends the signal path (/v1/traces, /v1/metrics, /v1/logs) to the base endpoint itself, so always give it the base URL, not the /v1/traces URL. For endpoint behavior in general, see the Opik OpenTelemetry overview.

$export OTEL_EXPORTER_OTLP_ENDPOINT=https://www.comet.com/opik/api/v1/private/otel
$export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=<your-workspace>,projectName=<your-project-name>'

Required headers:

  • Authorization
  • Comet-Workspace

Optional headers:

  • projectName (recommended; defaults to Default Project)

Configure Copilot Chat in VS Code

Copilot Chat reads its OTel configuration from VS Code settings, from environment variables, or from enterprise managed settings. Pick the one that matches how you roll out configuration.

The exporter settings live in settings.json, but exporter headers cannot be set in settings.json. Opik Cloud and Enterprise deployments need the Authorization and Comet-Workspace headers, so those still come from one environment variable.

  1. Add to your user or workspace settings.json:
settings.json
1{
2 "github.copilot.chat.otel.enabled": true,
3 "github.copilot.chat.otel.exporterType": "otlp-http",
4 "github.copilot.chat.otel.otlpEndpoint": "https://www.comet.com/opik/api/v1/private/otel",
5 "github.copilot.chat.otel.captureContent": true
6}
  1. Export the headers in your shell profile:
$export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=<your-workspace>,projectName=<your-project-name>'
  1. Launch VS Code from that shell with code ., then reload the window.

The same macOS note applies: a Dock or Spotlight launch does not inherit the variable.

Configure Copilot CLI

Copilot CLI uses the same exporter and the same environment variables as the Quick start. It exports OTLP over HTTP only (http/protobuf or http/json); gRPC is not supported.

$export COPILOT_OTEL_ENABLED=true
$export COPILOT_OTEL_EXPORTER_TYPE=otlp-http
$export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
$export OTEL_EXPORTER_OTLP_ENDPOINT=https://www.comet.com/opik/api/v1/private/otel
$export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=<your-workspace>,projectName=<your-project-name>'
$export COPILOT_OTEL_CAPTURE_CONTENT=true
$export OTEL_SERVICE_NAME=copilot-cli

Run copilot from that shell and traces are exported as you work.

Validation

  1. Reload VS Code (or start a new Copilot CLI session) after applying the configuration.
  2. Ask Copilot something that triggers a tool call, for example “list the files in this repo”.
  3. Open the target project in Opik. You should see an invoke_agent trace with nested chat and execute_tool spans, token counts on the chat spans, and, with content capture enabled, the prompt and response in the trace input and output.
  4. Open the Threads tab to see the conversation grouped by gen_ai.conversation.id.

If nothing arrives, set COPILOT_OTEL_LOG_LEVEL=debug and check the GitHub Copilot Chat output channel in VS Code for exporter errors. A 401 means the Authorization or Comet-Workspace header did not reach the exporter; the most common cause is VS Code not inheriting the environment variable.

Notes

  • Opik ingests the traces signal only. Copilot also sends metrics to /v1/metrics and events to /v1/logs on the same base endpoint; Opik has no receiver for those and rejects them, which is harmless and does not affect trace ingestion.
  • Keep content capture off unless your policy explicitly allows prompt and response export. Metadata, timing and token counts are still logged without it.
  • Cost on chat spans is calculated from gen_ai.usage.* when the gen_ai.request.model or gen_ai.response.model value matches a model in Opik’s price table (for example gpt-4.1). If a span shows no cost, check its model field and reach out so we can add the alias.

Source references