Observability for Spring AI (Java) with Opik

Spring AI is a framework designed to simplify the integration of AI and machine learning capabilities into Spring applications. It provides a familiar Spring-based programming model for working with AI models, vector stores, and AI-powered features, making it easier to build intelligent applications within the Spring ecosystem.

Spring AI’s primary advantage is its seamless integration with the Spring framework, allowing developers to leverage Spring’s dependency injection, configuration management, and testing capabilities while building AI-powered applications.

Getting started

To use the Spring AI integration with Opik, you will need to have Spring AI and the required OpenTelemetry packages installed. The easiest way to start is to use the OPIK SpringAI starter project.

Prerequisites

Before running the demo application, ensure you have the following installed:

• Java 21 or higher
• Maven 3.6+ for dependency management and building
• OpenAI API Key (only for Opik Cloud) - Sign up at OpenAI Platform
• OPIK API Key - Sign up at Comet OPIK

Installation

1. Clone the Repository

$
git clone git@github.com:comet-ml/opik-springai-demo.git
$
cd opik-springai-demo

2. Verify Java Installation

$
java --version

Ensure you have Java 21 or higher installed.

3. Verify Maven Installation

$
mvn --version

4. Install Dependencies

$
mvn clean install

Environment configuration

The application requires the following environment variables to be set:

Required Variables

• OPENAI_API_KEY: Your OpenAI API key
• OTEL_EXPORTER_OTLP_ENDPOINT: OPIK OpenTelemetry endpoint
• OTEL_EXPORTER_OTLP_HEADERS: Authorization headers for OPIK

Configure your environment variables based on your Opik deployment:

$
export OTEL_EXPORTER_OTLP_ENDPOINT=https://www.comet.com/opik/api/v1/private/otel
$
export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=default'

$
export OTEL_EXPORTER_OTLP_HEADERS='Authorization=<your-api-key>,Comet-Workspace=default,projectName=<your-project-name>'

Using Opik with Spring AI

Set up OpenTelemetry instrumentation for Spring AI in your application.yaml:

1
spring:
2
  application:
3
    name: spring-ai-opik-demo
4
  ai:
5
    openai:
6
      api-key: ${OPENAI_API_KEY}
7
      chat:
8
        options:
9
          model: gpt-4o
10
          temperature: 0.7
11
server:
12
  port: 8085
13
14
# Enable OpenTelemetry tracing
15
management:
16
  tracing:
17
    sampling:
18
      probability: 1.0 # Sample all traces
19
  opentelemetry:
20
    tracing:
21
      export:
22
        otlp:
23
          endpoint: ${OTEL_EXPORTER_OTLP_ENDPOINT}
24
          headers: ${OTEL_EXPORTER_OTLP_HEADERS}
25
26
# Disable metrics and logs exporters via OpenTelemetry to avoid putting an extra load on the OpenTelemetry collector
27
otel:
28
  metrics:
29
    exporter: none
30
  logs:
31
    exporter: none

Your Spring AI code will now automatically send traces to Opik:

1
import io.micrometer.tracing.Span;
2
import io.micrometer.tracing.Tracer;
3
import org.springframework.ai.chat.client.ChatClient;
4
import org.springframework.ai.chat.prompt.Prompt;
5
import org.springframework.lang.NonNull;
6
import org.springframework.stereotype.Service;
7
import org.springframework.util.CollectionUtils;
8
9
import java.util.List;
10
import java.util.Map;
11
import java.util.Objects;
12
13
/**
14
 * Service class responsible for handling chat-related operations.
15
 * Provides functionality to interact with an underlying LLM chat client.
16
 */
17
@Service
18
public class ChatService {
19
20
    private static final String TAGS_KEY = "opik.tags";
21
    private static final String METADATA_PREFIX = "opik.metadata.";
22
23
    private final Tracer tracer;
24
    private final ChatClient chatClient;
25
26
    public ChatService(ChatClient.Builder chatClientBuilder, Tracer tracer) {
27
        this.chatClient = chatClientBuilder.build();
28
        this.tracer = tracer;
29
    }
30
31
    public String askQuestion(@NonNull String question) {
32
        return chatClient
33
                .prompt(new Prompt(question))
34
                .call()
35
                .content();
36
    }
37
38
    public String askQuestion(@NonNull String question, List<String> tags, Map<String, String> metadata) {
39
40
        Span span = tracer.currentSpan();
41
        if (Objects.nonNull(span)) {
42
            setTags(span, tags);
43
            setMetadata(span, metadata);
44
        }
45
46
        return chatClient
47
                .prompt(new Prompt(question))
48
                .call()
49
                .content();
50
    }
51
52
    private void setTags(@NonNull Span span, List<String> tags) {
53
        if (!CollectionUtils.isEmpty(tags)) {
54
            span.tagOfStrings(TAGS_KEY, tags);
55
        }
56
    }
57
58
    private void setMetadata(@NonNull Span span, Map<String, String> metadata) {
59
        if ( !CollectionUtils.isEmpty(metadata) ) {
60
            // populate metadata
61
            metadata.forEach((String k, String v) ->
62
                    span.tag(METADATA_PREFIX + k, v));
63
        }
64
    }
65
}

Running the Demo Application

After cloning the OPIK SpringAI starter repository, you can run the demo application using one of the following methods:

Method 1: Using Maven Spring Boot Plugin

$
export OTEL_EXPORTER_OTLP_HEADERS='Comet-Workspace=default,projectName=otel-springai-test' \
>
export OPENAI_API_KEY=sk-proj-your-api-key \
>
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:5173/api/v1/private/otel
$
mvn spring-boot:run

Method 2: Using JAR File

$
mvn clean package
$
$
export OTEL_EXPORTER_OTLP_HEADERS='Comet-Workspace=default,projectName=otel-springai-test' \
>
export OPENAI_API_KEY=sk-proj-your-api-key \
>
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:5173/api/v1/private/otel
$
java -jar target/spring-ai-demo-opik-0.0.1-SNAPSHOT.jar

Method 3: Development Mode with Auto-reload

$
export OTEL_EXPORTER_OTLP_HEADERS='Comet-Workspace=default,projectName=otel-springai-test' \
>
export OPENAI_API_KEY=sk-proj-your-api-key \
>
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:5173/api/v1/private/otel
$
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Dspring.devtools.restart.enabled=true"

The application will start on http://localhost:8085

Testing the Demo Application using REST API

After that you can send a request to the application endpoints to interact with the chatbot:

$
curl --get --data-urlencode "question=How to integrate Spring AI with OpenAI for building chatbots?" http://localhost:8085/api/chat/ask-me

Or POST request to the /api/chat/ask-enhanced endpoint with TAGS and METADATA in the body:

$
curl -X POST \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "question": "What are the benefits of using Spring AI?",
>
    "tags": ["spring", "ai", "tutorial"],
>
    "metadata": {
>
      "userId": "user123",
>
      "sessionId": "session456",
>
      "category": "educational"
>
    }
>
  }' \
>
  http://localhost:8085/api/chat/ask-enhanced

After running the demo application, you can view the traces in Opik by navigating to the Traces tab in the Projects page.

Further improvements

If you have any questions or suggestions for improving the Spring AI integration, please open an issue on our GitHub repository.