Skip to content

Integrate with Gymnasium

Gymnasium defines a standard API for defining Reinforcement Learning environments. Comet provides a gymnasium.Wrapper which makes it easy to log the environment performance to the Comet Platform.

Open In Colab

Start logging

Wrap your gymnasium Enviornment with the CometLogger.

from comet_ml import Experiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

experiment = Experiment(
    api_key="YOUR_API_KEY",
    project_name="YOUR_PROJECT_NAME",
    workspace="YOUR_WORKSPACE",
)

env = gym.make('Acrobot-v1')
env = CometLogger(env, experiment)

for x in range(20):
    observation, info = env.reset()
    truncated = False
    terminated = False
    while not (truncated or terminated):
        observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
        env.render()

Log automatically

  • Metrics:
    • episode_reward: cumulative reward for an episode
    • episode_length: number of steps in an episode

Visualize your RL training videos

The RecordVideo wrapper records videos of the environment. If this wrapper is used before CometLogger, CometLogger will log all video files to Comet via env.close().

from comet_ml import Experiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

experiment = Experiment(
    api_key="YOUR_API_KEY",
    project_name="YOUR_PROJECT_NAME",
    workspace="YOUR_WORKSPACE",
)

env = gym.make('Acrobot-v1', render_mode="rgb_array")
env = gym.wrappers.RecordVideo(env, 'test')
env = CometLogger(env, experiment)

for x in range(20):
    observation, info = env.reset()
    truncated = False
    terminated = False
    while not (truncated or terminated):
        observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
        env.render()

env.close() #Uploads video folder 'test' to Comet

Use the Comet Video Viewer Panel, found in the Public Panels Section, to view the mp4 files within the Comet Platform. rl_video_panel.png

Continue your RL training

You can leverage ExistingExperiment to continue a training run. The experiment key of a previous run can be found in the Tabular view of the Experiments Pane of the Comet UI.

from comet_ml import ExistingExperiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

exp = ExistingExperiment(previous_experiment='YOUR_PREV_EXP_KEY')
env = gym.make('Acrobot-v1')
env = CometLogger(env, exp)

Try it out!

Don't just take our word for it, try it out for yourself.

Feb. 24, 2024