Integrate with Gymnasium¶
Gymnasium defines a standard API for defining Reinforcement Learning environments. Comet provides a gymnasium.Wrapper which makes it easy to log the environment performance to the Comet Platform.
Start logging¶
Wrap your gymnasium Enviornment with the CometLogger
.
from comet_ml import Experiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym
experiment = Experiment(
api_key="YOUR_API_KEY",
project_name="YOUR_PROJECT_NAME",
workspace="YOUR_WORKSPACE",
)
env = gym.make('Acrobot-v1')
env = CometLogger(env, experiment)
for x in range(20):
observation, info = env.reset()
truncated = False
terminated = False
while not (truncated or terminated):
observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
env.render()
Log automatically¶
- Metrics:
- episode_reward: cumulative reward for an episode
- episode_length: number of steps in an episode
Visualize your RL training videos¶
The RecordVideo wrapper records videos of the environment. If this wrapper is used before CometLogger
, CometLogger
will log all video files to Comet via env.close()
.
from comet_ml import Experiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym
experiment = Experiment(
api_key="YOUR_API_KEY",
project_name="YOUR_PROJECT_NAME",
workspace="YOUR_WORKSPACE",
)
env = gym.make('Acrobot-v1', render_mode="rgb_array")
env = gym.wrappers.RecordVideo(env, 'test')
env = CometLogger(env, experiment)
for x in range(20):
observation, info = env.reset()
truncated = False
terminated = False
while not (truncated or terminated):
observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
env.render()
env.close() #Uploads video folder 'test' to Comet
Use the Comet Video Viewer Panel, found in the Public Panels Section, to view the mp4 files within the Comet Platform.
Continue your RL training¶
You can leverage ExistingExperiment
to continue a training run. The experiment key of a previous run can be found in the Tabular view of the Experiments Pane of the Comet UI.
from comet_ml import ExistingExperiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym
exp = ExistingExperiment(previous_experiment='YOUR_PREV_EXP_KEY')
env = gym.make('Acrobot-v1')
env = CometLogger(env, exp)
Try it out!¶
Don't just take our word for it, try it out for yourself.
- For more examples using Gymnasium, see our examples GitHub repository.
- Run the end-to-end example above in Colab: