Pytorch Lightning<\/strong><\/h3>\n\n\n\nLightning<\/a> helps organize PyTorch code and decouple the science code from the engineering code. It\u2019s more of a style-guide than a framework. By organizing PyTorch code under a LightningModule, Lightning makes things like TPU, multi-GPU and 16-bit precision training (40+ other features) trivial.<\/p>\n\n\n\nHere\u2019s an example of PyTorch code organized in the Lightning format:<\/p>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-1.png\")
<\/figure>\n\n\n\n<\/p>\n\n\n\nPytorch to Lightning Conversion<\/figcaption><\/figure>\n\n\n\nComet<\/strong><\/h3>\n\n\n\nComet<\/a> is a powerful meta machine learning experimentation platform allowing users to automatically track their metrics, hyperparameters, dependencies, GPU utilization, datasets, models, debugging samples, and more, enabling much faster research cycles, and more transparent and collaborative data science.<\/p>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-2.png\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n\nComet Project UI<\/figcaption>\n<\/figure>\n\n\n\nComet provides best-in-class feature richness and flexibility\u2014including a built-in hyperparameter optimization service, interactive confusion matrices, full code tracking and reproducibility features, and a wide range of on-premise installation options.<\/p>\n\n\n\n
To learn how to get started with PyTorch Lightning and Comet, follow along below as we go through a simple code example using the two tools. If you\u2019d prefer to be more hands on, continue this demo in this Google Colab notebook<\/a> instead.<\/p>\n\n\n\nInstall and Load Dependencies<\/b><\/h3>\n\n\n\n
Be sure you\u2019ve pip installed the comet-ml and PyTorch-lightning libraries.<\/p>\n\n\n\n
pip install pytorch-lightning\npip install comet-ml<\/code><\/pre>\n\n\n\nNext, in your script or notebook, import Comet\u2019s Experiment<\/a> class and the CometLogger<\/a> and Trainer<\/a> classes from PyTorch Lightning.<\/p>\n\n\n\nfrom comet_ml import Experiment\nfrom pytorch_lightning.loggers import CometLogger\nfrom pytorch_lightning import Trainer<\/code><\/pre>\n\n\n\nCreate CometLogger<\/b><\/h3>\n\n\n\n
Once you\u2019ve imported your dependencies, initialize a CometLogger. You\u2019ll need to pass in your Comet API Key<\/a>, which you can easily access once you\u2019ve signed up for Comet with Github or your email address here<\/a>. Projects and Workspaces are optional but help you organize which experiments belong to which modeling projects you\u2019re working on.<\/p>\n\n\n\ncomet_logger = CometLogger(\n api_key=\"YOUR COMET API KEY\",\n workspace=\"YOUR COMET WORKSPACE\", # Optional\n project_name=\"YOUR COMET PROJECT\" # Optional\n # rest_api_key=os.environ[\"COMET_REST_KEY\"], # Optional\n # experiment_name=\"default\" # Optional\n)<\/code><\/pre>\n\n\n\nSet up PyTorch Lightning Template<\/b><\/h3>\n\n\n\n
It\u2019s time to define a PyTorch Lightning Model, which will define your architecture, forward and step functions, as well as your data loaders.<\/p>\n\n\n\n
class MNISTModel(pl.LightningModule):\n def __init__(self):\n super(MNISTModel, self).__init__()\n # not the best model...\n self.l1 = torch.nn.Linear(28 * 28, 10)\n def forward(self, x):\n return torch.relu(self.l1(x.view(x.size(0), -1)))\n def training_step(self, batch, batch_nb):\n # REQUIRED\n x, y = batch\n y_hat = self.forward(x)\n loss = F.cross_entropy(y_hat, y)\n tensorboard_logs = {'train_loss': loss}\n return {'loss': loss, 'log': tensorboard_logs}\n ...\n def train_dataloader(self):\n return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32) ...<\/code><\/pre>\n\n\n\nOnce you\u2019ve defined your Model class, create an instance for training.<\/p>\n\n\n\n
mnist_model = MNISTModel()<\/code><\/pre>\n\n\n\nCreate a trainer and pass in your CometLogger instance you created earlier.<\/p>\n\n\n\n
trainer = Trainer(logger=CometLogger)<\/code><\/pre>\n\n\n\nThat\u2019s it! You\u2019re ready to start training your model. Simply call .fit() on your trainer, passing in the mnist_model you created earlier.<\/p>\n\n\n\n
trainer.fit(mnist_model)<\/code><\/pre>\n\n\n\nVisualize Logs<\/b><\/h3>\n\n\n\n
Head to the Comet UI in the browser to visualize your training \u2014 metrics, parameters, code, system metrics, and more \u2014 in real time.<\/p>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-3.png\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n\nComet Project UI<\/figcaption>\n<\/figure>\n\n\n\nInspect Dataset Samples<\/b><\/h3>\n\n\n\n
Comet has dedicated modules for almost every type of data you may be modeling with. Using the Comet Graphics module, easily log and inspect dataset samples right in the UI.<\/p>\n\n\n\n
for _ in range(10):\n value = random.randint(0, len(images))\n tmp = images[value]\n img = tmp.numpy()[0]\n comet_logger.experiment.log_image(img, name=\"label :{}\".format(labels[value]))<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-4.png\")
<\/figure>\n\n\n\n\nComet Graphics Module<\/figcaption>\n<\/figure>\n\n\n\nGenerate Interactive Confusion Matrices for Model Debugging<\/b><\/h3>\n\n\n\n
Comet also allows you to create interactive confusion matrices in the UI, allowing you to visualize predictions and debug your model efficiently. (See the Google Colab notebook<\/a> for the code used to generate these confusion matrices.)<\/p>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-5.png\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n\nComet Confusion Matrix Module<\/figcaption>\n<\/figure>\n\n\n\nStore your Trained Model to the Comet Model Registry<\/b><\/h3>\n\n\n\n
Once you\u2019ve trained a model you\u2019d like to save for future use (training in the future, taking into production, etc.), use the Comet Model Registry to do just that. Once you\u2019ve saved your model, simply call experiment.log_model() to store your model in Comet\u2019s Model Registry.<\/p>\n\n\n\n
torch.save(mnist_model.state_dict(), 'my_model')\ncomet_logger.experiment.log_model('model_0', '.\/my_model')<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/image-6.png\")
<\/figure>\n\n\n\n\nComet Model Registry UI<\/figcaption>\n<\/figure>\n\n\n\nFor more information on PyTorch Lightning and the CometLogger, consult the docs here<\/a>. For more information on Comet\u2019s experiment management capabilities, check out Comet\u2019s docs here<\/a>.<\/p>\n\n\n\n
Here\u2019s an example of PyTorch code organized in the Lightning format:<\/p>\n\n\n\n <\/p>\n\n\n\n Comet<\/a> is a powerful meta machine learning experimentation platform allowing users to automatically track their metrics, hyperparameters, dependencies, GPU utilization, datasets, models, debugging samples, and more, enabling much faster research cycles, and more transparent and collaborative data science.<\/p>\n\n\n\n <\/p>\n\n\n\n Comet provides best-in-class feature richness and flexibility\u2014including a built-in hyperparameter optimization service, interactive confusion matrices, full code tracking and reproducibility features, and a wide range of on-premise installation options.<\/p>\n\n\n\n To learn how to get started with PyTorch Lightning and Comet, follow along below as we go through a simple code example using the two tools. If you\u2019d prefer to be more hands on, continue this demo in this Google Colab notebook<\/a> instead.<\/p>\n\n\n\n Be sure you\u2019ve pip installed the comet-ml and PyTorch-lightning libraries.<\/p>\n\n\n\n Next, in your script or notebook, import Comet\u2019s Experiment<\/a> class and the CometLogger<\/a> and Trainer<\/a> classes from PyTorch Lightning.<\/p>\n\n\n\n Once you\u2019ve imported your dependencies, initialize a CometLogger. You\u2019ll need to pass in your Comet API Key<\/a>, which you can easily access once you\u2019ve signed up for Comet with Github or your email address here<\/a>. Projects and Workspaces are optional but help you organize which experiments belong to which modeling projects you\u2019re working on.<\/p>\n\n\n\n It\u2019s time to define a PyTorch Lightning Model, which will define your architecture, forward and step functions, as well as your data loaders.<\/p>\n\n\n\n Once you\u2019ve defined your Model class, create an instance for training.<\/p>\n\n\n\n Create a trainer and pass in your CometLogger instance you created earlier.<\/p>\n\n\n\n That\u2019s it! You\u2019re ready to start training your model. Simply call .fit() on your trainer, passing in the mnist_model you created earlier.<\/p>\n\n\n\n Head to the Comet UI in the browser to visualize your training \u2014 metrics, parameters, code, system metrics, and more \u2014 in real time.<\/p>\n\n\n\n <\/p>\n\n\n\n Comet has dedicated modules for almost every type of data you may be modeling with. Using the Comet Graphics module, easily log and inspect dataset samples right in the UI.<\/p>\n\n\n\n Comet also allows you to create interactive confusion matrices in the UI, allowing you to visualize predictions and debug your model efficiently. (See the Google Colab notebook<\/a> for the code used to generate these confusion matrices.)<\/p>\n\n\n\n <\/p>\n\n\n\n Once you\u2019ve trained a model you\u2019d like to save for future use (training in the future, taking into production, etc.), use the Comet Model Registry to do just that. Once you\u2019ve saved your model, simply call experiment.log_model() to store your model in Comet\u2019s Model Registry.<\/p>\n\n\n\n For more information on PyTorch Lightning and the CometLogger, consult the docs here<\/a>. For more information on Comet\u2019s experiment management capabilities, check out Comet\u2019s docs here<\/a>.<\/p>\n\n\n\n<\/figure>\n\n\n\nComet<\/strong><\/h3>\n\n\n\n
<\/figure>\n\n\n\nInstall and Load Dependencies<\/b><\/h3>\n\n\n\n
pip install pytorch-lightning\npip install comet-ml<\/code><\/pre>\n\n\n\nfrom comet_ml import Experiment\nfrom pytorch_lightning.loggers import CometLogger\nfrom pytorch_lightning import Trainer<\/code><\/pre>\n\n\n\nCreate CometLogger<\/b><\/h3>\n\n\n\n
comet_logger = CometLogger(\n api_key=\"YOUR COMET API KEY\",\n workspace=\"YOUR COMET WORKSPACE\", # Optional\n project_name=\"YOUR COMET PROJECT\" # Optional\n # rest_api_key=os.environ[\"COMET_REST_KEY\"], # Optional\n # experiment_name=\"default\" # Optional\n)<\/code><\/pre>\n\n\n\nSet up PyTorch Lightning Template<\/b><\/h3>\n\n\n\n
class MNISTModel(pl.LightningModule):\n def __init__(self):\n super(MNISTModel, self).__init__()\n # not the best model...\n self.l1 = torch.nn.Linear(28 * 28, 10)\n def forward(self, x):\n return torch.relu(self.l1(x.view(x.size(0), -1)))\n def training_step(self, batch, batch_nb):\n # REQUIRED\n x, y = batch\n y_hat = self.forward(x)\n loss = F.cross_entropy(y_hat, y)\n tensorboard_logs = {'train_loss': loss}\n return {'loss': loss, 'log': tensorboard_logs}\n ...\n def train_dataloader(self):\n return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32) ...<\/code><\/pre>\n\n\n\nmnist_model = MNISTModel()<\/code><\/pre>\n\n\n\ntrainer = Trainer(logger=CometLogger)<\/code><\/pre>\n\n\n\ntrainer.fit(mnist_model)<\/code><\/pre>\n\n\n\nVisualize Logs<\/b><\/h3>\n\n\n\n
<\/figure>\n\n\n\nInspect Dataset Samples<\/b><\/h3>\n\n\n\n
for _ in range(10):\n value = random.randint(0, len(images))\n tmp = images[value]\n img = tmp.numpy()[0]\n comet_logger.experiment.log_image(img, name=\"label :{}\".format(labels[value]))<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nGenerate Interactive Confusion Matrices for Model Debugging<\/b><\/h3>\n\n\n\n
<\/figure>\n\n\n\nStore your Trained Model to the Comet Model Registry<\/b><\/h3>\n\n\n\n
torch.save(mnist_model.state_dict(), 'my_model')\ncomet_logger.experiment.log_model('model_0', '.\/my_model')<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\n