{"id":8082,"date":"2023-11-02T09:57:11","date_gmt":"2023-11-02T17:57:11","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=8082"},"modified":"2025-04-24T17:04:49","modified_gmt":"2025-04-24T17:04:49","slug":"empowering-comet-with-gitlab","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/empowering-comet-with-gitlab\/","title":{"rendered":"Empowering Comet with GitLab"},"content":{"rendered":"\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>
Photo by Lopez Robin<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n

Comet<\/a> is a meta Machine Learning experimentation platform, providing many features to track, compare, optimize and monitor experiments and models. In practice, Comet permits tracking datasets and code changes. It also provides tools to improve productivity and collaboration, including panels and reports.<\/strong><\/p>\n

Recently, Comet has been integrated with GitLab<\/a>, a very popular DevOps platform that permits the development of secure software in a single application. In practice, you can write the code related to your Machine Learning experiment locally, save it to GitLab and build reports and monitor experiment results in Comet.<\/strong><\/p>\n

Thanks to the combination between GitLab and Comet<\/a>, you can build CI\/CD (Continuous Integration and Continuous Delivery\/Continuous Deployment) applications easily and quickly.<\/p>\n

How can you achieve such a goal?<\/strong> Let\u2019s look at a practical example.<\/p>\n

Imagine that your team wants to build a Machine Learning model that classifies if a banknote is true or false. You write your code and then you want to track your experiment in Comet by monitoring the most popular evaluation metrics for your models, such as precision, recall, and accuracy. Imagine, also, that some members of your team improve your model and want to integrate the changes to your code. Before accepting your colleagues\u2019 code, you want to make sure that her changes improve the model. To solve your problem, you can exploit the power of GitLab.<\/p>\n

In this article, I describe how to set up the described scenario in Comet and how to integrate it in GitLab. For a more detailed experiment, you can refer to Dhruv Nair<\/a>\u2019s article, entitled <\/span>Streamline ML Model Development with GitLab\u2019s DevOps Platform and Comet<\/a>.<\/span><\/p>\n

Set up of an Experiment in Comet<\/h2>\n

Firstly I log in to the Comet Platform<\/a>. Then, I create a New Project, by clicking the top right button.<\/p>\n<\/div>\n<\/div>\n

\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n

Once created, I make a new Experiment by clicking on the top right \u2018Add\u2019 button, as shown in the Figure on the left.<\/p>\n

I copy the generated code. Each experiment is identified by its API key and refers to a specific workspace, which corresponds to your username.<\/p>\n

Now, I can write the code for my classifier. I create a new empty script, named classifier.py<\/code>.<\/p>\n

Firstly, I define two environment variables, which contain my Experiment secrets:<\/p>\n

COMET_API_KEY = os.environ.get(\"COMET_API_KEY\")\nCOMET_WORKSPACE = os.environ.get(\"COMET_WORKSPACE\")<\/span><\/pre>\n
The two variables will be configured later in GitLab, as explained later in this article.<\/p>\n
Then, I paste the code generated by my Comet experiment:<\/p>\n
from comet_ml import Experiment<\/span>experiment = Experiment(\n   api_key=COMET_API_KEY,\n   project_name=\"banknote\",\n   workspace=COMET_WORKSPACE,\n)<\/span><\/pre>\n
Now, I load the banknote dataset<\/a>:<\/p>\n
import pandas as pd<\/span>df = pd.read_csv('source\/data_banknote_authentication.csv')<\/span><\/pre>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n
I extract from it X<\/code> and Y<\/code>:<\/p>\n
cols = ['Variance', 'Skewness', 'Kurtosis', 'Entropy']\nX = df[cols]\nY = df['Class']<\/span><\/pre>\n
Since the input features are not scaled, I transform them through a Min-Max Scaler:<\/p>\n
from sklearn.preprocessing import MinMaxScaler<\/span>scaler = MinMaxScaler()\nX_scaled = scaler.fit_transform(X)<\/span><\/pre>\n
I split the dataset into training and test sets:<\/p>\n
from sklearn.model_selection import train_test_split<\/span>X_train, X_test, y_train, y_test = train_test_split(X_scaled, Y, test_size=0.33, random_state=42)<\/span><\/pre>\n
And I fit a K-Neighbors Classifier:<\/p>\n
from sklearn.neighbors import KNeighborsClassifier<\/span>model = KNeighborsClassifier(n_neighbors=3)\nmodel.fit(X_train,y_train)<\/span><\/pre>\n
Now I evaluate the model:<\/p>\n
from sklearn.metrics import classification_report<\/span>y_pred = model.predict(X_test)<\/span>target_names=['class 0', 'class 1']\nreport = classification_report(y_test, y_pred, target_names=target_names, output_dict=True)<\/span><\/pre>\n
Finally, I save results in my Comet Experiment:<\/p>\n
for label, metric in report.items():\n  try:\n    experiment.log_metrics(metric, prefix=label)\n  except:\n    experiment.log_metric(label, metric)<\/span><\/pre>\n
I also build a confusion matrix:<\/p>\n
experiment.log_confusion_matrix(y_test.tolist(), y_pred.tolist())<\/span><\/pre>\n
I save my script, and I run it:<\/p>\n
python classifier.py<\/span><\/pre>\n
When the script is completed, I can check the results in my Comet workspace. There is a table with all my metrics, as shown in the following figure:<\/p>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
In addition, the confusion matrix has been created:<\/p>\n
\n
\"\"<\/figure>
<\/picture><\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n
The experiment is ready and working. The next step involves its integration with GitLab.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
How does the team at Uber manage to keep their data organized and their team united? Comet\u2019s experiment tracking. Learn more from Uber\u2019s Olcay Cirit.<\/a><\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\n
Set-Up of Gitlab<\/h2>\n
Firstly, I log in to GitLab<\/a>. Then, I create a new project by clicking the top right button. I entitle my project banknote<\/code>.<\/p>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
Configure Environment Variables<\/h2>\n
Then, I must configure the two environment variables exploited by my script. I select my project, then, from the menu on the left, I select Settings \u2192 CI\/CD \u2192 Variables \u2192Expand. I can add two protected variables, named COMET_API_KEY<\/code> and COMET_WORKSPACE<\/code>, as shown in the following figure:<\/p>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
Only protected branches of the repository can access the protected variables. The main branch is considered protected by default. You can manage who can access protected variables in the Protected Branches tab, which can be accessed from Settings \u2192Repository \u2192Protected Branches \u2192Expand.<\/p>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
Add files to the Project<\/h2>\n
Once variables have been set, I clone the repository in my local file system. I copy the repository URL which is available by clicking the clone button and I write in a terminal the following command:<\/p>\n
git clone <my_repository_url><\/span><\/pre>\n
A new folder appears in my file system, called banknote<\/code>. I copy the classifier.py<\/code> file in this folder, as well as the CSV file. I also create a requirements.txt<\/code> file, containing the list of libraries needed to run my experiment:<\/p>\n
comet-ml\npandas\nscikit-learn<\/span><\/pre>\n
Then, I add them to the git repository and I save all the changes to my remote workspace in GitLab:<\/p>\n
cd banknote\ngit add classifier.py\ngit add requirements.txt\ngit add source\/*.csv\ngit commit -m \"Initial Import\"\ngit push origin<\/span><\/pre>\n
Run the Experiment on GitLab<\/h2>\n
Now I\u2019m ready to run my experiment in GitLab. I can exploit the CI\/CD module provided by GitLab: from the menu on the left, I select CI\/CD \u2192Editor \u2192Create a new CI\/CD pipeline. An editor opens, showing the default .gitlab-ci.yml<\/code> file. This script specifies the steps needed to run the experiment. I remove the default code and I paste the following one to my .gitlab-ci.yml<\/code>file:<\/p>\n
image: python:3.8<\/span>build-classification:\n  script:\n     - pip install -r requirements.txt\n     - python classifier.py<\/span><\/pre>\n
I commit changes and automatically the code is executed. When the pipeline is ready, I can click the View pipeline button, located in the top right of the page, and then on the build-classification button<\/p>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
A terminal opens showing the output of the script:<\/p>\n
\n
\n
\"\"<\/figure>
<\/picture><\/div>\n<\/div>\n
Image by Author<\/figcaption>\n<\/figure>\n
The results of the experiment run in GitLab are available in Comet. In addition, every time I run a new pipeline in the CI\/CD environment, a new experiment is created in Comet.<\/p>\n
If a colleague of yours proposes some changes to the code through a pull request, before accepting her changes to the code, you can verify the results in Comet and then decide whether to accept or not.<\/p>\n
Summary<\/h2>\n
Congratulations! You have just learned how to integrate Comet and GitLab to track your experiments! The process is simple and requires the following steps:<\/p>\n