{"id":8066,"date":"2023-11-02T09:17:34","date_gmt":"2023-11-02T17:17:34","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=8066"},"modified":"2025-04-24T17:04:58","modified_gmt":"2025-04-24T17:04:58","slug":"training-gradient-boosting-models-with-comet-ml","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\/","title":{"rendered":"Training Gradient Boosting Models With Comet ML"},"content":{"rendered":"\n<div class=\"fk fl fm fn fo\">\n<div class=\"ab ca\">\n<div class=\"ch bg ew ex ey ez\">\n<figure class=\"ml mm mn mo mp mq mi mj paragraph-image\">\n<div class=\"mr ms ee mt bg mu\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mv mw c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k\" alt=\"\" width=\"700\" height=\"1050\"><\/figure><div class=\"mi mj mk\"><picture><\/picture><\/div>\n<\/div><figcaption class=\"mx my mz mi mj na nb be b bf z dw\" data-selectable-paragraph=\"\">Photo by <a class=\"af nc\" href=\"https:\/\/unsplash.com\/@diane_soko?utm_source=medium&amp;utm_medium=referral\" target=\"_blank\" rel=\"noopener ugc nofollow\">Diane Picchiottino<\/a> on <a class=\"af nc\" href=\"https:\/\/unsplash.com\/?utm_source=medium&amp;utm_medium=referral\" target=\"_blank\" rel=\"noopener ugc nofollow\">Unsplash<\/a><\/figcaption><\/figure>\n<h1 id=\"8786\" class=\"nd ne fr be nf ng nh gr ni nj nk gu nl nm nn no np nq nr ns nt nu nv nw nx ny bj\" data-selectable-paragraph=\"\">Introduction<\/h1>\n<p id=\"98ea\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">Hyperparameters are among the most important aspects of any given model in Data Science and Machine Learning applications. The right combination of hyperparameters is essential when one desires to come up with a great model.<\/p>\n<p id=\"7a97\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">A pertinent problem that plagues engineers and coders is the fact that it is difficult to get a perfect set of hyperparameters that will give the most accurate results. This problem has a fix though. If you have a set of hyperparameters with a range of values, then you could use an algorithm that attempts multiple combinations of hyperparameters and rates the performance using evaluation metrics.<\/p>\n<p id=\"eb14\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">For this reason, multiple popular platforms and algorithms have arisen depending on the level of complexity of a problem. The variety allows someone to leverage the type of hyperparameter tuning algorithm that would computationally fit the problem at hand.<\/p>\n<h2 id=\"2576\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Optuna vs Comet<\/h2>\n<p id=\"e4e9\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">Optuna resonates strongly with the data science and machine learning community and it is often used to look for the best hyperparameters for gradient boosting models like xgboost.<\/p>\n<p id=\"c0fa\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">Someone with experience using Optuna might find it a little difficult to use <a class=\"af nc\" href=\"https:\/\/www.comet.com\/site\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet<\/a> for the same objective if they have never tried using it before. Comet provides a user interface that allows someone to easily track many experiments performed for a given training session, but also provides a built-in hyperparameter optimizer.<\/p>\n<p id=\"a2ab\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">I will now get to the technical details of hyperparameter tuning and experiment tracking on comet ml by making parallels with Optuna for simple migration.<\/p>\n<h2 id=\"ed59\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Requirements<\/h2>\n<p id=\"ad5e\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">There are a few requirements that you\u2019d need before you begin:<\/p>\n<ol class=\"\">\n<li id=\"e222\" class=\"nz oa fr be b gp ou oc od gs ov of og oh pq oj ok ol pr on oo op ps or os ot pt pu pv bj\" data-selectable-paragraph=\"\">A <a class=\"af nc\" href=\"https:\/\/www.comet.com\/site\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet ML account<\/a><\/li>\n<li id=\"22cc\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">An installation of Jupyter Notebook or a Google Colab account<\/li>\n<li id=\"0b78\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Python 3.9.X +<\/li>\n<li id=\"a2bd\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">XGBoost, Numpy, Pandas, Comet_ml and Scikit-learn libraries installed<\/li>\n<li id=\"dc7b\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Motivation to learn<\/li>\n<\/ol>\n<h2 id=\"0441\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Workflow Overview<\/h2>\n<p id=\"4ed3\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">A typical Optuna workflow involves a few standard steps. You would have to first implement a normal ML workflow to train our gradient boosting model, e.g. XGBoost:<\/p>\n<ol class=\"\">\n<li id=\"0eca\" class=\"nz oa fr be b gp ou oc od gs ov of og oh pq oj ok ol pr on oo op ps or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Import necessary libraries.<\/li>\n<li id=\"f5e8\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Define training and testing datasets.<\/li>\n<li id=\"7cff\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Define a function to be optimized that will be used to train on different hyperparameters.<\/li>\n<\/ol>\n<p id=\"b24d\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">For someone familiar with Optuna, only the third step will be different and that\u2019s where we will begin:<\/p>\n<pre class=\"ml mm mn mo mp qb qc qd bo qe ba bj\"><span id=\"01ca\" class=\"qf ne fr qc b bf qg qh l qi qj\" data-selectable-paragraph=\"\"><span class=\"hljs-keyword\">import<\/span> optuna\n<span class=\"hljs-keyword\">from<\/span> xgboost <span class=\"hljs-keyword\">import<\/span> XGBClassifier\n<span class=\"hljs-keyword\">from<\/span> optuna.integration <span class=\"hljs-keyword\">import<\/span> XGBoostPruningCallback\n\n<span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title.function\">objective<\/span>(<span class=\"hljs-params\">trial<\/span>):\n   <span class=\"hljs-comment\">#Hyperparameters with ranges from minimum value to maximum value<\/span>\n   parameter_grid = {\n\n        <span class=\"hljs-string\">\"n_estimators\"<\/span>: trial.suggest_categorical(<span class=\"hljs-string\">\"n_estimators\"<\/span>, [<span class=\"hljs-number\">10000<\/span>]),\n        <span class=\"hljs-string\">\"learning_rate\"<\/span>: trial.suggest_float(<span class=\"hljs-string\">\"learning_rate\"<\/span>, <span class=\"hljs-number\">0.01<\/span>, <span class=\"hljs-number\">0.3<\/span>),\n        <span class=\"hljs-string\">\"max_leaves\"<\/span>: trial.suggest_int(<span class=\"hljs-string\">\"max_leaves\"<\/span>, <span class=\"hljs-number\">20<\/span>, <span class=\"hljs-number\">3000<\/span>, step=<span class=\"hljs-number\">20<\/span>),\n        <span class=\"hljs-string\">\"max_depth\"<\/span>: trial.suggest_int(<span class=\"hljs-string\">\"max_depth\"<\/span>, <span class=\"hljs-number\">3<\/span>, <span class=\"hljs-number\">12<\/span>),\n        <span class=\"hljs-string\">\"max_bin\"<\/span>: trial.suggest_int(<span class=\"hljs-string\">\"max_bin\"<\/span>, <span class=\"hljs-number\">200<\/span>, <span class=\"hljs-number\">300<\/span>),\n        <span class=\"hljs-string\">\"lambda\"<\/span>: trial.suggest_int(<span class=\"hljs-string\">\"lambda\"<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">100<\/span>, step=<span class=\"hljs-number\">5<\/span>),\n        <span class=\"hljs-string\">\"alpha\"<\/span>: trial.suggest_int(<span class=\"hljs-string\">\"alpha\"<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">100<\/span>, step=<span class=\"hljs-number\">5<\/span>),\n        <span class=\"hljs-string\">\"gamma\"<\/span>: trial.suggest_float(<span class=\"hljs-string\">\"gamma\"<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">15<\/span>),\n    }\n<span class=\"hljs-comment\">#feeding the hyperparameters into the model<\/span>\nmodel = XGBClassifier(objective=<span class=\"hljs-string\">\"binary:logistic\"<\/span>, **parameter_grid)\n\n<span class=\"hljs-comment\">#Fitting<\/span>\nmodel.fit(X_train,\n          y_train,\n          eval_set=[X_test, y_test],\n          early_stopping_rounds=<span class=\"hljs-number\">100<\/span>,\n          callbacks = XGBoostPruningCallback(trial, <span class=\"hljs-string\">\"validation_0-logloss\"<\/span>))\n<span class=\"hljs-comment\">#Predict<\/span>\ny_preds = model.predict(X_test)\n\n<span class=\"hljs-comment\">#Creating study<\/span>\nstudy = optuna.create_study(direction=<span class=\"hljs-string\">\"minimize\"<\/span>, study_name=<span class=\"hljs-string\">\"Xgboost\"<\/span>)\n\n<span class=\"hljs-comment\">#Optimizing the study<\/span>\nstudy.optimize(objective, n_trials = <span class=\"hljs-number\">20<\/span>)\n\n<span class=\"hljs-comment\">#Printing the best hyperparameters<\/span>\n<span class=\"hljs-keyword\">for<\/span> key, value <span class=\"hljs-keyword\">in<\/span> study.best_params.items():\n    <span class=\"hljs-built_in\">print<\/span>(<span class=\"hljs-string\">f\"\\t\\t<span class=\"hljs-subst\">{key}<\/span>: <span class=\"hljs-subst\">{value}<\/span>\"<\/span>)<\/span><\/pre>\n<p id=\"0b16\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The above code snippet assumes that you already have your train and test datasets. Additionally, it gives a generalized view of the process that one undergoes when training with Optuna.<\/p>\n<p id=\"084a\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">Now that we have seen a basic Optuna workflow, we can implement this in Comet ML. We will also see how we can take these results and use them in our gradient-boosting algorithm.<\/p>\n<h2 id=\"d5bb\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Comet ML Implementation<\/h2>\n<p id=\"f06b\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">The Comet ML Python library and Comet account are necessary to interact with the Comet API.<\/p>\n<p id=\"91c7\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The first cell will include Comet ML\u2019s library call and an init method that would initialize the project:<\/p>\n<pre class=\"ml mm mn mo mp qb qc qd bo qe ba bj\"><span id=\"4132\" class=\"qf ne fr qc b bf qg qh l qi qj\" data-selectable-paragraph=\"\"><span class=\"hljs-keyword\">import<\/span> comet_ml\n<span class=\"hljs-comment\">#Similar to the initialization of a study in Optuna<\/span>\ncomet_ml.login(project_name=<span class=\"hljs-string\">\"xgboost\"<\/span>)<\/span><\/pre>\n<p id=\"5e3d\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">Running this will then ask you for an API key that you will copy from the user settings as highlighted in the <a class=\"af nc\" href=\"https:\/\/www.comet.com\/docs\/v2\/guides\/tracking-ml-training\/configuring-comet\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">docs<\/a>.<\/p>\n<p id=\"86d8\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">After ensuring that your API key has been input and it accepts it as valid, the next step would be to make a similar workflow to your Optuna:<\/p>\n<ol class=\"\">\n<li id=\"39ae\" class=\"nz oa fr be b gp ou oc od gs ov of og oh pq oj ok ol pr on oo op ps or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Place the optimization algorithm, the hyperparameters, the metric and objective in a dict and use the dict as an argument in Comet\u2019s optimizer.<\/li>\n<li id=\"3d9f\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Proceed with a normal training workflow and call the hyperparameters as arguments in the classifier.<\/li>\n<li id=\"70f0\" class=\"nz oa fr be b gp pw oc od gs px of og oh py oj ok ol pz on oo op qa or os ot pt pu pv bj\" data-selectable-paragraph=\"\">Perform an experiment run that will graph the outcome in multiple experiments in Comet.<\/li>\n<\/ol>\n<p id=\"c995\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The three steps will be accompanied by code below with comparisons on how it is done in Optuna.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"fk fl fm fn fo\">\n<div class=\"ab ca\">\n<div class=\"ch bg ew ex ey ez\">\n<blockquote class=\"qs\"><p id=\"ba42\" class=\"qt qu fr be qv qw qx qy qz ra rb ot dw\" data-selectable-paragraph=\"\">Struggling to track and reproduce complex experiment parameters? Artifacts are just one of the many tools in the Comet toolbox to help ease model management. <a class=\"af nc\" href=\"https:\/\/www.comet.com\/site\/blog\/debugging-your-machine-learning-models-with-comet-artifacts\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Read our PetCam scenario to learn more<\/a>.<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"fk fl fm fn fo\">\n<div class=\"ab ca\">\n<div class=\"ch bg ew ex ey ez\">\n<h2 id=\"36e3\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Step 1: Dictionary for Optimizer<\/h2>\n<p id=\"d5e7\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">The code will look as follows:<\/p>\n<pre class=\"ml mm mn mo mp qb qc qd bo qe ba bj\"><span id=\"40d1\" class=\"qf ne fr qc b bf qg qh l qi qj\" data-selectable-paragraph=\"\"><span class=\"hljs-keyword\">from<\/span> comet_ml <span class=\"hljs-keyword\">import<\/span> Optimizer\n\n<span class=\"hljs-comment\">#Dict that contains optimization algorithm, hyperparameters,metric and objective<\/span>\nconfig = {\n    <span class=\"hljs-comment\">#algorithm for optimization<\/span>\n    <span class=\"hljs-string\">\"algorithm\"<\/span>: <span class=\"hljs-string\">\"bayes\"<\/span>,\n    <span class=\"hljs-comment\">#parameters<\/span>\n    <span class=\"hljs-string\">\"parameters\"<\/span>:{\n        <span class=\"hljs-string\">\"learning_rate\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"float\"<\/span>,<span class=\"hljs-string\">\"min\"<\/span>:<span class=\"hljs-number\">0<\/span>, <span class=\"hljs-string\">\"max\"<\/span>:<span class=\"hljs-number\">1<\/span>},\n        <span class=\"hljs-string\">\"max_leaves\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"integer\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>: <span class=\"hljs-number\">20<\/span>, <span class=\"hljs-string\">\"max\"<\/span>:<span class=\"hljs-number\">3000<\/span>},\n        <span class=\"hljs-string\">\"max_depth\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"integer\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>: <span class=\"hljs-number\">3<\/span>, <span class=\"hljs-string\">\"max\"<\/span>: <span class=\"hljs-number\">12<\/span>},\n        <span class=\"hljs-string\">\"max_bin\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"integer\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>: <span class=\"hljs-number\">200<\/span>, <span class=\"hljs-string\">\"max\"<\/span>: <span class=\"hljs-number\">300<\/span>},\n        <span class=\"hljs-string\">\"lambda\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"integer\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>: <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-string\">\"max\"<\/span>: <span class=\"hljs-number\">100<\/span>},\n        <span class=\"hljs-string\">\"alpha\"<\/span>: {<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"integer\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>:<span class=\"hljs-number\">0<\/span>, <span class=\"hljs-string\">\"max\"<\/span>:<span class=\"hljs-number\">100<\/span>},\n        <span class=\"hljs-string\">\"gamma\"<\/span>:{<span class=\"hljs-string\">\"type\"<\/span>:<span class=\"hljs-string\">\"float\"<\/span>, <span class=\"hljs-string\">\"min\"<\/span>:<span class=\"hljs-number\">0<\/span>, <span class=\"hljs-string\">\"max\"<\/span>:<span class=\"hljs-number\">15<\/span>}\n    },\n    <span class=\"hljs-comment\">#What shall be optimized<\/span>\n    <span class=\"hljs-string\">\"spec\"<\/span>:{\n        <span class=\"hljs-string\">\"metric\"<\/span>: <span class=\"hljs-string\">\"loss\"<\/span>,\n        <span class=\"hljs-string\">\"objective\"<\/span>: <span class=\"hljs-string\">\"minimize\"<\/span>\n    },\n    <span class=\"hljs-string\">\"trials\"<\/span>: <span class=\"hljs-number\">1<\/span>,\n}\n<span class=\"hljs-comment\">#Will be used to feed information in the workflow<\/span>\nOpt_algo = Optimizer(config) <\/span><\/pre>\n<h2 id=\"6419\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Step 2: Normal Workflow with Comet ML integration<\/h2>\n<p id=\"7041\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">Here we will split the train data and the test data. The most important thing will be the training function that introduces the hyperparameters that we want to perform multiple experiments on.<\/p>\n<p id=\"31e0\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The code will be as follows:<\/p>\n<pre class=\"ml mm mn mo mp qb qc qd bo qe ba bj\"><span id=\"839f\" class=\"qf ne fr qc b bf qg qh l qi qj\" data-selectable-paragraph=\"\"><span class=\"hljs-keyword\">from<\/span> xgboost <span class=\"hljs-keyword\">import<\/span> XGBClassifier\n<span class=\"hljs-keyword\">import<\/span> pandas <span class=\"hljs-keyword\">as<\/span> pd\n<span class=\"hljs-keyword\">import<\/span> numpy <span class=\"hljs-keyword\">as<\/span> np\n<span class=\"hljs-keyword\">from<\/span> sklearn.model_selection <span class=\"hljs-keyword\">import<\/span> train_test_split\n\n<span class=\"hljs-comment\">#training dataset<\/span>\ntrain_df = pd.read_csv(<span class=\"hljs-string\">\"input file path in here\"<\/span>)\n\n<span class=\"hljs-comment\">#Performing splits<\/span>\nX_train,  X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=<span class=\"hljs-number\">0.2<\/span>, random_state = <span class=\"hljs-number\">1<\/span>)\n\n<span class=\"hljs-comment\">#training function<\/span>\n<span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title.function\">train_model<\/span>(<span class=\"hljs-params\">experiment<\/span>):\n    model = XGBClassifier(objective=<span class=\"hljs-string\">\"binary:logistic\"<\/span>,\n                           learning_rate = experiment.get_parameter(<span class=\"hljs-string\">\"learning_rate\"<\/span>),\n                           max_leaves = experiment.get_parameter(<span class=\"hljs-string\">\"max_leaves\"<\/span>),\n                           max_depth = experiment.get_parameter(<span class=\"hljs-string\">\"max_depth\"<\/span>),\n                           max_bin = experiment.get_parameter(<span class=\"hljs-string\">\"max_bin\"<\/span>),\n                           reg_lambda = experiment.get_parameter(<span class=\"hljs-string\">\"lambda\"<\/span>),\n                           alpha = experiment.get_parameter(<span class=\"hljs-string\">\"alpha\"<\/span>),\n                           gamma = experiment.get_parameter(<span class=\"hljs-string\">\"gamma\"<\/span>))\n    model.fit(\n            X_train,\n            y_train,\n            eval_set=[(X_val, y_val)],\n            early_stopping_rounds = <span class=\"hljs-number\">100<\/span>,\n            )\n    y_pred = model.predict(X_val)<\/span><\/pre>\n<p id=\"9c2e\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">In the above code, I would like to highlight the syntax in the XGBClassifier hyperparameters. It is not automatically obvious that the hyperparameters we used in the config dictionary correspond with those in XGBoost. Hence we need to map them with the <code class=\"cw rc rd re qc b\">get_parameter()<\/code> method. This is very similar to how Optuna uses <code class=\"cw rc rd re qc b\">trial.suggest_int()<\/code> without the difficulty of having to suggest types for those who may struggle with them.<\/p>\n<p id=\"1be9\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">Once we finish training then we can use this on a test set to predict its performance.<\/p>\n<h2 id=\"99da\" class=\"oz ne fr be nf pa pb pc ni pd pe pf nl oh pg ph pi ol pj pk pl op pm pn po pp bj\" data-selectable-paragraph=\"\">Step 3: Training Function with Comet ML page interaction<\/h2>\n<p id=\"052b\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">An added benefit of using Comet is the interface that allows one to easily track the experiments they have performed and the types of hyperparameters that would produce a given set of results.<\/p>\n<p id=\"1d9e\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">We will then call the model and put in a for loop and let it train multiple times.<\/p>\n<pre class=\"ml mm mn mo mp qb qc qd bo qe ba bj\"><span id=\"3a19\" class=\"qf ne fr qc b bf qg qh l qi qj\" data-selectable-paragraph=\"\"><span class=\"hljs-keyword\">for<\/span> experiment <span class=\"hljs-keyword\">in<\/span> opt.get_experiments():\n    <span class=\"hljs-comment\">#Training<\/span>\n    train_model(experiment)\n    <span class=\"hljs-comment\">#Ending the experiment<\/span>\n    experiment.end()<\/span><\/pre>\n<p id=\"361d\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">After running the above code, a project will be initialized on <a class=\"af nc\" href=\"https:\/\/www.comet.com\/site\" target=\"_blank\" rel=\"noopener ugc nofollow\">Comet ML<\/a>\u2019s homepage with the name you initialized with and will run multiple experiments in a short span of time. By pressing stop on the cell, one can stop further experiments from running.<\/p>\n<p id=\"3f2b\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">As seen below:<\/p>\n<figure class=\"ml mm mn mo mp mq mi mj paragraph-image\">\n<div class=\"mr ms ee mt bg mu\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mv mw c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png\" alt=\"\" width=\"700\" height=\"306\"><\/figure><div class=\"mi mj rf\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*DsgAHZ2kqX9C_tFpJ_NP_Q.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mx my mz mi mj na nb be b bf z dw\" data-selectable-paragraph=\"\">Screenshot by author<\/figcaption>\n<\/figure>\n<p id=\"957d\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">On the right, our XGBoost project is ready and it contains the 42 experiments I ran. Clicking \u201cView Project\u201d takes you to the next page where the experiments are.<\/p>\n<figure class=\"ml mm mn mo mp mq mi mj paragraph-image\">\n<div class=\"mr ms ee mt bg mu\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mv mw c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*IAx9b7PYZoWjlZNBAv6ozA.png\" alt=\"\" width=\"700\" height=\"312\"><\/figure><div class=\"mi mj rg\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*IAx9b7PYZoWjlZNBAv6ozA.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*IAx9b7PYZoWjlZNBAv6ozA.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*IAx9b7PYZoWjlZNBAv6ozA.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*IAx9b7PYZoWjlZNBAv6ozA.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*IAx9b7PYZoWjlZNBAv6ozA.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*IAx9b7PYZoWjlZNBAv6ozA.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*IAx9b7PYZoWjlZNBAv6ozA.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*IAx9b7PYZoWjlZNBAv6ozA.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mx my mz mi mj na nb be b bf z dw\" data-selectable-paragraph=\"\">Screenshot by author<\/figcaption>\n<\/figure>\n<p id=\"4329\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The above graph shows us the hyperparameters that led to a given loss function. As the objective was to minimize the function, we can interact with the graph and click on the lowest possible value of the loss function and extract the hyperparameters.<\/p>\n<figure class=\"ml mm mn mo mp mq mi mj paragraph-image\">\n<div class=\"mr ms ee mt bg mu\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mv mw c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*1XSQqakXGC0EcEh1CHWblg.png\" alt=\"\" width=\"700\" height=\"350\"><\/figure><div class=\"mi mj rf\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*1XSQqakXGC0EcEh1CHWblg.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*1XSQqakXGC0EcEh1CHWblg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*1XSQqakXGC0EcEh1CHWblg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*1XSQqakXGC0EcEh1CHWblg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*1XSQqakXGC0EcEh1CHWblg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*1XSQqakXGC0EcEh1CHWblg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*1XSQqakXGC0EcEh1CHWblg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*1XSQqakXGC0EcEh1CHWblg.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mx my mz mi mj na nb be b bf z dw\" data-selectable-paragraph=\"\">Screenshot by author<\/figcaption>\n<\/figure>\n<p id=\"0bf2\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">The experiment with the lowest possible value is named \u201creliable_aroma_2012\u201d and when we click it, we can see all the details that we seek.<\/p>\n<figure class=\"ml mm mn mo mp mq mi mj paragraph-image\">\n<div class=\"mr ms ee mt bg mu\" tabindex=\"0\" role=\"button\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"bg mv mw c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/1*sCoOl7jtjVxDRf-3EeVlgg.png\" alt=\"\" width=\"700\" height=\"350\"><\/figure><div class=\"mi mj rf\"><picture><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/format:webp\/1*sCoOl7jtjVxDRf-3EeVlgg.png 1400w\" type=\"image\/webp\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\"><source srcset=\"https:\/\/miro.medium.com\/v2\/resize:fit:640\/1*sCoOl7jtjVxDRf-3EeVlgg.png 640w, https:\/\/miro.medium.com\/v2\/resize:fit:720\/1*sCoOl7jtjVxDRf-3EeVlgg.png 720w, https:\/\/miro.medium.com\/v2\/resize:fit:750\/1*sCoOl7jtjVxDRf-3EeVlgg.png 750w, https:\/\/miro.medium.com\/v2\/resize:fit:786\/1*sCoOl7jtjVxDRf-3EeVlgg.png 786w, https:\/\/miro.medium.com\/v2\/resize:fit:828\/1*sCoOl7jtjVxDRf-3EeVlgg.png 828w, https:\/\/miro.medium.com\/v2\/resize:fit:1100\/1*sCoOl7jtjVxDRf-3EeVlgg.png 1100w, https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*sCoOl7jtjVxDRf-3EeVlgg.png 1400w\" sizes=\"(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px\" data-testid=\"og\"><\/picture><\/div>\n<\/div>\n<figcaption class=\"mx my mz mi mj na nb be b bf z dw\" data-selectable-paragraph=\"\">Screenshot by author<\/figcaption>\n<\/figure>\n<p id=\"59b4\" class=\"pw-post-body-paragraph nz oa fr be b gp ou oc od gs ov of og oh ow oj ok ol ox on oo op oy or os ot fk bj\" data-selectable-paragraph=\"\">And on this page, we can collect the hyperparameters that we desire and feed them into our algorithm for the performance we seek.<\/p>\n<h1 id=\"8cd6\" class=\"nd ne fr be nf ng nh gr ni nj nk gu nl nm nn no np nq nr ns nt nu nv nw nx ny bj\" data-selectable-paragraph=\"\">Conclusion<\/h1>\n<p id=\"a790\" class=\"pw-post-body-paragraph nz oa fr be b gp ob oc od gs oe of og oh oi oj ok ol om on oo op oq or os ot fk bj\" data-selectable-paragraph=\"\">The code above was an introduction to all parties that desire to migrate from Optuna to Comet ML for the many potential benefits that it holds. The workflow is similar when it comes to code optimization and the tweaks made will easily transition one used to Optuna to fully using Comet ML.<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Photo by Diane Picchiottino on Unsplash Introduction Hyperparameters are among the most important aspects of any given model in Data Science and Machine Learning applications. The right combination of hyperparameters is essential when one desires to come up with a great model. A pertinent problem that plagues engineers and coders is the fact that it [&hellip;]<\/p>\n","protected":false},"author":79,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[9,7],"tags":[],"coauthors":[176],"class_list":["post-8066","post","type-post","status-publish","format-standard","hentry","category-product","category-tutorials"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Training Gradient Boosting Models With Comet ML - Comet<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Training Gradient Boosting Models With Comet ML\" \/>\n<meta property=\"og:description\" content=\"Photo by Diane Picchiottino on Unsplash Introduction Hyperparameters are among the most important aspects of any given model in Data Science and Machine Learning applications. The right combination of hyperparameters is essential when one desires to come up with a great model. A pertinent problem that plagues engineers and coders is the fact that it [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2023-11-02T17:17:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:04:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k\" \/>\n<meta name=\"author\" content=\"Mwanikii Njagi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mwanikii Njagi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Training Gradient Boosting Models With Comet ML - Comet","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml","og_locale":"en_US","og_type":"article","og_title":"Training Gradient Boosting Models With Comet ML","og_description":"Photo by Diane Picchiottino on Unsplash Introduction Hyperparameters are among the most important aspects of any given model in Data Science and Machine Learning applications. The right combination of hyperparameters is essential when one desires to come up with a great model. A pertinent problem that plagues engineers and coders is the fact that it [&hellip;]","og_url":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2023-11-02T17:17:34+00:00","article_modified_time":"2025-04-24T17:04:58+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k","type":"","width":"","height":""}],"author":"Mwanikii Njagi","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Mwanikii Njagi","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\/"},"author":{"name":"Mwanikii Njagi","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c7043b3e6b992af7b3220aa1f27d2162"},"headline":"Training Gradient Boosting Models With Comet ML","datePublished":"2023-11-02T17:17:34+00:00","dateModified":"2025-04-24T17:04:58+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\/"},"wordCount":1059,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k","articleSection":["Product","Tutorials"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml\/","url":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml","name":"Training Gradient Boosting Models With Comet ML - Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k","datePublished":"2023-11-02T17:17:34+00:00","dateModified":"2025-04-24T17:04:58+00:00","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#primaryimage","url":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k","contentUrl":"https:\/\/miro.medium.com\/v2\/resize:fit:700\/0*jq76J8Fa50HXKw7k"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/training-gradient-boosting-models-with-comet-ml#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Training Gradient Boosting Models With Comet ML"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/c7043b3e6b992af7b3220aa1f27d2162","name":"Mwanikii Njagi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/1a3c516cf04aca9418dfb2213081f4df","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/cropped-1_2jy9gyk0G_yaniWm8gJFVA-1-96x96.webp","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2023\/08\/cropped-1_2jy9gyk0G_yaniWm8gJFVA-1-96x96.webp","caption":"Mwanikii Njagi"},"url":"https:\/\/www.comet.com\/site\/blog\/author\/freddynjagigmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8066","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/79"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=8066"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8066\/revisions"}],"predecessor-version":[{"id":15475,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/8066\/revisions\/15475"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=8066"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=8066"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=8066"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=8066"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}