{"id":345,"date":"2020-09-21T15:12:44","date_gmt":"2020-09-21T23:12:44","guid":{"rendered":"https:\/\/live-cometml.pantheonsite.io\/?p=345"},"modified":"2025-04-24T17:30:39","modified_gmt":"2025-04-24T17:30:39","slug":"logging-histograms-gradients-and-activations-with-comet","status":"publish","type":"post","link":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/","title":{"rendered":"Logging Histograms, Gradients and Activations with Comet"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Introduction<\/h2>\n\n\n\n<p>3D Histograms or Ridge Plots are a great way to visualize the training progress of your Neural Network. Histogram distributions of the weights, gradients, and activations allow us to get some intuition for the loss surface that we are trying to optimize.<\/p>\n\n\n\n<p>For example, your model may be approaching a local minima if you observe that your gradients are becoming smaller over time. On the flip side, a trend of large gradients with high variance might imply that you should reduce your learning rate, or use some form of regularization.<\/p>\n\n\n\n<p>The Comet SDK provides an easy way to visualize your weights, activations and gradients using the `log_histogram_3D` method. For this post we will provide examples of histogram logging with Comet using Tensorflow\u2019s Gradient Tape.<\/p>\n\n\n\n<p>You can explore the visualizations in this post&nbsp;<a href=\"https:\/\/www.comet.com\/cometpublic\/histograms\/view\/OtXVeqWm9SNvb7rz1GG3TFEdv\">here<\/a>. We have also included Colab Notebooks at the end of this post so that you can try out the histograms feature for yourself!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Logging Histograms<\/h2>\n\n\n\n<p>For this example, we are going to use a simple 2 layer perceptron and train it on the MNIST dataset. Let\u2019s start by loading in our data, and defining our model<\/p>\n\n\n\n<div>\n<div id=\"highlighter_804805\" class=\"syntaxhighlighter plain\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">1<\/div>\n<div class=\"line number2 index1 alt1\">2<\/div>\n<div class=\"line number3 index2 alt2\">3<\/div>\n<div class=\"line number4 index3 alt1\">4<\/div>\n<div class=\"line number5 index4 alt2\">5<\/div>\n<div class=\"line number6 index5 alt1\">6<\/div>\n<div class=\"line number7 index6 alt2\">7<\/div>\n<div class=\"line number8 index7 alt1\">8<\/div>\n<div class=\"line number9 index8 alt2\">9<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<div class=\"line number13 index12 alt2\">13<\/div>\n<div class=\"line number14 index13 alt1\">14<\/div>\n<div class=\"line number15 index14 alt2\">15<\/div>\n<div class=\"line number16 index15 alt1\">16<\/div>\n<div class=\"line number17 index16 alt2\">17<\/div>\n<div class=\"line number18 index17 alt1\">18<\/div>\n<div class=\"line number19 index18 alt2\">19<\/div>\n<div class=\"line number20 index19 alt1\">20<\/div>\n<div class=\"line number21 index20 alt2\">21<\/div>\n<div class=\"line number22 index21 alt1\">22<\/div>\n<div class=\"line number23 index22 alt2\">23<\/div>\n<div class=\"line number24 index23 alt1\">24<\/div>\n<div class=\"line number25 index24 alt2\">25<\/div>\n<div class=\"line number26 index25 alt1\">26<\/div>\n<div class=\"line number27 index26 alt2\">27<\/div>\n<div class=\"line number28 index27 alt1\">28<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"plain plain\">def get_dataset():<\/code><\/div>\n<div class=\"line number2 index1 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">num_classes = 10<\/code><\/div>\n<div class=\"line number3 index2 alt2\"><\/div>\n<div class=\"line number4 index3 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\"># the data, shuffled and split between train and test sets<\/code><\/div>\n<div class=\"line number5 index4 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">(x_train, y_train), (x_test, y_test) = mnist.load_data()<\/code><\/div>\n<div class=\"line number6 index5 alt1\"><\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_train = x_train.reshape(60000, 784)<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_test = x_test.reshape(10000, 784)<\/code><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_train = x_train.astype(\"float32\")<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_test = x_test.astype(\"float32\")<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_train \/= 255<\/code><\/div>\n<div class=\"line number12 index11 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">x_test \/= 255<\/code><\/div>\n<div class=\"line number13 index12 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">print(x_train.shape[0], \"train samples\")<\/code><\/div>\n<div class=\"line number14 index13 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">print(x_test.shape[0], \"test samples\")<\/code><\/div>\n<div class=\"line number15 index14 alt2\"><\/div>\n<div class=\"line number16 index15 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\"># convert class vectors to binary class matrices<\/code><\/div>\n<div class=\"line number17 index16 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">y_train = to_categorical(y_train, num_classes)<\/code><\/div>\n<div class=\"line number18 index17 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">y_test = to_categorical(y_test, num_classes)<\/code><\/div>\n<div class=\"line number19 index18 alt2\"><\/div>\n<div class=\"line number20 index19 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">return x_train, y_train, x_test, y_test<\/code><\/div>\n<div class=\"line number21 index20 alt2\"><\/div>\n<div class=\"line number22 index21 alt1\"><code class=\"plain plain\">def build_model_graph():<\/code><\/div>\n<div class=\"line number23 index22 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">model = Sequential()<\/code><\/div>\n<div class=\"line number24 index23 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">model.add(Dense(128, activation=\"sigmoid\", input_shape=(784,), name=\"dense1\"))<\/code><\/div>\n<div class=\"line number25 index24 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">model.add(Dense(64, activation=\"sigmoid\", name=\"dense2\"))<\/code><\/div>\n<div class=\"line number26 index25 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">model.add(Dense(10, activation=\"softmax\", name=\"output\"))<\/code><\/div>\n<div class=\"line number27 index26 alt2\"><\/div>\n<div class=\"line number28 index27 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">return model<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n\n\n\n<p>Tensorflow\u2019s GradientTape allows eager execution of model code without precomputing a static graph in which inputs are fed in through placeholders.<\/p>\n\n\n\n<p>This implies that at every training step, we will have to calculate the gradients of our parameters with respect to our loss, and apply them with the optimizer in order to perform backpropagation.<\/p>\n\n\n\n<div>\n<div id=\"highlighter_386846\" class=\"syntaxhighlighter plain\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">1<\/div>\n<div class=\"line number2 index1 alt1\">2<\/div>\n<div class=\"line number3 index2 alt2\">3<\/div>\n<div class=\"line number4 index3 alt1\">4<\/div>\n<div class=\"line number5 index4 alt2\">5<\/div>\n<div class=\"line number6 index5 alt1\">6<\/div>\n<div class=\"line number7 index6 alt2\">7<\/div>\n<div class=\"line number8 index7 alt1\">8<\/div>\n<div class=\"line number9 index8 alt2\">9<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"plain plain\">def step(model, X, y, gradmap={}, activations={}):<\/code><\/div>\n<div class=\"line number2 index1 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">with tf.GradientTape() as tape:<\/code><\/div>\n<div class=\"line number3 index2 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">pred = model(X)<\/code><\/div>\n<div class=\"line number4 index3 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">loss = categorical_crossentropy(y, pred)<\/code><\/div>\n<div class=\"line number5 index4 alt2\"><\/div>\n<div class=\"line number6 index5 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">grads = tape.gradient(loss, model.trainable_variables)<\/code><\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">opt.apply_gradients(zip(grads, model.trainable_variables))<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">gradmap = get_gradients(gradmap, grads, model)<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">activations = get_activations(activations, X, model)<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><\/div>\n<div class=\"line number12 index11 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">return loss.numpy().mean(), gradmap, activations<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n\n\n\n<p>We will use two dictionaries to store layerwise information about the gradients and activations. For each batch of data we will accumulate the values of the gradients and activations. At the end of every epoch we will scale these by the number of batches in an epoch, and log them to Comet.<\/p>\n\n\n\n<div>\n<div id=\"highlighter_604496\" class=\"syntaxhighlighter plain\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">1<\/div>\n<div class=\"line number2 index1 alt1\">2<\/div>\n<div class=\"line number3 index2 alt2\">3<\/div>\n<div class=\"line number4 index3 alt1\">4<\/div>\n<div class=\"line number5 index4 alt2\">5<\/div>\n<div class=\"line number6 index5 alt1\">6<\/div>\n<div class=\"line number7 index6 alt2\">7<\/div>\n<div class=\"line number8 index7 alt1\">8<\/div>\n<div class=\"line number9 index8 alt2\">9<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<div class=\"line number13 index12 alt2\">13<\/div>\n<div class=\"line number14 index13 alt1\">14<\/div>\n<div class=\"line number15 index14 alt2\">15<\/div>\n<div class=\"line number16 index15 alt1\">16<\/div>\n<div class=\"line number17 index16 alt2\">17<\/div>\n<div class=\"line number18 index17 alt1\">18<\/div>\n<div class=\"line number19 index18 alt2\">19<\/div>\n<div class=\"line number20 index19 alt1\">20<\/div>\n<div class=\"line number21 index20 alt2\">21<\/div>\n<div class=\"line number22 index21 alt1\">22<\/div>\n<div class=\"line number23 index22 alt2\">23<\/div>\n<div class=\"line number24 index23 alt1\">24<\/div>\n<div class=\"line number25 index24 alt2\">25<\/div>\n<div class=\"line number26 index25 alt1\">26<\/div>\n<div class=\"line number27 index26 alt2\">27<\/div>\n<div class=\"line number28 index27 alt1\">28<\/div>\n<div class=\"line number29 index28 alt2\">29<\/div>\n<div class=\"line number30 index29 alt1\">30<\/div>\n<div class=\"line number31 index30 alt2\">31<\/div>\n<div class=\"line number32 index31 alt1\">32<\/div>\n<div class=\"line number33 index32 alt2\">33<\/div>\n<div class=\"line number34 index33 alt1\">34<\/div>\n<div class=\"line number35 index34 alt2\">35<\/div>\n<div class=\"line number36 index35 alt1\">36<\/div>\n<div class=\"line number37 index36 alt2\">37<\/div>\n<div class=\"line number38 index37 alt1\">38<\/div>\n<div class=\"line number39 index38 alt2\">39<\/div>\n<div class=\"line number40 index39 alt1\">40<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"plain plain\">def train(model, X, y, epoch, steps_per_epoch, experiment):<\/code><\/div>\n<div class=\"line number2 index1 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">gradmap = {}<\/code><\/div>\n<div class=\"line number3 index2 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">activations = {}<\/code><\/div>\n<div class=\"line number4 index3 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">total_loss = 0<\/code><\/div>\n<div class=\"line number5 index4 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">with experiment.train():<\/code><\/div>\n<div class=\"line number6 index5 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\"># show the current epoch number<\/code><\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">print(\"[INFO] starting epoch {}\/{}...\".format(epoch, EPOCHS), end=\"\")<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">for i in range(0, steps_per_epoch):<\/code><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">start = i * BS<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">end = start + BS<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">curr_step = ((i + 1) * BS) * epoch<\/code><\/div>\n<div class=\"line number12 index11 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><\/div>\n<div class=\"line number13 index12 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">loss, gradmap, activations = step(<\/code><\/div>\n<div class=\"line number14 index13 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">model,<\/code><\/div>\n<div class=\"line number15 index14 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">X[start:end],<\/code><\/div>\n<div class=\"line number16 index15 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">y[start:end],<\/code><\/div>\n<div class=\"line number17 index16 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">gradmap,<\/code><\/div>\n<div class=\"line number18 index17 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">activations,<\/code><\/div>\n<div class=\"line number19 index18 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">)<\/code><\/div>\n<div class=\"line number20 index19 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">experiment.log_metric(<\/code><\/div>\n<div class=\"line number21 index20 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">\"batch_loss\", loss, step=curr_step<\/code><\/div>\n<div class=\"line number22 index21 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <\/code><\/div>\n<div class=\"line number23 index22 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><\/div>\n<div class=\"line number24 index23 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">total_loss += loss<\/code><\/div>\n<div class=\"line number25 index24 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><\/div>\n<div class=\"line number26 index25 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">experiment.log_metric(<\/code><\/div>\n<div class=\"line number27 index26 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">\"loss\", total_loss \/ steps_per_epoch, step=epoch * steps_per_epoch<\/code><\/div>\n<div class=\"line number28 index27 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">)<\/code><\/div>\n<div class=\"line number29 index28 alt2\"><\/div>\n<div class=\"line number30 index29 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\"># scale gradients<\/code><\/div>\n<div class=\"line number31 index30 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">for k, v in gradmap.items():<\/code><\/div>\n<div class=\"line number32 index31 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">gradmap[k] = v \/ steps_per_epoch<\/code><\/div>\n<div class=\"line number33 index32 alt2\"><\/div>\n<div class=\"line number34 index33 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\"># scale activations<\/code><\/div>\n<div class=\"line number35 index34 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">for k, v in activations.items():<\/code><\/div>\n<div class=\"line number36 index35 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">activations[k] = v \/ steps_per_epoch<\/code><\/div>\n<div class=\"line number37 index36 alt2\"><\/div>\n<div class=\"line number38 index37 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">log_weights(experiment, model, epoch * steps_per_epoch)<\/code><\/div>\n<div class=\"line number39 index38 alt2\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">log_histogram(experiment, gradmap, epoch * steps_per_epoch, prefix=\"gradient\")<\/code><\/div>\n<div class=\"line number40 index39 alt1\"><code class=\"plain spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"plain plain\">log_histogram(experiment, activations, epoch * steps_per_epoch, prefix=\"activation\")<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n\n\n\n<p>We can visualize these reported values under the Histograms tab in our Comet Experiment<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"562\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-1.png\" alt=\"\" class=\"wp-image-394\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-1.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-1-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-1-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><figcaption><strong>Figure 1. Logged Histograms in Comet<\/strong>&nbsp;(<a href=\"https:\/\/www.comet.com\/cometpublic\/histograms\/7eff52d72bdc4847b30e362d7cbb2d06?experiment-tab=histograms&amp;groupBy=false&amp;ignoreOutliers=0&amp;maxSteps=5&amp;orderBy=asc&amp;sortBy=name\">link to Comet Experiment<\/a>)<\/figcaption><\/figure>\n\n\n\n<p>We can sort and search the various histograms using the options in the menu. For example, we might want to group our histograms based on the reported names<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"563\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-2.png\" alt=\"\" class=\"wp-image-393\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-2.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-2-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-2-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"528\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3.png\" alt=\"\" class=\"wp-image-392\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3-300x155.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3-768x396.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><strong><strong>Figure 2. Selecting grouping for Histograms (<a href=\"https:\/\/www.comet.com\/cometpublic\/histograms\/7eff52d72bdc4847b30e362d7cbb2d06?experiment-tab=histograms&amp;groupBy=false&amp;ignoreOutliers=0&amp;maxSteps=5&amp;orderBy=asc&amp;sortBy=name\">link to Comet Experiment<\/a>)<\/strong><\/strong><figure><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-392 size-full\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3.png\" alt=\"\" width=\"1024\" height=\"528\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3-300x155.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-3-768x396.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"561\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4.png\" alt=\"\" class=\"wp-image-391\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4-300x164.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4-768x421.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><strong><strong>Figure 3. Histograms Grouped by Name (<a href=\"https:\/\/www.comet.com\/cometpublic\/histograms\/7eff52d72bdc4847b30e362d7cbb2d06?experiment-tab=histograms&amp;groupBy=fileName&amp;orderBy=asc&amp;sortBy=name\">link to Comet Experiment<\/a>)<\/strong><\/strong><figure><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-391 size-full\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4.png\" alt=\"\" width=\"1024\" height=\"561\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4-300x164.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-4-768x421.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\">\n<figcaption><strong><strong>Figure 4. Histograms Grouped by Name Expanded View<\/strong><\/strong>&nbsp;<\/figcaption>\n<\/figure>\n\n\n\n<p>For larger models, we may have many such histograms. Comet makes it possible to search for the specific histogram that you\u2019re interested in. In Figure 5 we can see how we are able to search for the histogram distribution of the gradients of a single layer.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"562\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-5.png\" alt=\"\" class=\"wp-image-390\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-5.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-5-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-5-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\">\n<figcaption><strong>Figure 5. Filtered Histograms based on Name<\/strong><\/figcaption>\n<\/figure>\n\n\n\n<p>Additionally, Comet provides&nbsp;<a href=\"https:\/\/www.comet.com\/site\/introducing-panels-custom-visualizations-for-machine-learning\/\">Custom Panels<\/a>&nbsp;that can visualize these histograms as a heatmap over time. This particular custom panel can be found in Panels Gallery \u2192 Public Panels \u2192 Histogram by Step. After adding the Panel to the project, we should see it above the Experiment Table.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"561\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-6.png\" alt=\"\" class=\"wp-image-389\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-6.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-6-300x164.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-6-768x421.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"562\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7.png\" alt=\"\" class=\"wp-image-388\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><strong><strong>Figure 6. Adding a custom panel to visualize histograms<\/strong><\/strong><figure><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-388 size-full\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7.png\" alt=\"\" width=\"1024\" height=\"562\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-7-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\">\n<figcaption><strong>Figure 7. Custom Panel to Visualize Histograms in the Project<\/strong>&nbsp;<\/figcaption>\n<\/figure>\n\n\n\n<p>This custom panel has drop downs for both the experiment ids and the histogram names. We can select a particular experiment from the dropdown and then visualize the individual histograms in that experiment as a heat map over time.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"450\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-8.png\" alt=\"\" class=\"wp-image-387\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-8.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-8-300x132.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-8-768x338.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"563\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9.png\" alt=\"\" class=\"wp-image-386\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><strong><strong>Figure 8. Selecting an experiment from the dropdown in a Custom Panel<\/strong><\/strong><figure><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-386 size-full\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9.png\" alt=\"\" width=\"1024\" height=\"563\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9-300x165.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Figure-9-768x422.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\">\n<figcaption><strong>Figure 9. Selecting a histogram in the experiment from the dropdown in a Custom Panel<\/strong><\/figcaption>\n<\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>In this post we demonstrated how Comet\u2019s histogram logging and Custom Panels can be used to visualize the weights, gradients, and activations of a Neural Network.<\/p>\n\n\n\n<p><strong>Colab Notebooks :<\/strong><\/p>\n\n\n\n<p><a href=\"https:\/\/colab.research.google.com\/drive\/1_V_yP5CQxhfuxVtvnTpgbHdtB7O4fFD9?usp=sharing\">TF Gradient Tape Example<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/colab.research.google.com\/drive\/1PJ_EoibXEM6Ogjc_YkPbE3v6eMyCznu-?usp=sharing\">Pytorch Example<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction 3D Histograms or Ridge Plots are a great way to visualize the training progress of your Neural Network. Histogram distributions of the weights, gradients, and activations allow us to get some intuition for the loss surface that we are trying to optimize. For example, your model may be approaching a local minima if you [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":346,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"customer_name":"","customer_description":"","customer_industry":"","customer_technologies":"","customer_logo":"","footnotes":""},"categories":[8,9],"tags":[],"coauthors":[128],"class_list":["post-345","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-comet-community-hub","category-product"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Logging Histograms, Gradients and Activations with Comet<\/title>\n<meta name=\"description\" content=\"Guide to logging histograms, gradients and activations with Comet. 3D Histograms or Ridge Plots visualize the training progress of your Neural Network.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Logging Histograms, Gradients and Activations with Comet\" \/>\n<meta property=\"og:description\" content=\"Guide to logging histograms, gradients and activations with Comet. 3D Histograms or Ridge Plots visualize the training progress of your Neural Network.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:published_time\" content=\"2020-09-21T23:12:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-24T17:30:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"562\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Dhruv Nair\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Cometml\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Dhruv Nair\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Logging Histograms, Gradients and Activations with Comet","description":"Guide to logging histograms, gradients and activations with Comet. 3D Histograms or Ridge Plots visualize the training progress of your Neural Network.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/","og_locale":"en_US","og_type":"article","og_title":"Logging Histograms, Gradients and Activations with Comet","og_description":"Guide to logging histograms, gradients and activations with Comet. 3D Histograms or Ridge Plots visualize the training progress of your Neural Network.","og_url":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_published_time":"2020-09-21T23:12:44+00:00","article_modified_time":"2025-04-24T17:30:39+00:00","og_image":[{"width":1024,"height":562,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png","type":"image\/png"}],"author":"Dhruv Nair","twitter_card":"summary_large_image","twitter_creator":"@Cometml","twitter_site":"@Cometml","twitter_misc":{"Written by":"Dhruv Nair","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#article","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/"},"author":{"name":"Matt Peternell","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/85aa446f8be987e848ea929ef524b67b"},"headline":"Logging Histograms, Gradients and Activations with Comet","datePublished":"2020-09-21T23:12:44+00:00","dateModified":"2025-04-24T17:30:39+00:00","mainEntityOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/"},"wordCount":603,"commentCount":0,"publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png","articleSection":["Comet Community Hub","Product"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/","url":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/","name":"Logging Histograms, Gradients and Activations with Comet","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png","datePublished":"2020-09-21T23:12:44+00:00","dateModified":"2025-04-24T17:30:39+00:00","description":"Guide to logging histograms, gradients and activations with Comet. 3D Histograms or Ridge Plots visualize the training progress of your Neural Network.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2022\/02\/Screen-Shot-2020-07-29-at-6.55.34-PM-1024x562-1.png","width":1024,"height":562,"caption":"Keras Gradient Image | Comet ML"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/blog\/logging-histograms-gradients-and-activations-with-comet\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Logging Histograms, Gradients and Activations with Comet"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]},{"@type":"Person","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/85aa446f8be987e848ea929ef524b67b","name":"Matt Peternell","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/person\/image\/da003ee51bbeeccfb95147ec69139879","url":"https:\/\/secure.gravatar.com\/avatar\/36058153d701caaf237a96d5d6fb9c2d1678325c3ed0d8e88bf5e487019a2a53?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/36058153d701caaf237a96d5d6fb9c2d1678325c3ed0d8e88bf5e487019a2a53?s=96&d=mm&r=g","caption":"Matt Peternell"},"description":"We re-implemented the architecture of this model to incorporate patient and study information. By comparing our updated model to the original Github repository, we were able to quantify the benefits of classifying by patient as opposed to classifying by individual X-ray. We observed a 0.0254 increase in AUROC when evaluating the DenseNet121 on patients instead of on individual scans.","sameAs":["http:\/\/atre.net"],"url":"https:\/\/www.comet.com\/site\/blog\/author\/mpeternellatre-net\/"}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=345"}],"version-history":[{"count":1,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/345\/revisions"}],"predecessor-version":[{"id":15693,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/posts\/345\/revisions\/15693"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/346"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/categories?post=345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/tags?post=345"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}