Skip to content

Integrate with New Relic¶

Comet integrates with New Relic.

New Relic is a full stack monitoring platform that allows you to visualize, analyze, and optimize your entire software stack from one place. With over 300 agents and integrations, you can ingest and store all your operational data, including logs, in one place.

Comet partnered with New Relic to extend full stack observability to include your machine learning models in production and their performance baselines established in training, so you can build better models, faster.

Data available in New Relic¶

Once the integration has been configured by your Comet customer success manager, you will get access in New Relic to the following metrics for each model:

MetricDescription
comet.hourly.nb_predictionsNumber of predictions made by the model
comet.hourly.input.data_driftData drift broken down by input feature
comet.hourly.prediction.valueDistribution of the model’s prediction
comet.hourly.prediction.probabilityDistribution of the model’s prediction probability. Available percentiles appear in the following rows.
comet.hourly.prediction.probability.minMinimum
comet.hourly.prediction.probability.p1010th percentile
comet.hourly.prediction.probability.p2525th percentile
comet.hourly.prediction.probability.p5050th percentile or median
comet.hourly.prediction.probability.p7575th percentile
comet.hourly.prediction.probability.p9090th percentile
comet.hourly.prediction.probability.maxMaximum

Use Comet metrics in New Relic¶

Comet metrics can be used to create new dashboards or augment existing ones. To get started, you can use the default Comet dashboard to start tracking the performance of your machine learning models in New Relic.

In addition, you can use Comet metrics with New Relic infrastructure metrics to correlate the performance of your machine learning models with the performance of their underlying infrastructure.

Debug machine learning models with New Relic and Comet¶

Comet metrics can be used as part of New Relic Applied Intelligence to find, troubleshoot and resolve infrastructure issues that could be impacting the performance of your machine learning models.

If the issue with the performance of your model is not related to underlying infrastructure, you can use the Comet MPM dashboard to identify performance issues on subsets of your data using MPM Segments. You can also use MPM Log Search to query individual predictions and identify outliers.

Get started¶

Already a Comet customer monitoring models in production? Setting up the integration with New Relic One can be completed quickly. Simply reach out to your Comet customer success manager to complete the onboarding.

Not a Comet customer yet? Contact Comet today to set up an account and get started.

Dec. 2, 2024