TruEra is happy to announce that it is partnering with New Relic, a leading provider of software stack observability, to provide MLOps leaders with the ability to easily monitor both model infrastructure and machine learning model quality and performance in one place. The partnership includes a new TruEra native data exporter that allows New Relic users to use TruEra as a diagnostics engine, with visibility and access to all TruEra Monitoring metrics. Users benefit from a “single pane of glass” across all MLOps sources.
New Relic is dedicated to making it easier for software engineers to instrument everything and better understand their digital systems. TruEra is dedicated to driving high quality machine learning models, both in development and in production. Together, we can provide MLOps leaders and data science leaders and stakeholders with comprehensive visibility into infrastructure health and real-world ML performance.
MLOps is managing more than just ops – it’s also monitoring AI Quality in real time
MLOps management is increasingly challenging, with constant demands not only regarding infrastructure status, but also critical questions about the integrity and results of the machine learning-based applications themselves. Managers are contending not only with issues like application responsiveness and uptime, but also critical issues such as – is the ML model still providing accurate results? Is there model drift, and how severe is it? Is the model still fair? This combination of TruEra and New Relic is exciting because, in a single place, MLOps leaders will be able to answer those critical operational questions along with the critical AI Quality questions, quickly and easily.
A single pane view into ML Ops: New Relic and TruEra
“New Relic is focused on making modern software great, with a connected, real-time view of all your operational data in one place. By partnering with TruEra, a leader in AI Quality management solutions, you can respond faster and better to optimize both your operational and model quality performance results,” said Guy Fighel, GM of applied intelligence and VP of product engineering at New Relic. “Now MLOps leaders can answer questions like – is my ML application running the way that we want it to, AND is it delivering the right results in the right way? TruEra is a great partner because they answer not only the model performance questions, but also point to how those issues can be solved, so that your models are running at best performance for the longest possible amount of time.”
New Relic: operational observability
Engineering teams rely on New Relic to visualize, analyze and troubleshoot their software. New Relic helps you to:
- Visualize, analyze, and optimize your entire software stack from one place
- Monitor your distributed services, applications, and serverless functions, no matter how or where they’re developed
- Understand what’s happening in your infrastructure, cloud resources, containers, and clusters
TruEra: real-time AI Quality performance management
With the integration of TruEra Monitoring into New Relic One, MLOps leaders can also get a real-time view into key machine learning quality analytics, such as:
- Stability – model score drift, distribution and stability score
- Accuracy – actual and estimated accuracy, decision trends
- Reliability – robustness analytics
- Bias/Fairness – disparate impact on groups
- Data Quality – data quality trends, rule violations
TruEra Monitoring lets you set customizable alerts on actionable metrics, with the ability to ensure that only relevant changes in metrics warrant alerting, to minimize alert fatigue. Further, when model performance goes sideways, users can use TruEra Diagnostics to get to the root cause of the issue with surgical precision.
Get to the root cause, so you can be back in production, fast
TruEra is based on highly accurate explainability technology. While most are familiar with explainability as a way of solving the “black box problem” by describing the drivers of model function, explainability is also critical to troubleshooting and debugging machine learning models. With TruEra, any highlighted trouble spots surfaced in monitoring can be quickly resolved, since the solution pinpoints the source of drift, for example, so that data science teams can resolve issues quickly and get your model back into production fast.
How to get started with TruEra on New Relic
The key steps are:
- Install the truera2newrelic interface.
- Generate the API key in the TruEra console.
- Generate the API key in the New Relic console.
- Verify the first data export run.
- Set up the data exporter.
For additional information, you can also see our step-by-step technical guide for a full walkthrough of the process.
Where to find TruEra
You can find TruEra in the New Relic I/O Marketplace.
Watch a video on New Relic and TruEra on YouTube.
If you’re completely new to TruEra and AI Quality management, learn more at the TruEra website: leader in AI Quality management.
