Developing a platform for production ML models retraining
What is ML cube Platform?
(ML)³ is a B2B innovative Machine Learning product for Life-cycle Management real-time Optimization, with the goal of preventing models’ obsolescence. It is offered as a SaaS and it provides monitoring functionalities, performance analysis, and recommended maintenance (re-training, adaptation, decommissioning) for production systems adopted by different clients independently from the application domain.
Why ML cube Platform?
The capability of Machine Learning models of acquiring knowledge from available data is one of the biggest opportunities for the technological and scientific advancement of our age. However, the non-stationary nature of many real-world processes poses the issue of guaranteeing the robustness of ML models over time, or more generally, by varying the conditions in which training data have been collected.
From the economic point of view, besides the risk of reducing the quality and effectiveness of the service provided, it is necessary to quantify the cost and the time to acquire and process new data, retrain the model, to possibly make manual adjustments and, in worst scenarios, to request the intervention of the original developer of the model. In conclusion, with the pervasive diffusion of ML systems, it will be necessary to adopt tools able to automatically regulate and control the costs of the entire life-cycle management during their application.
ML cube provides several functionalities aimed at maintaining high performance standards while minimizing management costs:
Online Performance Monitoring
ML cube periodically generates automatic reports about observed model’s performance. In case of anomalous deviations, it generates alerts also in asynchronous mode with respect to the reporting activity.
Automatic Retrain Triggering
In addition, ML cube learns a meta-model of the observed model’s performance level. Consequently, the Platform, via Reinforcement and Transfer Learning techniques, recommends the optimal time for retraining and, if possible, it automatically activates related procedures.
Cost Model Management
Thanks to additional information provided by the client, ML cube acquires complete knowledge about the cost model including risk estimation due to the loss of the service quality and time and costs estimation about the retraining activities. In this way, the optimization of the lifecycle management will not be based only on technical/scientific KPIs, but also on business metrics.
ML cube Platform Dashboard at a glance
The ML cube dashboard allows clients to visualize data metrics and measure real-time performance. It containes predictive forecasts that utilize advanced AI-algorithms and displays automatic retrain trigger alarms.
With this dashboard clients can track ML model behaviors, address crucial bottlenecks for their business, make better and faster desisions based on real-time insights and accurate future predictions.
ML cube Platform Life Cycle Optimization
ML cube exploits its meta-models and the acquired knowledge about the observed system to provide specific recommendations related to maintenance activities of the model. In the typical case, ML cube suggests the optimal time to retrain the model integrating the new available data.
How does it work?
ML cube offers an entire functional layer able to manage the lifecycle of ML models operating on a generic informatic system. Through its APIs, ML cube performs the following steps iteratively:
It acquires real-time predictions of the observed model
It compares them with real-data once they are available
It detects potential patterns of errors indicating a sharp deviation of the model from its expected performance