Why you Might Want to use Machine Learning

According to Statista Digital Economy Compass 2019, two major trends will disrupt the economy and our lives:

For the sake of consistency, we will use the term machine learning (ML), however, the concepts apply to both artificial intelligence and data science fields.

Every machine learning pipeline is a set of operations, which are executed to produce a model. An ML model is roughly defined as a mathematical representation of a real-world process. We might think of the ML model as a function that takes some input data and produces an output (classification, sentiment, recommendation, or clusters). The performance of each model is evaluated by using evaluation metrics, such as precision & recall, or accuracy.

Being a powerful tool, machine learning can solve many practical problems. Similar to any other software tools, we would need to identify the “right” nail (use-case or problem) to use this “hammer” (machine learning algorithms).

We are interested in including machine learning into software systems because ML might solve some problems, which can be too complex to be solved traditionally. For such problems, a probabilistic (stochastic) solution that is implemented by using machine learning, might be the right way to pursue. For example, Perceptive problems in conversational UIs can be solved with techniques such as speech recognition or sentiment analysis. Machine learning (deep learning) appears to be the most appropriate one because such problems have a large number of elements with different representations. Another type of problems that suitable for ML, are multi-parameters problems. For example, we apply machine learning approaches to generate a stock prices prediction, which is a foundation for a stock trading decisions.

Placing models into production means making your models available to the software systems. Practically, by deploying the ML model, we can provide the following functionality:

In the following Table, we summarize the ML/AI capabilities:

ML Capabilities

Table Source: “The AI Organization” by David Carmona

Deployment Gap

More and more enterprises are experimenting with ML. Getting a model into the real world involves more than just building it. In order to take full advantage of the built ML model by making it available to our core software system, we would need to incorporate the trained ML model into the core codebase. That means, we need to deploy the ML model into production. By deploying models, other software systems can supply data to these and get predictions, which are in turn populated back into the software systems. Therefore, the full advantage of ML models is only possible through the ML model deployment.

However, according to a report by Algorithmia “2020 State of Enterprise Machine Learning”, many companies haven’t figured out how to achieve their ML/AI goals. Because bridging the gap between ML model building and practical deployments is still a challenging task. There’s a fundamental difference between building a ML model in the Jupyter notebook model and deploying an ML model into a production system that generates business value. Although AI budgets are on the rise, only 22 percent of companies that use machine learning have successfully deployed an ML model into production.

ML Deployment Gap

Figure Source

The “2020 State of Enterprise Machine Learning” report is based on a survey of nearly 750 people including machine learning practitioners, managers for machine learning projects, and executives at tech firms. Half of the respondents answered that it takes their company between a week and three months to deploy an ML model. About 18 percent stated that it takes from three months to a year. According to the report “The main challenges people face when developing ML capabilities are scale, version control, model reproducibility, and aligning stakeholders”.

Scenarios of Change That Need to be Managed

The reason for the previously described deployment gap is that the development of the machine learning-based applications is fundamentally different from the development of the traditional software. The complete development pipeline includes three levels of change: Data, ML Model, and Code. This means that in machine learning-based systems, the trigger for a build might be the combination of a code change, data change, or model change. This is also known as “Changing Anything Changes Everything” principle.

Three levels of change

In the following, we list some scenarios of possible changes in machine learning applications:

Additionally, three common issues influence the value of ML models once they’re in production.

The first is data quality: since ML models are built on data, they are sensitive to the semantics, amount and completeness of incoming data.

The second is model decay: the performance of ML models in production degenerate over time because of changes in the real-life data that has not been seen during the model training.

The third is locality: when transferring ML models to new business customers, these models, which have been pre-trained on different user demographics, might not work correctly according to quality metrics.

Since ML/AI is expanding into new applications and shaping new industries, building successful ML projects remains a challenging task. As shown, there is a need to establish effective practices and processes around designing, building, and deploying ML models into production - MLOps.

Further reading: Why is DevOps for Machine Learning so Different?

MLOps Definition

We saw what real-world problems might be solved by applying machine learning. We established the challenges of getting the ML models into production.

Finally, we are set up to define the term MLOps:

The term MLOps is defined as “the extension of the DevOps methodology to include Machine Learning and Data Science assets as first-class citizens within the DevOps ecology” Source: MLOps SIG.

Alternatively, we can use the definition of Machine Learning Engineering (MLE), where MLE is the use of scientific principles, tools, and techniques of machine learning and traditional software engineering to design and build complex computing systems. MLE encompasses all stages from data collection, to model building, to make the model available for use by the product or the consumers.” (by A.Burkov).

MLOps, like DevOps, emerges from the understanding that separating the ML model development from the process that delivers it — ML operations — lowers quality, transparency, and agility of the whole intelligent software.

The Evolution of the MLOps

In the early 2000s, when businesses needed to implement machine learning solutions, they used the vendors’ licensed software such as SAS, SPSS, and FICO. With the rise of open-source software and the availability of data, more software practitioners started using Python or R libraries for training ML models. However, the usage of the models in production was still a problem. As the containerization technology was emerging, the deployment of the model in a scalable way was solved by using Docker containers and Kubernetes. Recently, we see the evolution of those solutions into ML deployment platforms that cover the whole iteration of model experimentation, training, deployment, and monitoring. The following Figure visualizes the evolution of the MLOps.

The Evolution of MLOps