MLOps Infrastructure Stack

Before any machine learning model can be put in production, many experimentation cycles are needed to identify the right ML model to achieve the business goal. This experimentation phase adds additional complexity in any ML project because it includes three main artifacts: Data, Model, and Code. To manage this complexity, we need well-defined structures, processes, and proper software tools that manage ML artifacts and cover the machine learning cycle.

MLOps must be language-, framework-, platform-, and infrastructure-agnostic practice. MLOps should follow a “convention over configuration” implementation.

The MLOps technology stack should include tooling for the following tasks:

To solve MLOps challenges there are many choices ranging from using an off-the-shelf machine-learning platform to implement an in-house solution by composing open-source libraries. Existing cloud providers already working on offering machine learning platforms such as AI Platform by Google Cloud, AzureML, and SageMaker by AWS. Initiated at Google, the Kubeflow project presents an option to manage a set of open-source tools for MLOps and assemble them on Kubernetes. The adoption of such ML platform depends on the cloud strategy of the organization. In case, when an in-house hosted solution for MLOps is preferred, non-cloud systems like MLFlow, Sacred, or DVC might be a tool of choice.

An example of the technology stack might include the following open source tools:

MLOps Setup Components Tools
Source Control Git
Test & Build Services PyTest & Make
Deployment Services Git, DVC
Model & Dataset Registry DVC[aws s3]
Feature Store project code library
ML Metadata Store DVC
ML Pipeline Orchestrator DVC & Make

Depending on the requirements or restrictions, organizations can implement their own in-house MLOps system by combining existing open-source libraries. To build such a system one should have an overview of the MLOps tool landscape. This landscape is rapidly developing and changing on a daily basis because different tools specialise in different niches. The Linux Foundation’s LF AI project created a visualization for the ML/AI and MLOps tools. Another curated list of the production machine learning tools is maintained by the Institute for Ethical AI.

AI Landscape

Figure Source

Further References About the Technical Stack for MLOps