Applying MLflow to an existing machine learning project

2 min readJun 9, 2020

What is MLflow?

MLflow is an open-source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. MLflow currently offers four components:

MLflow Tracking
MLflow Projects
MLflow Models
MLflow Registry

In this example, we are going to use an existing Keras Tensorflow example and add MLflow support to it.

Example: Collaborative Filtering for Movie Recommendations” with Keras and Tensorflow 2.0.0

The example we have selected is “Collaborative Filtering for Movie Recommendations” with Keras and Tensorflow 2.0.0. This example is created by Siddhartha Banerjee and the example Source URL is as below: https://keras.io/examples/structured_data/collaborative_filtering_movielens/

The GitHub source for the sample example is here:

https://github.com/keras-team/keras-io/blob/master/examples/structured_data/collaborative_filtering_movielens.py

Creating necessary files for MLflow:

Download the source code on your local machine. Run the code locally to make sure it works and you understand it

$ python collaborative_filtering_movielens.py

Create conda.yaml as below:

name: tf_keras_recom_example
channels:
— defaults
— anaconda
— conda-forge
dependencies:
— python=3.6
— numpy=1.14.2
— pandas
— pip:
— mlflow
— tensorflow==2.0.0

Create MLProject as below:

name: movielens-recommendations-keras-tf2
conda_env: conda.yaml
entry_points:
main:
command: “python collaborative_filtering_movielens.py”

Now you must have 3 files as below:

Editing the code file for MLflow:

Add the following python modules

import mlflow.tensorflow
mlflow.tensorflow.autolog()

Comment matplotlib specific code in the project so plotting UI specific thread will not block the code execution.

# import matplotlib.pyplot as plt
# plt.plot(history.history[“loss”])
# plt.plot(history.history[“val_loss”])
# plt.title(“model loss”)
# plt.ylabel(“loss”)
# plt.xlabel(“epoch”)
# plt.legend([“train”, “test”], loc=”upper left”)
# plt.show()

That’s all. Now we will run the code in 2 separate terminal environments.