_cmlAsset 4

Continuous Machine Learning (CML) is CI/CD for Machine Learning Projects

Get Started

GitFlow for data science

Use GitLab or GitHub to manage ML experiments, track who trained ML models or modified data and when. Codify data and models with DVC instead of pushing to your Git repo.

Auto reports for ML experiments

Auto-generate reports with metrics and plots in each Git Pull Request. Rigorous engineering practices help your team make informed, data-driven decisions.

No additional services

Build your own ML platform using just GitHub or GitLab and your favorite cloud services: AWS, Azure, GCP. No databases, services or complex setup needed.

CML Use Cases

The simplest case of using CML, and a clear way for any user to get started, is to generate a simple report. Add the following .yaml to your project repository and commit to get started

.gitlab-ci.yml

stages:
- cml_run
cml:
stage: cml_run
image: dvcorg/cml-py3:latest
script:
- pip3 install -r requirements.txt
- python train.py
- cat metrics.txt >> report.md
- cml-publish confusion_matrix.png --md >> report.md
- cml-send-comment report.md
GitLab

CML Report

GitLab Base report example

.github/workflows/cml.yaml

name: train-my-model
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
container: docker://dvcorg/cml-py3:latest
steps:
- uses: actions/[email protected]
- name: cml_run
env:
repo_token: ${{ secrets.GITHUB_TOKEN }}
run: |
pip3 install -r requirements.txt
python train.py
cat metrics.txt >> report.md
cml-publish confusion_matrix.png --md >> report.md
cml-send-comment report.md
GitHub

CML Report

.gitlab-ci.yml

stages:
- cml_run
cml:
stage: cml_run
image: dvcorg/cml-py3:latest
script:
- dvc pull data
- pip install -r requirements.txt
- dvc repro
# Compare metrics to master
- git fetch --prune
- dvc metrics diff --show-md master >> report.md
# Visualize loss function diff
- dvc plots diff
--target loss.csv --show-vega master > vega.json
- vl2png vega.json | cml-publish --md >> report.md
- cml-send-comment report.md

CML Report

.github/workflows/cml.yaml

name: train-my-model
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
container: docker://dvcorg/cml-py3:latest
steps:
- uses: actions/[email protected]
- name: cml_run
env:
repo_token: ${{ secrets.GITHUB_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.KEY }}
run: |
dvc pull data
pip install -r requirements.txt
dvc repro
# Compare metrics to master
git fetch --prune
dvc metrics diff --show-md master >> report.md
# Visualize loss function diff
dvc plots diff \
--target loss.csv --show-vega master > vega.json
vl2png vega.json | cml-publish --md >> report.md
cml-send-comment report.md
GitHub

CML Report

.gitlab-ci.yml

stages:
- cml_run
cml:
stage: cml_run
image: dvcorg/cml-py3:latest
script:
- pip install -r requirements.txt
- cml-tensorboard-dev \
--logdir logs \
--name "Go to tensorboard" \
--md >> report.md
- cml-send-comment report.md
- python train.py

CML Report

.github/workflows/cml.yaml

name: train-my-model
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
container: docker://dvcorg/cml-py3:latest
steps:
- uses: actions/[email protected]
- name: cml_run
env:
repo_token: ${{ secrets.GITHUB_TOKEN }}
TB_CREDENTIALS: ${{ secrets.TB_CRED }}
run: |
pip install -r requirements.txt
cml-tensorboard-dev \
--logdir logs \
--name "Go to tensorboard" \
--md >> report.md
cml-send-comment report.md
python train.py
GitHub

CML Report

.gitlab-ci.yml

deploy_aws:
image: dvcorg/cml:latest
script:
- echo "Deploying..."
- RUNNER_LABELS="cml,aws"
- MACHINE="cml$(date +%s)"
- docker-machine create
--driver amazonec2
--amazonec2-access-key $AWS_ACCESS_KEY_ID
--amazonec2-secret-key $AWS_SECRET_ACCESS_KEY
--amazonec2-instance-type g3s.xlarge
--amazonec2-vpc-id vpc-76f1f01e
--amazonec2-region us-east-2
--amazonec2-zone a
--amazonec2-ssh-user ubuntu
--amazonec2-ami ami-0b9391ca3ebd62e15
--amazonec2-root-size 130
$MACHINE
- eval "$(docker-machine env --shell sh $MACHINE)"
- (
docker-machine ssh $MACHINE "sudo mkdir -p /docker_machine &&
sudo chmod 777 /docker_machine" &&
docker-machine scp -r -q ~/.docker/machine/
$MACHINE:/docker_machine &&
docker run --name runner --gpus all -d
-v /docker_machine/machine:/root/.docker/machine
-e DOCKER_MACHINE=$MACHINE
-e repo_token=$repo_token
-e RUNNER_LABELS=$RUNNER_LABELS
-e RUNNER_REPO=$CI_PROJECT_URL
-e RUNNER_IDLE_TIMEOUT=120
dvcorg/cml-py3:latest &&
sleep 20 && echo "Deployed $MACHINE"
) || (docker-machine rm -f $MACHINE && exit 1)
train_aws:
tags:
- cml
- aws
script:
- apt-get update -y
- apt-get install python3-dev -y
- apt install imagemagick -y
- pip install -r requirements.txt
# DVC reproduce neural style transfer training
- dvc repro
# Compare master and workspace image results
- git fetch --prune
- git show origin/master:final_owl.png > master_owl.png
- convert +append final_owl.png master_owl.png out.png
- convert out.png -resize 75% out_shrink.png
- echo "# Style transfer" >> report.md
- echo "### Workspace vs. Master" >> report.md
- cml-publish out_shrink.png --md >> report.md
# Report training parameters
- echo "## Training parameter diffs" >> report.md
- dvc params diff master --show-md >> report.md
- echo >> report.md
# Report GPU details
- echo "## GPU info" >> report.md
- cat gpu_info.txt >> report.md
- cml-send-comment report.md

CML Report

.github/workflows/cml.yaml

name: style-transfer
on: [push]
jobs:
deploy-cloud-runner:
runs-on: [ubuntu-latest]
container: docker://dvcorg/cml
steps:
- name: deploy
env:
repo_token: ${{ secrets.REPO_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SECRET_ACCESS_KEY }}
run: |
echo "Deploying..."
RUNNER_LABELS="cml,aws"
RUNNER_REPO="https://github.com/${GITHUB_REPOSITORY}"
MACHINE="cml$(date +%s)"
docker-machine create \
--driver amazonec2 \
--amazonec2-instance-type g3s.xlarge \
--amazonec2-vpc-id vpc-76f1f01e \
--amazonec2-region us-east-2 \
--amazonec2-zone a \
--amazonec2-ssh-user ubuntu \
--amazonec2-ami ami-07b11e59bc74961b5 \
--amazonec2-root-size 64 \
$MACHINE
eval "$(docker-machine env --shell sh $MACHINE)"
(
docker-machine ssh $MACHINE "sudo mkdir -p \
/docker_machine && \
sudo chmod 777 /docker_machine" && \
docker-machine scp -r -q ~/.docker/machine/ \
$MACHINE:/docker_machine && \
docker run --name runner --gpus all -d \
-v /docker_machine/machine:/root/.docker/machine \
-e DOCKER_MACHINE=$MACHINE \
-e repo_token=$repo_token \
-e RUNNER_LABELS=$RUNNER_LABELS \
-e RUNNER_REPO=$RUNNER_REPO \
-e RUNNER_IDLE_TIMEOUT=120 \
dvcorg/cml-py3:latest && \
sleep 20 && echo "Deployed $MACHINE"
) || (docker-machine rm -y -f $MACHINE && exit 1)
train:
needs: deploy-cloud-runner
runs-on: [self-hosted,cml]
steps:
- uses: actions/[email protected]
- name: cml_run
env:
repo_token: ${{ secrets.REPO_TOKEN}}
run: |
apt-get update -y
apt-get install python3-dev -y
apt install imagemagick -y
pip install -r requirements.txt
# DVC reproduce neural style transfer training
dvc repro
# Compare master and workspace image results
git fetch --prune
git show origin/master:final_owl.png > master_owl.png
convert +append final_owl.png master_owl.png out.png
convert out.png -resize 75% out_shrink.png
echo "# Style transfer" >> report.md
echo "### Workspace vs. Master" >> report.md
cml-publish out_shrink.png --md >> report.md
# Report training parameters
echo "## Training parameter diffs" >> report.md
dvc params diff master --show-md >> report.md
# Report GPU details
echo "## GPU info" >> report.md
cat gpu_info.txt >> report.md
cml-send-comment report.md
GitHub

CML Report

The MLOps Ecosystem

MLOps isn't a platform- it's an ecosystem of tools. CML helps you bring your favorite DevOps tools to machine learning.

  • Continuous integration for ML

    CML

  • Manage environments

    Docker and Packer

  • Infrastructure as code

    Terraform and Docker-Machine

  • Data as code

    DVC

Subscribe for updates. We won't spam you.
_cmlAsset 4
Powered by