Usage
A GitLab, GitHub, or Bitbucket account is required. Familiarity with GitHub Actions, GitLab CI/CD, or Bitbucket Pipelines may also be beneficial.
The key file in any CML project is .github/workflows/cml.yaml:
name: CML
on: [push]
jobs:
train-and-report:
runs-on: ubuntu-latest
# optionally use a convenient Ubuntu LTS + DVC + CML container
# container: docker://ghcr.io/iterative/cml:0-dvc2-base1
steps:
# may need to setup Node.js & Python3 on e.g. self-hosted
# - uses: actions/setup-node@v2
# with:
# node-version: '16'
# - uses: actions/setup-python@v4
# with:
# python-version: '3.x'
- uses: iterative/setup-cml@v1
- uses: actions/checkout@v3
- name: Train model
run: |
# Your ML workflow goes here
pip install -r requirements.txt
python train.py # generate plot.png
- name: Create CML report
env:
REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Post CML report as a comment in GitHub
cat metrics.txt >> report.md
echo '' >> report.md
cml comment create report.mdThis generates visual reports in pull requests (see cml comment create):
We helpfully provide CML and other useful libraries pre-installed on our
custom Docker images. In the above
example, uncommenting the
container: docker://ghcr.io/iterative/cml:0-dvc2-base1 field will make the
runner pull the CML Docker image. The image already has Node.js, Python 3, DVC
and CML set up on an Ubuntu LTS base for convenience.
Example projects
Integrate CML to your
pipeline configuration
using the .gitlab-ci.yml file, for example:
train-model:
# use a convenient Ubuntu LTS + DVC + CML container
image: iterativeai/cml:0-dvc2-base1
script:
- pip install -r requirements.txt
- python train.py # generate plot.png
create-CML-report:
needs: [train-model]
image: iterativeai/cml:0-dvc2-base1
script:
# Post CML report as a comment in GitLab
- cat metrics.txt >> report.md
- echo '' >> report.md
- cml comment create report.mdYou must provide a
personal or project access token (PAT)
via a REPO_TOKEN variable.
This generates visual reports in pull requests (see cml comment create):
We helpfully provide CML and other useful libraries pre-installed on our
custom Docker images. In the above
example, the image: iterativeai/cml:0-dvc2-base1 field will make the runner
pull the CML Docker image. The image already has Node.js, Python 3, DVC and CML
set up on an Ubuntu LTS base for convenience.
Example projects
Integrate CML to your
pipeline configuration
using the bitbucket-pipelines.yml file, for example:
image: iterativeai/cml:0-dvc2-base1
pipelines:
default:
- step:
name: Train model
script:
- pip install -r requirements.txt
- python train.py # generate plot.png
- step:
name: Create CML report
script:
# Post CML report as a comment in Bitbucket
- cat metrics.txt >> report.md
- echo '' >> report.md
- cml comment create report.mdYou must provide
access credentials via a
REPO_TOKEN variable.
This generates visual reports in pull requests (see cml comment create):
CML works with Bitbucket Cloud, where you can use the Bitbucket Pipelines CI/CD system to run workflows automatically on triggering events. Bitbucket Server is not yet supported.
Example projects
CML Commands
Below is a list of CML commands for starting cloud compute runners, writing and publishing Markdown reports to your CI/CD system.
∞ runner
Launch a runner hosted by a cloud compute provider or locally on-premise (see self-hosted runners)
e.g. cml runner launch --cloud={aws,azure,gcp,kubernetes} ...
∞ pr
Commit specified files to a new branch and create a pull request
e.g. cml pr create "**/*.json" "**/*.py" --md >> report.md
∞ comment
Post a Markdown report as a commit comment
e.g. cml comment create report.md
∞ check
Post a Markdown report as a GitHub check
e.g. cml check create report.md
∞ tensorboard
Return a link to a https://tensorboard.dev page
e.g. cml tensorboard connect --logdir=./logs --md >> report.md
CML Reports
The cml comment create command can be used to post reports. CML reports are
written in Markdown (GitHub,
GitLab, or
Bitbucket
flavors). That means they can contain images, tables, formatted text, HTML
blocks, code snippets and more — really, what you put in a CML report is up to
you. Some examples:
📝 Text Write to your report using whatever method you prefer. For example, copy the contents of a text file containing the results of ML model training:
$ cat results.txt >> report.md🖼️ Images Display images using the markdown or HTML. Note that if an image
is an output of your ML workflow (i.e. it is produced by your workflow), you can
use markdown to embed it in a CML report. For example, if plot.png is output
by python train.py, run:
$ echo '' >> report.md
$ cml comment create report.md