by iterative.ai
Get started
Edit on GitHub

Using CML

A GitLab, GitHub, or Bitbucket account is required. Familiarity with Github Actions or GitLab CI/CD may also be beneficial.

The key file in any CML project is .github/workflows/cml.yaml:

name: CML
on: [push]
jobs:
  run:
    runs-on: ubuntu-latest
    # optionally use a convenient Ubuntu LTS + DVC + CML container
    # container: docker://ghcr.io/iterative/cml:0-dvc2-base1
    steps:
      # may need to setup Node.js & Python3 on e.g. self-hosted
      # - uses: actions/setup-node@v2
      #   with:
      #     node-version: '16'
      # - uses: actions/setup-python@v2
      #   with:
      #     python-version: '3.x'
      - uses: iterative/setup-cml@v1
      - uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: Train model
        run: |
          # Your ML workflow goes here
          pip install -r requirements.txt
          python train.py
      - name: Create CML report
        env:
          REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          # Post reports as comments in GitHub PRs
          cat results.txt >> report.md
          cml send-comment report.md

The example above generates visual reports in pull requests: cml first report

We helpfully provide CML and other useful libraries pre-installed on our custom Docker images. In the above example, uncommenting the container: docker://ghcr.io/iterative/cml:0-dvc2-base1 field will make the runner pull the CML Docker image. The image already has Node.js, Python 3, DVC and CML set up on an Ubuntu LTS base for convenience.

Example projects

The key file in any CML project is .gitlab-ci.yml:

train-model:
  # use a convenient Ubuntu LTS + DVC + CML container
  image: iterativeai/cml:0-dvc2-base1
  script:
    - pip install -r requirements.txt
    - python train.py
create-CML-report:
  needs: train-model
  image: iterativeai/cml:0-dvc2-base1
  script:
    - cat metrics.txt >> report.md
    - cml publish plot.png --md >> report.md
    - cml send-comment report.md

⚠️ You must provide a personal or project access token (PAT) via a REPO_TOKEN variable.

The example above generates visual reports in merge requests: GitLab CML report

We helpfully provide CML and other useful libraries pre-installed on our custom Docker images. In the above example, the image: iterativeai/cml:0-dvc2-base1 field will make the runner pull the CML Docker image. The image already has Node.js, Python 3, DVC and CML set up on an Ubuntu LTS base for convenience.

Example projects

The key file in any CML project is bitbucket-pipelines.yml:

image: iterativeai/cml:0-dvc2-base1
pipelines:
  default:
    - step:
        name: Train model
        script:
          - pip install -r requirements.txt
          - python train.py
    - step:
        name: Create CML report
        script:
          - cat metrics.txt > report.md
          - echo >> report.md
          - cml publish plot.png --md >> report.md
          - cml send-comment report.md

⚠️ You must provide access credentials via a REPO_TOKEN variable.

The example above generates visual reports in pull requests: bitbucket cloud pr

⚠️ CML works with Bitbucket Cloud, where you can use the Bitbucket Pipelines CI/CD system to run workflows automatically on triggering events. Bitbucket Server is not yet supported.

Example projects

CML Commands

CML provides a number of commands to help package the outputs of ML workflows (including numeric data and visualizations about model performance) into a CML report.

Below is a list of CML commands for starting cloud compute runners, writing and publishing markdown reports to your CI/CD system.

runner
Launch a runner hosted by a cloud compute provider or locally on-premise (see self-hosted runners)
e.g. cml runner --cloud={aws,azure,gcp,kubernetes} ...

publish
Publicly host an image for displaying in a CML report
e.g. cml publish plot.png --md >> report.md

pr
Commit specified files to a new branch and create a pull request
e.g. cml pr "**/*.json" "**/*.py" --md >> report.md

send-comment
Post a markdown report as a commit comment
e.g. cml send-comment report.md

send-github-check
Post a markdown report as a GitHub check
e.g. cml send-github-check report.md

tensorboard-dev
Return a link to a https://tensorboard.dev page
e.g. cml tensorboard-dev --logdir=./logs --md >> report.md

CML Reports

The cml send-comment command can be used to post reports. CML reports are written in markdown (GitHub, GitLab, or Bitbucket flavors). That means they can contain images, tables, formatted text, HTML blocks, code snippets and more — really, what you put in a CML report is up to you. Some examples:

📝 Text Write to your report using whatever method you prefer. For example, copy the contents of a text file containing the results of ML model training:

$ cat results.txt >> report.md

🖼️ Images Display images using the markdown or HTML. Note that if an image is an output of your ML workflow (i.e., it is produced by your workflow), you will need to use the cml publish command to include it a CML report. For example, if plot.png is output by python train.py, run:

$ cml publish plot.png --md >> report.md
Content

🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

Have a question? Join our chat, we will help you:

Discord Chat