Edit on GitHub

Using CML

A GitLab, GitHub, or Bitbucket account is required. Familiarity with Github Actions or GitLab CI/CD may also be beneficial.

GitLab

Please see our docs on CML with GitLab CI/CD and in particular the personal access token requirement.

Bitbucket

Please see our docs on CML with Bitbucket Cloud. Bitbucket Server support estimated to arrive by mid 2021.

GitHub

The key file in any CML project is .github/workflows/cml.yaml:

name: your-workflow-name
on: [push]
jobs:
  run:
    runs-on: ubuntu-latest
    # optionally use a convenient Ubuntu LTS + DVC + CML container
    # container: docker://ghcr.io/iterative/cml:0-dvc2-base1
    steps:
      - uses: actions/checkout@v2
      # may need to setup NodeJS & Python3 on e.g. self-hosted
      # - uses: actions/setup-node@v2
      #   with:
      #     node-version: '12'
      # - uses: actions/setup-python@v2
      #   with:
      #     python-version: '3.x'
      - uses: iterative/setup-cml@v1
      - name: Train model
        run: |
          # Your ML workflow goes here
          pip install -r requirements.txt
          python train.py
      - name: Write CML report
        env:
          REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          # Post reports as comments in GitHub PRs
          cat results.txt >> report.md
          cml-send-comment report.md

We helpfully provide CML and other useful libraries pre-installed on our custom Docker images. In the above example, uncommenting the container: docker://ghcr.io/iterative/cml:0-dvc2-base1 field will make the runner pull the CML Docker image. The image already has NodeJS, Python 3, DVC and CML set up on an Ubuntu LTS base for convenience.

CML Functions

CML provides a number of functions to help package the outputs of ML workflows (including numeric data and visualizations about model performance) into a CML report.

Below is a table of CML functions for writing markdown reports and delivering those reports to your CI system.

FunctionDescriptionExample Inputs
cml-runnerLaunch a runner locally or hosted by a cloud providerSee Arguments
cml-publishPublicly host an image for displaying in a CML report<path to image> --title <image title> --md
cml-send-commentReturn CML report as a comment in your GitLab/GitHub workflow<path to report> --head-sha <sha>
cml-send-github-checkReturn CML report as a check in GitHub<path to report> --head-sha <sha>
cml-prCommit the given files to a new branch and create a pull request<path>...
cml-tensorboard-devReturn a link to a Tensorboard.dev page--logdir <path to logs> --title <experiment title> --md

CML Reports

The cml-send-comment command can be used to post reports. CML reports are written in markdown (GitHub, GitLab, or Bitbucket flavors). That means they can contain images, tables, formatted text, HTML blocks, code snippets and more โ€” really, what you put in a CML report is up to you. Some examples:

๐Ÿ“ Text Write to your report using whatever method you prefer. For example, copy the contents of a text file containing the results of ML model training:

cat results.txt >> report.md

๐Ÿ–ผ๏ธ Images Display images using the markdown or HTML. Note that if an image is an output of your ML workflow (i.e., it is produced by your workflow), you will need to use the cml-publish function to include it a CML report. For example, if graph.png is output by python train.py, run:

cml-publish graph.png --md >> report.md
Content

๐Ÿ› Found an issue? Let us know! Or fix it:

Edit on GitHub

โ“ Have a question? Join our chat, we will help you:

Discord Chat

    About

  • Install
  • Use cases
  • Blog
  • Github

By iterative.ai - An open platform to operationalize AI