Get started
Edit on GitHub

Command Reference: runner

cml runner launch [options]

Starts a runner (either via any supported cloud compute provider or locally on-premise).


Any generic option in addition to:

  • --labels=<...>: One or more (comma-delimited) labels for this runner [default: cml].
  • --name=<...>: Runner name displayed in the CI [default: cml-{ID}].
  • --idle-timeout=<seconds>: Seconds to wait for jobs before terminating. Set to -1 to disable timeout [default: 300].
  • --no-retry: Don't restart the workflow when terminated due to instance disposal or GitHub Actions timeout.
  • --single: Terminate runner after one workflow run.
  • --reuse: Don't launch a new runner if an existing one has the same name or overlapping labels. If an existing matching (same name or overlapping labels) instance is busy, it'll still be reused.
  • --reuse-idle: Creates a new runner only if the matching labels don't exist or are already busy.
  • --cloud={aws,azure,gcp,kubernetes}: Cloud compute provider to host the runner.
  • --cloud-type={m,l,xl,m+k80,m+v100,...}: Instance type. Also accepts native types such as t2.micro.
  • --cloud-gpu={nogpu,k80,v100,tesla}: GPU type.
  • --cloud-hdd-size=<...>: Disk storage in GB.
  • --cloud-spot: Request a preemptible spot instance.
  • --cloud-spot-price=<...>: Maximum spot instance USD bidding price, [default: current price].
  • --cloud-region={us-west,us-east,eu-west,eu-north,...}: Region where the instance is deployed. Also accepts native AWS/Azure region or GCP zone [default: us-west].
  • --cloud-permission-set=<...>: AWS instance profile or GCP instance service account. More details below.
  • --cloud-metadata=<...>: key=value pair to associate with cloud runner instances. May be specified multiple times.
  • --cloud-startup-script=<...>: Run the provided Base64-encoded Linux shell script during the instance initialization. More details below.
  • --cloud-ssh-private=<key>: Private SSH RSA key [default: auto-generate throwaway key]. Only supported on AWS and Azure; intended for debugging purposes. More details below.
  • --cloud-aws-security-group=<...>: AWS security group identifier.
  • --cloud-aws-subnet=<...>: AWS subnet identifier.
  • --docker-volumes=<...>: Volume mount to pass to Docker, e.g. /var/run/docker.sock:/var/run/docker.sock for Docker-in-Docker support. May be specified multiple times. Only supported by GitLab.

FAQs and Known Issues


  • GitHub Actions timeout after a few hours.

    You can request up to 35 days via timeout-minutes: 50400. CML will helpfully restart GitHub Actions workflows approaching 35 days (you'd need to write your code to save intermediate results to take advantage of this).


Using --cloud-permission-set


The associated cloud credentials must grant access to resources needed for managing compute instances.

An AWS ARN to an instance-profile:

  • arn:aws:iam::1234567890:instance-profile/dvc-s3-access
$ cml runner launch \
  --cloud-permission-set=arn:aws:iam::1234567890:instance-profile/dvc-s3-access \

A GCP service account email & list of scopes:

$ cml runner launch \,scopes=storage-rw,datastore \

Common Permissions

It's recommended to use provider-managed policies/roles and then explicitly limit the permissions further if possible.

For example this could potentially be further limited to:

ec2:CreateSecurityGroup -- (Firewall and SSH Access Management)
ec2:CreateTags -- (General Resource Management)
ec2:RunInstances -- (EC2 Instance Management)
ec2:DescribeSpotInstanceRequests -- (Optionally needed for Spot Access)

GCP Managed Roles:

  • roles/compute.admin
  • roles/iam.serviceAccountUse

For example this could potentially be further limited to:


You may also require additional permissions specific to your application (for example: object storage, private docker registries, and other cloud services). These additional permissions should be managed separately, and exposed either as independent credentials or via --cloud-permission-set

Currently this feature is only available on AWS & GCP clouds.

A set of permissions for a cml runner instance can be predefined a via an AWS role or a GCP service account.

This could, for example, enable credential-free access to AWS s3 & GCP gs DVC remotes, or grant access to AWS' Elastic Container Registry & GCP's Artifact Registry (to push and pull custom docker images).

Other AWS examples include accessing data in:

  • Secrets Manager
  • DynamoDB
  • Redshfit

Example "Permission Sets"

Policy example:
  "Version": "2012-10-17",
  "Statement": [
      "Sid": "DVCAccess",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::mydvcbucket/*"

Trust relationships:

  "Version": "2012-10-17",
  "Statement": [
      "Sid": "CMLRunnerInstance",
      "Effect": "Allow",
      "Principal": {
        "Service": ""
      "Action": "sts:AssumeRole"

Using --cloud-permission-set will likely require:

  • an additional role be added to your cml runner credentials roles/ServiceAccountUser,
  • ensuring the invoker has the permission iam.serviceAccount.actAs on the targeted Service Account.

Using --cloud-startup-script

A base64-encoded script to execute during cloud instance provisioning (after cml runner does its initial setup but before the runner becomes available to the CI/CD provider).

This script counts towards the total provisioning time. The total exceeding 10 minutes is considered a failure, resulting in cml runner terminating the instance and exiting with an error.

For example:

$ cml runner launch \
  --cloud-startup-script=IyEvYmluL2Jhc2gKCmVjaG8gImhlbGxvIHdvcmxkIgo= \

where echo IyEvYmluL2Jhc2gKCmVjaG8gImhlbGxvIHdvcmxkIgo= | base64 -d is:


echo "hello world"

This can be used for debugging, for example allowing SSH access for a GitHub user:

$ cml runner launch \
  --cloud-startup-script=$(echo 'echo "$(curl${{ }}.keys)" >> /home/ubuntu/.ssh/authorized_keys' | base64 -w 0) \

GitHub Actions will replace ${{ }} with the username of the person who triggered the workflow. Conveniently, GitHub (and GitLab) provide a URL to access a user's public SSH keys. In effect the above command runs:

$ curl >> ~/.ssh/authorized_keys

in the cloud instance.

This enables easy SSH access into the runner for debugging as well as experimentation.

By comparison, --cloud-ssh-private relies on a local user-generated private key and is only supported on AWS and Azure.

Using --cloud-ssh-private

  1. Generate a new RSA PEM private key for debugging purposes:

    $ ssh-keygen -t rsa -m pem -b 4096 -f key.pem
  2. Pass the contents of the generated private key file when invoking the cml runner command:

    $ cml runner launch --cloud=... --cloud-ssh-private="$(cat key.pem)"
  3. Access the instance from your local system by using the generated key as an identity file:

    $ ssh -i key.pem ubuntu@IP_ADDRESS

    replacing the IP_ADDRESS placeholder with the instance address returned by cml runner (search the output logs for instanceIp).


See the section self-hosted runners


🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

Have a question? Join our chat, we will help you:

Discord Chat