ClearML

Welcome to the ClearML Agent Management API documentation. Here, you'll find comprehensive guidance on how to deploy and manage ClearML agents seamlessly using our API.

Why deploy ClearML agents on CX?

• Scalability: Access a vast array of GPU compute resources, allowing dynamic scalability of your ClearML agents.

• Cost-Effectiveness: Get the scalability of the cloud at a fraction of the cost.

• Security: Benefit from a fortified environment for ClearML agent operations, equipped with state-of-the-art authentication and authorization features.

• Ease of Use: Deploy GPU resources efficiently to process tasks from your ClearML frontend using a single endpoint.

Table of Contents

1. Quick Start

2. ClearML Integration

3. Endpoints Overview

   • Create a ClearML Agent

   • List All ClearML Agents

   • Fetching Agent Status

   • Deleting an Agent

Quick Start

To get started:

1. Authenticate using the /login endpoint.

2. Deploy a ClearML agent using the /create_agent endpoint.

3. List, delete, and check the status of your agent using the endpoints described below.

ClearML Integration

ClearML is a robust open-source platform that streamlines machine learning workflows. It aids data scientists and engineers in tracking, reproducing, and analyzing experiments across various tools, frameworks, and codebases.

By integrating ClearML with CX, you can:

• Efficient Deployment: Launch multiple ClearML agents seamlessly.

• Customizable Specs: Define the machine specifications for each agent according to your needs.

• Replica Control: Manage the number of replicas for each agent effortlessly.

• Auto-Scaling: The system scales to zero automatically when idle, ensuring cost-efficiency.

How it works

Each customer is assigned a unique namespace in the ComputeX back end. ComputeX supports deploying ClearML Agents and Serving instances on the ComputeX GPU cloud to enable scalable and cost-effective machine learning workflows.

The ClearML scheduler and front end components can either be self-hosted in the Client AWS ecosystem, hosted entirely within CX, or use ClearML's Paid hosted service.

The diagram above outlines a pipeline where the Client chooses to self-host ClearML in AWS, while leveraging ComputeX for GPU Compute on ClearML Agent and ClearML Serving instances. Custom IAM Role and IAM User configurcredentials can be configured in the ClearML Agent deployment to enable access to the Client's AWS resources.

Leveraging GPU Diversity: With CX's extensive GPU offerings, you can create a separate queue for each desired GPU specification. This allows for targeted task assignments from the ClearML frontend simply by specifying the appropriate queue name.

Endpoints Overview

Creating a ClearML Agent

• URL: /create_agent

• Method: POST

Deploy a ClearML agent on the CX backend by supplying the requisite configuration.

Sample Request:

curl -X 'POST' \
    'https://api.computex.co/api/v1/clearml/create_agent' \
    -H 'accept: application/json' \
    -H 'Authorization: Bearer <YOUR-TOKEN>' \
    -H 'Content-Type: application/json' \
    -d {
    "key": "ClearML-Key",
    "secret": "ClearML-Secret",
    "queue_name": "a40-queue",
    "num_cpu_cores": 10,
    "num_gpu": 1,
    "gpu_sku": "A40",
    "memory": 64,
    "file_server_url": "https://files.clear.ml",
    "api_server_url": "https://api.clear.ml",
    "web_server_url": "https://app.clear.ml",
    "replica_count": 1
    }

Listing ClearML Agents

• URL: /agents

• Method: GET

Retrieve an array of all active ClearML agents.

Sample Request

curl -X 'GET' \
    'http://localhost:8000/api/v1/clearml/agents' \
    -H 'Authorization: Bearer <YOUR-TOKEN>' \

Fetching Agent Status

• URL: /status

• Method: GET

Acquire the current operational status of a designated ClearML agent using its unique ID.

Sample Request

curl -X 'GET' \
    'http://localhost:8000/api/v1/clearml/status?id=121' \
    -H 'Authorization: Bearer <YOUR-TOKEN>' \

Deleting an Agent

• URL: /{agent_id}

• Method: DELETE

Terminate a specific ClearML agent by referencing its unique ID.

Sample Request

curl -X 'DELETE' \
    'http://localhost:8000/api/v1/clearml/<AGENT-ID>' \
    -H 'Authorization: Bearer <YOUR-TOKEN>' \