Turbo Management System

Turbo

Turbo serves as a computational cluster management platform enabling users to orchestrate and schedule tasks across multiple machines. ByteCompute GPU Clusters come with Turbo pre-installed for immediate distributed training capabilities, while also permitting custom scheduler integration. Computing tasks are submitted to Turbo's central control node, where the scheduler dynamically allocates resources to available GPU nodes according to current utilization.

Turbo Basic Concepts

1. Jobs: Computational workloads submitted to the cluster, encompassing scripts, executable programs, or other task-based operations.
2. Nodes: Individual computing units within the cluster that execute jobs, which may be implemented as physical hardware or virtualized environments.
3. Head Node: The central access point preconfigured in every ByteCompute GPU Cluster, where users authenticate to create, submit, and retrieve computational tasks.
4. Partitions: Logical groupings of nodes designated for job execution, configurable with distinct attributes including node quantity and memory allocation.
5. Priorities: Ranking mechanisms that establish job execution order, granting precedence to higher-priority tasks over lower-priority ones.

Using Turbo

1. Job Submission: Jobs can be submitted to the cluster using the sbatch command. Jobs can be submitted in batch mode or interactively using the srun command.
2. Job Monitoring: Jobs can be monitored using the squeue command, which displays information about the jobs that are currently running or waiting to run.
3. Job Control: Jobs can be controlled using the scancel command, which allows users to cancel or interrupt jobs that are running.

Troubleshooting Turbo

1. Error Messages: Diagnostic notifications generated by Turbo to assist in problem identification and resolution.
2. Log Files: System-generated records enabling cluster status monitoring and issue diagnosis through operational history analysis.