The Boole High Performance Computing (HPC) Platform is operated by CloudCIX. It provides compute resources for interactive data exploration, batch processing, parallel workloads, GPU jobs, containers, and object storage access.
This guide takes you from your first job to running across many nodes. It is hands-on: most topics have a job script you submit on Boole, an output file you read, and an exercise where you change something and see what happens. All sample output comes from real runs on Boole.
It is written for researchers and students who can run a Python script on their own computer and now want to run it on HPC. You need basic shell commands (cd, ls, cat). You don’t need to know Slurm or MPI beforehand. The examples are in Python, but the ideas apply to any language.
Before accessing Boole HPC, create an HPC account in CloudCIX:
In the Membership app, open My Membership and select HPC Accounts.
Click Create HPC Account.
Enter a username and password, then click Save.
You can then sign in to hpc.cloudcix.com using these credentials.
We recommend watching the short Boole HPC Basics video for an overview of the platform:
The main access point is the Boole HPC Portal (Open OnDemand):
From the portal, you can upload data, launch jobs, and start interactive applications. Available apps include:
Application |
Use |
|---|---|
Boole Shell Access |
Browser-based terminal access to the cluster. |
Remote Desktop |
Full remote desktop environment for GUI-based workflows. |
CloudCIX AI Lab |
AI and machine learning workspace. |
Jupyter Notebook |
Python, data analysis, and scientific computing. |
Jupyter + Spark |
Jupyter environment with Spark for distributed data processing. |
RStudio Server |
Browser-based R environment for statistics, analysis, and visualization. Bioconductor is pre-installed. |
VS Code Server |
Browser-based Visual Studio Code development environment. |
ParaView |
Visualization and analysis for scientific datasets. |
nf-core pipelines |
Bioinformatics workflow pipelines. |
Each interactive app runs as a Slurm job that holds the cores, memory and GPU you choose until you end it. See Interactive Apps in Open OnDemand: Ask for What You Use for what to request.
You can also connect directly to the HPC system using SSH or transfer files via rsync from HEAnet & CloudCIX IP addresses only.
Every example script and job template in this guide can be downloaded in one go. Open a terminal on Boole, for example through Boole Shell Access, and run:
cd ~
curl -sL https://docs.cloudcix.com/hpc_examples.tar.gz | tar xz
cd ~/hpc_examples
This creates ~/hpc_examples with two folders: examples/ (the scripts used in these pages) and templates/ (job scripts to copy for your own work). Run the commands in this guide from ~/hpc_examples; output files appear there.
To get the latest version later, run the same command again. It overwrites the example files, so copy any you have changed to another folder first.
Each script is also shown on the page that uses it, with its own download link.
Word |
Meaning |
|---|---|
Login node |
The machine your terminal opens on. Shared by everyone; for editing and submitting, not heavy work. |
Compute node |
A machine that runs jobs. |
Slurm |
The scheduler that queues jobs and gives them compute nodes. |
Partition |
A queue for one type of node: |
Job / job ID |
One request to Slurm, and the number |
Module |
A piece of installed software you switch on with |
Socket |
One physical processor chip in a node. A node can have more than one. |
Core |
One independent processing unit in a processor. |
Hyperthread |
Half of a core that the system shows as a separate CPU. Not as fast as a real core. |
Process / thread |
A running program with its own memory / a second line of work inside one process, sharing its memory. |
Task / rank |
One copy of your program started by Slurm. In MPI it is called a rank. |
MPI |
The standard way for processes on different nodes to cooperate by sending messages. |
Speedup / efficiency |
How many times faster with N cores / speedup divided by N (100% is perfect). |
Python virtual environment (venv) |
A folder holding its own Python packages, switched on with |
Data-loader worker |
A process that prepares training data on the CPU while the GPU trains. |
By the end you can… |
Read |
|---|---|
Submit, monitor and cancel interactive and batch jobs, and request CPUs, memory and time |
|
Load software with modules, manage Conda environments, and run containers |
|
Move data to and from CloudCIX Object Storage |
|
Tell real cores from hyperthreads, use every core you asked for, run job arrays and MPI jobs, measure scaling, and check whether a job was efficient |
|
Size a GPU job so the GPU stays busy without holding a whole node |
|
Run a real MPI application, OpenFOAM, in parallel, and judge how far it scales |
|
Size Jupyter, VS Code and other interactive sessions, and end them when done |
|
Turn your own work into a job script from a template |
|
Practise all of the above, step by step |
The core of this guide is getting the most out of the CPUs and memory you request. Asking for more cores never makes a program faster by itself. These pages show how to make your work actually use them, and how to check that it does.
Not covered: writing or optimising numerical code, GPU programming itself (CUDA, or how to use PyTorch), compiled languages such as C, C++ or Fortran (the ideas carry over, but the examples are Python), and managing or administering the cluster.
Boole HPC Guides