Boole HPC Platform Guide

Introduction

The Boole High Performance Computing (HPC) Platform is operated by CloudCIX. It provides compute resources for interactive data exploration, batch processing, parallel workloads, GPU jobs, containers, and object storage access.

This guide takes you from your first job to running across many nodes. It is hands-on: most topics have a job script you submit on Boole, an output file you read, and an exercise where you change something and see what happens. All sample output comes from real runs on Boole.

It is written for researchers and students who can run a Python script on their own computer and now want to run it on HPC. You need basic shell commands (cd, ls, cat). You don’t need to know Slurm or MPI beforehand. The examples are in Python, but the ideas apply to any language.

Create an HPC Account

Before accessing Boole HPC, create an HPC account in CloudCIX:

  1. In the Membership app, open My Membership and select HPC Accounts.

  2. Click Create HPC Account.

    The HPC Accounts option in the My Membership menu
  3. Enter a username and password, then click Save.

    The New HPC Account form with username and password fields

You can then sign in to hpc.cloudcix.com using these credentials.

Video

We recommend watching the short Boole HPC Basics video for an overview of the platform:

Access Methods

The main access point is the Boole HPC Portal (Open OnDemand):

From the portal, you can upload data, launch jobs, and start interactive applications. Available apps include:

Application

Use

Boole Shell Access

Browser-based terminal access to the cluster.

Remote Desktop

Full remote desktop environment for GUI-based workflows.

CloudCIX AI Lab

AI and machine learning workspace.

Jupyter Notebook

Python, data analysis, and scientific computing.

Jupyter + Spark

Jupyter environment with Spark for distributed data processing.

RStudio Server

Browser-based R environment for statistics, analysis, and visualization. Bioconductor is pre-installed.

VS Code Server

Browser-based Visual Studio Code development environment.

ParaView

Visualization and analysis for scientific datasets.

nf-core pipelines

Bioinformatics workflow pipelines.

Each interactive app runs as a Slurm job that holds the cores, memory and GPU you choose until you end it. See Interactive Apps in Open OnDemand: Ask for What You Use for what to request.

You can also connect directly to the HPC system using SSH or transfer files via rsync from HEAnet & CloudCIX IP addresses only.

Get the Example Files

Every example script and job template in this guide can be downloaded in one go. Open a terminal on Boole, for example through Boole Shell Access, and run:

cd ~
curl -sL https://docs.cloudcix.com/hpc_examples.tar.gz | tar xz
cd ~/hpc_examples

This creates ~/hpc_examples with two folders: examples/ (the scripts used in these pages) and templates/ (job scripts to copy for your own work). Run the commands in this guide from ~/hpc_examples; output files appear there.

To get the latest version later, run the same command again. It overwrites the example files, so copy any you have changed to another folder first.

Each script is also shown on the page that uses it, with its own download link.

Words You Will Meet

Word

Meaning

Login node

The machine your terminal opens on. Shared by everyone; for editing and submitting, not heavy work.

Compute node

A machine that runs jobs.

Slurm

The scheduler that queues jobs and gives them compute nodes.

Partition

A queue for one type of node: cloud, physical or physical-gpu.

Job / job ID

One request to Slurm, and the number sbatch gives it.

Module

A piece of installed software you switch on with module load.

Socket

One physical processor chip in a node. A node can have more than one.

Core

One independent processing unit in a processor.

Hyperthread

Half of a core that the system shows as a separate CPU. Not as fast as a real core.

Process / thread

A running program with its own memory / a second line of work inside one process, sharing its memory.

Task / rank

One copy of your program started by Slurm. In MPI it is called a rank.

MPI

The standard way for processes on different nodes to cooperate by sending messages.

Speedup / efficiency

How many times faster with N cores / speedup divided by N (100% is perfect).

Python virtual environment (venv)

A folder holding its own Python packages, switched on with source <folder>/bin/activate.

Data-loader worker

A process that prepares training data on the CPU while the GPU trains.

What This Guide Covers

By the end you can…

Read

Submit, monitor and cancel interactive and batch jobs, and request CPUs, memory and time

Slurm Basics

Load software with modules, manage Conda environments, and run containers

Software: Modules, Conda and Containers

Move data to and from CloudCIX Object Storage

Accessing Object Storage from the Boole HPC Platform

Tell real cores from hyperthreads, use every core you asked for, run job arrays and MPI jobs, measure scaling, and check whether a job was efficient

Parallel Computing on Boole: Cores, Nodes and MPI

Size a GPU job so the GPU stays busy without holding a whole node

Getting the Most from Boole’s GPUs

Run a real MPI application, OpenFOAM, in parallel, and judge how far it scales

Running OpenFOAM in Parallel on Boole

Size Jupyter, VS Code and other interactive sessions, and end them when done

Interactive Apps in Open OnDemand: Ask for What You Use

Turn your own work into a job script from a template

Job Script Templates

Practise all of the above, step by step

Hands-on Exercises

The core of this guide is getting the most out of the CPUs and memory you request. Asking for more cores never makes a program faster by itself. These pages show how to make your work actually use them, and how to check that it does.

Not covered: writing or optimising numerical code, GPU programming itself (CUDA, or how to use PyTorch), compiled languages such as C, C++ or Fortran (the ideas carry over, but the examples are Python), and managing or administering the cluster.