> For the complete documentation index, see [llms.txt](https://docs.aisl.science/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.aisl.science/learning-and-resources/training-apertus/unsloth-on-cscs-clariden.md).

# Unsloth on CSCS Clariden

This page documents the **currently validated interim setup** for running Unsloth-based QLoRA/SFT workloads on **CSCS Clariden**. The approach reuses a CSCS-provided PyTorch uenv and layers the remaining Python dependencies into a persistent virtual environment, so no custom container image is required.

{% hint style="info" %}
**Validated on 7 September 2026:** The stack below successfully completed NF4 quantization, gated Hugging Face access, 4-bit Apertus loading and inference, LoRA adapter attachment, backward propagation, and one optimizer step with completion-only SFT on an NVIDIA GH200 120 GB GPU.
{% endhint %}

## Validated stack

| Component    | Tested version / configuration |
| ------------ | ------------------------------ |
| CSCS uenv    | `pytorch/v2.9.1:v2`            |
| Architecture | `aarch64`                      |
| Python       | `3.12.12`                      |
| PyTorch      | `2.9.1`                        |
| CUDA Toolkit | `12.9`                         |
| GPU          | NVIDIA GH200 120 GB            |
| Triton       | `3.5.1`                        |
| Unsloth      | `2026.9.2`                     |
| Unsloth Zoo  | `2026.9.1`                     |
| bitsandbytes | `0.50.2`                       |
| torchao      | `0.15.0`                       |
| Transformers | `5.5.0`                        |
| TRL          | `0.24.0`                       |
| PEFT         | `0.20.0`                       |
| Datasets     | `4.3.0`                        |
| Accelerate   | `1.14.0`                       |

{% hint style="warning" %}
This is a **tested version combination, not a general compatibility guarantee**. Revalidate the stack before changing the PyTorch uenv or upgrading major dependencies.
{% endhint %}

## Environment setup

The tested approach keeps PyTorch in the CSCS uenv and installs only the additional ML libraries into a persistent venv.

{% stepper %}
{% step %}

## Pull the PyTorch uenv

```bash
uenv image pull pytorch/v2.9.1:v2
```

{% endstep %}

{% step %}

## Start a short compute session

Replace `<PROJECT>` with your allocated Slurm account:

```bash
srun \
  --account=<PROJECT> \
  --partition=debug \
  --time=00:30:00 \
  --uenv=pytorch/v2.9.1:v2 \
  --view=default \
  --pty bash
```

Do not install or run training workloads directly on the login node.
{% endstep %}

{% step %}

## Create the persistent venv

```bash
mkdir -p "$SCRATCH/venvs"
python -m venv --system-site-packages "$SCRATCH/venvs/apertus-sft"
source "$SCRATCH/venvs/apertus-sft/bin/activate"
python -m pip install -U pip
```

`--system-site-packages` lets the venv reuse the PyTorch stack supplied by the CSCS uenv instead of installing a second PyTorch distribution.
{% endstep %}

{% step %}

## Install Unsloth and fix the tested torchao compatibility issue

The sequence used for the validated environment was:

```bash
python -m pip install "bitsandbytes==0.50.2"
python -m pip install "unsloth==2026.9.2"

python -m pip uninstall -y torchao
python -m pip install --no-deps "torchao==0.15.0"

python -m pip check
```

The Unsloth resolver installed the remaining libraries shown in the version table above.

{% hint style="warning" %}
**Keep `torchao==0.15.0` pinned with PyTorch 2.9.1.** During testing, the initial Unsloth install selected `torchao 0.18.0`; that failed against PyTorch 2.9.1 with an import error around `torch.nn.functional.ScalingType`. Pinning `torchao==0.15.0` resolved the issue.
{% endhint %}

To record the local venv packages after a successful setup:

```bash
python -m pip list --local --format=freeze > "$HOME/apertus-sft-2026-09-07.txt"
```

{% endstep %}
{% endstepper %}

## bitsandbytes and CUDA 12.9

With this stack, bitsandbytes reports:

```
No prebuilt binary for CUDA 12.9, loading CUDA 12.8 instead.
```

This warning was **not a blocker**. An actual NF4 quantize/dequantize operation completed successfully on the GH200 with `bitsandbytes 0.50.2`.

## Hugging Face authentication

The text-focused Apertus checkpoint used in the AISL workflow is gated. Each authorized user must authenticate before downloading it:

```bash
hf auth login
hf auth whoami
```

Never place Hugging Face tokens directly in notebooks, scripts, repositories, screenshots, or shared chats.

## Validated Apertus inference

The following model was successfully loaded in 4-bit mode with Unsloth:

[`andreasmartin/apertus-v1.5-8b-text`](https://huggingface.co/andreasmartin/apertus-v1.5-8b-text)

The test covered model download, native Apertus chat-template rendering, 4-bit loading, and text generation on a GH200.

{% hint style="info" %}
Import `unsloth` **before** `transformers`, `trl`, or `peft` so Unsloth can apply its patches before those libraries are initialized.
{% endhint %}

## Validated QLoRA/SFT smoke test

A one-step training smoke test completed successfully with:

* 4-bit Apertus loading;
* one visible GH200 GPU;
* LoRA adapters on attention and MLP projection layers;
* prompt/completion examples using the native Apertus chat template;
* `completion_only_loss=True`;
* BF16 training;
* forward pass, backward pass, gradient computation, and one optimizer step.

The smoke test attached approximately **19.9 million trainable LoRA parameters**, about **0.25%** of the 8B model parameters.

{% hint style="warning" %}
The synthetic one-step smoke test proves that the **training stack works**. It does not demonstrate model improvement. Real SFT experiments should establish a base-model baseline and compare the adapted model against the same held-out evaluation set.
{% endhint %}

## Single-GPU testing

During the smoke test, the process was restricted to one visible GPU with:

```bash
CUDA_VISIBLE_DEVICES=0 python train.py
```

This avoids accidental multi-GPU model placement during small 8B QLoRA tests. It does **not** change the resources allocated by Slurm or JupyterHub.

## Execution paths

The persistent environment can potentially be reused from different Clariden access paths:

* **SSH + Slurm** — validated for the current smoke tests and suitable for reproducible batch training;
* **JupyterLab** — attractive for interactive notebook-based teaching and experimentation;
* **FirecREST** — potentially useful for submitting training jobs programmatically from another notebook or application.

{% hint style="info" %}
At this stage, the **Unsloth stack itself has been validated through Slurm on Clariden**. Reusing the same environment from JupyterLab and automating the workflow through FirecREST are logical next validation steps, but should not yet be treated as validated paths.
{% endhint %}

## Known non-fatal warnings

The tested stack currently emits some warnings that did not block inference or QLoRA/SFT:

* bitsandbytes uses its CUDA 12.8 binary with the CSCS CUDA 12.9 PyTorch stack;
* the optional CUDA-fused xIELU extension is not installed, so Unsloth uses a fallback implementation;
* `HF_HUB_ENABLE_HF_TRANSFER` is deprecated in current Hugging Face Hub tooling;
* `PYTORCH_CUDA_ALLOC_CONF` is deprecated in favour of `PYTORCH_ALLOC_CONF`;
* generation should explicitly pass an attention mask when the pad token and EOS token are identical.

## Related resources

* [Training Apertus](https://docs.aisl.science/docs/learning-and-resources/training-apertus)
* [CSCS Clariden](https://docs.aisl.science/docs/external-tools-and-services/cscs-clariden)
* [Apertus 1.5 SFT student notebook on Kaggle](https://www.kaggle.com/code/andreasmartinch/sft-student-template-apertus-1-5-8b)
* [Apertus 1.5 8B text-focused conversion](https://huggingface.co/andreasmartin/apertus-v1.5-8b-text)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.aisl.science/learning-and-resources/training-apertus/unsloth-on-cscs-clariden.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.