> For the complete documentation index, see [llms.txt](https://docs.aisl.science/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.aisl.science/learning-and-resources/training-apertus.md).

# Training Apertus

This page is a practical starting point for **training and supervised fine-tuning (SFT) of Apertus 1.5**. The recommended workflow starts with a small, reproducible experiment on Kaggle and can later be moved to larger compute such as CSCS Clariden when access has been allocated.

{% hint style="info" %}
**Start small and establish a baseline first.** Use a small dataset and a short QLoRA/SFT run to verify the complete workflow before increasing training steps, dataset size, context length, or compute resources.
{% endhint %}

## Recommended learning path

{% stepper %}
{% step %}

## Start with the Kaggle notebook

Use the [Apertus 1.5 SFT student notebook](https://www.kaggle.com/code/andreasmartinch/sft-student-template-apertus-1-5-8b) as the primary entry point.

The notebook is designed to make the training workflow visible and reproducible before moving to larger infrastructure.
{% endstep %}

{% step %}

## Use the text-focused Apertus checkpoint

For text-only SFT exercises, use [`andreasmartin/apertus-v1.5-8b-text`](https://huggingface.co/andreasmartin/apertus-v1.5-8b-text).

It is a text-only derivative of the official Apertus 1.5 8B model, repackaged as a standard `ApertusForCausalLM` checkpoint for text-generation tooling and SFT experiments. It is a convenience derivative for teaching and text workflows, not an official Swiss AI Initiative checkpoint.
{% endstep %}

{% step %}

## Fine-tune and compare against the base model

Treat SFT as an experiment rather than only as a training exercise:

1. define a small evaluation set;
2. run the base model as a baseline;
3. fine-tune with a small training subset;
4. evaluate the adapted model on the same held-out examples;
5. compare quality, format adherence, and task-specific metrics.

This makes it possible to determine whether the adaptation actually improved the target behaviour.
{% endstep %}

{% step %}

## Scale only when needed

Once the workflow works on Kaggle, the core training code can be exported to a `.py` file and scheduled on larger compute.

If your course, project, or event provides CSCS access, use [CSCS Clariden](https://docs.aisl.science/docs/external-tools-and-services/cscs-clariden) for general infrastructure access. The currently validated Apertus/Unsloth runtime and QLoRA smoke-test setup is documented separately in [Unsloth on CSCS Clariden](https://docs.aisl.science/docs/learning-and-resources/training-apertus/unsloth-on-cscs-clariden).
{% endstep %}
{% endstepper %}

## Why a text-focused checkpoint?

{% hint style="info" %}
**Apertus 1.5 tooling status — 7 September 2026:** The official [`swiss-ai/Apertus-v1.5-8B`](https://huggingface.co/swiss-ai/Apertus-v1.5-8B) is released with the multimodal `apertus1p5` architecture: image, audio, and text input with text output. Its integration into released Hugging Face Transformers is still in progress; [Transformers PR #47662](https://github.com/huggingface/transformers/pull/47662) remains open as of this date.

The official multimodal architecture is therefore not yet recognized by some downstream tooling such as Unsloth. For **text-only SFT teaching**, [`andreasmartin/apertus-v1.5-8b-text`](https://huggingface.co/andreasmartin/apertus-v1.5-8b-text) keeps the Apertus 1.5 text backbone while exposing it through the widely supported `apertus` / `ApertusForCausalLM` format.
{% endhint %}

## Credentials and secrets

{% hint style="warning" %}
Never place Hugging Face tokens, CSCS credentials, private SSH keys, passwords, or other secrets directly in a notebook, training script, repository, screenshot, or shared chat. Use the secret-management mechanism provided by the execution environment, such as Kaggle Secrets or environment variables.
{% endhint %}

## Related resources

* [Apertus 1.5 SFT student notebook on Kaggle](https://www.kaggle.com/code/andreasmartinch/sft-student-template-apertus-1-5-8b)
* [Apertus 1.5 8B text-focused conversion](https://huggingface.co/andreasmartin/apertus-v1.5-8b-text)
* [Official Apertus 1.5 8B model](https://huggingface.co/swiss-ai/Apertus-v1.5-8B)
* [Unsloth on CSCS Clariden](https://docs.aisl.science/docs/learning-and-resources/training-apertus/unsloth-on-cscs-clariden)
* [Hack Apertus](https://hackapertus.ch/)
* [CSCS Clariden](https://docs.aisl.science/docs/external-tools-and-services/cscs-clariden)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.aisl.science/learning-and-resources/training-apertus.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.