Tycho - User contributions [en]

Visual Studio Remote Development

2023-11-15T14:17:05Z

Ricolo:

[https://github.com/Microsoft/vscode-remote-release#readme Remote development ] is a set of extensions for Visual Studio Code that allows you to interact with a remote server on VSCode as if it is running on your local computer. For more details, refer to Microsoft's documentation [https://code.visualstudio.com/docs/remote/remote-overview here].

== Enabling and configuring Remote - SSH ==
=== Installing Remote - SSH ===
First you will need to install a SSH client compatible with <code>OpenSSH</code>. Refer to [https://code.visualstudio.com/docs/remote/troubleshooting#_installing-a-supported-ssh-client this page] for more instructions ('''note:''' if you are running macOS, you do not have to do this).

Follow the instructions [https://code.visualstudio.com/docs/editor/extension-marketplace here] for searching and installing extensions on VSCode. Search for and install the extension "Remote - SSH" (or if you are feeling fancy, install "Remote Development" that includes "Remote - SSH" instead). Alternatively, just click [https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh here].

=== Adding a remote host/server ===
After reloading/reopening VSCode, on the activity bar (on the left) there should now be one more item "Remote Explorer". Click on that and a separate tab should pop up. Hover over "SSH" and a "+" symbol should show up. After clicking the "+" symbol, a prompt will show up and ask you to enter SSH connection command as if you are using a terminal app, such as
<pre>
ssh your_username@astro02.hpc.ku.dk
</pre>
Refer to [[Accessing Tycho]] for more explanations.

You should now be able to browse/edit files on the cluster as if it is your local machine by clicking "Open Folder" or using "Explorer" on the activity bar.

You can also create a new terminal session by clicking "Terminal" on the top and then select "New Terminal".

== Installing more extensions on a remote machine for extra features ==

Visual Studio Remote Development

2023-11-15T14:12:37Z

Ricolo: /* Enabling and configuring Remote - SSH */

Visual Studio Remote Development

2023-11-15T14:10:37Z

Ricolo: /* Enabling and configuring Remote - SSH */

[https://github.com/Microsoft/vscode-remote-release#readme Remote development ] is a set of extensions for Visual Studio Code that allows you to interact with a remote server on VSCode as if it is running on your local computer. For more details, refer to Microsoft's documentation [https://code.visualstudio.com/docs/remote/remote-overview here].

== Enabling and configuring Remote - SSH ==
=== Installing Remote - SSH ===
First you will need to install a SSH client compatible with <code>OpenSSH</code>. Refer to [https://code.visualstudio.com/docs/remote/troubleshooting#_installing-a-supported-ssh-client this page] for more instructions ('''note:''' if you are running macOS, you do not have to do this).

Follow the instructions [https://code.visualstudio.com/docs/editor/extension-marketplace here] for searching and installing extensions on VSCode. Search for and install the extension "Remote - SSH" (or if you are feeling fancy, install "Remote Development" that includes "Remote - SSH" instead). Alternatively, just click [https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh here].

=== Adding a remote host/server ===
After reloading/reopening VSCode, on the activity bar (on the left) there should now be one more item "Remote Explorer". Click on that and a separate tab should pop up. Hover over "SSH" and a "+" symbol should show up. After clicking the "+" symbol, a prompt will show up and ask you to enter SSH connection command as if you are using a terminal app, such as
<pre>
ssh your_username@astro02.hpc.ku.dk
</pre>
Refer to [[Accessing Tycho]] for more explanations.

You should now be able to browse/edit files on the cluster as if it is your local machine.

== Installing more extensions on a remote machine for extra features ==
=== for python ===
=== for julia ===
=== for jupyter notebook ===

Visual Studio Remote Development

2023-11-15T14:08:32Z

Ricolo:

[https://github.com/Microsoft/vscode-remote-release#readme Remote development ] is a set of extensions for Visual Studio Code that allows you to interact with a remote server on VSCode as if it is running on your local computer. For more details, refer to Microsoft's documentation [https://code.visualstudio.com/docs/remote/remote-overview here].

== Enabling and configuring Remote - SSH ==
=== Installing Remote - SSH ===
First you will need to install a SSH client compatible with <code>OpenSSH</code>. Refer to [https://code.visualstudio.com/docs/remote/troubleshooting#_installing-a-supported-ssh-client this page] for more instructions ('''note:''' if you are running macOS, you do not have to do this).

Follow the instructions [https://code.visualstudio.com/docs/editor/extension-marketplace here] for searching and installing extensions on VSCode. Search for and install the extension "Remote - SSH" (or if you are feeling fancy, install "Remote Development" that includes "Remote - SSH" instead). Alternatively, just click [https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh here]

=== Adding a remote host/server ===
After reloading/reopening VSCode, on the activity bar (on the left) there should now be one more item "Remote Explorer". Click on that and a separate tab should pop up. Hover over "SSH" and a "+" symbol should show up. After clicking the "+" symbol, a prompt will show up and ask you to enter SSH connection command as if you are using a terminal app, such as
<pre>
ssh your_username@astro02.hpc.ku.dk
</pre>
Refer to [[Accessing Tycho]] for more explanations.

== Installing more extensions on a remote machine for extra features ==
=== for python ===
=== for julia ===
=== for jupyter notebook ===

Visual Studio Remote Development

2023-11-15T13:52:43Z

Ricolo:

Visual Studio Remote Development

2023-11-15T13:47:22Z

Ricolo:

Visual Studio Remote Development

2023-11-15T13:39:05Z

Ricolo: /* Enabling and configuring Remote Development */

Visual Studio Remote Development

2023-11-15T13:37:11Z

Ricolo: /* Enabling and configuring Remote Development */

Visual Studio Remote Development

2023-11-15T13:35:50Z

Ricolo: /* Enabling and configuring Remote Development */

Visual Studio Remote Development

2023-11-15T13:28:27Z

Ricolo:

Visual Studio Remote Development

2023-11-15T13:27:41Z

Ricolo:

Visual Studio Remote Development

2023-11-15T13:18:51Z

Ricolo: Created page with "[https://github.com/Microsoft/vscode-remote-release#readme Remote development ] is a set of extensions for Visual Studio Code that allows you to interact with a remote server on VSCode as if it is running on your local computer. == Installing the Remote Development extension pack == == Configuring Visual Studio Code == == Installing even more extensions for extra features == === for python === === for julia === === for jupyter notebook ==="

[https://github.com/Microsoft/vscode-remote-release#readme Remote development ] is a set of extensions for Visual Studio Code that allows you to interact with a remote server on VSCode as if it is running on your local computer.

== Installing the Remote Development extension pack ==

== Configuring Visual Studio Code ==

== Installing even more extensions for extra features ==
=== for python ===
=== for julia ===
=== for jupyter notebook ===

Using GPUs

2023-11-15T12:53:10Z

Ricolo: /* Installing CUDA */

Here is a detailed guide on how to leverage the GPUs on the NBI cluster.

== Preparation work: making sure that your software is GPU-aware ==

Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+).

=== Installing CUDA ===

Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g. <code>cuda/11.2</code>; note: you will need to first load the <code>astro</code> module) pre-installed on the system.

Here we take a different route -- we install our own (and a newer version of) CUDA for greater control. Usually you would want to install the latest CUDA that your GPUs support, but as of the time of this writing, <code>torch</code> lacks the support for the latest CUDA version 12.x so we opt for an earlier release (11.8).

To install CUDA via <code>conda</code>, do

<pre>
conda install cuda -c nvidia/label/cuda-11.8.0
</pre>

You should check that your installation works by running
<pre>
nvcc --version
</pre>
This should match the version that you just installed.

('''NOTE:''' This number can be different from the number reported in <code>nvidia-smi</code>, since the number in <code>nvidia-smi</code> is the latest CUDA that is supported by the driver installed. In other words, you should make sure that the CUDA you installed is 'at most' that version)

=== Installing torch ===

Once you have CUDA properly installed, everything else should be a breeze. To install <code>torch</code> with CUDA awareness, simply do
<pre>
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</pre>

Test your installation with the following simple code snippet
<pre>
import torch

torch.zeros(100).cuda()
</pre>
If there is no error message, that means you have now installed torch with CUDA support successfully.

=== Installing cupy ===

Again, if you have CUDA installed, the installation of <code>cupy</code> is very straightforward. Simply run
<pre>
conda install -c conda-forge cupy cudnn cutensor nccl
</pre>

Note that <code>conda</code> will intelligently (and hopefully) detect the proper version to installed with your current installation of CUDA.

Test your installation with the following simple code snippet
<pre>
import cupy

cupy.random.rand(100).device
</pre>
It should say something like <code><CUDA Device 0></code>.

=== Installing jax ===

Installation of <code>jax</code> with CUDA is also simple. Run

<pre>
pip install --upgrade "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
</pre>

Test your installation with
<pre>
import jax

jax.devices()
</pre>
It should say something like <code>[cuda(id=0)]</code>

== Running a job directly on a GPU-equipped headnode ==

The GPU-equipped headnode/frontend is <code>astro02</code> (the node is accessible with <code>astro02.hpc.ku.dk</code>). There are physically 3 Nvidia-A30 GPUs. One of them is virutally split into 3 smaller and independent virtual GPUs (in Nvidia's term -- MIG or Multi-Instance GPU), one split into 2 smaller MIGs, and one remains 'unsplit'.

To specify which GPU to use, set the environment variable <code>CUDA_VISIBLE_DEVICES</code>. To see the list of 'compute instances' available, run
<pre>
nvidia-smi -L
</pre>

On <code>astro02</code>, you should see something like
<pre>
GPU 0: NVIDIA A30 (UUID: GPU-654aa619-952d-3f17-01ec-0c050ac8df88)
MIG 1g.6gb Device 0: (UUID: MIG-3868837f-57d0-5089-9887-19240a8809b4)
MIG 1g.6gb Device 1: (UUID: MIG-d28bcf9f-db13-5ad0-9be2-62d0e25c92a9)
MIG 1g.6gb Device 2: (UUID: MIG-e175ec33-0f38-5952-98d5-1c118bd9d398)
MIG 1g.6gb Device 3: (UUID: MIG-53cc4525-2ae7-5c11-9680-302d1d4177ba)
GPU 1: NVIDIA A30 (UUID: GPU-cb8c2438-a361-3e30-4ff5-4481d43c9e83)
MIG 2g.12gb Device 0: (UUID: MIG-0a768004-2ded-55f6-ac2b-4dd3f696a222)
MIG 2g.12gb Device 1: (UUID: MIG-0296d938-ea26-5174-a884-cd3c686bf660)
GPU 2: NVIDIA A30 (UUID: GPU-9bcd54bd-5a72-2e7b-90c8-3e3719d09e5c)
MIG 4g.24gb Device 0: (UUID: MIG-a8cb1bd5-6f68-54a1-8e88-ca2fa4ef80c0)
</pre>

For example, if we want to use the third <code>MIG 1g.6gb</code> instance with the UUID <code>MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>, set the environment variable
<code>export CUDA_VISIBLE_DEVICES=MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>

Then running the same test code for <code>torch</code> and checking with <code>nvidia-smi</code>, we see that
<pre>
+---------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+--------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Memory-Usage | Vol| Shared |
| ID ID Dev | BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+================================+===========+=======================|
| 0 3 0 0 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 4 0 1 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 5 0 2 | 107MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 2MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 6 0 3 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 1 0 0 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 2 0 1 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 2 0 0 0 | 1MiB / 24062MiB | 56 0 | 4 0 4 1 1 |
| | 1MiB / 32768MiB | | |
+------------------+--------------------------------+-----------+-----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 5 0 530009 C ...nda3/envs/igwn-py310/bin/python3.10 88MiB |
+---------------------------------------------------------------------------------------+
</pre>
Indeed we are using the desired MIG.

== Submitting a job to the GPU partition with slurm ==

Simply specify the GPU partition, <code>astro2_gpu</code>, and how many ‘generic resources (GRES)’ (in this case, GPU), that you want to use when submitting a job with <code>slurm</code>.

An example command is
<pre>
srun -p astro2_gpu --gres=gpu:1 nvidia-smi
</pre>
This should show the GPU (not the virtual one/MIG) that is being assigned to you.

As far as I know, there are 11 Nvidia A100 GPUs in this partition.

Using GPUs

2023-11-15T12:52:39Z

Ricolo: /* Installing jax with CUDA awareness */

Here is a detailed guide on how to leverage the GPUs on the NBI cluster.

== Preparation work: making sure that your software is GPU-aware ==

Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+).

=== Installing CUDA ===

Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g. <code>cuda/11.2</code>; note: you will need to first load the <code>astro</code> module) pre-installed on the system.

Here we take a different route -- we install our own (and a newer version of) CUDA for greater control. Usually you would want to install the latest CUDA that your GPUs support, but as of the time of this writing, <code>torch</code> lacks the support for the latest CUDA version 12.x so we opt for an earlier release (11.8).

To install CUDA via <code>conda</code>, do

<pre>
conda install cuda -c nvidia/label/cuda-11.8.0
</pre>

You should check that your installation works by running
<pre>
nvcc --version
</pre>
This should match the version that you just installed.

(*NOTE:* This number can be different from the number reported in <code>nvidia-smi</code>, since the number in <code>nvidia-smi</code> is the latest CUDA that is supported by the driver installed. In other words, you should make sure that the CUDA you installed is 'at most' that version)

=== Installing torch ===

Once you have CUDA properly installed, everything else should be a breeze. To install <code>torch</code> with CUDA awareness, simply do
<pre>
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</pre>

Test your installation with the following simple code snippet
<pre>
import torch

torch.zeros(100).cuda()
</pre>
If there is no error message, that means you have now installed torch with CUDA support successfully.

=== Installing cupy ===

Again, if you have CUDA installed, the installation of <code>cupy</code> is very straightforward. Simply run
<pre>
conda install -c conda-forge cupy cudnn cutensor nccl
</pre>

Note that <code>conda</code> will intelligently (and hopefully) detect the proper version to installed with your current installation of CUDA.

Test your installation with the following simple code snippet
<pre>
import cupy

cupy.random.rand(100).device
</pre>
It should say something like <code><CUDA Device 0></code>.

=== Installing jax ===

Installation of <code>jax</code> with CUDA is also simple. Run

<pre>
pip install --upgrade "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
</pre>

Test your installation with
<pre>
import jax

jax.devices()
</pre>
It should say something like <code>[cuda(id=0)]</code>

== Running a job directly on a GPU-equipped headnode ==

The GPU-equipped headnode/frontend is <code>astro02</code> (the node is accessible with <code>astro02.hpc.ku.dk</code>). There are physically 3 Nvidia-A30 GPUs. One of them is virutally split into 3 smaller and independent virtual GPUs (in Nvidia's term -- MIG or Multi-Instance GPU), one split into 2 smaller MIGs, and one remains 'unsplit'.

To specify which GPU to use, set the environment variable <code>CUDA_VISIBLE_DEVICES</code>. To see the list of 'compute instances' available, run
<pre>
nvidia-smi -L
</pre>

On <code>astro02</code>, you should see something like
<pre>
GPU 0: NVIDIA A30 (UUID: GPU-654aa619-952d-3f17-01ec-0c050ac8df88)
MIG 1g.6gb Device 0: (UUID: MIG-3868837f-57d0-5089-9887-19240a8809b4)
MIG 1g.6gb Device 1: (UUID: MIG-d28bcf9f-db13-5ad0-9be2-62d0e25c92a9)
MIG 1g.6gb Device 2: (UUID: MIG-e175ec33-0f38-5952-98d5-1c118bd9d398)
MIG 1g.6gb Device 3: (UUID: MIG-53cc4525-2ae7-5c11-9680-302d1d4177ba)
GPU 1: NVIDIA A30 (UUID: GPU-cb8c2438-a361-3e30-4ff5-4481d43c9e83)
MIG 2g.12gb Device 0: (UUID: MIG-0a768004-2ded-55f6-ac2b-4dd3f696a222)
MIG 2g.12gb Device 1: (UUID: MIG-0296d938-ea26-5174-a884-cd3c686bf660)
GPU 2: NVIDIA A30 (UUID: GPU-9bcd54bd-5a72-2e7b-90c8-3e3719d09e5c)
MIG 4g.24gb Device 0: (UUID: MIG-a8cb1bd5-6f68-54a1-8e88-ca2fa4ef80c0)
</pre>

For example, if we want to use the third <code>MIG 1g.6gb</code> instance with the UUID <code>MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>, set the environment variable
<code>export CUDA_VISIBLE_DEVICES=MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>

Then running the same test code for <code>torch</code> and checking with <code>nvidia-smi</code>, we see that
<pre>
+---------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+--------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Memory-Usage | Vol| Shared |
| ID ID Dev | BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+================================+===========+=======================|
| 0 3 0 0 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 4 0 1 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 5 0 2 | 107MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 2MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 6 0 3 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 1 0 0 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 2 0 1 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 2 0 0 0 | 1MiB / 24062MiB | 56 0 | 4 0 4 1 1 |
| | 1MiB / 32768MiB | | |
+------------------+--------------------------------+-----------+-----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 5 0 530009 C ...nda3/envs/igwn-py310/bin/python3.10 88MiB |
+---------------------------------------------------------------------------------------+
</pre>
Indeed we are using the desired MIG.

== Submitting a job to the GPU partition with slurm ==

Simply specify the GPU partition, <code>astro2_gpu</code>, and how many ‘generic resources (GRES)’ (in this case, GPU), that you want to use when submitting a job with <code>slurm</code>.

An example command is
<pre>
srun -p astro2_gpu --gres=gpu:1 nvidia-smi
</pre>
This should show the GPU (not the virtual one/MIG) that is being assigned to you.

As far as I know, there are 11 Nvidia A100 GPUs in this partition.

Using GPUs

2023-11-15T12:51:18Z

Ricolo: /* Preparation work: making sure that your software is GPU-aware */

Here is a detailed guide on how to leverage the GPUs on the NBI cluster.

== Preparation work: making sure that your software is GPU-aware ==

Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+).

=== Installing CUDA ===

Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g. <code>cuda/11.2</code>; note: you will need to first load the <code>astro</code> module) pre-installed on the system.

Here we take a different route -- we install our own (and a newer version of) CUDA for greater control. Usually you would want to install the latest CUDA that your GPUs support, but as of the time of this writing, <code>torch</code> lacks the support for the latest CUDA version 12.x so we opt for an earlier release (11.8).

To install CUDA via <code>conda</code>, do

<pre>
conda install cuda -c nvidia/label/cuda-11.8.0
</pre>

You should check that your installation works by running
<pre>
nvcc --version
</pre>
This should match the version that you just installed.

(*NOTE:* This number can be different from the number reported in <code>nvidia-smi</code>, since the number in <code>nvidia-smi</code> is the latest CUDA that is supported by the driver installed. In other words, you should make sure that the CUDA you installed is 'at most' that version)

=== Installing torch ===

Once you have CUDA properly installed, everything else should be a breeze. To install <code>torch</code> with CUDA awareness, simply do
<pre>
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</pre>

Test your installation with the following simple code snippet
<pre>
import torch

torch.zeros(100).cuda()
</pre>
If there is no error message, that means you have now installed torch with CUDA support successfully.

=== Installing cupy ===

Again, if you have CUDA installed, the installation of <code>cupy</code> is very straightforward. Simply run
<pre>
conda install -c conda-forge cupy cudnn cutensor nccl
</pre>

Note that <code>conda</code> will intelligently (and hopefully) detect the proper version to installed with your current installation of CUDA.

Test your installation with the following simple code snippet
<pre>
import cupy

cupy.random.rand(100).device
</pre>
It should say something like <code><CUDA Device 0></code>.

=== Installing jax with CUDA awareness ===

Installation of <code>jax</code> with CUDA is also simple. Run

<pre>
pip install --upgrade "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
</pre>

Test your installation with
<pre>
import jax

jax.devices()
</pre>
It should say something like <code>[cuda(id=0)]</code>

== Running a job directly on a GPU-equipped headnode ==

The GPU-equipped headnode/frontend is <code>astro02</code> (the node is accessible with <code>astro02.hpc.ku.dk</code>). There are physically 3 Nvidia-A30 GPUs. One of them is virutally split into 3 smaller and independent virtual GPUs (in Nvidia's term -- MIG or Multi-Instance GPU), one split into 2 smaller MIGs, and one remains 'unsplit'.

To specify which GPU to use, set the environment variable <code>CUDA_VISIBLE_DEVICES</code>. To see the list of 'compute instances' available, run
<pre>
nvidia-smi -L
</pre>

On <code>astro02</code>, you should see something like
<pre>
GPU 0: NVIDIA A30 (UUID: GPU-654aa619-952d-3f17-01ec-0c050ac8df88)
MIG 1g.6gb Device 0: (UUID: MIG-3868837f-57d0-5089-9887-19240a8809b4)
MIG 1g.6gb Device 1: (UUID: MIG-d28bcf9f-db13-5ad0-9be2-62d0e25c92a9)
MIG 1g.6gb Device 2: (UUID: MIG-e175ec33-0f38-5952-98d5-1c118bd9d398)
MIG 1g.6gb Device 3: (UUID: MIG-53cc4525-2ae7-5c11-9680-302d1d4177ba)
GPU 1: NVIDIA A30 (UUID: GPU-cb8c2438-a361-3e30-4ff5-4481d43c9e83)
MIG 2g.12gb Device 0: (UUID: MIG-0a768004-2ded-55f6-ac2b-4dd3f696a222)
MIG 2g.12gb Device 1: (UUID: MIG-0296d938-ea26-5174-a884-cd3c686bf660)
GPU 2: NVIDIA A30 (UUID: GPU-9bcd54bd-5a72-2e7b-90c8-3e3719d09e5c)
MIG 4g.24gb Device 0: (UUID: MIG-a8cb1bd5-6f68-54a1-8e88-ca2fa4ef80c0)
</pre>

For example, if we want to use the third <code>MIG 1g.6gb</code> instance with the UUID <code>MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>, set the environment variable
<code>export CUDA_VISIBLE_DEVICES=MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>

Then running the same test code for <code>torch</code> and checking with <code>nvidia-smi</code>, we see that
<pre>
+---------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+--------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Memory-Usage | Vol| Shared |
| ID ID Dev | BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+================================+===========+=======================|
| 0 3 0 0 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 4 0 1 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 5 0 2 | 107MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 2MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 6 0 3 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 1 0 0 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 2 0 1 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 2 0 0 0 | 1MiB / 24062MiB | 56 0 | 4 0 4 1 1 |
| | 1MiB / 32768MiB | | |
+------------------+--------------------------------+-----------+-----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 5 0 530009 C ...nda3/envs/igwn-py310/bin/python3.10 88MiB |
+---------------------------------------------------------------------------------------+
</pre>
Indeed we are using the desired MIG.

== Submitting a job to the GPU partition with slurm ==

Simply specify the GPU partition, <code>astro2_gpu</code>, and how many ‘generic resources (GRES)’ (in this case, GPU), that you want to use when submitting a job with <code>slurm</code>.

An example command is
<pre>
srun -p astro2_gpu --gres=gpu:1 nvidia-smi
</pre>
This should show the GPU (not the virtual one/MIG) that is being assigned to you.

As far as I know, there are 11 Nvidia A100 GPUs in this partition.

Using GPUs

2023-11-15T12:50:13Z

Ricolo: /* Installing torch */

Here is a detailed guide on how to leverage the GPUs on the NBI cluster.

== Preparation work: making sure that your software is GPU-aware ==

Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+).

=== Installing CUDA ===

Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g. <code>cuda/11.2</code>; note: you will need to first load the <code>astro</code> module) pre-installed on the system.

Here we take a different route -- we install our own (and a newer version of) CUDA for greater control. Usually you would want to install the latest CUDA that your GPUs support, but in my case, <code>torch</code> lacks the support for the lastest CUDA version 12.x so I opt for an earlier release (11.8).

To install CUDA via <code>conda</code>, do

<pre>
conda install cuda -c nvidia/label/cuda-11.8.0
</pre>

You should check that your installation works by running
<pre>
nvcc --version
</pre>
This should match the version that you just installed.

(*NOTE:* This number can be different from the number reported in <code>nvidia-smi</code>, since the number in <code>nvidia-smi</code> is the latest CUDA that is supported by the driver installed. In other words, you should make sure that the CUDA you installed is 'at most' that version)

=== Installing torch ===

Once you have CUDA properly installed, everything else should be a breeze. To install <code>torch</code> with CUDA awareness, simply do
<pre>
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</pre>

Test your installation with the following simple code snippet
<pre>
import torch

torch.zeros(100).cuda()
</pre>
If there is no error message, that means you have now installed torch with CUDA support successfully.

=== Installing cupy ===

Again, if you have CUDA installed, the installation of <code>cupy</code> is very straightforward. Simply run
<pre>
conda install -c conda-forge cupy cudnn cutensor nccl
</pre>

Note that <code>conda</code> will intelligently (and hopefully) detect the proper version to installed with your current installation of CUDA.

Test your installation with the following simple code snippet
<pre>
import cupy

cupy.random.rand(100).device
</pre>
It should say something like <code><CUDA Device 0></code>.

=== Installing jax with CUDA awareness ===

Installation of <code>jax</code> with CUDA is also simple. Run

<pre>
pip install --upgrade "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
</pre>

Test your installation with
<pre>
import jax

jax.devices()
</pre>
It should say something like <code>[cuda(id=0)]</code>

== Running a job directly on a GPU-equipped headnode ==

The GPU-equipped headnode/frontend is <code>astro02</code> (the node is accessible with <code>astro02.hpc.ku.dk</code>). There are physically 3 Nvidia-A30 GPUs. One of them is virutally split into 3 smaller and independent virtual GPUs (in Nvidia's term -- MIG or Multi-Instance GPU), one split into 2 smaller MIGs, and one remains 'unsplit'.

To specify which GPU to use, set the environment variable <code>CUDA_VISIBLE_DEVICES</code>. To see the list of 'compute instances' available, run
<pre>
nvidia-smi -L
</pre>

On <code>astro02</code>, you should see something like
<pre>
GPU 0: NVIDIA A30 (UUID: GPU-654aa619-952d-3f17-01ec-0c050ac8df88)
MIG 1g.6gb Device 0: (UUID: MIG-3868837f-57d0-5089-9887-19240a8809b4)
MIG 1g.6gb Device 1: (UUID: MIG-d28bcf9f-db13-5ad0-9be2-62d0e25c92a9)
MIG 1g.6gb Device 2: (UUID: MIG-e175ec33-0f38-5952-98d5-1c118bd9d398)
MIG 1g.6gb Device 3: (UUID: MIG-53cc4525-2ae7-5c11-9680-302d1d4177ba)
GPU 1: NVIDIA A30 (UUID: GPU-cb8c2438-a361-3e30-4ff5-4481d43c9e83)
MIG 2g.12gb Device 0: (UUID: MIG-0a768004-2ded-55f6-ac2b-4dd3f696a222)
MIG 2g.12gb Device 1: (UUID: MIG-0296d938-ea26-5174-a884-cd3c686bf660)
GPU 2: NVIDIA A30 (UUID: GPU-9bcd54bd-5a72-2e7b-90c8-3e3719d09e5c)
MIG 4g.24gb Device 0: (UUID: MIG-a8cb1bd5-6f68-54a1-8e88-ca2fa4ef80c0)
</pre>

For example, if we want to use the third <code>MIG 1g.6gb</code> instance with the UUID <code>MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>, set the environment variable
<code>export CUDA_VISIBLE_DEVICES=MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>

Then running the same test code for <code>torch</code> and checking with <code>nvidia-smi</code>, we see that
<pre>
+---------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+--------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Memory-Usage | Vol| Shared |
| ID ID Dev | BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+================================+===========+=======================|
| 0 3 0 0 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 4 0 1 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 5 0 2 | 107MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 2MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 6 0 3 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 1 0 0 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 2 0 1 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 2 0 0 0 | 1MiB / 24062MiB | 56 0 | 4 0 4 1 1 |
| | 1MiB / 32768MiB | | |
+------------------+--------------------------------+-----------+-----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 5 0 530009 C ...nda3/envs/igwn-py310/bin/python3.10 88MiB |
+---------------------------------------------------------------------------------------+
</pre>
Indeed we are using the desired MIG.

== Submitting a job to the GPU partition with slurm ==

Simply specify the GPU partition, <code>astro2_gpu</code>, and how many ‘generic resources (GRES)’ (in this case, GPU), that you want to use when submitting a job with <code>slurm</code>.

An example command is
<pre>
srun -p astro2_gpu --gres=gpu:1 nvidia-smi
</pre>
This should show the GPU (not the virtual one/MIG) that is being assigned to you.

As far as I know, there are 11 Nvidia A100 GPUs in this partition.

Using GPUs

2023-11-15T12:49:12Z

Ricolo: Created page with "Here is a detailed guide on how to leverage the GPUs on the NBI cluster. == Preparation work: making sure that your software is GPU-aware == Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+). === Installing CUDA === Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g...."

Here is a detailed guide on how to leverage the GPUs on the NBI cluster.

== Preparation work: making sure that your software is GPU-aware ==

Before proceeding, it is recommended to have your local installation of (ana-/mini-)conda with newer python version (preferably 3.10+).

=== Installing CUDA ===

Normally, you would/should use the system-wide CUDA installation to make sure that it is compatible with the GPUs. In fact, there are environment modules for CUDA (e.g. <code>cuda/11.2</code>; note: you will need to first load the <code>astro</code> module) pre-installed on the system.

Here we take a different route -- we install our own (and a newer version of) CUDA for greater control. Usually you would want to install the latest CUDA that your GPUs support, but in my case, <code>torch</code> lacks the support for the lastest CUDA version 12.x so I opt for an earlier release (11.8).

To install CUDA via <code>conda</code>, do

<pre>
conda install cuda -c nvidia/label/cuda-11.8.0
</pre>

You should check that your installation works by running
<pre>
nvcc --version
</pre>
This should match the version that you just installed.

(*NOTE:* This number can be different from the number reported in <code>nvidia-smi</code>, since the number in <code>nvidia-smi</code> is the latest CUDA that is supported by the driver installed. In other words, you should make sure that the CUDA you installed is 'at most' that version)

=== Installing torch ===

Once you have CUDA properly installed, everything else should be a breeze. To install `torch` with CUDA awareness, simply do
<pre>
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</pre>

Test your installation with the following simple code snippet
<pre>
import torch

torch.zeros(100).cuda()
</pre>
If there is no error message, that means you have now installed torch with CUDA support successfully.

=== Installing cupy ===

Again, if you have CUDA installed, the installation of <code>cupy</code> is very straightforward. Simply run
<pre>
conda install -c conda-forge cupy cudnn cutensor nccl
</pre>

Note that <code>conda</code> will intelligently (and hopefully) detect the proper version to installed with your current installation of CUDA.

Test your installation with the following simple code snippet
<pre>
import cupy

cupy.random.rand(100).device
</pre>
It should say something like <code><CUDA Device 0></code>.

=== Installing jax with CUDA awareness ===

Installation of <code>jax</code> with CUDA is also simple. Run

<pre>
pip install --upgrade "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
</pre>

Test your installation with
<pre>
import jax

jax.devices()
</pre>
It should say something like <code>[cuda(id=0)]</code>

== Running a job directly on a GPU-equipped headnode ==

The GPU-equipped headnode/frontend is <code>astro02</code> (the node is accessible with <code>astro02.hpc.ku.dk</code>). There are physically 3 Nvidia-A30 GPUs. One of them is virutally split into 3 smaller and independent virtual GPUs (in Nvidia's term -- MIG or Multi-Instance GPU), one split into 2 smaller MIGs, and one remains 'unsplit'.

To specify which GPU to use, set the environment variable <code>CUDA_VISIBLE_DEVICES</code>. To see the list of 'compute instances' available, run
<pre>
nvidia-smi -L
</pre>

On <code>astro02</code>, you should see something like
<pre>
GPU 0: NVIDIA A30 (UUID: GPU-654aa619-952d-3f17-01ec-0c050ac8df88)
MIG 1g.6gb Device 0: (UUID: MIG-3868837f-57d0-5089-9887-19240a8809b4)
MIG 1g.6gb Device 1: (UUID: MIG-d28bcf9f-db13-5ad0-9be2-62d0e25c92a9)
MIG 1g.6gb Device 2: (UUID: MIG-e175ec33-0f38-5952-98d5-1c118bd9d398)
MIG 1g.6gb Device 3: (UUID: MIG-53cc4525-2ae7-5c11-9680-302d1d4177ba)
GPU 1: NVIDIA A30 (UUID: GPU-cb8c2438-a361-3e30-4ff5-4481d43c9e83)
MIG 2g.12gb Device 0: (UUID: MIG-0a768004-2ded-55f6-ac2b-4dd3f696a222)
MIG 2g.12gb Device 1: (UUID: MIG-0296d938-ea26-5174-a884-cd3c686bf660)
GPU 2: NVIDIA A30 (UUID: GPU-9bcd54bd-5a72-2e7b-90c8-3e3719d09e5c)
MIG 4g.24gb Device 0: (UUID: MIG-a8cb1bd5-6f68-54a1-8e88-ca2fa4ef80c0)
</pre>

For example, if we want to use the third <code>MIG 1g.6gb</code> instance with the UUID <code>MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>, set the environment variable
<code>export CUDA_VISIBLE_DEVICES=MIG-e175ec33-0f38-5952-98d5-1c118bd9d398</code>

Then running the same test code for <code>torch</code> and checking with <code>nvidia-smi</code>, we see that
<pre>
+---------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+--------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Memory-Usage | Vol| Shared |
| ID ID Dev | BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+================================+===========+=======================|
| 0 3 0 0 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 4 0 1 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 5 0 2 | 107MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 2MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 0 6 0 3 | 12MiB / 5952MiB | 14 0 | 1 0 1 0 0 |
| | 0MiB / 8191MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 1 0 0 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 1 2 0 1 | 25MiB / 11968MiB | 28 0 | 2 0 2 0 0 |
| | 0MiB / 16383MiB | | |
+------------------+--------------------------------+-----------+-----------------------+
| 2 0 0 0 | 1MiB / 24062MiB | 56 0 | 4 0 4 1 1 |
| | 1MiB / 32768MiB | | |
+------------------+--------------------------------+-----------+-----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 5 0 530009 C ...nda3/envs/igwn-py310/bin/python3.10 88MiB |
+---------------------------------------------------------------------------------------+
</pre>
Indeed we are using the desired MIG.

== Submitting a job to the GPU partition with slurm ==

Simply specify the GPU partition, <code>astro2_gpu</code>, and how many ‘generic resources (GRES)’ (in this case, GPU), that you want to use when submitting a job with <code>slurm</code>.

An example command is
<pre>
srun -p astro2_gpu --gres=gpu:1 nvidia-smi
</pre>
This should show the GPU (not the virtual one/MIG) that is being assigned to you.

As far as I know, there are 11 Nvidia A100 GPUs in this partition.