Tycho - User contributions [en]

FAQs

2023-11-15T16:14:05Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the [http://en.wikipedia.org/wiki/RSA_(cryptosystem)#readme RSA] or [http://en.wikipedia.org/wiki/Digital_Signature_Algorithm# readme DSA] encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with [http://en.wikipedia.org/wiki/Ssh-keygen#readme ssh-keygen] on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system ([http://en.wikipedia.org/wiki/SSHFS#readme SSHFS]). To use SSHFS you need to install [http://en.wikipedia.org/wiki/Filesystem_in_Userspace#readme FUSE]. For Linux you can install [http://fuse.sourceforge.net/#readme fuse] and for Mac there is [https://osxfuse.github.io/#readem osxfuse]. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

ENVIRONMENTS

* What are 'modules' and how do I use them?
* What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

Use the .sh for bash and .csh for tcsh.
To undo the default and start from scratch, use

<pre>
module purge
</pre>

SOFTWARE

* What software do I have available?
* Do I have access
* What is SLURM?
* Can I submit SLURM jobs from anywhere?
* How do I find out which queues I can submit to?

STORAGE

* Where should I put data from large simulations?
* How much disk space can I claim?
* How long can I have stuff on disk?
* Are my data backed up, and where?

FAQs

2023-11-15T15:49:59Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

ENVIRONMENTS

* What are 'modules' and how do I use them?
* What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

Use the .sh for bash and .csh for tcsh.
To undo the default and start from scratch, use

<pre>
module purge
</pre>

SOFTWARE

* What software do I have available?
* Do I have access
* What is SLURM?
* Can I submit SLURM jobs from anywhere?
* How do I find out which queues I can submit to?

STORAGE

* Where should I put data from large simulations?
* How much disk space can I claim?
* How long can I have stuff on disk?
* Are my data backed up, and where?

FAQs

2023-11-15T15:47:58Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
* What are 'modules' and how do I use them?
* What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

Use the .sh for bash and .csh for tcsh.
To undo the default and start from scratch, use

<pre>
module purge
</pre>

===SOFTWARE===

* What software do I have available?
* Do I have access
* What is SLURM?
* Can I submit SLURM jobs from anywhere?
* How do I find out which queues I can submit to?

===STORAGE===

* Where should I put data from large simulations?
* How much disk space can I claim?
* How long can I have stuff on disk?
* Are my data backed up, and where?

FAQs

2023-11-15T15:47:27Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
- What are 'modules' and how do I use them?
- What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

Use the .sh for bash and .csh for tcsh.
To undo the default and start from scratch, use

<pre>
module purge
</pre>

===SOFTWARE===

* What software do I have available?
* Do I have access
* What is SLURM?
* Can I submit SLURM jobs from anywhere?
* How do I find out which queues I can submit to?

===STORAGE===

* Where should I put data from large simulations?
* How much disk space can I claim?
* How long can I have stuff on disk?
* Are my data backed up, and where?

FAQs

2023-11-15T15:46:32Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
- What are 'modules' and how do I use them?
- What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

Use the .sh for bash and .csh for tcsh
To undo the default and start from scratch, use

<pre>
module purge
</pre>

FAQs

2023-11-15T15:46:19Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
- What are 'modules' and how do I use them?
- What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

# Use the .sh for bash and .csh for tcsh
To undo the default and start from scratch, use

<pre>
module purge
</pre>

FAQs

2023-11-15T15:45:51Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
- What are 'modules' and how do I use them?
- What is my default environment setup?
- MPI
- libraries
- compilers
-...

* Is there a standard module to load which lays out everything - no sweat?
Yes, several standard modules are loaded by default via the same

<pre>
source
</pre>

command that makes the module command available:

<pre>
source /software/astro/startup.{sh,csh}
</pre>

# Use the .sh for bash and .csh for tcsh
To undo the default and start from scratch, use

<pre>
module purge
</pre>

FAQs

2023-11-15T15:43:54Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

===ENVIRONMENTS===
# What are 'modules' and how do I use them?
# What is my default environment setup?
- MPI
- libraries
- compilers
-...

FAQs

2023-11-15T15:43:09Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''
# login nodes are unreachable: mailto:support@hpc.ku.dk
# login nodes are alive, but I don't have a homedir: mailto:support@hpc.ku.dk
# I don't have access to the software I need: mailto:support@hpc.ku.dk
# forgot my password: mailto:support@hpc.ku.dk
* Is there a sharing policy?
# cluster queues
# disk space
# analysis servers
* Can I run codes interactively, fx on a front end, or on an analysis server?

FAQs

2023-11-15T15:42:38Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

===GENERAL & SUPPORT===
'''Whom should I ask which questions?'''

FAQs

2023-11-15T15:42:04Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

and then start the tunnel with

<pre>
ssh tunnel -fN
</pre>

FAQs

2023-11-15T15:41:14Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk
LocalForward 5911
localhost:5911
</pre>

FAQs

2023-11-15T15:40:47Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

SSH tunneling: To create a tunnel for for example display :11 on astro06 using ssh, do

<pre>
ssh astro06 -L 5911:localhost:5911
</pre>

You can do the same without starting a remote shell by doing

<pre>
ssh astro06 -L 5911:localhost:5911 -fN
</pre>

If you succeed in always using display :11 you can add the tunnel configuration as part of the ~/.ssh/config file:

<pre>
Host tunnel
User username
HostName astro06.hpc.ku.dk LocalForward 5911 localhost:5911
</pre>

FAQs

2023-11-15T15:38:26Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will arrive at your home directory /groups/astro/username. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

You can cut this down to

<pre>
ssh astro06
</pre>

by adding these lines to the file ~/.ssh/config:

<pre>
Host astro06
User username
HostName astro06.hpc.ku.dk
</pre>

Do not use the ssh option -Y, which "enables trusted X11 forwarding". This means, basically, that you turn off some essential X security features and say "I trust the remote host completely". Forwarding of X should work without any extra options.

'''How can I setup a passwordless ssh?'''

You can avoid to type in your password each time for the authenticating of SSH sessions, by using an SSH key with the RSA or DSA encryption, in combination with ssh-add and ssh-agent. Therefore, you generate with ssh-keygen on your local laptop/computer a pair of a personal (id_rsa) and public key (id_rsa.pub).

<pre>
ssh-keygen -t rsa
</pre>

With the option -t you specify the encryption type. Here we use RSA, but you can also use DSA. You will asked for the filename, you can just press enter and use the default. Then, you will be asked for a passphrase to protect your personal key.
You should NOT under any circumstances use an empty pass phrase; it is not necessary for convenience reasons (see below), and could endanger your access to remote supercomputers. Use instead really a "pass phrase"; a relatively long expression, which need not have the character of a password (such as "I really love Putin" or something equally ridiculous ;-).
There will be stored two keys (id_rsa and id_rsa.pub) in the hidden folder ~/.ssh/ on your client (laptop or remote host you want to be a client of another host). You need to copy your public key to the accepted key list of on your host machine (authorized_keys).

<pre>
cat ~/.ssh/id_rsa.pub | ssh username@astro06.hpc.ku.dk 'cat >> .ssh/authorized_keys'
</pre>

When you connect now, the server will request your pass phrase from the client (the one where you issued the ssh-keygen command). However, rather than wait for the ssh prompt, you should use "ssh-add" to give the pass phrase, once and for all (e.g. just after rebooting of you client).
If your ssh command is issued in a window inside a VNC session, or in a window on your local laptop, chances are you can just type "ssh-add" to store the password in a process called ssh-agent, which typically is already running (most X-sessions -- such as the ones you start under VNC -- are automatically started as child processes of an ssh-agent).

If an ssh-agent process is already running (and your ssh command is a descendant of it), it stores the credentials created by your once-per-reboot ssh-add command and automatically answers requests from hosts you try to connect to. If for some reason no ssh-agent is running on an intermediate host, then as an alternative you can forward your ssh-credentials from your laptop, by using "ssh -A" to login to astro0X, and continuing an ssh from there (having stored the id_rsa.pub from your laptop on the remote supercomputer).
Finally, if for some reason you start out from a place (such a just a command window) where no ssh-agent is running, you can just start one, by doing

<pre>
ssh-agent bash
</pre>

This starts an ssh-agent, which starts bash (or tcsh) as a descendant. Then execute "ssh-add", and you're free from typing the pass phrase, for as long as you keep the ssh-agent running.

'''How do I get my IP white-listed'''

On any of the astro0X hosts, use the command

<pre>
hpc-setup-firewall.sh
</pre>

You have three personal slots, but other people's slots also work for you. The first time you login from home, or a new place, you may need to login via muon.nbi.dk, or some other NBI host, from which astro0X is already open. To find your IP-number, either use one of the web-services (but beware of spam-ware), or just type "finger -m $user" after logging in to an NBI host.
SSHFS.

You can locally access remote data on the frontend very conveniently by the SSH file system (SSHFS). To use SSHFS you need to install FUSE. For Linux you can install fuse and for Mac there is osxfuse. You need to create an empty file as mount point on your local laptop/computer, e.g.

<pre>
makedir ~/nbi
</pre>

Then you mount your by specifying the host and the mount point.

<pre>
sshfs username@astro06.hpc.ku.dk:/astro/username/ ~/nbi/ -oauto_cache,reconnect,volname=nbi
</pre>

You can unmount the filesystem with

<pre>
fusermount -u ~/nbi
</pre>

FAQs

2023-11-15T15:37:33Z

David.oneill:

FAQs

2023-11-15T15:36:50Z

David.oneill:

FAQs

2023-11-15T15:35:34Z

David.oneill:

FAQs

2023-11-15T15:32:36Z

David.oneill:

FAQs

2023-11-15T15:31:29Z

David.oneill:

FAQs

2023-11-15T15:31:10Z

David.oneill:

FAQs

2023-11-15T14:53:24Z

David.oneill:

FAQs

2023-11-15T14:47:08Z

David.oneill:

FAQs

2023-11-15T14:46:00Z

David.oneill:

FAQs

2023-11-15T14:45:18Z

David.oneill:

FAQs

2023-11-15T14:45:03Z

David.oneill:

FAQs

2023-11-15T14:43:59Z

David.oneill:

FAQs

2023-11-15T14:43:23Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

From which you will be prompted to enter your password (Note: If the connection times out, it's likely that your IP address is not recognised). By entering the correct password, you will be arrive at your home directory

FAQs

2023-11-15T14:36:30Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data (see [[Hardware]] for an up to date list of available frontends). It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtual memory).

For example, you can login to the astro06 machine entering the following in the command line

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

FAQs

2023-11-15T14:34:35Z

David.oneill:

'''How do I access the HPC Cluster?'''

You can remotely access the frontend machines astro01-09 via SSH to submit jobs or to analyze data. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtiual memory).

<pre>
ssh username@astro06.hpc.ku.dk
</pre>

Login
You can login to the astro06 machine for example by

FAQs

2023-11-15T14:27:39Z

David.oneill: Created page with "* How do I access the HPC Cluster? Enter the following in the command terminal SSH username@astroXX.hpc.ku.dk You can remotely access the frontend machines astro06-09 via SSH to submit jobs or to analyze data. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtiual memory). Login You can lo..."

* How do I access the HPC Cluster?
Enter the following in the command terminal

SSH username@astroXX.hpc.ku.dk

You can remotely access the frontend machines astro06-09 via SSH to submit jobs or to analyze data. It may be a good idea to check the load factor after logging in, using "top", and choose a different frontend if the CPU or memory use is already high (use "<" or ">" in top to temporarily change from sorting on CPU to memory / virtiual memory).
Login
You can login to the astro06 machine for example by

Tycho Technical Documentation

2023-11-15T14:26:05Z

David.oneill:

Welcome to the technical documentation for the Tycho high performance computing resources. See [https://nbi.ku.dk/english/research_infrastructure/tycho-supercomputer/ the Infrastructure page] at NBI for a non-technical overview of the cluster.

Tycho contain in total more 10,000 CPU cores and 13 data center class GPUs. The cluster is complemented by a 1,300 TB data storage archive and a number of powerful analysis machines used as frontends for the cluster and for pre- and post-prcessing. Current peak performance is 443 TFlops from the CPUs and 93 TFlops from the GPUs.

Tycho is hosted at the [http://www.dcsc.ku.dk/: High Performance Computing center] at the faculty of SCIENCE, University of Copenhagen.

===First steps===
Please visit the [[first steps]] page to get started

===Overview===
* [[Acknowledging the use of Tycho in articles and presentations]]
* [[Getting Help]]
* [[Mattermost discussion forum]]
* [[Being a good HPC user]]
* [[Accessing Tycho]]
* [[Using Jupyter notebooks on the frontends]]
* [[Visual Studio Remote Development]]
* [[Virtual Desktop]]
* [[Hardware]]
* [[Erda]]
* [[Using GPUs]]
* [[Compilers]]
* [[Debugging and Profiling]]
* [[MPI Libraries]]
* [[Running batch jobs]]
* [[Examples of SLURM scripts]]
* [[Codes]]
* [[Adding a second IP Address]]
* [[Setting up One-Time-Password Access before travelling]]
* [[FAQs]]

===Scientific Software===

* [[Module system]]
* [[Running Mathematica on compute nodes]]
* [[GRChombo]]

*************************

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/postorius/lists/mediawiki-announce.lists.wikimedia.org/ MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

MPI Libraries

2023-11-15T14:24:25Z

David.oneill: Created page with "Normally, the choice of MPI-libraries does not need to be made explicitly, but is instead chosen by selecting a specific mpi module (see "Software->Modules"). Notice that the MPI module has to match the compiler. The default setup is to use the Intel compiler suite together with the Intel MPI library. With the default setup, the program is automatically linking against the SLURM process manager, and it is easy to get optimal performance when running batch jobs (see "Ru..."

Normally, the choice of MPI-libraries does not need to be made explicitly, but is instead chosen by selecting a specific mpi module (see "Software->Modules"). Notice that the MPI module has to match the compiler.

The default setup is to use the Intel compiler suite together with the Intel MPI library.

With the default setup, the program is automatically linking against the SLURM process manager, and it is easy to get optimal performance when running batch jobs (see "Running Batch Jobs"). It is slightly more complicated to run MPI jobs on the frontend machines (see "Running on frontends").

For historical reasons, many other compiler+MPI library combinations exists, but it is not recommended to use them.

Erda

2023-11-15T14:19:03Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located [https://erda.dk/public/ucph-erda-user-guide.pdf#readme here ]

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install [http://en.wikipedia.org/wiki/Filesystem_in_Userspace#readme FUSE ]
For Linux you can install [http://fuse.sourceforge.net/#readme fuse] and for Mac there is [https://osxfuse.github.io/#readme osxfuse].

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

<pre>
Host io.erda.dk
Hostname io.erda.dk
User username@nbi.ku.dk
Port 22
</pre>

Where username is your primary KU mail alias that is associated with your ERDA account.

For authentication two options exist. You can either specify a password or use a SSH key with RSA encryption.
To do this go to the [http://erda.ku.dk/#readme ERDA website] and go to Settings > SFTP.
Here you can either add your SSH key to the list of authorized keys or enter the password you wish to use.
To learn how to create an SSH key look in the FAQ section. Once this is done we can move on.

You need to create an empty folder as mount point on your local laptop/computer, e.g.

<pre>
mkdir ~/erda
</pre>

In the following commands we assume that you choose the folder name erda. Once the empty folder is created you mount with the following command:

<pre>
sshfs io.erda.dk: erda -o idmap=user -o big_writes -o reconnect
</pre>

You can unmount the filesystem with

<pre>
umount ~/erda
</pre>

On the cluster umount does not work. As a quick and dirty solution, instead kill processes referencing erda in the name to close the mount -- beware that this may have side consequences if you happen to have another process with erda in the name.

<pre>
pkill erda
</pre>

Erda

2023-11-15T14:18:40Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located [https://erda.dk/public/ucph-erda-user-guide.pdf#readme here ]

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install [http://en.wikipedia.org/wiki/Filesystem_in_Userspace#readme FUSE ]
For Linux you can install [http://fuse.sourceforge.net/#readme fuse] and for Mac there is [https://osxfuse.github.io/#readme osxfuse].

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

<pre>
Host io.erda.dk
Hostname io.erda.dk
User username@nbi.ku.dk
Port 22
</pre>

Where username is your primary KU mail alias that is associated with your ERDA account.

For authentication two options exist. You can either specify a password or use a SSH key with RSA encryption.
To do this go to the [http://erda.ku.dk/#readme ERDA website] and go to Settings > SFTP.
Here you can either add your SSH key to the list of authorized keys or enter the password you wish to use.
To learn how to create an SSH key look in the FAQ section.

Once this is done we can move on.

You need to create an empty folder as mount point on your local laptop/computer, e.g.

<pre>
mkdir ~/erda
</pre>

In the following commands we assume that you choose the folder name erda. Once the empty folder is created you mount with the following command:

<pre>
sshfs io.erda.dk: erda -o idmap=user -o big_writes -o reconnect
</pre>

You can unmount the filesystem with

<pre>
umount ~/erda
</pre>

On the cluster umount does not work. As a quick and dirty solution, instead kill processes referencing erda in the name to close the mount -- beware that this may have side consequences if you happen to have another process with erda in the name.

<pre>
pkill erda
</pre>

Erda

2023-11-15T14:15:12Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located [https://erda.dk/public/ucph-erda-user-guide.pdf#readme here ]

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install [http://en.wikipedia.org/wiki/Filesystem_in_Userspace#readme FUSE ]
For Linux you can install [http://fuse.sourceforge.net/#readme fuse] and for Mac there is [https://osxfuse.github.io/#readme osxfuse].

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

<pre>
Host io.erda.dk
Hostname io.erda.dk
User username@nbi.ku.dk
Port 22
</pre>

Where username is your primary KU mail alias that is associated with your ERDA account.

For authentication two options exist. You can either specify a password or use a SSH key with RSA encryption.
To do this go to the [http://erda.ku.dk/#readme ERDA website] and go to Settings > SFTP.
Here you can either add your SSH key to the list of authorized keys or enter the password you wish to use.
To learn how to create an SSH key look in the FAQ section.

Once this is done we can move on.

You need to create an empty folder as mount point on your local laptop/computer, e.g.

Erda

2023-11-15T14:13:52Z

David.oneill:

Erda

2023-11-15T14:12:34Z

David.oneill:

Erda

2023-11-15T14:11:34Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located [https://erda.dk/public/ucph-erda-user-guide.pdf#readme here ]

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install [http://en.wikipedia.org/wiki/Filesystem_in_Userspace#readme FUSE ]
For Linux you can install fuse and for Mac there is osxfuse.

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

Erda

2023-11-15T14:10:45Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located [https://erda.dk/public/ucph-erda-user-guide.pdf#readme here ]

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install FUSE (http://en.wikipedia.org/wiki/Filesystem_in_Userspace).
For Linux you can install fuse and for Mac there is osxfuse.

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

Erda

2023-11-15T14:09:16Z

David.oneill:

'''ERDA Is A File System Provided By The University Of Copenhagen:'''
It can be accessed through your webbrowser at http://erda.ku.dk/.
It is very easy to create an account using your KU credentials (Swedish number plate).
The full documentation for ERDA is currently located at "https://erda.dk/public/ucph-erda-user-guide.pdf/"

===Mount ERDA In Dolphin On The Astro Cluster===
You can mount your ERDA home folder in Dolphin (The default file manager on the cluster). To add ERDA to Dolphin you should:
#'''Enable the Places panel in Dolphin'''- To enable the Places panel you either hit F9 or go to the dropdown menu View > Panels and check the Places option.
#'''Add entry'''- Once the Places panels is enabled right-click anywhere inside the panel and select Add entry
#'''Add entry dialog box'''- In the dialog box enter the following path into the Location field: davs://io.erda.dk:443 and optionally enter a name for the entry in the Description field. Click OK
#'''Enter credentials and establish connection'''- Once you click on your ERDA entry in the Places panel a dialog box will appear. In the Username field enter your ERDA username (usually the same as your KUnet mail). In the Password field you must enter your WebDAVS password. You can choose this password under Settings > WebDAVS on the ERDA website

===Import/Export Files And Folders Between Astro Cluster And ERDA===
Import/Export from the astro cluster to ERDA. To setup an import request, log on to http://erda.ku.dk/. From the Files tab select the Manage data transfer icon in the top right. We recommend setting up a transfer key, but one can also use a password instead. '''To set up a transfer key, switch to the Manage transfer keys tab.'''

First choose a name for the key
Click generate key
Copy the key to the host from which you wish to authorize transfer (could be your workstation or the cluster).
Copy the key to ./ssh/authorized_keys on the desired transfer target host.
Once the key is setup switch to the Manage data transfers tab to setup a transfer.

Select Import or Export.
Choose an optional transfer name.
Choose a transfer protocal. In the following we use SFTP.
Enter the host address. For the cluster this could be astro06.hpc.ku.dk. The port should be 22, which should be default
Choose login with key and select the key created earlier.
Choose the source file or directory. In my experience it only works with the complete path, e.g /lustre/astro/XXXX/XXXX
Enter the destination folder on the ERDA drive.
Optionally enter email for notification of completion.
You can restart/redo a previous transfer request (fx for backup purposes) by clicking on the far-right icon of a previous transfer
request in the list on the Manage data transfer site.

===Mount ERDA With SSHFS===
NOTE: This option now works on the cluster with the upgrade to CentOS7.

One can use SSHFS to mount ERDA as an local partition
which can be accessed through both terminal and graphical file managers.
To use SSHFS you need to install FUSE (http://en.wikipedia.org/wiki/Filesystem_in_Userspace).
For Linux you can install fuse and for Mac there is osxfuse.

We recommend adding the ERDA host to your ssh config. To do this add the following information to your ssh config file, usually located in ~/.ssh/config:

Erda

2023-11-15T14:07:38Z

David.oneill:

Erda

2023-11-15T14:07:19Z

David.oneill:

Erda

2023-11-15T14:06:51Z

David.oneill:

Erda

2023-11-15T14:06:36Z

David.oneill:

Erda

2023-11-15T14:06:07Z

David.oneill:

Erda

2023-11-15T14:04:50Z

David.oneill:

Erda

2023-11-15T14:04:23Z

David.oneill:

Erda

2023-11-15T14:03:41Z

David.oneill:

Erda

2023-11-15T14:02:54Z

David.oneill:

Erda

2023-11-15T14:01:59Z

David.oneill: