Commit b1d15b33 authored by Mani Tofigh's avatar Mani Tofigh

deleted: account/README.md

        deleted:    account/account.md
        deleted:    account/accounting.rst
        deleted:    account/login.rst
        deleted:    account/uitquota.rst
        modified:   help/contact.md
        modified:   help/faq.md
        deleted:    help/hpc-qa-sessions.md
        modified:   help/rtfm.png
        deleted:    help/staff.md
        modified:   help/writing-support-requests.md
        modified:   jobs/README.md
        modified:   quickstart/about-star.md
        modified:   quickstart/quickstart.md
parent 3a249a48
---
sort: 2
---
# Account
{% include list.liquid all=true %}
# Getting an account
## Getting started on the machine (account, quota, password)
Before you start using the Star cluster, please read Hofstra University's [Acceptable Use Guidelines](http://www.hofstra.edu/scs/aug).
A general introduction to the Star HPC cluster, research community, and support group be found at [starhpc.hofstra.io](https://starhpc.hofstra.io).
## How to get an account and a CPU quota on Star
To be able to work on the Star cluster, you must have an account and you must have been granted CPU time on the system. There are two ways to achieve this:
### Hofstra Quota
“Local users” (i.e. users from Hofstra or affiliated with Hofstra in some
way) can apply for an account and a quota from Hofstra’s share of Star. If
you want to apply for such a quota follow the instructions here:
* [How to get a local account on Star](huquota.html)
### Research Council Quota
Regional users (including users from Hofstra) may apply for an account and a
CPU quota from the Research Council's share of Star. If you want to
apply for such a quota please use this [regional quota form](https://www.example.com/application/project/).
.. _accounting:
=============================
CPU-hour quota and accounting
=============================
CPU quota
=========
To use the batch system you have to have a cpu quota, either local or
national. For every job you submit we check that you have sufficient
quota to run it and you will get a warning if you do not have sufficient
cpu-hours to run the job. The job will be submitted to queue, but will
not start until you have enough cpu-hours to run it.
Resource charging
=================
We charge for used resources, both cpu and memory.
The accounting system charges for used processor equivalents (PE)
times used walltime, so if you ask for more than (total memory pr. node)/(total cores pr. node)
- in the example below 32GB/20cores = 1.6 GB.
The concept of PE defines a processor equivalent as the resource unit 1 core and 1 unit of memory.
For a node with 2 GB og memory pr core, i.e. 32 GB of memory and 16 cores - 1 PE equals to 1 core and
2GB of memory. Currently there is no common definition on the memory unit, other than the one specified
above.
The best way to explain the concept of PE is by example:
Assume that you have a node with 20 cpu-cores and 32 GB memory::
if you ask for less than 1.6GB memory per core then PE will equal the cpu count.
if you ask for 3.2GB memory per core then PE will be twice the cpu-count.
if you ask for 32 GB memory then PE=20 as you only can run one cpu per compute node.
The available memory and core settings for Stallo are explained here: :ref:`about_stallo`
Inspecting your quota
=====================
You can use the cost command to check how much cpu-hours are left on
your allocation:
::
[user@stallo-1 ~]$ cost
Id Name Amount Reserved Balance CreditLimit Deposited Available Percentage
--- ------- --------- -------- -------- ----------- --------- --------- ----------
272 nnXXXXk 168836.65 96272.00 72564.65 0.00 290000.00 72564.65 58.22
10 nnYYYYk 4246.04 0.00 4246.04 0.00 150000.00 4246.04 2.83
The column meaning is
Amount:
The number of hours available for new jobs.
Reserved:
The number of hours reserved for currently running jobs.
Balance:
Amount - Reserved.
CreditLimit:
Allocated low-pri quota.
Deposited:
Allocated normal quota
Inspecting historic use
-----------------------
You can view the accounting history of your projects using::
gstatement --hours --summarize -s YYYY-MM-DD -e YYYY-MM-DD -p nnXXXXk
for more detail see::
gstatement --man
.. _login:
=============================
Logging in for the first time
=============================
Log in with SSH
===============
An *SSH* client (Secure SHell) is the required tool to connect to Stallo. An *SSH* client provides secure encrypted communications between two hosts over an insecure network.
If you already have *ssh* installed on your UNIX-like system, have a user account and password on a Notur system, login may be as easy as typing
::
ssh <machine name> (for instance: ssh stallo.uit.no)
into a terminal window.
If your user name on the machine differs from your user name on the local machine, use the -l option to specify the user name on the machine to which you connect. For example:
::
ssh <machine name> -l [username]
And if you need X-forwarding (for instance, if you like to run Emacs in it's own window) you must log in like this:
::
ssh -X -Y <machine name>
No matter how you login, you will need to confirm that the connection shall be trusted. The SHA256 key fingerprint of ``stallo.uit.no`` is:
::
SHA256:YJpwZ91X5FNXTc/5SE1j9UR1UAAI4FFWVwNSoWoq6Hc
So you should get precisely this message the first time you login via ssh:
::
The authenticity of host 'stallo.uit.no (129.242.2.68)' can't be established.
RSA key fingerprint is SHA256:YJpwZ91X5FNXTc/5SE1j9UR1UAAI4FFWVwNSoWoq6Hc.
Are you sure you want to continue connecting (yes/no)?
If you see this message with precisely this code, you can continue by typing ``yes`` and pressing *Enter*. If you connect to Stallo for the first time and ssh does *not* show you this key, please contact support@metacenter.no immediately.
Log in with an ssh key
----------------------
To avoid entering your password every time you login and to increase security, you can log in with an ssh keypair. This keypair consists of a private key that you have to store on your computer and a public key that you can store on Stallo. On Linux or OSX simply type:
::
ssh-keygen
and follow the instructions on the screen. Please use a good passphrase. You will have to enter this passphrase the first time you log in after a reboot of the computer, but not anymore afterwards. To copy the public key to Stallo, type:
::
ssh-copy-id <username>@stallo.uit.no
To learn more about ssh keys, have a look at `this <https://wiki.archlinux.org/index.php/SSH_keys>`_ page.
On Windows, you can use PuTTYgen that comes with PuTTY. More information on `ssh.com <https://www.ssh.com/ssh/putty/windows/puttygen>`_.
SSH clients for Windows and Mac
-------------------------------
At the `OpenSSH page <https://www.openssh.com>`_ you will find several *SSH* alternatives for both Windows and Mac.
Please note that Mac OS X comes with its own implementation of *OpenSSH*, so you don't need to install any third-party software to take advantage of the extra security *SSH* offers. Just open a terminal window and jump in.
Learning more about SSH
-----------------------
To learn more about using SSH, please also consult the `OpenSSH page <https://www.openssh.com>`_ page and take a look at the manual page on your system (*man ssh*).
Obtain a new password
=====================
When you have been granted an account on stallo.uit.no, your username and password is sent to you separately.
The username by email and the password by SMS. The password you receive by SMS is temporally and only valid for 7 days.
The passwd command does not seem to work. My password is reset back to the old one after a while. Why is this happening?
------------------------------------------------------------------------------------------------------------------------
The Stallo system is using a centralised database for user management.
This will override the password changes done locally on Stallo.
The password can be changed `here <https://www.metacenter.no/user/password/>`_, log in using your
username on stallo and the NOTUR domain.
Logging on the compute nodes
============================
Information on how to log in on a compute node.
Some times you may want to log on a compute node (for instance to check
out output files on the local work area on the node), and this is also
done by using SSH. From stallo.uit.no you can log in to
compute-x-y the following way:
::
ssh -Y compute-x-y (for instance: ssh compute-5-8)
or short
::
ssh -Y cx-y (for instance: ssh c5-8)
If you don't need display forwarding you can omit the "-Y" option
above.
If you for instance want to run a program interactively on a compute
node, and with display forwarding to your desktop you should in stead do
something like this:
#. first login to Stallo with display forwarding,
#. then you should reserve a node through the
queuing system
Below is an example on how you can do this:
::
1) Log in on Stallo with display forwarding.
$ ssh -Y stallo.uit.no
2) Reserve and log in on a compute node with display forwarding.
(start an interactive job.)
$ srun -N 1 -t 1:0:0 --pty bash -i
3) Open a new terminal window, type squeue -j <jobid> (it shows you which node(s) was allocated
to that specific job). Then ssh -Y <nodename> to that node and start your preferred gui.
Graphical logon to Stallo
=========================
If you want to run applications with graphical user interfaces we recommend you to use the
`remote desktop service <http://stallo-gui.uit.no/vnc/>`_
on Stallo.
If you have a new account and you have never logged in on Stallo before, first log in with a classical ssh connection (see above). Afterwards you can use the graphical logon. Otherwise it can happen that your /home will not be created and you will get an error message of the type: "Could not chdir to home directory /home/your_user_name: No such file or directory"
**Important:**
If you are connecting from outside the networks of UNINETT and partners you need to log into
stallo-gui.uit.no with ssh to verify that you have a valid username and password on the Stallo system.
After a successful login the service will automatically allow you to connect from the ip-address
your client currently has. The connection will be open for at least 5 minutes after you log in.
There is no need to keep the ssh-connection open after you have connected to the remote desktop,
in fact you will be automatically logged off after 5 seconds.
*************************************
How to get a local account on Stallo
*************************************
To get a local account on Stallo, you need to provide us with:
--------------------------------------------------------------
* Your full name, date of birth, and nationality.
* Your position (master student, PhD, PostDoc, staff member, visitor/guest).
* Your mobile phone number. This is necessary for recovery of passwords.
* Your institutional mail address (i.e. your work email at the research institution to which you belong)
* The name and address of the instruction you belong to; also including name of the center, institute etc.
* A preferred username. Note: If you already have a Notur user account name, please enter that one. A username is defined as a sequence of two to thirty lowercase alphanumeric characters where the first letter may only be a lowercase letter.
* Necessary additional group and account memberships.
* Optional: If you know in advance, please let us know: how many CPU hours you expect to use, how much long-term storage space (GB) you will need, and what software you will use. Partial answers are also welcome. The more we know about the needs of our users, the better services we can provide and the better we can plan for the future.
**If you are a staff member and need to get a local project,** we need information about the project:
* Name of the project
* Brief description of the project
* Field of science for the project
* Name of additonal members of the project
**If you are a student, PhD or post-doc,** you need to also provide us with the name of your advisor and name of the project you are to be a member of.
Compile this list in a proper manner and send to support@metacenter.no with a comment that the application is for a local account on Stallo.
---
sort: 3
---
# Contact
# Contact Us
If you need help, please file a support request via support@starhpc.hofstra.io,
If you need help, please file a support request via <a href="#"> support@starhpc.hofstra.io</a>,
and our local team of experts will try to assist you as soon as possible.
![rtfm]({{ site.baseurl }}/help/rtfm.png "rtfm")
Credit: https://xkcd.com/293/
<!-- ![rtfm]({{ site.baseurl }}/help/rtfm.png "rtfm") -->
<!-- ![rtfm]({{ site.baseurl }}/help/rtfm2.png "rtfm2") -->
<!-- ![rtfm]({{ site.baseurl }}/help/rtfm3.png "rtfm3") -->
![rtfm]({{ site.baseurl }}/help/rtfm4.png "rtfm4")
\ No newline at end of file
---
sort: 1
---
# Frequently asked questions
## Passwords
......
# Open Question & Answer Sessions for All Users
## Learn new tricks and ask & discuss questions
Meet the HPC staff, discuss problems and project ideas, give feedback or
suggestions on how to improve services, and get advice for your
projects.
Join us on [Zoom](https://uit.zoom.us/j/65284253551), get yourself a
coffee or tea and have a chat. It doesn’t matter from which institute or
university you are, you are **welcome to join at any time**.
This time we will start with a **short seminar about “Helpful Tools &
Services we offer you might not know about”**. Afterwards we have the
open question and answer session.
We can talk about: - Problems/questions regarding the Star
shutdown and migration - Questions regarding data storage and
management - Help with programming and software management - Help with
project organization and data management - Anything else If you think
you might have a challenging question or topics for us, you can also
send them to us before, so we can come prepared and help you better. If
you have general or specific questions about the event:
<jorn.dietze@uit.no>
## Next event
- **2020-10-13, 13:00 - 15:00**, [online Zoom
meeting](https://uit.zoom.us/j/65284253551)
### Past events
- 2020-10-10, 13:00 - 15:00, online Zoom meeting
- 2020-04-23, 13:00 - 15:00, online Zoom meeting
- 2020-02-19, 10:00 - 12:00, [main kantina](http://bit.ly/36Fhd9y)
- 2019-11-12, 14:00 - 16:00, [main kantina](http://bit.ly/36Fhd9y)
- 2019-09-11, 10:00 - 12:30, MH bygget atrium
### Similar events which serve as inspiration
- <https://scicomp.aalto.fi/news/garage.html>
- <https://openworking.wordpress.com/2019/02/04/coding-problems-just-pop-over/>
help/rtfm.png

41 KB | W: | H:

help/rtfm.png

43.5 KB | W: | H:

help/rtfm.png
help/rtfm.png
help/rtfm.png
help/rtfm.png
  • 2-up
  • Swipe
  • Onion skin
# Support staff
## Our vision
Give our users a competitive advantage in the international science race.
### Goal
Contribute to make our scientists the most efficient users of High Performance
Computing (HPC), by creating the best support environment for any HPC user,
and present them to the most efficient solutions to support their highest
demands in HPC.
### Services
The Star HPC support group works within three areas:
* Technical and operational computing resource support.
* Basic to advanced user support for projects and research groups that
utilize HPC or is expecting to do so in the future.
### Regional resource site
Since 2000 Hofstra has been a Resource Site in the National High Performance
Computing Consortium (NOTUR), a joint mission between the four Norwegian
Universities in Bergen, Oslo, Trondheim and Tromsø. Since December 2014
UNINETT Sigma2 manages the national infrastructure and offers High Performance
Computing and Data Storage.
The HPC group at Hofstra also has a role to play as a regional resource site
within HPC, offering HPC services to other institutions in Northern Norway. A
strategic collaboration between Hofstra and the Norwegian Polar Institute within
HPC has been ongoing since 1998.
## Operations Team
* Alexander Rosenberg
* Mani Tofigh
## The Board
* Edward H. Currie
* Daniel P. Miller
* Adam C. Durst
* Jason D. Williams
* Thomas G. Re
* Oren Segal
* John Ortega
---
sort: 2
---
# How to write good support requests
Writing good support requests is not only good for the support team, it
......
......@@ -2,6 +2,6 @@
sort: 4
---
# Slurm Jobs
# Jobs
source: `{{ page.path }}`
---
sort: 1
---
# About Star
## Overview
Star HPC Cluster is a computing facility designed for a variety of research and computational tasks. Combining advanced computing `nodes` and high-speed `storage systems` with a suite of `software applications`.
Star HPC Cluster is a computing facility designed for a variety of research and computational tasks. Combining advanced computing **nodes** and high-speed **storage systems** with a suite of **software applications**.
SLURM (Simple Linux Utility for Resource Management) is our chosen job scheduler and queueing system that efficiently manages resource allocation, ensuring everyone gets the right amount of resources at the right time.
......@@ -23,24 +27,36 @@ The cluster also supports various software applications tailored to different ne
### Storage System
## Software
## Our vision
Making complex and time-intensive calculations simple and accessible.
### Services
The Star HPC support group works within three areas:
### SLURM
* Technical and operational computing resource support.
* Basic to advanced user support for projects and research groups that utilize HPC or is expecting to do so in the future.
### Singularity/Apptainer
### Python
### Our Goal
### Jupyter
Our heart is set on creating a vibrant community where our High-Performance Computing (HPC) cluster is a beacon of collaboration and discovery. We are wishing to provide a supportive space where researchers and students can express their scientific ideas and explore unchanted areas. We are here to make the complicated world of computational research a shared path of growth, learning, and significant discoveries for the ones that are eager to learn.
### MATLAB
### R
## Operations Team
### OpenMPI
* Alexander Rosenberg
* Mani Tofigh
### Anaconda
### NetCDF
## The Board
### Hadoop/Spark
* Edward H. Currie
* Daniel P. Miller
* Adam C. Durst
* Jason D. Williams
* Thomas G. Re
* Oren Segal
* John Ortega
---
sort: 2
---
# Quick Start Guide
## Account Access
......@@ -10,6 +14,29 @@ Access to the cluster comes with responsibilities and certain privileges. Your a
Users should understand the policies on data privacy and user responsibilities.
## Getting started on the machine (account, quota, password)
Before you start using the Star cluster, please read Hofstra University's [Acceptable Use Guidelines](http://www.hofstra.edu/scs/aug).
A general introduction to the Star HPC cluster, research community, and support group be found at [starhpc.hofstra.io](https://starhpc.hofstra.io).
## How to get an account and a CPU quota on Star
To be able to work on the Star cluster, you must have an account and you must have been granted CPU time on the system. There are two ways to achieve this:
### Hofstra Quota
“Local users” (i.e. users from Hofstra or affiliated with Hofstra in some
way) can apply for an account and a quota from Hofstra’s share of Star. If
you want to apply for such a quota follow the instructions here:
* [How to get a local account on Star](huquota.html)
### Research Council Quota
Regional users (including users from Hofstra) may apply for an account and a
CPU quota from the Research Council's share of Star. If you want to
apply for such a quota please use this [regional quota form](https://www.example.com/application/project/).
## Login node
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment