Computational resources

If you need to run experiments that won’t fit on your desktop/laptop computer, you can use:

  1. Local computing resources from Lamsade and Miles. There are enough CPUs and GPUs to run small scale experiments.
  2. Grid5000: the national grid for High Performance Computing (HPC), the grid includes enough GPU to run medium to large scale deep learning experiments.
  3. Jean-Zay: a large cluster computer dedicated for HPC and IA

1. Lamsade/Miles computing resources

Available resources
  • Lamcalcul: 24 cores / 24 threads / 19GiB
  • Ourasi: 20 cores / 40 threads / 32GiB RAM / 2x Nvidia RTX A6000
  • Kaisertrot: 20 cores / 40 threads / 32GiB RAM / 2x Nvidia RTX A6000
  • Boldeagle: 8 cores / 16 threads / 32GiB RAM / 2x Nvidia GTX 1080 Ti
  • Readycash: 8 cores / 16 threads / 32GiB RAM / 2x Nvidia RTX 2080 Ti
  • Upnquick: 20 cores / 40 threads / 120GiB RAM / 2x Nvidia A100
  • Coktailjet: 20 cores / 40 threads / 120GiB RAM / 2x Nvidia A40

Check server status HERE.

How to get access

If you have an account at the Lamsade, you already have an access to lamcalcul. If you need more and in particular if you need GPUs to run your experiments, you have to be added to the GPU unix group.  Please contact Olivier and me at olivier.rouyer@lamsade.dauphine.fr and benjamin.negrevergne@dauphine.psl.eu if you want to be added to the GPU unix group.

How to connect

If you want to connect to the servers from outside the Lamsade, you first need to connect on lamgate before you can connect on whatever server you would like to use.

Note: connecting from eduroam is the same as connecting from outside the Lamsade.

ssh username@ssh.lamsade.dauphine.fr -p 5022

Of course you need to replace username with your actual username, the one that was given to you by Olivier Rouyer when you joined the lab.

Once you’re inside the network, or if you’re already inside, you simply need to log on the server you like with ssh.

     ssh servername

We strongly encourage you to setup ssh keys in order to avoid unsafe password authentication.

You can also setup your ssh-config file using proxycommand to avoid having to connect manually to lamgate first.
Here the content of my .ssh/config file.

Host lamgate
Hostname ssh.lamsade.dauphine.fr
User bnegrevergne
Port 5022


Host lamcalcul
Hostname lamcalcul
User bnegrevergne
ProxyCommand ssh lamgate "nc -q0 %h %p"
2. Jean-Zay
Available resources
  • 1528 computing nodes XA730i, with  2 Intel Cascade Lake 6248 with 20 cores @ 2,5 Ghz, (40 cores per nodes)
  • 261 computing nodes XA780i, with 2 Intel Cascade Lake 6248 and 4 GPUs Nvidia V100 SXM2 32 Go

More info HERE

How to get access

First, you need to obtain an administrative account on the eDari platform.
Here: https://www.edari.fr/before_register

Once you have an administrative account, you can ask a computing account on Jean-Zay via.
Log on eDari, and click on “Déclaration de compte calcul”, and fill the form.

Here is what you need to complete the form:

  1. A fix IP address to access Jean-Zay. You can use the one of Lamsade:
    IP address: 193.48.71.230 et 193.48.71.233
    FQN: gw1.lamsade.dauphine.fr et ssh.lamsade.dauphine.fr
  2. A 8 char password stub, that will be used to genrate your final user password:
    Any random sequence will do.
  3. Various info about the security manager (Olivier Rouyer):
    Firstname/Lastname: Olivier Rouyer
    Email address: rouyer@dauphine.fr
    Phone number: 0144054472

Sign the declaration, and upload it via the web application.

At this point, you have an account but no hours on it. You can either ask for hours by writing a new project, or join an existing project (recommended). To join an existing project, you will need to project number. Currently (jan. 2020), we have hours on a project whose id is: AD011011296. Please inform Benjamin Negrevergne for more details.

 

Getting started with slurm: https://www.irit.fr/wp-content/uploads/2022/01/20211215_Pres_Jean_Zay_IRIT.pdf

More help: http://www.idris.fr/jean-zay/

3. Grid5000

Available resources
  • 8 sites
  • 32 clusters
  • 711 nodes
  • 12522 CPU cores
  • 140 GPUs
  • 221 SSDs and 1046 HDDs on nodes (total: 1087.29 TB)

See HERE for a detailed list of resources or HERE for a brief summary (not always up to date)

How to get access

Fill the registration form here, HERE with the following information

Manager: bnegrevergne
Institutional Affiliation:
 Team: Miles
Laboratory: Lamsade
Institute/Organization: Paris Dauphine — PSL
Site: Guest

The rest is up to you.

Getting started

(Once you have your account)
https://www.grid5000.fr/mediawiki/index.php/Getting_Started

Status (to check the grid status)
https://www.grid5000.fr/mediawiki/index.php/Status

Reserve and run an interactive shell on any available computer at Rennes

ssh username@access.grid5000.fr
ssh rennes
oarsub -I -lwalltime=01:00:00

Reserve one node and 4 cores for a minute, and execute the content of the script.

run.sh

#!/bin/sh 
#OAR -l nodes=1/core=4,walltime=00:01:00 
#OAR -O /home/bnegrevergne/runs/run_%jobid%.output 
#OAR -E /home/bnegrevergne/runs/run_%jobid%.error 
set -xv 
# to specify the name of the cluster 
# -p cluster='parasilo' 
# to create a series of OAR jobs from a file with parameters on each lie
# --array-param-file fichierparams.txt 

MSG="salut" 

echo 
echo "=============== RUN ===============" 
echo "yeah it works!" 
echo $MSG 
echo "=============== FIN ==============="

And execute on one site using

oarsub -S ./run.sh