Computing Workshops 2019-09-27 High Memory Jobs

class: center, middle

Computing Workshops  
# High Memory Jobs

Friday September 27th 2019  
by Oliver Stueker

---

### Let's first talk about the difference between "Storage" and "Memory"

Memory:
* RAM
* DRAM 
* DDR1, ... DDR4 RAM

...all these basically mean the same

Examples for Storage:
* Hard Disk Drives (HDD)
* Solid State Drives (SSD)
* Optical Media (CD-ROM, DVD, Bluray, ...)
* Flash-Memory (USB sticks)
* Tape

---

## Data-types and Units of storage

### A selection of common data-types

name             | abbr   | size [bit] | size [byte]
---------------- | ------ | ---------- | -----------
character        | char   |      8     |      1 
integer          | int    |     32     |      4 
long integer     | long   |     64     |      8 
floating point   | float  |     32     |      4 
double precision | double |     64     |      8

---
## Example: Gaussian using localscatch

Jobscript:
```bash
#!/bin/bash
#SBATCH --mem-per-cpu=16G     # memory-per-cpu
#SBATCH --cpus-per-task=2     # cpus as defined by %nprocs
#SBATCH --time=03-00:00       # expect run time (DD-HH:MM)
module load gaussian/g09.e01 
# use localscatch:
export GAUSS_SCRDIR=$SLURM_TMPDIR

g09 < mycoolmolecule.gjf  > mycoolmolecule.log
```

mycoolmolecule.gjf:
```
%NprocShared=2
%MEM=30GB
%CHK=mycoolmolecule.chk
#N MP2/6-31G(d)  OPT  FREQ MaxDisk=400GB

MyCoolMolecule

0  1
 C    0.         0.         0.
...
```

---
## Example: Gaussian using (net)scatch

g09 < mycoolmolecule.gjf  > mycoolmolecule.log
```

mycoolmolecule.gjf:
```
%NprocShared=2
%MEM=30GB
%CHK=mycoolmolecule.chk
#N MP2/6-31G(d) OPT FREQ MaxDisk=400GB

MyCoolMolecule

0  1
 C    0.         0.         0.
...
```

---
## Example: Gaussian restart

g09 < mycoolmolecule_restart.gjf  > mycoolmolecule_restart.log
```

mycoolmolecule.gjf:
```
%RWF=/scratch/user/mycoolmolecule_mp2.rwf
%NoSave
%nprocshared=2
%mem=30GB
%OldCHK=mycoolmolecule.chk
%CHK=mycoolmolecule_restart.chk
#N MP2/6-31G(d)  FREQ  MaxDisk=400GB  geom=Checkpoint

MyCoolMolecule

0  1
```

---
class: split-50
### Implications of using high-memory jobs: Core Equivalents

<img height="150px"  src="https://docs.computecanada.ca/mediawiki/images/c/c9/Core_equivalent_diagram_GP.png" />
<br/>Core Equivalent diagram
<hr>
--
.column[
<img height="150px"  src="https://docs.computecanada.ca/mediawiki/images/1/18/Two_core_equivalents.png" />
<br/>2 Core Equivalents
]
--
.column[
<img height="150px"  src="https://docs.computecanada.ca/mediawiki/images/9/90/Two_and_a_half_core_equivalents.png" />
<br/>2.5 Core Equivalents
]
[https://docs.computecanada.ca/wiki/Allocations_and_resource_scheduling](https://docs.computecanada.ca/wiki/Allocations_and_resource_scheduling#What_is_a_core_equivalent_and_how_is_it_used_by_the_scheduler.3F)
---

## XKCD #2207 "Math Work"

[![](https://imgs.xkcd.com/comics/math_work.png)](http://xkcd.com/2207/)
<br/>
I could type this into a solver, which MIGHT help, but would also mean I have to get a lot of parentheses right...