Overview of CSC Supercomputing Services at Vaasa University

1. CSC Supercomputing Services
Olli-Pekka Lehto
Development Manager
Computing Platforms
olli-pekka.lehto@csc.fi
@ople

2. Supercomputers back in the day..

3. Supercomputers today
Commodity technologies rule
– Clusters of x86 Linux servers
– Ethernet, InfiniBand networks
– DIY software stacks
Special systems in high-end
– BlueGene, Cray, NEC
– Increasingly commodity-based
Cloud computing increasing
CSC: >4000 customers, ~500 projects
http://www.top500.org

4. Use of supercomputers today
Use of supercomputers diversifying
– Finance
– Digital product design
– Digital humanities (“socionomics”)
– Linguistics
Huge potential
– Able to solve old problems in large scale
– New approaches to existing problems
– Dreaming up new approaches

5. Supercomputer architecture today
EdustasolmuEdustasolmut
Frontend nodes
Kytkentäverkko
Interconnect networkLaskentasolmut
Compute nodes
Levypalvelimet
Storage servers
Levyjärjestelmä
Storage system
Internet
Hallintasolmut
Management nodes
Hallintaverkko
Hallintasolmut
Management nodes

6. CSC Computing Services
Performance Capacity Accelerated Cloud Hosting
Sisu
Massive
parallelism
Fast
interconnect
Taito
General use
Large memory
>100
applications
Taito
extension
Visualization
Special codes
Nvidia GPU
Intel Xeon Phi
cPouta
Build your
own
Openstack
IaaS
Kajaani
Espoo
Efficient and
secure
datacenters
Virtual and
physical
servers
Storage Services
Backup Archiving
Fast parallel storage

7. CSC Computing Services
Performance Capacity Accelerated Cloud Hosting
Sisu
Cray XC40
40512
cores
1700 TFlops
Taito
HP Cluster
18880
cores
580 TFlops
Taito
extension
Bull cluster
76 Nvidia K40
GPU
90 Intel Xeon
Phi 7120X
240 TFlops
cPouta
Dynamically
provisioned
from Taito
Kajaani
Espoo
Storage Services
>4PB, ~ 100GB/s

8. New: Xeon Haswell E5 CPUs
Intel Xeon E5-2690v3 2,6GHz
– Launched 9/2014
– 12 cores/CPU (+50%)
– AVX2 instructions (2x max flops/GHz)
– DDR4 memory
– “Energy-to-solution” at best 1/3 vs.
Sandy Bridge
CSC one of earliest adopters
– Sisu upgraded 7/2014
– Taito upgraded 12/2014

9. Total performance:
2.7 PFlop/s
CSC is the most powerful academic
computing facility in the Nordics

10. Datacenter Kajaani
Old paper warehouse
– Renforsin Ranta
Advantages
– Cool climate
– Cheap & reliable power
– Expandable
– High energy-efficiency
– Local expertise

11. Getting started
Academic use (OKM)
Activate an account at CSC
– https://research.csc.fi/accounts-and-projects
Initial quota for testing & minor work
– Taito access, core hours, storage
Other use (non-OKM, commercial)
Paid packages
– Annual base package + additional compute
and storage quota options
Send mail to asiakaspalvelu@csc.fi for
details

12. Accessing Taito & Sisu
NoMachine NX –virtual desktop
Unix shell & X-Windows
Scientists’ User Interface
https://sui.csc.fi

13. Usage Options
1. Use preinstalled apps (>100)
– MATLAB, Maple, Mathematica, Abaqus,
Ansys, Fluent, SUI web apps etc.
– https://research.csc.fi/software
2. Install your own apps
– (RedHat) Linux –compatibility needed
3. Build your own code
– Popular languages C/C++, Fortran, Python, R
GPU: CUDA, OpenCL, OpenACC
– Parallelization: MPI, OpenMP, PGAS

14. Which system to use?
Various levels of capability
– From fast workstation
– ..to world-class supercomputing
Various levels of expertise needed

15. 1. Scientist’s User Interface
https://sui.csc.fi/

16. 1. Taito-shell – Get started quickly
Instant access
– ssh into taito-shell.csc.fi
– You will be logged in to a shared server
– Run programs directly
Resources are not guaranteed
– 4 cores, 128GB RAM as “best effort”
– We scale node pool according to usage
Not recommended for heavy computing
https://research.csc.fi/sisu-user-guide

17. 2. Taito – For general use
Guaranteed resources
– ssh into taito.csc.fi (frontend node)
– Prepare your job on the fronend node
– Submit your job into the batch job system (SLURM)
– SLURM allocates the resources and runs the job
Guaranteed resources
– 1-100s of cores, 64-256GB RAM per node
– 2 hugemem nodes: 1.5TB RAM per node
https://research.csc.fi/taito-user-guide

18. 3. Taito extension - Novel technology
Specialized processors for computing
– Intel Xeon Phi 7120X
– Nvidia Tesla K40 GPU (also for visualization)
Usage is similar to Taito (SLURM etc.)
Potential for high performance (>2x CPU)
– Applications must support Phi/GPU
– Performance depends on the application
Future of supercomputing?
https://research.csc.fi/taito-gpu
https://research.csc.fi/taito-mic

19. 4. Sisu – Extreme scaling
When you need even more cores
– 100s to 1000s of cores
– At this point scaling is challenging
Cray’s fast interconnect network helps out a lot
Usage similar to Taito (SLURM etc.)
Apply for access
– Demsonstrate that your application can scale
– ..or use a preinstalled application
https://research.csc.fi/sisu-user-guide

20. 5. Grand Challenges & PRACE
If even 8000 cores is not enough?
Grand Challenge calls
– For cutting-edge research that requires huge
resources
– Assessed by a scientific customer panel
PRACE calls
– Access to the largest systems in Europe
https://www.csc.fi/en/web/guest/grand-challenge-proposals
http://www.prace-ri.eu/

21. Scalable computing solutions
Taito-shell
SUI
Taito
Sisu
Grand Challenges
PRACETaito
extension
Capability
Common storage for all systems
All systems x86 Linux –based

23. New: Accelerators
Special processors for computing
– 72 Nvidia Tesla K40 GPGPU
– 90 Intel Xeon Phi 7120X
Very efficient and powerful
– ..if the application supports it
GPUs can be used for visualization
Integrated into the Taito-cluster
– Direct-liquid cooled racks by Bull-Atos
https://research.csc.fi/taito-gpu
https://research.csc.fi/taito-mic

24. Why use CSC computing services?
High-end compute HW, storage, networks
Free for academia
Secure operations
Data stays in Finland
Large library of applications
Specialist support & consulting
Highly developed HPC software stack
Services for all usage levels
– You can also mix & match!

25. Energy-efficiency
0
0.5
1
1.5
2
2.5
3
3.5
Vuori Sisu P1 Taito P1 Sisu P2 Taito P2 Bull
GFlop/W
Taito-extension

26. Cray XC blade
4 dual CPU nodes (96 cores)
64GB RAM per node
Aries Router
(500GB/s switching capability)
Power
Net

27. Blade in XC rack
48 blades
384 CPUs
4608 cores

28. Party in the back

29. Aries Interconnect
2 dimensional
all-to-all network
in a group
All-to-all network
between groups
Source:
Robert Alverson, Cray
Hot Interconnects 2012 keynote
Optical uplinks to
inter-group net
29CSC presentation

30. Cray interconnect bandwidth
9 000 000 x 1,75 x
Avg european consumer IP
traffic (~4TB/s) 2013
OR
1080p Netflix streams
~7 TB/s
=