Dr Matthew Berryman Managing Director, Across the Cloud Chair, High Performance Steering Committee, University of Wollongong Background slides by Adrian White, Manager, APAC Research & Technical Computing, AWS

1. Real-world High Performance & High Throughput
Computing on AWS
Dr Matthew Berryman
Managing Director, Across the Cloud
Chair, High Performance Steering Committee, University of Wollongong
Background slides by Adrian White, Manager, APAC Research & Technical Computing, AWS
5/12/2017

2. Tools should fit your workflows
Collect WorkflowsEvents Integrate Discover
Validate &
Share
Real-time
Batch
Amazon
Kinesis Amazon EMR
Streaming
Amazon S3
AWS Lambda
AWS Batch
Amazon S3 Amazon S3
AWS CLI &
SDKs
Amazon
EMR
Amazon
Redshift
Amazon
Athena
HPC cluster
Amazon API
Gateway
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

3. A cluster in the cloud is an ephemeral tool
Amazon S3
Source data IN
Data product OUT
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

4. Compute in the cloud is flexible
M4
General
purpose
Compute
optimized
C4
C3
Storage and I/O
optimized
I3
G2
GPU or FPGA
enabled
Memory
optimized
D2
M3
X1
P2
F1
R4
R3
C5
I2 HS1
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

5. Instances types within a family
Model vCPUs Memory (GiB) Networking
Performance
r4.xlarge 2 15.25 Up to 10 Gbps
r4.xlarge 4 30.5 Up to 10 Gbps
r4.2xlarge 8 61 Up to 10 Gbps
r4.4xlarge 16 122 Up to 10 Gbps
r4.8xlarge 32 244 10 Gbps
r4.16xlarge 64 488 20 Gbps
R4
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

6. Clusters in the cloud are fit for purpose
Amazon S3
R4
P2 P2 P2
P2 P2 P2 R4
C5
C5
C5 C5
C5 C5
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

7. Clusters can scale and are elastic
W = 1, C =1 W = n, C = n W = 0, C ~ 0
t
c c
t
c
t
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

8. Everyone consumes S3
Collect Share
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

9. Keep storage simple – S3 and POSIX cache
M M M
S S S S
Mgt
N.B. Data lifecycle is
required
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

10. HPC+HTC Tools

11. A few HPC and HTC tools on AWS
CfnCluster AWS Batch Amazon EMR+
CfnCluster is
provided by AWS to
quickly provision
configurable HPC
and HTC cluster
environments
Alces Flight is available
in the AWS Marketplace
and bundles 1000+
commonly used
scientific applications
https://aws.amazon.com
/marketplace/
AWS Batch provides
compute resources via
Docker containers
with user-definable
queues and an
optimised job
scheduler
Amazon EMR
provides a managed
Hadoop framework
supporting Apache
Spark, HBase, Presto,
and Flink on Amazon
EC2 and EC2 Spot
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

12. A closer look at Alces Flight
1000+ popular scientific applications
• Multiple versions, complete with libraries and
various compiler optimizations, ready to run
• Supports Docker and Singularity
• Slurm default scheduler (also PBS Pro, SGE
etc)
Available via the AWS Marketplace
http://alces-flight.com/ for more information
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

13. R4
…
OpenDroneMap on Alces Flight architecture
r4.8xlarge
32 vCPUs
244 GiB RAM
10 Gbps

14. R4
R4
…
OpenDroneMap on Alces Flight architecture

15. Demo: Push-button HPC + aerial imagery
processing on Alces Flight HPC

16. How’d that magic happen?

17. So, how much did this cost?
On-demand / hr Spot / hr Running total
Login node
(r4.2xlarge)
$1.91 $1.91
Compute nodes
(r4.8xlarge x 4)
$8.52 $1.20
(85% saving)
$3.60
Shared storage
(1TB general purpose SSD
via NFS)
$0.17 $0.17
Data transfer + S3
(egress)
$0.07 $0.07
Processing time: 3 hours
GRAND TOTAL: $5.75

18. Evolving the compute paradigm
Physical Virtualization Containerization Serverless
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

19. Simplifying and cutting costs
further: OpenDroneMap-ECS

20. Copy

21. Run

22. Demo: OpenDroneMap-ECS

23. AWS Batch
Take a look at https://www.nextflow.io/

24. “HPC” on Lambda?
def my_function(b):
x = np.random.normal(0, b, 1024)
A = np.random.normal(0, b, (1024, 1024))
return np.dot(A, x)
pwex = pywren.default_executor()
res = pwex.map(my_function, np.linspace(0.1, 100, 1000))
PyWren.io
PyWren lets you run your existing
python code at massive scale via
AWS Lambda
CSIRO have built GT-
Scan2 for CRISPR/Cas9
analysis on AWS Lambda

25. Before you go home…

26. Do two things!
+
Register and enroll in the
AWS Research Cloud Program
https://aws.amazon.com/rcp
Launch your own personal cluster
Using Alces Flight
http://alces-flight.com/community
1. 2.
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

27. More Information
• High Performance Computing on AWS
(whitepapers, customer examples, tech overviews)
• AWS Compute Blog
• Research & Technical Computing on AWS
• AWS Research Cloud Program
HPC + HTC Tools
CfnCluster, AWS Batch, Alces Flight
Pywren: Terraflops and microservices
© 2017, Amazon Web Services, Inc. or its Affiliates, All rights reserved.

28. Source code
• IoT push button HPC:
https://github.com/AcrossTheCloud/iot-button-hpc
• Alces Flight Open Drone Map:
https://github.com/AcrossTheCloud/alces-flight-odm
• OpenDroneMap-ECS:
https://github.com/opendronemap/opendronemap-ecs
• The slowest Internet-connected computer:
https://github.com/matthewberryman/brunsviga (live at
brunsviga.io )

29. Thank you!