In this presentation from the Dell booth at SC13, Will Law from Stanford describes how Big Data gets handled for HPC at the University. "As science drives a rapidly growing need for storage, existing environments face increasing pressure to expand capabilities while controlling costs. Many researchers, scientists and engineers find that they are outgrowing their current system, but fear their organizations may be too small to cover the cost and support needed for more storage. Join these experts for a lively discussion on how you can take control and solve the HPC data deluge." Watch the video presentation: http://insidehpc.com/2013/12/03/panel-discussion-solving-hpc-data-deluge/

1. HPC Storage at Stanford
University
!
William Law
Stanford Research Computing
Center
wlaw@stanford.edu
!

2. We are a new center
!
Working across disciplines, across the university.
!

3. The grand tradition is for each research cluster to be a stand alone
island.
!

4. Backups? Who needs ‘em.
!
Oh, we do.
!

5. Storage
!
• AFS
• box.com
• GPFS
• Lustre
• NFS
• Full HW solutions (panasas, isilon, netapp)
!

6. Data Transfer
!
• Aspera (Enterprise, Faspex)
• Globus Online
• box.com
!

7. Concerns
!
•
IOPS
•
Storage Growth
•
Sharing/Transfer/Workflows
•
Data Integrity
•
Backups
•
Long Term Data Retention/Sustainability (who
pays?)
•
Security and Compliance (and dealing with PHI)
•
Cost
!

8. Path Forward
!
Standardize platforms
Use open protocols for access
Provide access to common storage pools
across resources
!