My introductory talk at GlobusWorld 2012, in which I review the progress made with Globus Online during the last year, and introduce our new Globus Storage, Globus Collaborate, and Globus Integrate services. A great demo showed it all working.

1. GlobusWORLD 2012
Innovating Research Data Management
Ian Foster – April 2012

2. The data deluge
www.globustoolkit.org www.globusonline.org

3. Run a research project from a coffee shop?
www.globustoolkit.org 3 www.globusonline.org

4. Towards “research IT as a service”
• Dark Energy Survey • SBGrid structural biology consortium
• Galaxy genomics • NCAR climate data applications
• LIGO observatory • Land use change; economics
www.globustoolkit.org www.globusonline.org

5. Let’s rethink how we provide research IT
Accelerate discovery and innovation worldwide by
providing research IT as a service
Leverage the cloud to
• provide millions of researchers with unprecedented
access to powerful tools;
• enable a massive shortening of cycle times in
time-consuming research processes; and
• reduce research IT costs dramatically via
economies of scale
www.globustoolkit.org www.globusonline.org

6. 2011: Grid meets Cloud
Globus Toolkit Globus Online
Build the Grid Use the Grid
Components for building Reliable file transfer
custom grid solutions Software-as-a-Service
globustoolkit.org globusonline.org
www.globustoolkit.org 6 www.globusonline.org

7. Globus Online – The first 18 months
• >4,500 registered users; adding ~300/month
• >3.5 PB moved; averaging ~50 TB/week
Data Transferred (TB/week) Support Requests Received
200 250
200
150
150
100
100
50 50
0 0
Jan 2011 Feb 2012 Dec 2010 Jan 2011
www.globustoolkit.org 7 www.globusonline.org

8. Globus Online – The first 18 months
• 514 active endpoints (year-to-date) including
62 at Tier 1 research institutions
• >390,000,000 files moved
• >100,000 transfer requests
• 99.9% uptime in 2012
• 18 identity providers supported, using multiple
protocols: X.509, MyProxy OAuth, OpenID
www.globustoolkit.org 8 www.globusonline.org

9. Our user communities
www.globustoolkit.org 9 www.globusonline.org

10. 10
www.globustoolkit.org www.globusonline.org

11. 2012: Move. Store. Collaborate.
globus online
Globus Globus Globus
Transfer Storage Collaborate
Globus Integrate Globus Connect Multi User
Globus Connect
Globus Toolkit Globus Nexus
www.globustoolkit.org 11 www.globusonline.org

12. Towards “research IT as a service”
Research Data Management-as-a-Service
Globus Globus Globus Globus …SaaS
Transfer Storage Collaborate Catalog
Globus Integrate …PaaS
www.globustoolkit.org www.globusonline.org

13. Globus Storage: For when you want to …
• Place your data
where you want
• Access it from
Globus Transfer, HTTP/REST, Desktop sync
anywhere via different
protocols
• Update it, version it, Globus
Storage
and take snapshots volume
• Share versions with
who you want
• Synchronize among Commercial National Campus
storage service research computin
locations provider center g center
www.globustoolkit.org www.globusonline.org

14. Globus Collaborate: For when you want to
Join with a few or many people to:
• Share documents
• Track tasks
• Send email
• Share data
• Do whatever
With:
• Common groups
• Delegated
management
www.globustoolkit.org www.globusonline.org

15. Globus Integrate: For when you want to…
Write programs that access/manage user
identities, profiles, groups, resources—and data …
Globus Globus Globus
Transfer Storage Collaborate
Globus Integrate Globus Connect Multi User
Globus Connect
Globus Toolkit Globus Nexus
… via REST APIs and command line programs
www.globustoolkit.org www.globusonline.org

16. Earth System Grid – Portal integration
• Outsource data transfer to Globus
– Data download to user machine
from search
– Data transfer to another server
by user
– Replication of data between sites
by administrator
• No ESGF client software needed
www.globustoolkit.org 16 www.globusonline.org

17. www.globustoolkit.org www.globusonline.org

18. Let’s take a look at what’s coming…
Traumatic Brain Injury (TBI)
Diffusion Tensor Imaging (DTI)
www.globustoolkit.org 18 www.globusonline.org

19. Let’s take a look at what’s coming…
Globus Connect
Bryce Move DTI results to PADS
Bryce’s laptop Compute
DTI Group Cluster
- Kyle
- Bryce Globus Storage Globus Transfer
Create snapshot to Copy TBI data to
share with group compute cluster
Globus Nexus Globus Transfer
Add Bryce to TBI Move DTI results
collaboration to shared volume
Globus Collaborate
Publish DTI data to TBI
web site
Amazon S3
Globus Storage
Create volume and
share with TBI group SDSC
UChicago
Cloud
Kyle “TBI” Object
Globus Connect volume Store
Move MRI files to Cornell
TBI shared volume Red Cloud
www.globustoolkit.org 19 www.globusonline.org

20. Globus Toolkit update
GT Releases 2011-2012
5/4/11 GT 5.1.0
5/18/11 GT 5.0.4
8/2/11 GT 5.1.1 Highlights
11/16/11 GT 5.1.3 (5.2 beta) • Native packaging
12/15/11 GT 5.2.0 • GridFTP: DCSC support
3/8/12 GT 5.0.5 • New GRAM5 version;
Likely last 5.0.x release in use on OSG
4/5/12 GT 5.2.1rc1
5.2.1 final planned for April
www.globustoolkit.org 20 www.globusonline.org

21. Globus Toolkit update – GridFTP
• GT 5.2.0
– Support for DCSC command
• Allows client to specify credentials used to secure data
channel connection
• Utilized by Globus Transfer for seamless data movement
across multiple security domains
– Server administrators may now restrict client access
to a set of paths
• Coming in 5.2.1
– MLSC – stream directory listings over control channel
• Enables simple (control channel-only) implementation of
Globus Storage ftp server
• Can help directory listings for gridftp servers behind
www.globustoolkit.org restrictive firewalls 21 www.globusonline.org

22. Delivery plans
Globus Transfer Globus Storage Globus Collaborate
• Generally available • Early release • Initial projects at
• Service and Web UI • Generally available in UChicago
enhancements Fall 2012 • Early release sometime
continue before year-end 2012
Globus Integrate
Globus Connect Multi User (GA)
• Transfer API available
• APIs for Storage, Collaborate
planned after app release Globus Connect (GA)
Globus Toolkit (GA V5.2) Globus Nexus (Alpha)
www.globustoolkit.org www.globusonline.org

23. Delivery plans
• Globus Storage
– Current: Early access – contact us if interested
– Open beta planned in late Spring; GA in late Summer
– V1.0: UChicago and Amazon S3 object stores
• Globus Collaborate
– Current: Pre-release with two UChicago/CI groups
– First release planned around year-end
www.globustoolkit.org 23 www.globusonline.org

24. Premium offerings
Motivated by sustainability: resource cost
recovery, user support, future development
• Globus Transfer plans (resource providers)
– www.globusonline.org/premium-plans/
• Globus Storage premium plans
– Plans variables include: size, object store
type, durability, support level, personalization
• Globus Collaborate premium plans
– Plans variables include: branding capabilities, support
level, integration (e.g. identity)
www.globustoolkit.org 24 www.globusonline.org

25. Premium offerings
Motivated by sustainability: resource cost
recovery, user support, future development
• Globus Transfer plans (resource providers)
– www.globusonline.org/premium-plans/
• Globus Storage premium plans
– Plans variables include: size, object store
type, durability, support level, personalization
• Globus Collaborate premium plans
– Plans variables include: branding capabilities, support
level, integration (e.g. identity)
www.globustoolkit.org 25 www.globusonline.org

26. The Globus yeam
Bryce Allen Rachana Lisa
Ananthakris Childers
hnan
Martin Tom Howe Raj Jack Kordas Lukasz
Feller Kettimuthu Lacinski
Lee Liming JP Navarro Karl Pickett
Andrew Peter Zich
Zich
www.globustoolkit.org 26 www.globusonline.org

27. Thank you!
www.globustoolkit.org 27 www.globusonline.org