09.02.20 Tour, Calit2 at UCSD TTI-Vanguard The Strongly Coupled LambdaCloud La Jolla, CA

1. The Strongly Coupled
LambdaCloud
Tour
TTI-Vanguard
Calit2@UCSD
February 20, 2009
Dr. Larry Smarr
Director, California Institute for Telecommunications and
Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD

2. Two New Calit2 Buildings Provide
Laboratories for “Living in the Future”
• “Convergence” Laboratory Facilities
– Nanotech, BioMEMS, Chips, Radio, Photonics
– Virtual Reality, Digital Cinema, HDTV, Gaming
• Over 1000 Researchers in Two Buildings
– Linked via Dedicated Optical Networks
UC San Diego
www.calit2.net
Preparing for a World in Which
Distance is Eliminated…

3. The OptIPuter Creates a 10Gbps LambdaCloud:
Enabling Collaborative Data-Intensive e-Research
www.evl.uic.edu/cavern/sage
“OptIPlanet: The OptIPuter
Global Collaboratory” –
Special Section of
Future Generations
50 OptIPortals Worldwide -- Computer Systems,
Volume 25, Issue 2,
Campus CI Now the Bottleneck February 2009
Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI
Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST
Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent

4. Academic Research “OptIPlatform” Cyberinfrastructure:
A 10Gbps Lightpath Cloud
HD/4k Video Cams
HD/4k Telepresence
Instruments
End User HPC
OptIPortal
10G
Lightpath
National LambdaRail
Campus
Optical
Switch
Data Repositories & Clusters HD/4k Video Images

5. 1/3 Billion Pixel OptIPortal: Viewing Very Large Images
or Many Simultaneous Images
Spitzer Space Telescope (Infrared)
NASA Earth
Satellite Images
Bushfires
October 2007
San Diego
Source: Falko Kuester, Calit2@UCSD

6. Just in Time OptIPlanet Collaboratory:
Live Session with NASA Ames from Calit2
From Start to
This Image in
Less Than 2 Weeks!
View from NASA Ames
Lunar Science Institute
Mountain View, CA
Virtual Handshake
HD compressed 6:1
Visit Yesterday by
Acting
NASA Administrator
Chris Scolese
Source: Falko Kuester, Calit2; Michael Sims, NASA

7. Ultra Resolution Virtual Reality:
Toward a 3D Global Collaboratory
Connected at 50 Gb/s to Quartzite 15 Meyer Sound
Speakers +
Subwoofer
30 HD
Projectors!
Varrier Showing
360 degree
Mars Rover Images StarCAVE Showing
Biomolecules and
GreenLight Project
Source: Tom DeFanti, Greg Dawe, Calit2
Cluster with 30 Nvidia 5600 cards-60 GB Texture Memory

8. The GreenLight Project:
Instrumenting the Energy Cost of Cloud Computing
• Focus on 5 Communities with At-Scale Computing Needs:
– Metagenomics
– Ocean Observing
– Microscopy
– Bioinformatics
– Digital Media
• Measure, Monitor, & Web Publish
Real-Time Sensor Outputs
– Via Service-oriented Architectures
– Allow Researchers Anywhere To Study Computing Energy Cost
• Develop Middleware that Automates Optimal Choice
of Compute/RAM Power Strategies for Desired Greenness
• Partnering With Minority-Serving Institutions
Cyberinfrastructure Empowerment Coalition
See the Virtual GreenLight
Machine Room in the StarCAVE
Source: Tom DeFanti, Calit2; GreenLight PI

9. Calit2/EVL Varrier --
60 Screen Panorama OptIPortal
Photo:
Amy Bennion
360 Degree Mars Landscape x 23,000 pixels
Mars Rendered at 46,000
Rover Spirit at McMurdo 2006
Dan Sandin, Greg Dawe, Tom Peterka, Tom DeFanti, Jason Leigh, Jinghua Ge, Javier
Girado, Bob Kooima, Todd Margolis, Lance Long, Alan Verlo, Maxine Brown,
Jurgen Schulze, Qianby 4096 pixels
16384 Liu, Ian Kaufman, Bryan Glogowski

10. Calit2 VirtuLab-
Our Visual Skunkworks
4k VTC 4k on OptIPortal
Autostereo 3D TV
Source: Tom DeFanti, Calit2

11. Creating a California Cyberinfrastructure of
OptIPuter “On-Ramps” to NLR, I2DC, & TeraGrid
UC Davis
UC Berkeley
UC San Francisco
UC Merced
UC Santa Cruz
Creating a Critical Mass of
OptIPuter End Users on
a Secure LambdaGrid
UC Los Angeles
UC Santa Barbara UC Riverside
UC Irvine CENIC Workshop at Calit2
UC San Diego
Sept 15-16, 2008

12. CENIC’s New “Hybrid Network” - Traditional Routed IP
and the New Switched Ethernet and Optical Services
~ $14M
Invested
in
Upgrade
Now
Campuses
Need to
Upgrade
Source: Jim Dolgonas, CENIC

13. The “Golden Spike” UCSD Experimental Optical Core:
Ready to Couple Users to CENIC L1, L2, L3 Services
Quartzite Communications
To 10GigE cluster Goals by 2008:
Core Year 3
node interfaces
>= 60 endpoints at 10 GigE CENIC L1, L2
>= 30 Packet switched Wavelength
Quartzite Selective
Services
Corewavelengths
.....
Switch
>= 30 Switched
>= 400 Connected endpoints
Lucent To 10GigE cluster
node interfaces and
other switches
To cluster nodes
.....
Glimmerglass
Approximately 0.5 Tbps .....
To cluster nodes
GigE Switch with
Dual 10GigE Upliks
Arrive at the “Optical”
Production
OOO
Switch
To cluster nodes
Center of Hybrid Campus
32 10GigE
.....
Switch GigE Switch with
Dual 10GigE Upliks
Force10
...
To Packet Switch CalREN-HPR
GigE Switch with
Dual 10GigE Upliks other Research
nodes
Cloud
GigE
Funded by
10GigE
NSF MRI Campus Research
4 GigE
4 pair fiber Grant Cloud
Cisco 6509
Juniper T320
OptIPuter Border Router
Source: Phil Papadopoulos, SDSC/Calit2
(Quartzite MRI PI, OptIPuter co-PI)

14. Calit2 Sunlight
Optical Exchange Contains Quartzite
Maxine 10:45 am
Brown, Feb. 21,
EVL, UIC
OptIPuter
2008
Project
Manager

15. Block Layout of UCSD
Quartzite/OptIPuter Network
Glimmerglass
OOO Switch
~60 10 Gbps Lightpaths
Quartzite
Application Specific
Embedded Switches

16. PI Larry Smarr

17. Calit2 Microbial Metagenomics Science Cloud Server
Moving Compute to the Data--Web and Optical Available
Source: Phil Papadopoulos, SDSC, Calit2
512 Processors ~200TB
~5 Teraflops Sun
1GbE X4500
~ 200 Terabytes Storage and Storage
10GbE
Switched 10GbE
/ Routed
Core
2500 Users From
70 Countries!

18. UCSD Campus Research CI:
Design Team Architecture for Fiber Shared Resources
UCSD Triton Components
HPC System
Cluster
Condo
PetaScale
Data
Analysis UCSD Storage
Facility
UC Grid
Pilot
Digital
Collections
DNA Arrays,
Lifecycle Research
Mass Spec.,
Microscopes,
Management Cluster OptiPortal
Research
Genome
Instrument
Sequencers
N x 10Gbe
Source: Phil Papadopoulos, SDSC/Calit2

19. Current Planning for UCSD Triton
“OptIPuter on Steroids” *
Large Memory Shared Resource
PSDAF Cluster
• 256 GB/Node • 16 – 32 GB/Node
• 8TB Total • 4 - 8TB Total
• 128 GB/sec • 256 GB/sec
• ~ 10 TF • ~ 25 TF
x256
x32
UCSD Research Labs
Large Scale Storage
• 2 – 4 PB
• 75 – 150 GB/sec
• 3000 – 6000 disks
Campus Research
Triton
Resource
Network
Source: *Phil Papadopoulos, SDSC/Calit2

20. UCSD Planned Optical Networked
Biomedical Researchers and Instruments
• Connects at 10 Gbps :
CryoElectron Natural Triton – Microarrays
Microscopy Facility Sciences
Building San Diego – Genome Sequencers
Supercomputer – Mass Spectrometry
Center
– Light and Electron
Microscopes
– Whole Body Imagers
– Computing
Cellular & Molecular
– Storage
Medicine East
Calit2@UCSD
Bioengineering
Radiology
Imaging Lab UCSD Research Park
National
Center for
Creates Campus–Wide
Microscopy Instrument “Data Utility” -
& Imaging Center for
Molecular Genetics Lab Portals to Triton
Pharmaceutical
Sciences Building Cellular & Molecular
Biomedical Research Medicine West

21. Open Cloud OptIPuter Testbed
Manage and Compute Large Datasets
CENIC NLR C-Wave Dragon
Open Source SW
HW Phase 1 (2008)  Hadoop
• 4 racks  Sector/Sphere
MREN
– 120 Nodes
 Thrift, GPB
– 480 Cores
 Eucalyptus
• 10+ Gb/s WAN
 Benchmarks
Phase 2 (2009) will add additional racks
to current sites and increase number of sites
21
Source: Robert Grossman, UIC

22. Sorting 10 Billion Records (1.2 TB) at 4 Sites (120 Nodes)
Sustaining >5 Gbps--Only 5% Distance Penalty
Supercomputing 2009
http://angle.ncdm.uic.edu/simnetup/

23. Cisco CWave for CineGrid: A New Cyberinfrastructure
for High Resolution Media Streaming*
Source: John (JJ) Jamison, Cisco
PacificWave
1000 Denny Way
(Westin Bldg.)
Seattle
StarLight
Northwestern Univ
Level3 Chicago
1360 Kifer Rd. McLean
Sunnyvale 2007
Equinix
818 W. 7th St.
Los Angeles CENIC Wave
Cisco Has Built 10 GigE Waves on CENIC, PW,
& NLR and Installed Large 6506 Switches for
Calit2 Access Points in San Diego, Los Angeles,
San Diego Sunnyvale, Seattle, Chicago and McLean
CWave core PoP for CineGrid Members
Some of These Points are also GLIF GOLEs
10GE waves on NLR and CENIC (LA to SD)
* May 2007

24. The Commercial Market for High Resolution
Teleconference Systems in Rapidly Expanding
Changing the Way We Work, Live, Play and Learn
• 320 Cisco • 216K TelePresence • 31K+ meetings avoided
TelePresence meetings scheduled to travel
date. • Conservative estimate of
major cities globally
• US/Canada: 101 CTS 3000, 79
• 48% Average Utilization cost savings:
CTS 1000, 3 CTS 3200, 13 CTS • 279K hours (average •~$278M to date
500 meeting is 1.25 hrs) • Cubic meters of emissions
• APAC: 24 CTS 3000, 20 CTS • 22K+ meetings with saved: 90 million
1000, 3 CTS 500 customers to discuss • Equal to >15,500+ cars off
• Japan: 5 CTS 3000, 2 CTS Cisco Technology over the road
1000, 1 CTS 500,1 CTS3200 TelePresence
• Europe: 30CTS 3000, 19 CTS
1000, 1 CTS3200, 5 CTS500
• Emerging: 13 CTS 3000,
1CTS1000
Experience
Live
Telepresence
on Calit2
5th Floor
Source: Cisco-Updated January 5, 2009

25. The GreenLight Project Focuses on
Minimizing Energy for Key User Communities
• Microbial Metagenomics
• Ocean Observing
• Microscopy
• Bioinformatics
• Digital Media—CineGrid Project
– Calit2 will Host TB of Media Assets in GreenLight
CineGrid Cloud Exchange to Measure and Propose
Reductions in the “Carbon Footprint” Generated by:
– File Transfers and
– Computational Tasks
– Required for Digital Cinema and Other High Quality
Digital Media Applications

26. CineGrid Exchange:
Using Optical Fibers to Create Remote Storage
Global Warming will Drive
Cloud Computing!