Cloud Data Integration Best Practices
•
2 likes•2,812 views
While many enterprises consider cloud computing the savior of their data strategy, there is a process they should be following when looking to leveraging database-as-a-service. This includes understanding their own data requirements, selecting the right cloud computing candidate, and then planning for the migration and operations. A huge number of issues and obstacles will inevitably arise, but fortunately best practices are emerging. This presentation will take you through the process of moving data to cloud computing providers.
![Today’s Agenda ,[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to Cloud Data Integration Best Practices
Similar to Cloud Data Integration Best Practices (20)
More from Darren Cunningham
More from Darren Cunningham (20)
Recently uploaded
Recently uploaded (20)
Cloud Data Integration Best Practices
- 1. Cloud Data Integration Best Practices Kurt Messersmith, Amazon Web Services David S. Linthicum, www.davidlinthicum.com Darren Cunningham, Informatica Cloud
- 3. AMAZON WEB SERVICES Kurt Messersmith, Sr Manager, AWS
- 4. AMAZON’S THREE BUSINESSES Consumer (Retail) Business Tens of millions of active customer accounts Seven countries: US, UK, Germany, Japan, France, Canada, China Seller Business Sell on Amazon websites Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network Developers & IT Professionals On-demand infrastructure for hosting web-scale solutions Hundreds of thousands of registered customers
- 5. AWS USAGE GRAPH 2007: AWS bandwidth usage surpassed Amazon.com global websites Today: AWS bandwidth usage 30% greater than Amazon.com global websites Bandwidth Usage:
- 6. CLOUD ATTRIBUTES Abstract Resources Not tied to physical hardware and can be flexible as your needs demand. On-Demand Provisioning Ask for what you need, exactly when you need it. Pay only for what you use. Scalability Scale up or down depending on usage needs. No Up-Front Costs No contracts or long-term commitments. Pay only for what you use. Efficiency of Experts Utilize the skills, knowledge and resources of experts.
- 8. PREDICTIONS COST MONEY Infrastructure Cost $ time Large Capital Expenditure You just lost customers Predicted Demand Traditional Hardware Actual Demand Automated Virtualization
- 9. AGILITY EXAMPLE—COST NEUTRAL EQUATION This graphic compares running the same 10,000 jobs on 2 servers versus 1000 servers. The cost is the same for either scenario in using AWS (and RightScale), but the difference in elapsed time is 499 hours. (assuming each server can process 10 jobs/hour) 2 server cloud 10,000 jobs 10,000 jobs 1000 server cloud Output data Output data Total processing time:500 hours Total processing time:1 hour
- 11. DIVERSE ENTERPRISE CUSTOMER ROSTER
- 12. Data Integration for “Cloud.” David S. Linthicum www.davidlinthicum.com [email_address]
- 14. Understand Cloud Provider Interfaces New Accounts Commission Calculation Data Cleaning Sales Order Update Finance/ Operations Sales
- 15. Public Cloud Traditional Data Center Evolving Migration to Public Cloud Computing Providers
- 16. Public Cloud Traditional Data Center Evolving Migration to Public Cloud Computing Providers
- 17. SaaS IaaS PaaS APIs ERP Legacy Data On Premise On Demand Data Integration
- 24. Create the Information Model Ontologies Understand Ontologies Understand the Data Data Dictionary & Metadata Catalog the Data Data Catalog Legacy Metadata External Metadata (B2B) Build Information Model Information Model
- 26. The Informatica Cloud www.informaticacloud.com Darren Cunningham, Informatica Cloud Marketing
- 27. Replicate Data Primary Cloud Integration Use Cases: Your Company Load Data Synchronize Data Cleanse Data
- 28. Cloud Integration Options Outsource Cloud Services On-Premise Tools 3 4 2 Hand Code You need to consider integration for what it is: the mother of all single points of failure . “ ” David Linthicum Author, Cloud Computing and SOA Convergence in Your Enterprise 1
- 29. The Informatica Cloud The Industry’s Broadest Cloud Integration Portfolio Informatica Cloud Services Business Managers Migrate Validate Monitor Synch Replicate Informatica Cloud Editions & Options IT Informatica Cloud Platform SIs, ISVs, Developers Custom
- 31. Data Replication as a Cloud Service We’re using Informatica Cloud Services to replicate millions of rows of data from Salesforce to a centralized database running on Amazon EC2. ” “
Editor's Notes
- First, it’s useful to provide the context that the way we think about what Amazon.com is, the way we think about it at the highest levels of the Company, is that we have three macro and distinct businesses: our Consumer/Retail business, our Seller business, and our Developer business.
- Stanford: Stanford University uses Moonwalk’s Enterprise Data Management System to store and backup their data into Amazon S3. Moonwalk is an ISV partner of AWS. CA: Management solutions from CA provide a business driven management approach and add comprehensive support for Amazon EC2 with real-time automation, provisioning, application performance, service, and database management. Adobe: Adobe offers their LiveCycle Developer Express program on AWS, giving their their enterprise developer community ready access to their document workflow solution for developing solutions. They are also now offering ColdFusion 9, their application development platform, on AWS. Microsoft: Microsoft has used AWS over the years for various tasks, including software delivery, human intelligence tasks, and application hosting. Mailtrust (Rackspace): Mailtrust archives their mail servers in S3. NYTimes: NYTimes have used AWS extensively over the years for data processing pipelines, data analysis, application hosting, etc. Early projects included the TimesMachine, etc. SanDisk: Sandisk uses S3 as the cloud-based storage mechanism to backup and share data from their flash drive products. NASDAQ: NASDAQ uses S3 to store and delivery their ticker symbol data for the NASDAQ Market Replay application. ESPN: ESPN uses EC2 and S3 to host several of their social networking and mobile properties. Intuit: Intuit used SOASTA’s CloudTest service to run load testing on their tax software, utilizing 2200 EC2 cores. They also host some applications one EC2 during tax season, including Intuit TaxCaster. Netflix: Netflix is using AWS for a variety of mission critical applications and services, and will continue to look for ways to leverage the Amazon cloud to service their customers. Autodesk: Autodesk hosts several applications on EC2, including Autodesk Seek. Autodesk Seek is the online source for architects and building engineers to search and download manufacturer design information (i.e., reusable CAD models that can be dropped directly into design projects) Pfizer: Pfizer has done antibody docking for drug design on EC2. They leveraged up to 500 c1.mediums at a time to do the modeling. New York Life has created a financial planning application. This application will help their employees do 'what-if' scenarios for their customers. It will look at things like income, debt, expenses, etc, and come up with a customized plan.
- What should you expect from this approach Key to adaptability and agility Investments required in strategy not technology – how to use current technology to achieve goals. Investment in re-orienting the thinking of IT staff How do we provide services the same way FedEx enables overnight versus 2 day. FedEx has built flexible architecture to enable business level services
- So the cloud is taking off…but it’s also become the #1 driver of Data Fragmentation in the enterprise. As one of our customers said, “a SaaS application without integration is like a beautiful island that nobody can actually get to.” In fact, Forrester Research has shown that 65% of IT managers recognize integration issues as the top barriers to success.
- So the cloud is taking off…but it’s also become the #1 driver of Data Fragmentation in the enterprise. As one of our customers said, “a SaaS application without integration is like a beautiful island that nobody can actually get to.” In fact, Forrester Research has shown that 65% of IT managers recognize integration issues as the top barriers to success.
- That’s why Informatica has put such a focus on the cloud. We’re actually now delivering deployment options across the different flavors of cloud computing – SaaS, PaaS, and Infrastructure as a Service. Informatica Cloud Services are purpose-built applications that are designed for non-technical line of business users (often the SaaS administrator for example). We’ve initally focused on a set of specific use cases that we see as the primary requirements today for SaaS application customers: data migration (loading data in), data synchronization (keeping systems and processes unified on a real-time basis), data quality, and data replication (keeping a local copy of cloud data – typically for on-premise business intelligence). Last year we extended our cloud offerings in two important ways: We introduced the Informatica Cloud Platform, which allows our customers to build and share more complex mappings and functions as a custom cloud service and… Support for IaaS deployments such as Amazon EC2. This means you can sign up to use Informatica PowerCenter or Data Quality on an hourly basis or deploy your software directly on their servers.
- http://www.destinationcrm.com/Articles/Web-Exclusives/Viewpoints/Want-SaaS-Get-Integration-First-51857.aspx By 2010, 76% of US organizations will use at least one SaaS-delivered application for business.