Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018

•

2 gostaram•510 visualizações

Corteva Agriscience, the agricultural division of DowDuPont, produces as much DNA sequence data every six hours as existed in the entire public sphere in 2008. On-premises processing and storage could not scale to meet the business demand. Partnering with Sogeti (part of Capgemini), Corteva replatformed their existing Hadoop-based genome processing systems into AWS using a serverless, cloud-native architecture. In this session, learn how Corteva Agriscience met current and future data processing demands without maintaining any long-running servers by using AWS Lambda, Amazon S3, Amazon API Gateway, Amazon EMR, AWS Glue, AWS Batch, and more. This session is brought to you by AWS partner, Capgemini America.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Petabytes of Data & No Servers: Corteva
Scales DNA Analysis to Meet Increasing
Business Demand
Ryan Smith
Software Development Leader –
Bioinformatics
Corteva Agriscience
E N T 2 1 8 - S
Scott Warren
Cloud Architect
Sogeti USA

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DNA Sequencing Technology
• Lab uses Illumina sequencing
machines
• Data generated for analysis

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sequence Alignment

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where We Started
• Every 6 hours Corteva produces as much Genetic data as
existed in the entire public sphere in 2008
• On premises compute and storage demands were
becoming unsustainable
• 35 node Hadoop cluster with 2PB of storage
• Significant increase in future demand

10
Why AWS?
• Understood research needs
• Amazon service offerings mirrored on
premises Hadoop system
• Amazon Elastic Map Reduce
• Amazon Simple Storage Solution
(Amazon S3)
• Cost efficiency
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Uses
• Genome-wide variation screening
• Transformation assay
• Quality control
• Whole genome assembly

Our Applications
SNPFinder
• Whole genome alignment of short reads
• Looking for single nucleotide
polymorphisms (SNPs)
• Input data size 50-500+GB
Vector Quality Control (VQC)
• Synthesize a DNA fragment to create
a transgenic event
• Synthesis needs to be quality
controlled
• Regulatory requirements
• Input data size <10MB
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Project Theseus

User Interaction
SNPFinder:
• Pipeline transforms data into queryable
state
• Analysis is done ad-hoc through a user
interface or API layer
VQC
• All processing is completed when data
enters the application
• Users are viewing these results to
inform decision making
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Guiding Principles
User Patterns
• Time Sensitive Workloads
• Small User Base
Technical
• Serverless
• Immutable Infrastructure
• Automate Everything
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Difference in Design
• Both application use the same input data
• Type of processing, outputs and technical
requirements are very different
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SNP Calling Pipeline
• Align short reads
• Decide if SNP or sequencing error
• Transform into queryable format
(Parquet)

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SNPFinder
Pipeline
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

SNPFinder Queries
• Position
• Purity/Coverage
• Neighborhood
• Other SNPs
• GC%
• Repetitive Sequence
• Annotations
AAATTGAGTACGCGAGCTAGCGAGCTAGAGCGATG
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SNPFinder Query

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SNPFinder User Interface

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VQC Architecture

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VQC Architecture – Data Ingestion

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VQC Architecture – Query

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VQC User Interface

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture Comparison
Many small jobs - VQC
A few big jobs - SNPFinder

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Glue
• Introduced later in the project
• Using for data cleanup
• Move data without having to fully
reprocess

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Results – Autoscaling

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Results – Data Import

Results – Business Impact
• Eliminate resource contention
• Disaster recovery
• Our data is now stored in many different physical locations
• Lab growth enabled
• Data storage is no longer an issue
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ryan Smith
ryand.smith@pioneer.com
Corteva Agriscience
Scott Warren
scott.warren@us.sogeti.com
Sogeti USA

Mais conteúdo relacionado

Mais procurados

Amazon Kinesis Video Streams makes it easy to capture live video, play it back, and store it for real-time and batch-oriented ML-driven analytics. In this session, we first dive deep on the top five best practices for getting started and scaling with Amazon Kinesis Video Streams. Next, we demonstrate a streaming video from a standard USB camera connected to a laptop, and we perform a live playback on a standard browser within minutes. We also have on stage members of Amazon Go, who are building the next generation of physical retail store experiences powered by their "just walk out" technology. They walk through the technical details of their integration with Kinesis Video Streams and highlight their successes and difficulties along the way.

Serverless Video Ingestion & Analytics with Amazon Kinesis Video Streams (ANT...

Amazon Web Services

If you want to rapidly release a mobile app, leveraging cross-platform development techniques using a commonly known language is a proven strategy. In this session, we introduce React Native, a cross-platform JavaScript framework for building native iOS and Android apps. We also show you how to easily build cloud-connected data-driven apps using React Native, GraphQL, and mobile-optimized AWS services.

Develop Cross-Platform Mobile Apps with React Native, GraphQL, & AWS (MOB324)...

Amazon Web Services

Migrating database to cloud

Amazon Web Services

Data is eating the world. Don't let it consume your organization! In this talk, we take a deep dive into the various methods of ingesting the ever-increasing amount of data that your organization is generating, and we discuss how to effectively leverage that data as part of your data analytics pipeline. We cover topics such as batch and real-time ingestion with AWS services and open source tools. We also dive into some reference architecture implementations of common ingestion patterns. Finally, we discuss how you can make this data useful through catalogs and governance.

Keeping the Pace with Data Ingestion (GPSCT402) - AWS re:Invent 2018

Amazon Web Services

In this session, Amazon Sumerian leaders provide an overview of the evolution of the augmented reality/virtual reality (AR/VR) industry. They discuss what has changed since Sumerian was announced at re:invent 2017. They also talk about key market trends, and they highlight areas with the highest adoption and potential. This is an immersive session, with examples and demonstrations of immersive experiences created using Amazon Sumerian.

What's New in AR & VR: State of the World Report (ARV203) - AWS re:Invent 2018

Amazon Web Services

Get hands on with the main components of AWS IoT Core. You will learn how to connect and manage your devices, secure device connections and data, process and act upon device data, and read and set device state at any time. You will see how the Device Gateway serves as the entry point for IoT devices connecting to AWS. The Device Gateway supports the MQTT, WebSockets, and HTTP 1.1 protocols. You will work with the high throughput Message Broker to securely transmit messages to and from all of your IoT devices and applications. The flexible nature of the Message Broker’s topic structure allows you to send messages to, or receive messages from, as many devices as you would like. It supports messaging patterns ranging from one-to-one command and control messaging, to one-to-one million (or more!) broadcast notification systems. You will set up the Registry to track device attributes and metadata and create a persistent, virtual version of each device known as the Device Shadow. Finally, you will explore the Rules Engine to author rules within the management console or write rules using a SQL-like syntax. Join us if you are a cloud architect, device engineer, software developer, or firmware engineer.

AWS IoT Core Workshop (IOT305-R1) - AWS re:Invent 2018

Amazon Web Services

AWS IoT Analytics collects, processes, and analyzes IoT data quickly and easily so you can gain operational insights. To collect data and prepare it for analysis, you use Channels. Channels listen to devices and pull device messages into AWS IoT Analytics. Once messages are flowing, pipelines process messages and turn them into useful data using techniques like transforms and filters. Pipelines also enrich your IoT data with data from other AWS service and external data sources. After it's processed, data is stored in an IoT-optimized data store. Once data is ready to be analyzed, datasets are created by querying the data or by using built-in Jupyter notebook templates for sophisticated analytics, such as machine learning. AWS IoT Analytics integrates with Amazon SageMaker so you can build machine learning models, such as predictive analytics. Join us for a deep dive on these components.

Building IoT Analytics (IOT327-R1) - AWS re:Invent 2018

Amazon Web Services

Overview of Redis with Search and Graph (DAT334) - AWS re:Invent 2018

Amazon Web Services

Whether it’s distributing configurations and customer settings, launching instances, or responding to surges in load, having a great control plane is key to the success of any system or service. Come hear about the techniques we use to build stable and scalable control planes at Amazon. We dive deep into the designs that power the most reliable systems at AWS. We share hard-earned operational lessons and explain academic control theory in easy-to-apply patterns and principles that are immediately useful in your own designs.

Closing Loops and Opening Minds: How to Take Control of Systems, Big and Smal...

Amazon Web Services

DynamoDB transactions enables developers to maintain correctness of their data at scale by adding atomicity and isolation guarantees for multi-item conditional updates. With transactions, you can perform a batch of conditional operations including, PutItem, UpdateItem, and DeleteItem with guarantees. Come to this session to learn about how DynamoDB transactions works, the primary use cases in enables, and how to build modern applications that require transactions.

[NEW LAUNCH!] Building modern applications using Amazon DynamoDB transactions...

Amazon Web Services

As financial institutions look to accelerate and scale their use of machine learning, they need to address questions related to specific results, such as the version of the code and the data that lead to a particular inference. The use of disparate and increasingly non-traditional data sources for activities such as targeted marketing, fraud detection, and improved returns is driving a need for structured development of machine learning models. In this session, we’ll discuss how we can use a combination of AWS services including Amazon SageMaker, AWS CodeCommit, AWS CodeBuild, and AWS CodePipeline to create a workflow that will help financial institutions meet their requirements and drive business results.

Industrialize Machine Learning Using CI/CD Techniques (FSV304-i) - AWS re:Inv...

Amazon Web Services

Modernization involves implementing business processes and technology that provide your business applications with high availability, agility, and elasticity. Nowhere is this more important than in breaking apart the monolith. Modernizing an application as part of a migration can be extremely successful if you follow the AWS migration methodology of “discover, plan, migrate, and optimize” as you move that application to the cloud. In this session, we share what we learned from over 400 successful migrations. We also show you how to virtually break a monolith to a modernized architecture as part of the planning phase and accelerate your migration using container technologies and application discovery tools.

Breaking Up the Monolith While Migrating to AWS (GPSTEC320) - AWS re:Invent 2018

Amazon Web Services

91APP 之API 經濟學與API Gateway與導入之旅

Amazon Web Services

AIOps: Steps Towards Autonomous Operations (DEV301-R1) - AWS re:Invent 2018

Amazon Web Services

In recent years, there has been explosive growth in the number of connected devices and real-time data sources. Data is being produced continuously and its production rate is accelerating. Businesses can no longer wait for hours or days to use this data. To gain the most valuable insights, they must use this data immediately so they can react quickly to new information. In this chalk talk, we discuss how to take advantage of streaming data sources to analyze and react in near-real time. In addition, we present different options for how to solve a real-world scenario and walk through those solutions.

Don’t Wait Until Tomorrow: From Batch to Streaming (ANT360) - AWS re:Invent 2018

Amazon Web Services

In this session, Verizon shares how it uses AWS Systems Manager for inventory, compliance, and patch management solutions. Learn about the challenges that large enterprises face when they attempt to retrofit legacy solutions for cloud environments, and discover best practices for using AWS Systems Manager for minimal access policies, custom Amazon Machine Images, tagging policies, encryption, and more.

Managing Modern Infrastructure in Enterprises (ENT227-R1) - AWS re:Invent 2018

Amazon Web Services

Vehicle mobility is evolving, from traditional rental and fleet services, to car sharing, ride hailing, and future driverless services. Mobility providers need an agile, scalable, digital platform to manage all aspects of their fleet and its usage. In this session, Avis Budget Group (ABG) and Slalom walk through their serverless mobility platform using the AWS connected vehicle reference architecture, Amazon SageMaker, Amazon Kinesis Data Analytics, and AWS Lambda. Learn the practical application of using AWS IoT to connect vehicles and Amazon SageMaker to apply machine learning to uncover insights for use cases, including vehicle inventory, shuttling efficiency, driver behavior, and vehicle trajectory analysis to identify fraudulent vehicle usage. We dive deep into the overall solution and services mentioned above, as well as the operations dashboard ABG created with Uber's open source framework, deck.gl.

Easy Rider: How ML, Serverless, and IoT Drive Mobility as a Service (AMT302) ...

Amazon Web Services

Amazon Kinesis makes it easy to speed up the time it takes for you to get valuable, real-time insights from your streaming data. In this session, we walk through the most popular applications that customers implement using Amazon Kinesis, including streaming extract-transform-load, continuous metric generation, and responsive analytics. Our customer Autodesk joins us to describe how they created real-time metrics generation and analytics using Amazon Kinesis and Amazon Elasticsearch Service. They walk us through their architecture and the best practices they learned in building and deploying their real-time analytics solution.

Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...

Amazon Web Services

Security is an optimization problem. A security team's goal is to enable the delivery of maximum business value at minimum risk and at minimum cost. Over investment in security is inefficient and slows the business down. Under investment in security can lead to expensive breaches and slows the business down. Striking the right balance requires careful risk management decisions and judgement calls. There are rarely absolutes when dealing with security, yet we've found a few situations where it's useful to apply absolutes to a field filled with shades of grey. Join this session to learn what they are.

0x32 Shades of #7f7f7f: The Tension Between Absolutes and Ambiguity in Securi...

Amazon Web Services

Modernizing your data warehouse can unlock new insights while substantially improving query and data load performance, increasing scalability, and saving costs. In this chalk talk, we discuss how to migrate your Oracle data warehouse to Amazon Redshift and achieve agility and faster time to insights while reducing costs. Pfizer joins us to share their journey in building the Scientific Data Cloud—a Redshift-powered data lake that provides unprecedented analytical capabilities in R&D as well as a focus on near real-time access to R&D continuous manufacturing.

Migrating Workloads from Oracle to Amazon Redshift: Best Practices with Pfize...

Amazon Web Services

Mais procurados (20)

Serverless Video Ingestion & Analytics with Amazon Kinesis Video Streams (ANT...

Develop Cross-Platform Mobile Apps with React Native, GraphQL, & AWS (MOB324)...

Migrating database to cloud

Keeping the Pace with Data Ingestion (GPSCT402) - AWS re:Invent 2018

What's New in AR & VR: State of the World Report (ARV203) - AWS re:Invent 2018

AWS IoT Core Workshop (IOT305-R1) - AWS re:Invent 2018

Building IoT Analytics (IOT327-R1) - AWS re:Invent 2018

Overview of Redis with Search and Graph (DAT334) - AWS re:Invent 2018

Closing Loops and Opening Minds: How to Take Control of Systems, Big and Smal...

[NEW LAUNCH!] Building modern applications using Amazon DynamoDB transactions...

Industrialize Machine Learning Using CI/CD Techniques (FSV304-i) - AWS re:Inv...

Breaking Up the Monolith While Migrating to AWS (GPSTEC320) - AWS re:Invent 2018

91APP 之API 經濟學與API Gateway與導入之旅

AIOps: Steps Towards Autonomous Operations (DEV301-R1) - AWS re:Invent 2018

Don’t Wait Until Tomorrow: From Batch to Streaming (ANT360) - AWS re:Invent 2018

Managing Modern Infrastructure in Enterprises (ENT227-R1) - AWS re:Invent 2018

Easy Rider: How ML, Serverless, and IoT Drive Mobility as a Service (AMT302) ...

Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...

0x32 Shades of #7f7f7f: The Tension Between Absolutes and Ambiguity in Securi...

Migrating Workloads from Oracle to Amazon Redshift: Best Practices with Pfize...

Semelhante a Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018

Petabytes of Data and No Servers: Corteva Scales DNA Analysis to Meet Increas...

Capgemini

In this workshop, we build and deploy an end-to-end analytics solution for monitoring the video streaming experience. We integrate an open source video player with Amazon Kinesis Data Streams to capture events in real time. We explore the data available for capture and a variety of use cases: from generating alerts on poor experience to content recommendations based on user behavior. We also show you how this real-time data can be archived in a data lake and further used to generate reports of aggregate performance and experience across a number of dimensions.

Build an AWS Analytics Solution to Monitor the Video Streaming Experience (MA...

Amazon Web Services

100 Billion Data Points With Lambda_AWSPSSummit_Singapore

Amazon Web Services

We are a lean team consisting of developers, lead architects, business analysts, and a project manager. To scale our applications and optimize costs, we need to reduce the amount of undifferentiated heavy lifting (e.g., patching, server management) from our projects. We have identified AWS serverless services that we will use. However, we need approval from a security and cost perspective. We need to build a business case to justify this paradigm shift for our entire technology organization. In this session, we learn to migrate existing applications and build a strategy and financial model to lay the foundation to build everything in a truly serverless way on AWS.

From Monolith to Modern Apps: Best Practices (SRV322-R2) - AWS re:Invent 2018

Amazon Web Services

Introducing Performance Insights - Cloud-Based Database Performance Monitorin...

Amazon Web Services

Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With Serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more. In this session, we will learn how to get started with Serverless computing using AWS Lambda, which lets you run code without provisioning or managing servers.

Introduction to Serverless computing and AWS Lambda - Floor28

Boaz Ziniman

Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more.In this session, we will learn how to get started with serverless computing using AWS Lambda, which lets you run code without provisioning or managing servers.

Introduction to Serverless computing and AWS Lambda | AWS Floor28

Amazon Web Services

Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more. In this session, we will learn how to get started with serverless computing with AWS Lambda, which lets you run code without provisioning or managing servers.

Introduction to Serverless on AWS - Builders Day Jerusalem

Amazon Web Services

Performance insights twitch

Kyle Hailey

Do you have an idea for an app but don't know where to start? In this session, we walk you through the process of taking your idea to reality, and we show you all the infrastructure you need to understand along the way. We also show you how AWS platform services and SDKs can help you get to a quality release faster and then scale for success with serverless technologies. In addition, we demonstrate how you can build a scalable production-ready app quickly with GraphQL and machine language capabilities. Please join us for a speaker meet-and-greet following this session at the Speaker Lounge (ARIA East, Level 1, Willow Lounge). The meet-and-greet starts 15 minutes after the session and runs for half an hour.

From Idea to Customers: Developing Modern Cloud-Enabled Apps with AWS (MOB201...

Amazon Web Services

From Monolithic to Modern Apps: Best Practices

Tom Laszewski

The service mesh is becoming the most critical component of the cloud-native stack, with users ranging from small startups to Internet giants and traditional enterprises. While still early in terms of adoption, this new infrastructure layer has massive implications for the way companies build and operate distributed systems. In this session, SignalFx provides an overview of distributed systems and service instrumentation, then examines how a service mesh approach addresses inter-service communication, testing, and other fundamental challenges of adopting microservices architecture. Learn about the considerations for monitoring and observability, as well as the trade-offs, of implementing service mesh. This session is brought to you by AWS partner, SignalFx.

Fully Realizing the Microservices Vision with Service Mesh (DEV312-S) - AWS r...

Amazon Web Services

With over $5 trillion in assets under management, Vanguard requires a secure, flexible, and fast data and analytics platform. By deploying Tableau on AWS, Vanguard was able to move from their private cloud to the AWS Cloud, and significantly reduce the administrative workload of Vanguard's IT team, allowing them to focus on innovating. In this session, learn how mission-critical processes such as configuring deployments, adding nodes, and creating backups are automated via scripts. Expect to leave with a clear picture of how a global deployment of Tableau on AWS supports peak surges and avoids the sunk costs of on-premises hardware. This session is brought to you by AWS partner, Tableau Software.

Vanguard's Journey with Tableau to the AWS Cloud (FSV307-S) - AWS re:Invent 2018

Amazon Web Services

As Amazon's consumer business continues to grow, so does the volume of data and the number and complexity of the analytics done in support of the business. In this session, we talk about how Amazon.com uses AWS technologies to build a scalable environment for data and analytics. We look at how Amazon is evolving the world of data warehousing with a combination of a data lake and parallel, scalable compute engines, such as Amazon EMR and Amazon Redshift.

Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...

Amazon Web Services

국내 미디어 고객사의 AWS 활용 사례 - POOQ 서비스, 콘텐츠연합플랫폼::조휘열::AWS Summit Seoul 2018

Amazon Web Services Korea

Chaos Engineering

Amazon Web Services

Building Highly Scalable Retail Order Management Systems with Serverless

Amazon Web Services

Learn how Fox and Discovery modernized their media processing workflows to positively impact operations and business results. In this session, we examine each company's production architecture and learn how they utilize AWS services such as AWS Elemental Media Services, AWS Lambda, AWS Step Functions, Amazon API Gateway, and container toolsets. You also get insights into new business capabilities enabled by their AWS serverless architecture, including automation of content assembly and quality control as well as increased customer engagement with personalization and improved processing performance.

Modernizing Media Supply Chains with AWS Serverless (API301) - AWS re:Invent ...

Amazon Web Services

Digital Transformation Through APIs (SRV323) - AWS re:Invent 2018

Amazon Web Services

The unique global cloud infrastructure offered by AWS helps customers build reliable, available, secure, scalable, and fault-tolerant applications. AWS has more experience operating global cloud infrastructures that enables customers run business critical workloads in the public cloud than anyone else. In this session, learn how AWS is continuously enhancing and expanding the AWS global infrastructure through more Regions and Availability Zones, custom hardware, purpose-built global network backbone, and innovative energy management systems to deliver to our customers lower latency, greater reliability, greater scalability, and operational efficiencies.

AWS Global Infrastructure Foundations

Amazon Web Services

Semelhante a Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018 (20)

Petabytes of Data and No Servers: Corteva Scales DNA Analysis to Meet Increas...

Build an AWS Analytics Solution to Monitor the Video Streaming Experience (MA...

100 Billion Data Points With Lambda_AWSPSSummit_Singapore

From Monolith to Modern Apps: Best Practices (SRV322-R2) - AWS re:Invent 2018

Introducing Performance Insights - Cloud-Based Database Performance Monitorin...

Introduction to Serverless computing and AWS Lambda - Floor28

Introduction to Serverless computing and AWS Lambda | AWS Floor28

Introduction to Serverless on AWS - Builders Day Jerusalem

Performance insights twitch

From Idea to Customers: Developing Modern Cloud-Enabled Apps with AWS (MOB201...

From Monolithic to Modern Apps: Best Practices

Fully Realizing the Microservices Vision with Service Mesh (DEV312-S) - AWS r...

Vanguard's Journey with Tableau to the AWS Cloud (FSV307-S) - AWS re:Invent 2018

Under the Hood: How Amazon Uses AWS Services for Analytics at a Massive Scale...

국내 미디어 고객사의 AWS 활용 사례 - POOQ 서비스, 콘텐츠연합플랫폼::조휘열::AWS Summit Seoul 2018

Chaos Engineering

Building Highly Scalable Retail Order Management Systems with Serverless

Modernizing Media Supply Chains with AWS Serverless (API301) - AWS re:Invent ...

Digital Transformation Through APIs (SRV323) - AWS re:Invent 2018

AWS Global Infrastructure Foundations

Mais de Amazon Web Services

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Amazon Web Services

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Amazon Web Services

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Esegui pod serverless con Amazon EKS e AWS Fargate

Amazon Web Services

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Costruire Applicazioni Moderne con AWS

Amazon Web Services

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Come spendere fino al 90% in meno con i container e le istanze spot

Amazon Web Services

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

Open banking as a service

Amazon Web Services

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Amazon Web Services

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Amazon Web Services

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Amazon Web Services

Computer Vision con AWS

Amazon Web Services

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Amazon Web Services

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

API moderne real-time per applicazioni mobili e web

Amazon Web Services

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Amazon Web Services

Tools for building your MVP on AWS

Amazon Web Services

How to Build a Winning Pitch Deck

Amazon Web Services

Building a web application without servers

Amazon Web Services

Fundraising Essentials

Amazon Web Services

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Amazon Web Services

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

Introduzione a Amazon Elastic Container Service

Amazon Web Services

Mais de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Esegui pod serverless con Amazon EKS e AWS Fargate

Costruire Applicazioni Moderne con AWS

Come spendere fino al 90% in meno con i container e le istanze spot

Open banking as a service

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Computer Vision con AWS

Database Oracle e VMware Cloud on AWS i miti da sfatare

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

API moderne real-time per applicazioni mobili e web

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Tools for building your MVP on AWS

How to Build a Winning Pitch Deck

Building a web application without servers

Fundraising Essentials

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Introduzione a Amazon Elastic Container Service

Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand Ryan Smith Software Development Leader – Bioinformatics Corteva Agriscience E N T 2 1 8 - S Scott Warren Cloud Architect Sogeti USA

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Where We Started • Every 6 hours Corteva produces as much Genetic data as existed in the entire public sphere in 2008 • On premises compute and storage demands were becoming unsustainable • 35 node Hadoop cluster with 2PB of storage • Significant increase in future demand

10. 10 Why AWS? • Understood research needs • Amazon service offerings mirrored on premises Hadoop system • Amazon Elastic Map Reduce • Amazon Simple Storage Solution (Amazon S3) • Cost efficiency © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. Our Applications SNPFinder • Whole genome alignment of short reads • Looking for single nucleotide polymorphisms (SNPs) • Input data size 50-500+GB Vector Quality Control (VQC) • Synthesize a DNA fragment to create a transgenic event • Synthesis needs to be quality controlled • Regulatory requirements • Input data size <10MB © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. User Interaction SNPFinder: • Pipeline transforms data into queryable state • Analysis is done ad-hoc through a user interface or API layer VQC • All processing is completed when data enters the application • Users are viewing these results to inform decision making © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. Guiding Principles User Patterns • Time Sensitive Workloads • Small User Base Technical • Serverless • Immutable Infrastructure • Automate Everything © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. Difference in Design • Both application use the same input data • Type of processing, outputs and technical requirements are very different © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. SNPFinder Queries • Position • Purity/Coverage • Neighborhood • Other SNPs • GC% • Repetitive Sequence • Annotations AAATTGAGTACGCGAGCTAGCGAGCTAGAGCGATG © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

44. Results – Business Impact • Eliminate resource contention • Disaster recovery • Our data is now stored in many different physical locations • Lab growth enabled • Data storage is no longer an issue © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018

Semelhante a Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018 (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Petabytes of Data & No Servers: Corteva Scales DNA Analysis to Meet Increasing Business Demand (ENT218-S) - AWS re:Invent 2018