Developing Unit Testable Software with Hadoop at Expedia

•Transferir como PPTX, PDF•

2 gostaram•1,162 visualizações

Unit testing is an established software engineering practice, yet it can be more challenging to apply in the Hadoop domain. This talk examines what testing options are available with some popular frameworks and discusses in detail how we test our Hive and Cascading applications at Hotels.com.

Tecnologia

Developing unit-testable
software with Hadoop
HUG UK - Jan 13th 2015

12
Big data +
High stakes +
Complexity +
Change +
Unknowns =
Fear

13
Benefits of unit testing
• Build confidence
• Enable change
• Describe behaviour
• Accelerate development

14
Tests need to be:
• Simple and quick to write
• Simple and quick to run

15
Hadoop testing challenges
• Framework modularization issues
• Heavyweight execution engine
• Availability of testing utilities

16
Hadoop testing challenges
• Framework modularization issues
• Heavyweight execution engine
• Availability of testing utilities

25
Hadoop testing challenges
• Framework modularization issues
• Heavyweight execution engine
• Availability of testing utilities

26
Local execution
Framework Local engine
Hive Local-mode
Cascading LocalFlowConnector
Pig Local mode
Crunch MemPipeline

27
Hadoop testing challenges
• Framework modularization issues
• Heavyweight execution engine
• Availability of testing utilities

28
Helper libraries
Framework Library
Hive HiveRunner
Cascading cascading-test
Plunger
Pig PigUnit
Crunch MemPipeline

30
Hive example with HiveRunner
[https://github.com/klarna/HiveRunner]

31
Hive + HiveRunner: pros
• Write/test Hive apps in the same environment
• Seamless UDF development

32
Hive + Runner: cons
• Slow execution
• CSV data – hard to maintain
• Assertions on CSV strings is brittle
• Hadoop compatibility issues

33
Cascading example with Plunger
[https://github.com/HotelsDotCom/plunger]

34
Cascading + Plunger: pros
• Compact tests, well defined scope
• Use standard Java tools
• Fast

35
Cascading + Plunger: cons
• Some tools only appear to work
• False sense of security

36
Measuring coverage
• Identify activated branches
• No tools do this

37
Conclusions
• Testing possible with most frameworks
• Efficacy largely influenced by framework
• Tooling is immature

38
We’re hiring
Java developers
Hadoop developers

40
Attribution
https://flic.kr/p/7XQdXm - Chris Campbell - CC BY-NC 2.0
https://flic.kr/p/4hpX7j - Andrew_Writer - CC BY-NC-ND 2.0
http://bit.ly/1BVe8xH - Prokofiev - CC BY-SA 3.0
Resources
HiveRunner
https://github.com/klarna/HiveRunner
Plunger
https://github.com/HotelsDotCom/plunger

Mais conteúdo relacionado

Mais procurados

Kubernetes provides an automated platform to deployment, scaling and operations of applications across a cluster of hosts. Complementing Kubernetes with a series of build scripts in conjunction with Travis-CI, GitHub, Artifactory, and Google Cloud Platform, we can take code from a merged pull request to a deployed environment with no manual intervention on a highly scaleable and robust infrastructure.

GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...

Oleg Shalygin

Beyond Relational

Lynn Langit

Spca2014 7 tenets of highly scalable applications kapic

NCCOMMS

Benchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease

Lynn Langit

Reliable, Scalable Kubernetes on AWS

Applatix

What do you do when your usual setup or turnkey solution isn’t suited for your workload? Most of the documentation and user feedback that you can find about OpenStack is written for the use-case of running a public facing cloud serving several external customers. When you want to host a single tenant with a single application the problem is completely different, you don't want publicly exposed APIs. You want to ensure optimal resource allocation to maximize your application performance. You want to leverage the fact that you own the infrastructure layer to optimize your instance placement strategy, and to get the best latency and to avoid creating SPOFs using affinity (or anti affinity rules). This talk will focus on what we learned during a two years journey; from getting OpenStack up and running reliably, to investigating performance bottlenecks, to maximizing the performance of our private cloud.

Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...

Pierre GRANDIN

Many applications are initially developed using relational database technology before upsizing to Cassandra as they mature. This presentation will examine the indicators that it is time to start considering a migration away from relational and factors to consider when making the decision to move. We will then discuss different approaches for implementing the migration and how to plan, estimate and manage the work along with key risks and gotchas you may encounter.

Instaclustr: When and how to migrate from a relational database to Cassandra

DataStax Academy

Are you wondering how to solve repetitive tasks with software automation, but you struggle every time you see the word “developer” or “code?” Do you know what APIs are and how they make these tasks easy to solve? In this session, we’ll explore the framework that can apply to mundane tasks (i.e. PowerShell, Pester), and we’ll discuss what open source tools are available to help solve these problems. Walk away with the advice you need to get started!

Automation for Anyone at Nutanix NEXT 2017 US

Chris Wahl

Campus days Azure HDInsight automation

Kenneth Michael Nielsen

Kubernetes is much more than a runtime platform for Docker containers. Through its API not only can you create custom clients, but you can also extend Kubernetes. Those custom Controllers are called Operators and work with application-specific custom resource definitions. Not only can you write those Kubernetes operators in Go, but you can also do this in Java. Within this talk, you will be guided through setting up and your first explorations of the Kubernetes API within a plain Java program. We explore the concepts of resource listeners, programmatic creation of deployments and services and how this can be used for your custom requirements.

Creating a Kubernetes Operator in Java

Rudy De Busscher

Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...

DevOpsDays Tel Aviv

Presentation for Dutch Microsoft TechDays 2015: With ASP.NET 5 comes MVC 6 with a programming model that unifies Web Pages, MVC and Web API. Each of these has been rebuilt to reflect Microsoft's vision of lean and composable web applications. In this session you will see the changes that have been made to the programming model. We will cover topics such as the new POCO controllers, View Components, dependency injection and much more. Plus, you are going to see the significant changes to the ASP.NET runtime on which MVC 6 is built.

MVC 6 - the new unified Web programming model

Alex Thissen

Greetings from AWS User Group Taiwan

Cliff Chao-kuan Lu

Jolt: Distributed, fault-tolerant test running at scale using Mesos

Mesosphere Inc.

ECS19 - Erwin Van Hunen - PnP Provisioning on Steroids

European Collaboration Summit

CCCNA17 Dynamic Roles in CloudStack

ShapeBlue

Breaking the Monolith - Microservice Extraction at SoundCloud

Jan Kischkel

DevOps, Cloud, and the Death of Backup Tape Changers

ke4qqq

The Cloud Native Stack

QAware GmbH

AWS reInvent 2016 recap Taiwan

Shuen-Huei Guan

Mais procurados (20)

GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...

Beyond Relational

Spca2014 7 tenets of highly scalable applications kapic

Benchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease

Reliable, Scalable Kubernetes on AWS

Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...

Instaclustr: When and how to migrate from a relational database to Cassandra

Automation for Anyone at Nutanix NEXT 2017 US

Campus days Azure HDInsight automation

Creating a Kubernetes Operator in Java

Developing the Stratoscale System at Scale - Muli Ben-Yehuda, Stratoscale - D...

MVC 6 - the new unified Web programming model

Greetings from AWS User Group Taiwan

Jolt: Distributed, fault-tolerant test running at scale using Mesos

ECS19 - Erwin Van Hunen - PnP Provisioning on Steroids

CCCNA17 Dynamic Roles in CloudStack

Breaking the Monolith - Microservice Extraction at SoundCloud

DevOps, Cloud, and the Death of Backup Tape Changers

The Cloud Native Stack

AWS reInvent 2016 recap Taiwan

Semelhante a Developing Unit Testable Software with Hadoop at Expedia

DevOps for Big Data - Data 360 2014 Conference

Grid Dynamics

Building the Web with Gradle

Eric Wendelin

A long time ago in a galaxy far, far away only the chosen few could deploy and operate a fully functional Hadoop cluster. Vendors were taking pride in rationalizing this experience to their customers by creating various distributions including Apache Hadoop. It all changed when Cloudera decided to support Apache Bigtop as the first 100% community driven bigdata management distribution of Apache Hadoop. Today, most major commercial distribution of Apache Hadoop are based on Bigtop. Bigtop has won the Hadoop distributions war and is offering a superset of packaged components. In this talk we will focus on practical advice of how to deploy and start operating a Hadoop cluster using Bigtop’s packages and deployment code. We will dive into the details of using packages of Hadoop ecosystem provided by Bigtop and how to build data management pipelines in support your enterprise applications.

Apache Bigtop: a crash course in deploying a Hadoop bigdata management platform

rhatr

12-factor-jruby

Joe Kutner

Neotys organized its first Performance Advisory Council in Scotland, the 14th & 15th of November. With 15 Load Testing experts from several countries (UK, France, New-Zeland, Germany, USA, Australia, India…) we explored several theme around Load Testing such as DevOps, Shift Right, AI etc. By discussing around their experience, the methods they used, their data analysis and their interpretation, we created a lot of high-value added content that you can use to discover what will be the future of Load Testing. You want to know more about this event ? https://www.neotys.com/performance-advisory-council

Neotys PAC - Ian Molyneaux

Neotys_Partner

With all the buzz about DevOps we tend to believe that everyone is doing it when in fact most companies are not because it’s hard to do in a legacy world. And according to IDC –the vast majority of enterprise apps are on-premise. And even in 2018, only 27.8% of the worldwide enterprise applications market are expected to be SaaS-based compared to 16.6% of the market in 2013. During this talk I will point out the difficulties on achieving DevOps Zen for On-Premise applications, and the way that those in the silent majority can get out off the bench and into the DevOps game.

Legacy On Premise Apps Got You Down? No Problem - DevOps for All

Muly Gottlieb

Continuous delivery with Jenkins Enterprise and Deployit

XebiaLabs

Introduction to Google Cloud Services / Platforms

Nilanchal

Self Healing blue/green Deployments with Dynatrace and Keptn

Florian Bacher

HadoopCon- Trend Micro SPN Hadoop Overview

Yafang Chang

Drupal In The Cloud

Jochen Lillich

Making BD Work~TIAS_20150622

Anthony Potappel

DrupalSouth 2015 - Performance: Not an Afterthought

Nick Santamaria

Operating a High Velocity Large Organization with Spring Cloud Microservices

Noriaki Tatsumi

PushToTest enhances overall TestMaker usability with powerful new features: Test Controller Windows, a whole new way to control tests and drill-down into the live results TestMaker Object Designer 2, a whole new quick way to record and playback Web application tests New Results Analysis Options, log individual requests and responses for functional testing, store notes in charts, and combination reports that mix resource monitoring with transaction times Test Object Repositories, efficiently reuse and version control tests among teams of users Join the special live Webinar to see Frank Cohen, CEO and founder at PushToTest, demonstrate the new features and answer your questions.

What Is New In TestMaker 6.5

Clever Moe

Calculating the Savings of Moving Your Drupal Site to the Cloud

Acquia

DrupalCon 2011 Highlight

Supakit Kiatrungrit

Active Cloud DB at CloudComp '10

Chris Bunch

VMworld 2013: Three Advantages of Running Cloud Foundry in a VMware Private C...

VMworld

In this presentation we explain how we use Watir, Ruby, Cumcumber and other supporting technologies to allow end to end testing in MyHeritage. These are the links to resource mentioned in the presentation: Ruby - https://www.ruby-lang.org/en/ Watir - http://watirwebdriver.com/ page-object - https://github.com/cheezy/page-object Selenium Grid - https://github.com/SeleniumHQ/selenium/wiki/Grid2 Selenium-Grid-Extras - https://github.com/groupon/Selenium-Grid-Extras Jenkins - https://jenkins-ci.org/ We also explain how QA automation engineers are an integral part of the Continuous Deployment process at MyHeritage

MyHeritage - End 2 End testing Infra

MatanGoren

Semelhante a Developing Unit Testable Software with Hadoop at Expedia (20)

DevOps for Big Data - Data 360 2014 Conference

Building the Web with Gradle

Apache Bigtop: a crash course in deploying a Hadoop bigdata management platform

12-factor-jruby

Neotys PAC - Ian Molyneaux

Legacy On Premise Apps Got You Down? No Problem - DevOps for All

Continuous delivery with Jenkins Enterprise and Deployit

Introduction to Google Cloud Services / Platforms

Self Healing blue/green Deployments with Dynatrace and Keptn

HadoopCon- Trend Micro SPN Hadoop Overview

Drupal In The Cloud

Making BD Work~TIAS_20150622

DrupalSouth 2015 - Performance: Not an Afterthought

Operating a High Velocity Large Organization with Spring Cloud Microservices

What Is New In TestMaker 6.5

Calculating the Savings of Moving Your Drupal Site to the Cloud

DrupalCon 2011 Highlight

Active Cloud DB at CloudComp '10

VMworld 2013: Three Advantages of Running Cloud Foundry in a VMware Private C...

MyHeritage - End 2 End testing Infra

Mais de huguk

As Hadoop became mainstream, the need to simplify and speed up analytics processes grew rapidly. Data wrangling emerged as a necessary step in any analytical pipeline, and is often considered to be its crux, taking as much as 80% of an analyst's time. In this presentation we will discuss how data wrangling solutions can be leveraged to streamline, strengthen and improve data analytics initiatives on Hadoop, including use cases from Trifacta customers. Bio: Olivier is EMEA Solutions Lead at Trifacta. He has 7 years experience in analytics with prior roles as technical lead for business analytics at Splunk and quantitative analyst at Accenture and Aon.

Data Wrangling on Hadoop - Olivier De Garrigues, Trifacta

huguk

Stephen Taylor is the community manager for Ether Camp. They provide an analysis tool for the Ethereum blockchain, ‘Block Explorer’ and also an ‘Intergrated Development Environment’ (I.D.E) that empowers developers to build, test and deploy applications in a sandbox environment. This November they are launching their second annual hackathon, hack.ether.camp which is aiming to deliver a more sustained approach to the hackathon ideology, by utilising blockchain technology.

ether.camp - Hackathon & ether.camp intro

huguk

At Google Cloud Platform, we're combining the Apache Spark and Hadoop ecosystem with our software and hardware innovations. We want to make these awesome tools easier, faster, and more cost-effective, from 3 to 30,000 cores. This presentation will showcase how Google Cloud Platform is innovating with the goal of bringing the Hadoop ecosystem to everyone. Bio: "I love data because it surrounds us - everything is data. I also love open source software, because it shows what is possible when people come together to solve common problems with technology. While they are awesome on their own, I am passionate about combining the power of open source software with the potential unlimited uses of data. That's why I joined Google. I am a product manager for Google Cloud Platform and manage Cloud Dataproc and Apache Beam (incubating). I've previously spent time hanging out at Disney and Amazon. Beyond Google, love data, amateur radio, Disneyland, photography, running and Legos."

Google Cloud Dataproc - Easier, faster, more cost-effective Spark and Hadoop

huguk

Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...

huguk

Big organisations have a wealth of rich customer data which opens up huge new opportunities. However, they have the challenge of how to extract value from this data while protecting the privacy of their individual customers. He will talk about the risks organisations face, and what they should do about it. He will survey the techniques which can be used to make data safe for analysis, and talk briefly about how they are solving this problem at Privitar.

Extracting maximum value from data while protecting consumer privacy. Jason ...

huguk

IBM is developing the Watson Ecosystem to leverage its Developer Cloud, APIs, Content Store and Talent Hub. This is part of IBM's recent announcement of the $1B investment in Watson as a new business unit including Silicon Alley NYC headquarters. For the first time, IBM will open up Watson as a development platform in the Cloud to spur innovation and fuel a new ecosystem of entrepreneurial software app providers who will bring forward a new generation of applications infused with Watson's cognitive computing intelligence.

Intelligence Augmented vs Artificial Intelligence. Alex Flamant, IBM Watson

huguk

In this talk about Apache Flink we will touch on three main things, an introductory look at Flink, a look under the hood and a demo. * In the introduction we will briefly look at the history of Flink and then go on to the API and different use cases. Here we will also see how it can be deployed in practice and what some of the pitfalls in a cluster setting can be. * In the second section we will look at the streaming execution engine that lies at the heart of Flink. Here we will see what makes it tick and also what distinguishes it from other approaches, such as the mini-batch execution model. Ufuk Celebi - PMC member at Apache Flink and co-founder and software engineer at data Artisans * In the final section we will see a live demo of a fault-tolerant streaming job that performs analysis of the wikipedia edit-stream.

Streaming Dataflow with Apache Flink

huguk

Sean Owen – Director of Data Science @Cloudera Building machine learning models is all well and good, but how do they get productionized into a service? It's a long way from a Python script on a laptop, to a fault-tolerant system that learns continuously, serves thousands of queries per second, and scales to terabytes. The confederation of open source technologies we know as Hadoop now offers data scientists the raw materials from which to assemble an answer: the means to build models but also ingest data and serve queries, at scale. This short talk will introduce Oryx 2, a blueprint for building this type of service on Hadoop technologies. It will survey the problem and the standard technologies and ideas that Oryx 2 combines: Apache Spark, Kafka, HDFS, the lambda architecture, PMML, REST APIs. The talk will touch on a key use case for this architecture -- recommendation engines.

Lambda architecture on Spark, Kafka for real-time large scale ML

huguk

Martin Oberhuber and Eliano Marques, Senior Data Scientists @Think Big International In this talk Think Big International Lead Data Scientists will discuss the options that exist today for engineering and data science teams aiming to use big data patterns to solve new business problems. With the enterprise adoption of the Hadoop ecosystem and the emerging momentum of open source projects like Spark it is becoming mandatory to have an approach that solves for business results but remains flexible to adapt and change with the open source market.

Today’s reality Hadoop with Spark- How to select the best Data Science approa...

huguk

Jonathon Southam: Venture Capital, Funding & Pitching

huguk

Signal Media: Real-Time Media & News Monitoring

huguk

Dean Bryen: Scaling The Platform For Your Startup

huguk

Peter Karney: Intro to the Digital catapult

huguk

Cytora: Real-Time Political Risk Analysis

huguk

Cubitic: Predictive Analytics

huguk

Bird.i: Earth Observation Data Made Social

huguk

Aiseedo: Real Time Machine Intelligence

huguk

This talk will cover the design and implementation decisions that have been key to the success of Apache Spark over other competing cluster computing frameworks. It will be delving into the whitepaper behind Spark and cover the design of Spark RDDs, the abstraction enables the Spark execution engine to be extended to support a wide variety of use cases: Spark SQL, Spark Streaming, MLib and GraphX. RDDs allow Spark to outperform existing models by up to 100x in multi-pass analytics.

Secrets of Spark's success - Deenar Toraskar, Think Reactive

huguk

Technical developments in the area of data warehousing have allowed companies to push their analysis a step further and, therefore, allowed data scientists to deliver more value to business areas. In that session, we will focus on the case of performance marketing at King and demonstrate how we use Hadoop capabilities to exploit user-level data efficiently. That approach results in obtaining a more holistic view in a return-on-investment analysis of TV advertisement.

TV Marketing and big data: cat and dog or thick as thieves? Krzysztof Osiewal...

huguk

Hadoop - Looking to the Future By Arun Murthy (Founder of Hortonworks, Creator of YARN) The Apache Hadoop ecosystem began as just HDFS & MapReduce nearly 10 years ago in 2006. Very much like the Ship of Theseus (http://en.wikipedia.org/wiki/Ship_of_Theseus), Hadoop has undergone incredible amount of transformation from multi-purpose YARN to interactive SQL with Hive/Tez to machine learning with Spark. Much more lies ahead: whether you want sub-second SQL with Hive or use SSDs/Memory effectively in HDFS or manage Metadata-driven security policies in Ranger, the Hadoop ecosystem in the Apache Software Foundation continues to evolve to meet new challenges and use-cases. Arun C Murthy has been involved with Apache Hadoop since the beginning of the project - nearly 10 years now. In the beginning he led MapReduce, went on to create YARN and then drove Tez & the Stinger effort to get to interactive & sub-second Hive. Recently he has been very involved in the Metadata and Governance efforts. In between he founded Hortonworks, the first public Hadoop distribution company.

Hadoop - Looking to the Future By Arun Murthy

huguk

Mais de huguk (20)

Data Wrangling on Hadoop - Olivier De Garrigues, Trifacta

ether.camp - Hackathon & ether.camp intro

Google Cloud Dataproc - Easier, faster, more cost-effective Spark and Hadoop

Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...

Extracting maximum value from data while protecting consumer privacy. Jason ...

Intelligence Augmented vs Artificial Intelligence. Alex Flamant, IBM Watson

Streaming Dataflow with Apache Flink

Lambda architecture on Spark, Kafka for real-time large scale ML

Today’s reality Hadoop with Spark- How to select the best Data Science approa...

Jonathon Southam: Venture Capital, Funding & Pitching

Signal Media: Real-Time Media & News Monitoring

Dean Bryen: Scaling The Platform For Your Startup

Peter Karney: Intro to the Digital catapult

Cytora: Real-Time Political Risk Analysis

Cubitic: Predictive Analytics

Bird.i: Earth Observation Data Made Social

Aiseedo: Real Time Machine Intelligence

Secrets of Spark's success - Deenar Toraskar, Think Reactive

TV Marketing and big data: cat and dog or thick as thieves? Krzysztof Osiewal...

Hadoop - Looking to the Future By Arun Murthy

Último

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

ICT role in 21st century education and its challenges

rafiqahmad00786416

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Vector Search -An Introduction in Oracle Database 23ai.pptx

Remote DBA Services

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

FWD Group - Insurer Innovation Award 2024

The Digital Insurer

CNIC Information System with Pakdata Cf In Pakistan

danishmna97

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Architecting Cloud Native Applications

WSO2

MS Copilot expands with MS Graph connectors

Nanddeep Nachan

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Developing Unit Testable Software with Hadoop at Expedia

1. Developing unit-testable software with Hadoop HUG UK - Jan 13th 2015

8. 8 108 Keywords 1010 Web hits

9. 9 Big data + High stakes = Exciting

10.

11.

12. 12 Big data + High stakes + Complexity + Change + Unknowns = Fear

13. 13 Benefits of unit testing • Build confidence • Enable change • Describe behaviour • Accelerate development

14. 14 Tests need to be: • Simple and quick to write • Simple and quick to run

15. 15 Hadoop testing challenges • Framework modularization issues • Heavyweight execution engine • Availability of testing utilities

16. 16 Hadoop testing challenges • Framework modularization issues • Heavyweight execution engine • Availability of testing utilities

17. 17 Sequential modularization

18. 18 Modularization via encapsulation

19. 19 Hive modularity

20. 20 Cascading modularity

21. 21 Pig modularity

22. 22 Crunch modularity

23. 23 Mapreduce modularity

24.

25. 25 Hadoop testing challenges • Framework modularization issues • Heavyweight execution engine • Availability of testing utilities

26. 26 Local execution Framework Local engine Hive Local-mode Cascading LocalFlowConnector Pig Local mode Crunch MemPipeline

27. 27 Hadoop testing challenges • Framework modularization issues • Heavyweight execution engine • Availability of testing utilities

28. 28 Helper libraries Framework Library Hive HiveRunner Cascading cascading-test Plunger Pig PigUnit Crunch MemPipeline

29. 29 Example Topic | Subtopic

30. 30 Hive example with HiveRunner [https://github.com/klarna/HiveRunner]

31. 31 Hive + HiveRunner: pros • Write/test Hive apps in the same environment • Seamless UDF development

32. 32 Hive + Runner: cons • Slow execution • CSV data – hard to maintain • Assertions on CSV strings is brittle • Hadoop compatibility issues

33. 33 Cascading example with Plunger [https://github.com/HotelsDotCom/plunger]

34. 34 Cascading + Plunger: pros • Compact tests, well defined scope • Use standard Java tools • Fast

35. 35 Cascading + Plunger: cons • Some tools only appear to work • False sense of security

36. 36 Measuring coverage • Identify activated branches • No tools do this

37. 37 Conclusions • Testing possible with most frameworks • Efficacy largely influenced by framework • Tooling is immature

38. 38 We’re hiring Java developers Hadoop developers

39. 39 Questions?

40. 40 Attribution https://flic.kr/p/7XQdXm - Chris Campbell - CC BY-NC 2.0 https://flic.kr/p/4hpX7j - Andrew_Writer - CC BY-NC-ND 2.0 http://bit.ly/1BVe8xH - Prokofiev - CC BY-SA 3.0 Resources HiveRunner https://github.com/klarna/HiveRunner Plunger https://github.com/HotelsDotCom/plunger

Developing Unit Testable Software with Hadoop at Expedia

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Developing Unit Testable Software with Hadoop at Expedia

Semelhante a Developing Unit Testable Software with Hadoop at Expedia (20)

Mais de huguk

Mais de huguk (20)

Último

Último (20)

Developing Unit Testable Software with Hadoop at Expedia