Introduction to Apache Hive

•

34 gostaram•7,211 visualizações

Tapan Avasthi

Tecnologia

APACHE HIVE
(Apache Hadoop Sub Project)

Agenda:
 Story – Making of Apache Hive
 What is Apache Hive
 Physical Layout
 Hive CLI
 Hive QL

Can Elephants Fly?

Concern: Can hadoop be used more efficiently/fruitfully by developers?

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 3

Thinking…. ?
Step 1. Give him Wings

Mr. Hadoop energizing himself.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 5

Thinking… ?
Step 2. Pray to Gravity

Thanks to gravity, sky never fell down on us ;)
But wait 2012 is not yet over. Keep Praying.

Mr. Hadoop enjoying his first air ride.

“God did not create the universe, gravity did” - Stephen Hawking

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 6

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 7

Upshot of the down-fall

Victims Mr. Hadoo
p – The Fly
ing Elephan
t

Blame Gravity! The Fall will have a huge impact.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 8

Saving Life…
Step1. Shrink

BEFORE -

ACME Elephant Shrinker

AFTER -

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 10

Saving Life…
Step2. Genetic Engineering & a bit of magic
BEFORE AFTER

Mr. Hadoop

Ms. Hive

Injecting Insecto-receptors

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 11

Behind the scenes…?

Hive was initially developed by Facebook.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 13

 Hive is a datawarehouse infrastructure built
on top of hadoop.
 Supports analysis of large datasets stored in
Hadoop compatible file systems like HDFS,
Amazon S3 fs.
 Provides SQL-like query language called
HiveQL.
 To accelerate queries, it provides indexing.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 14

 Warehouse directory in hdfs
 /user/hive/warehouse
 Tables ~ Subdirectories of warehouse
 Partitions ~ Subdirectories of corresponding
Table directory.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 15

 Hive Queries are implicitly converted to map-
reduce code by hive engine.
 Compiler translates all the queries into a
directed acyclic graph of map-reduce jobs.
 These map-reduce jobs are sent to hadoop
for execution.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 16

 /user/hive directory is created automatically as soon
as hive session is started first time.
 /user/hive/warehouse directory shall be accessible
by all.
 hadoop dfs -chmod –R 1777 /user/hive/warehouse
 Recommended to activate sticky bit if supported by
the hadoop version installed on cluster.
 /tmp directory shall also be made as a sticky
directory.
 hadoop dfs –chmod –R 1777 /tmp

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 17

 Hive CLI(Command Line Interface) can be
invoked by hive command.
 % hive

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 18

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 19

 DML’s
▪ Select
 DDL’s
▪ SHOW TABLES
▪ CREATE TABLE
▪ ALTER TABLE
▪ DROP TABLE

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 21

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 23

 Normal Tables are created under warehouse
directory. (source Data migrates to warehouse)
 Normal Tables are directly visible through hdfs
directory browsing.
 On Dropping a normal table, the source data and
table meta data both are deleted.
 External Tables read directly from hdfs files.
 External tables not visible in warehouse
directory.
 On Dropping an external table, only the meta
data is deleted but not the source data.

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 24

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 25

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 26

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 27

 Hive QL supports Joins on only equality
expressions. Complex boolean expressions,
inequality conditions are not supported.
 More than 2 tables can be joined.
 Number of map-reduce jobs generated for a
join depend on the columns being used.
 If same col is used for all the tables, then n=1
 Otherwise n>1

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 28

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 29

 HiveQL Doesn’t follow SQL-92 standard
 Lack support
 No Materialized views
 No Transaction level support
 Limited Sub-query support

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 30

Hadoop – Entering into the new world!

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 31

Reach me

Tapan Avasthi
Associate Software Developer Intern, Travelocity Global
tapan.avasthi@travelocity.com
tapan.k.avasthi@gmail.com

© 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 32

Mais conteúdo relacionado

Mais procurados

Hive paris

Szehon Ho

Hive is the main data transformation tool at Criteo, and hundreds of analysts and thousands of automated jobs run Hive queries every day. We evolved Criteo’s Hive platform from an error-prone add-on installed on some spare machines to a best-in-class installation capable of self-healing and automatically scaling to handle its growing load. The resulting platform is based on Mesos. Mesos has allowed Criteo to scale per demand and better utilize resources, iterate on development much faster than on bare metal, and roll out new versions seamlessly without downtime for our users.

Hive on mesos Strata

Szehon Ho

As Hadoop applications move into cloud deployments, object stores become more and more the source and destination of data. But object stores are not filesystems: sometimes they are slower; security is different, What are the secret settings to get maximum performance from queries against data living in cloud object stores? That's at the filesystem client, the file format and the query engine layers? It's even how you lay out the files —the directory structure and the names you give them. We know these things, from our work in all these layers, from the benchmarking we've done —and the support calls we get when people have problems. And now: we'll show you. This talk will start from the ground up "why isn't an object store a filesystem?" issue, showing how that breaks fundamental assumptions in code, and so causes performance issues which you don't get when working with HDFS. We'll look at the ways to get Apache Hive and Spark to work better, looking at optimizations which have been done to enable this —and what work is ongoing. Finally, we'll consider what your own code needs to do in order to adapt to cloud execution.

Dancing elephants - efficiently working with object stores from Apache Spark ...

DataWorks Summit

Big Data Warehousing: Pig vs. Hive Comparison

Caserta

Oracle Migration to Postgres in the Cloud

EDB

Improving Python and Spark Performance and Interoperability with Apache Arrow

Julien Le Dem

The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...

DataWorks Summit/Hadoop Summit

Authoring and Hosting Applications on YARN using Slider

DataWorks Summit

Enabling Diverse Workload Scheduling in YARN

DataWorks Summit

Big Data Certification

Adam Doyle

SQL et in-memory sur Hadoop avec Pivotal et HAWQ

Modern Data Stack France

Hd insight essentials quick view

Rajesh Nadipalli

Internet of things Crash Course Workshop

DataWorks Summit

Hortonworks SmartSense provides proactive recommendations that improve cluster performance, security and operations. And since 30% of issues are configuration related, Hortonworks SmartSense makes an immediate impact on Hadoop system performance and availability, in some cases boosting hardware performance by two times. Learn how SmartSense can help you increase the efficiency of your Hadoop hardware, through customized cluster recommendations. View the on-demand webinar: https://hortonworks.com/webinar/boosts-hadoop-hardware-performance-2x-smartsense/

Double Your Hadoop Hardware Performance with SmartSense

Hortonworks

High-level Programming Languages: Apache Pig and Pig Latin

Pietro Michiardi

Big Data" šodien ir viens no populārākajiem mārketinga saukļiem, kas tiek pamatoti un nepamatoti izmantots, runājot par (lielu?) datu uzglabāšanu un apstrādi. Prezentācijā es aplūkošu, kas tad patiesībā ir "big data" no tehnoloģijju viedokļa, kādi ir galvenie izmantošanas scenāriji un ieguvumi. Prezentācijā apskatīšu tādas tehnoloģijas kā Hadoop, HDFS, MapReduce, Impala, Sparc, Pig, Hive un citas. Tāpat tiks apskatīta integrācija ar tradicionālām DBVS un galvenie izmantošanas scenāriji.

Big data overview by Edgars

Andrejs Vorobjovs

Part five in a five-part series, this webcast will be a demonstration of the integration of Apache Zeppelin and Pivotal HDB. Apache Zeppelin is a web-based notebook that enables interactive data analytics. You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala and more. This webinar will demonstrate the configuration of the psql interpreter and the basic operations of Apache Zeppelin when used in conjunction with Hortonworks HDB.

How to Use Apache Zeppelin with HWX HDB

Hortonworks

Introduction to pig

Ravi Mutyala

DeathStar: Easy, Dynamic, Multi-Tenant HBase via YARN

DataWorks Summit

S3Guard: What's in your consistency model?

Hortonworks

Mais procurados (20)

Hive paris

Hive on mesos Strata

Dancing elephants - efficiently working with object stores from Apache Spark ...

Big Data Warehousing: Pig vs. Hive Comparison

Oracle Migration to Postgres in the Cloud

Improving Python and Spark Performance and Interoperability with Apache Arrow

The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...

Authoring and Hosting Applications on YARN using Slider

Enabling Diverse Workload Scheduling in YARN

Big Data Certification

SQL et in-memory sur Hadoop avec Pivotal et HAWQ

Hd insight essentials quick view

Internet of things Crash Course Workshop

Double Your Hadoop Hardware Performance with SmartSense

High-level Programming Languages: Apache Pig and Pig Latin

Big data overview by Edgars

How to Use Apache Zeppelin with HWX HDB

Introduction to pig

DeathStar: Easy, Dynamic, Multi-Tenant HBase via YARN

S3Guard: What's in your consistency model?

Semelhante a Introduction to Apache Hive

Large Scale Performance Monitoring for ElasticSearch, HBase, Solr, SenseiDB, ...

Sematext Group, Inc.

Hadoop2 new and noteworthy SNIA conf

Sujee Maniyam

Hadoop Overview

EMC

Track B-2: Advancing Collaboration & eLearning to Achieve Mission Goals, ...

scoopnewsgroup

The Evolution and Future of Hadoop Storage （Hadoop Conference Japan 2016キーノート...

Hadoop / Spark Conference Japan

Building infrastructure for Big Data

PromptCloud

Node.js and Photoshop Generator - JSConf Asia 2013

Andy Hall

Paremus Cloud and OSGi Beyond the VM - OSGi Cloud Workshop March 2012

mfrancis

Hadoop-as-a-Service for Lifecycle Management Simplicity

DataWorks Summit

Go daddy.com Cloud Storage Solution (Adam Knapp)

Ontico

HBase and Hadoop at Adobe

Cosmin Lehene

Greenplum Database on HDFS

DataWorks Summit

OWF12/Java Sacha labourey

Paris Open Source Summit

Machine Learning and Hadoop: Present and Future

Data Science London

Hadoop Summit 2012 - Deployment and Operations track Everyone hears about large clusters with thousands of machines and petabytes of storage yet not everyone starts their first Hadoop deployment with dozens of cabinets of equipment. What do you do when you don`t have quite as large of a deployment? What decisions should you make now and which should you postpone for later? This session is for SysAdmins that have not yet or just recently jumped into the Hadoop fray. You will be presented with the knowledge gained from two years of operational experience at a (currently) small Hadoop site. We will discuss things that are initially important for a small (10-100 node) cluster and what happens when you outgrow your first deployment.

Hadoop Operations: Starting Out Small / So Your Cluster Isn't Yahoo-sized (yet)

Michael Arnold

Hadoop operations

DataWorks Summit

Oop2012 keynote Design Driven Development

Michael Chaize

Hadoop Performance at LinkedIn

Allen Wittenauer

eFolder Webinar: How One Partner Leverages Dell AppAssure and StorageCraft

Dropbox

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Cloudera, Inc.

Semelhante a Introduction to Apache Hive (20)

Large Scale Performance Monitoring for ElasticSearch, HBase, Solr, SenseiDB, ...

Hadoop2 new and noteworthy SNIA conf

Hadoop Overview

Track B-2: Advancing Collaboration & eLearning to Achieve Mission Goals, ...

The Evolution and Future of Hadoop Storage （Hadoop Conference Japan 2016キーノート...

Building infrastructure for Big Data

Node.js and Photoshop Generator - JSConf Asia 2013

Paremus Cloud and OSGi Beyond the VM - OSGi Cloud Workshop March 2012

Hadoop-as-a-Service for Lifecycle Management Simplicity

Go daddy.com Cloud Storage Solution (Adam Knapp)

HBase and Hadoop at Adobe

Greenplum Database on HDFS

OWF12/Java Sacha labourey

Machine Learning and Hadoop: Present and Future

Hadoop Operations: Starting Out Small / So Your Cluster Isn't Yahoo-sized (yet)

Hadoop operations

Oop2012 keynote Design Driven Development

Hadoop Performance at LinkedIn

eFolder Webinar: How One Partner Leverages Dell AppAssure and StorageCraft

Hadoop's Impact on the Future of Data Management | Amr Awadallah

Último

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Scaling API-first – The story of a global engineering organization

Radu Cotescu

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Introduction to Apache Hive

1. APACHE HIVE (Apache Hadoop Sub Project) Agenda:  Story – Making of Apache Hive  What is Apache Hive  Physical Layout  Hive CLI  Hive QL

6. Thinking… ? Step 2. Pray to Gravity Thanks to gravity, sky never fell down on us ;) But wait 2012 is not yet over. Keep Praying. Mr. Hadoop enjoying his first air ride. “God did not create the universe, gravity did” - Stephen Hawking © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 6

12.

14.  Hive is a datawarehouse infrastructure built on top of hadoop.  Supports analysis of large datasets stored in Hadoop compatible file systems like HDFS, Amazon S3 fs.  Provides SQL-like query language called HiveQL.  To accelerate queries, it provides indexing. © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 14

15.  Warehouse directory in hdfs  /user/hive/warehouse  Tables ~ Subdirectories of warehouse  Partitions ~ Subdirectories of corresponding Table directory. © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 15

16.  Hive Queries are implicitly converted to map- reduce code by hive engine.  Compiler translates all the queries into a directed acyclic graph of map-reduce jobs.  These map-reduce jobs are sent to hadoop for execution. © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 16

17.  /user/hive directory is created automatically as soon as hive session is started first time.  /user/hive/warehouse directory shall be accessible by all.  hadoop dfs -chmod –R 1777 /user/hive/warehouse  Recommended to activate sticky bit if supported by the hadoop version installed on cluster.  /tmp directory shall also be made as a sticky directory.  hadoop dfs –chmod –R 1777 /tmp © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 17

20.

22.

24.  Normal Tables are created under warehouse directory. (source Data migrates to warehouse)  Normal Tables are directly visible through hdfs directory browsing.  On Dropping a normal table, the source data and table meta data both are deleted.  External Tables read directly from hdfs files.  External tables not visible in warehouse directory.  On Dropping an external table, only the meta data is deleted but not the source data. © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 24

28.  Hive QL supports Joins on only equality expressions. Complex boolean expressions, inequality conditions are not supported.  More than 2 tables can be joined.  Number of map-reduce jobs generated for a join depend on the columns being used.  If same col is used for all the tables, then n=1  Otherwise n>1 © 2012 Sabre Holdings Pvt. Ltd. | All rights reserved 28

Introduction to Apache Hive

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Introduction to Apache Hive

Semelhante a Introduction to Apache Hive (20)

Último

Último (20)

Introduction to Apache Hive