WebHDFS at King - May 2014 Hadoop MeetUp

•

1 gostou•1,185 visualizações

huguk

The latest developments at King on their work with WebHDFS .

Tecnologia Esportes

2
How to turbo charge
your data transfers with
WebHDFS
Andy Done, Data Platform Lead
andy.done@king.com

Old way
hadoop fs –cat /some/path/* | bulk_load my_table
WebHDFS

WebHDFS way
IMPORT INTO TABLE my_table FROM
FILE ‘http://namenode/webhdfs/v1/some/path/file_1’
FILE ‘http://namenode/webhdfs/v1/some/path/file_2’
…
FILE ‘http://namenode/webhdfs/v1/some/path/file_n’
WebHDFS

WebHDFS benefits
•  Simple
•  Efficient
•  Ubiquitous
•  Parallelisable
•  Bidirectional
•  Fast
WebHDFS

Mais conteúdo relacionado

Mais procurados

GlusterFS As an Object StorageKeisuke Takahashi

Windows Server 2016 WebinarMen and Mice

dNFS for DBA'sMarcin Przepiórowski

Apache Traffic Serversupertom

5-WebServers.pptwebhostingguy

are available herewebhostingguy

Cf camp 2019 cfconfig - a new way to manage your cold-fusion engine configOrtus Solutions, Corp

Rubyspec y el largo camino hacia Ruby 1.9David Calavera

[MathWorks] Versioning InfrastructurePerforce

RedisMarc Beaupré-Pham

Apache Traffic Server & LuaKit Chan

WE18_Performance_Up.pptwebhostingguy

HAProxy tech talkicebourg

Curl Tutorial Ankireddy Polu

Hands On Gluster with Jeff DarcyGluster.org

What is new in BIND 9.11?Men and Mice

Web Server Load BalancerMobME Technical

Mini-Training: To cache or not to cacheBetclic Everest Group Tech Team

Clug 2012 March web server optimisationgrooverdan

NGINX: High Performance Load BalancingNGINX, Inc.

Mais procurados (20)

GlusterFS As an Object Storage

Windows Server 2016 Webinar

dNFS for DBA's

Apache Traffic Server

5-WebServers.ppt

are available here

Cf camp 2019 cfconfig - a new way to manage your cold-fusion engine config

Rubyspec y el largo camino hacia Ruby 1.9

[MathWorks] Versioning Infrastructure

Redis

Apache Traffic Server & Lua

WE18_Performance_Up.ppt

HAProxy tech talk

Curl Tutorial

Hands On Gluster with Jeff Darcy

What is new in BIND 9.11?

Web Server Load Balancer

Mini-Training: To cache or not to cache

Clug 2012 March web server optimisation

NGINX: High Performance Load Balancing

Mais de huguk

Data Wrangling on Hadoop - Olivier De Garrigues, Trifactahuguk

ether.camp - Hackathon & ether.camp introhuguk

Google Cloud Dataproc - Easier, faster, more cost-effective Spark and Hadoophuguk

Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...huguk

Extracting maximum value from data while protecting consumer privacy. Jason ...huguk

Intelligence Augmented vs Artificial Intelligence. Alex Flamant, IBM Watsonhuguk

Streaming Dataflow with Apache Flink huguk

Lambda architecture on Spark, Kafka for real-time large scale MLhuguk

Today’s reality Hadoop with Spark- How to select the best Data Science approa...huguk

Jonathon Southam: Venture Capital, Funding & Pitchinghuguk

Signal Media: Real-Time Media & News Monitoringhuguk

Dean Bryen: Scaling The Platform For Your Startuphuguk

Peter Karney: Intro to the Digital catapulthuguk

Cytora: Real-Time Political Risk Analysishuguk

Cubitic: Predictive Analyticshuguk

Bird.i: Earth Observation Data Made Socialhuguk

Aiseedo: Real Time Machine Intelligencehuguk

Secrets of Spark's success - Deenar Toraskar, Think Reactive huguk

TV Marketing and big data: cat and dog or thick as thieves? Krzysztof Osiewal...huguk

Hadoop - Looking to the Future By Arun Murthyhuguk

Mais de huguk (20)

Data Wrangling on Hadoop - Olivier De Garrigues, Trifacta

ether.camp - Hackathon & ether.camp intro

Google Cloud Dataproc - Easier, faster, more cost-effective Spark and Hadoop

Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...

Extracting maximum value from data while protecting consumer privacy. Jason ...

Intelligence Augmented vs Artificial Intelligence. Alex Flamant, IBM Watson

Streaming Dataflow with Apache Flink

Lambda architecture on Spark, Kafka for real-time large scale ML

Today’s reality Hadoop with Spark- How to select the best Data Science approa...

Jonathon Southam: Venture Capital, Funding & Pitching

Signal Media: Real-Time Media & News Monitoring

Dean Bryen: Scaling The Platform For Your Startup

Peter Karney: Intro to the Digital catapult

Cytora: Real-Time Political Risk Analysis

Cubitic: Predictive Analytics

Bird.i: Earth Observation Data Made Social

Aiseedo: Real Time Machine Intelligence

Secrets of Spark's success - Deenar Toraskar, Think Reactive

TV Marketing and big data: cat and dog or thick as thieves? Krzysztof Osiewal...

Hadoop - Looking to the Future By Arun Murthy

Último

Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University

The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech

"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3

DMCC Future of Trade Web3 - Special EditionDubai Multi Commodity Centre

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada

Commit 2024 - Secret Management made easyAlfredo García Lavilla

What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett

SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero

TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey

DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy

Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB

How to write a Business Continuity PlanDatabarracks

H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati

Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst

Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity

Artificial intelligence in cctv survelliance.pptxhariprasad279825

Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge

Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar

"ML in Production",Oleksandr BaganFwdays