Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher

INVERTING 80/20: BEYOND BESPOKE
BIG DATA
Ari Gesher
Palantir Technologies
Wednesday, March 26, 14
SLIDES BY 3/14 at NOON - TBD.....

INVERTING 80/20
Beyond Bespoke Big Data
(or how operating systems change everything)
mailto: agesher@palantir.com
twitter: @alephbass
Ari Gesher, Engineering Ambassador
! •! Hiring, hiring, hiring - the only way to have this problem (and it’s a good one to have) is to have a hiring machine that’s kicking ass.
! •! Design for growth - make sure your systems (IT, food, etc) can handle exponential growth. Build small, but design scalable
! •! Be most worried about long-lead time resources: hiring, leadership, and real estate
! •! Culture: how it changes, how to preserve it
! •! Discard your old biases - project managers enable scale, but process hinders at small scale

Artillery tables, calculated by people called ‘computers’

Eniac, 1945

OS/360 becomes
THE computer

Yak Shavers, Inc
BEFORE OPERATING SYSTEMS

Yak Shavers, Inc
• Programmers had to write the minutiae of hardware, error codes, etc

Yak Shavers, Inc
• Writing directly for the hardware of a single computer

Yak Shavers, Inc
• Programmers started with nothing but primitive functionality

Yak Shavers, Inc
• Programmers started with nothing but primitive functionality
• Very little of the eﬀort went into building the actual intent of the
software

yak shaving, n

yak shaving, n
1. Any apparently useless activity which, by
allowing you to overcome intermediate
diﬃculties, allows you to solve a
larger problem.

yak shaving
Any apparently useless activity which, by allowing
you to overcome intermediate diﬃculties, allows
you to solve a larger problem.

Yaks keep hair
AFTER OPERATING SYSTEMS

Yaks keep hair
• OS API is a clean abstraction, hardware agnostic

Yaks keep hair
• Hardware can change and code still works

Yaks keep hair
• Common functionality (GUI, networking, ﬁles) shared by all programs

Yaks keep hair
• Common functionality (GUI, networking, ﬁles) shared by all programs
• Majority of eﬀort goes into primary purpose

THE RIGHT ABSTRACTION

abstraction creates
efficiency

abstraction creates
SAFETY

abstraction creates
innovation

big data, n
1. Data sets that are too large and complex to
manipulate or interrogate with standard
methods or tools (OED, 2013).
2. A set of technologies and tools designed to
address large-scale data analysis problems.
3. A IT buzzword from the early teens of the
21st century.

OPERATING SYSTEM ABSTRACTIONS

• Device Drivers

• Device Drivers
• Virtual Machine

• Device Drivers
• Virtual Machine
• File system

• Device Drivers
• Virtual Machine
• File system
• Permissions / Access Controls

• Device Drivers
• Virtual Machine
• File system
• Networking

• Device Drivers
• Virtual Machine
• File system
• Networking
• GUI Toolkits / UI Frameworks

Yaks keep hair
BIG DATA ABSTRACTIONS

Yaks keep hair
• Data Integration Standards / Pipelines

Yaks keep hair
• Virtual object model / Composition Rules

Yaks keep hair
• Object revisioning

Yaks keep hair
• Permissions / Access Control

Yaks keep hair
• Data Exchange / Translation

Yaks keep hair
• Data Exchange / Translation
• Interactive, integrated, extensible visualization & analysis environment

MOVIE ON NEXT SLIDE - NO AUDIO

MOVIE FILE _ NO AUDIO

FAMILIAR

INTERACTIVE

EXPRESSIVE

Questions?
agesher@palantir.com

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher

Semelhante a Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher (20)

Mais de Gigaom

Mais de Gigaom (20)

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher