My mom told me that Git doesn’t scale by Vicent Martí

These are the
things you don’t
care about
Saturday, May 11, 13

githubGit hosting:
No longer a pain in the ass

githubGit hosting:
No longer a pain in the ass
for you.
Not for us.
Because, goddamnit,
if I ever ﬁnd the guy who invented
this thing I’m going to hang him from
a fence by his underwear and.Saturday, May 11, 13

Let’s host some
Git repos!
 ﬁle.c
 src
 ﬁle.h
 README.md
 COPYING.md
 .git
 Bare Repository
 HEAD
 index
 objects
 refs
git-daemon

OK, now about
the web...
grit Ruby - Git
interface

OK, now about
the web...
grit  Bare Repo
 Bare Repo
 Bare Repo
Ruby - Git
interface

1VM grit
📽storage
rails app

nVM 📽storage

nVM 📽storage
(GFS)

Rails was making us slow.

Literally.

Time to move to
Real Hardware

ﬁleservers
frontends
📸
db

ﬁleservers
frontends
📸
db
?????????

bert(binary Erlang term)

bert(binary Erlang term)ernie
(not an acronym)

📸chimney
(Redis)
frontend
ﬁleserver
smoke
griterniegrit

Vertical Scaling
#realtalk

bottleneck:
grit

bottleneck:
grit
solution:
gitshell out to

bottleneck:
gitshell out to

bottleneck:
git
solution:
gitshell out to
shell out to

bottleneck:
git
solution:
gitshell out to
shell out to
properly

GUISE
GUISE

GUISE
GUISE
GUISE

GUISE
GUISE
GUISE
...what?

Why don’t we take

Why don’t we take
the Git binary...

Why don’t we take
the Git binary...
yeah?

Why don’t we take
the Git binary...
yeah? and compile it as

Why don’t we take
the Git binary...
a library

Why don’t we take
the Git binary...
a library
oh... go on...

Why don’t we take
the Git binary...
a library
oh... go on...
and link that into

Why don’t we take
the Git binary...
a library
oh... go on...
and link that into
our server

Scientiﬁc
Graph™

MemoryUsage
Time
Scientiﬁc
Graph™

Well, we didn’t think about
freeing memory, but...

THIS IS THE KIND
OF PROBLEM
WE COULD SOLVE
WITH CGI

THIS IS THE KIND
OF PROBLEM
WE COULD SOLVE
WITH CGI
IN 1995

What do you mean
the server died?

die("BUG: non-INDEX attr direction
in a bare repo");
die("a bad revision is needed");
die("'%s' is not a valid
branch name.", name); die("Empty patch.
Aborted.");
die("unable to read index file");
What do you mean
the server died?

libgit2
the “2” means this one
frees memory

libgit2
the “2” means this one
frees memory
NOT ENOUGH
ABSTRACT
FACTORIES

JGit
the “J” means this oneis in Java
...not our thing.

Javaa brief timeline
New companies
don’t use Java
because it’s
not like Unix
1995
New companies
use Java
because it’s
new and shiny
1997
New companies
don’t use Java
because it’s
ooooooold
2005
New companies
use the JVM
because WEBSCALE
2011

Javaa brief timeline
New companies
don’t use Java
because it’s
not like Unix
1995
New companies
use Java
because it’s
new and shiny
1997
New companies
don’t use Java
because it’s
ooooooold
2005
New companies
use the JVM
because WEBSCALE
2011
github

If you think you understand
the JVM, you are either:

a) Very smart

a) Very smart
b) Very wrong

Some people think
that github is a
Rails shop
Ruby shop.or even a

Some people think
that github is a
Rails shop
Ruby shop.or even a
github is a
Unix shop
and everything else is
just a detail.

libgit2So,

Good Heavens,
just look at the
time.
It’s NoSQL o’clock
NoSQL
NoSQL NoSQL
NoSQL
NoSQL NoSQL
NoSQLNoSQL

...do you even

...do you even
mongo?

a b r i e f i n t r o d u c t i o n
t o t h e G i t d a t a m o d e l

 ﬁle.c
 src
 ﬁle.h
 README.md
 COPYING.md

 file.c
 src
 file.h
 README.md
 COPYING.md
tree
src/
README.md
COPYING.md
tree
file.c
file.h
blob
blob
blob
blob

commit
parent
tree T
metadata

commit
T
commit
T
commit
T
commit
T
commit
T
commit
T
Behold,
a graph.

Well that was easy.

master
Oh
god
kill
me

Li le known
torture
methods:

warning:
the rabbit
hole is
pretty deep

Git doesn’t give
a #!%$ about CAP

Number of hops on a complex query
1,000,000

1,000,000
Required hops for a successful query
1,000,000

1,000,000
Required hops for a successful query
1,000,000
Replica count to ensure 100% availability
a metric shitton

We could ﬁx it.

We could ﬁx it.
But we won’t.

GitRPC
Less.

GitRPC
Rugged
libgit2
server
Ruby
Ruby
C

📸chimney
(Redis)
frontend
ﬁleserver
smoke
grit
ernie-
corn
grit
GitRPC GitRPC

📸chimney
(Redis)
frontend ﬁleserver
GitRPC GitRPC
serverclient

evolutionary(disappointing?)

We’ve had a
lot of

We’ve had a
lot of hard
engineering
challenges

We tackled them by:

Using the
most reliable
tools we know.

Challenging ourselves to build
the simplest thing.
Not because it’s easy,
but because it works.

Innovating
where it really ma ers.

revolutionary
product
building a
revolutionary
backend.
not a

Having fun

My mom told me that Git doesn’t scale by Vicent Martí

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a My mom told me that Git doesn’t scale by Vicent Martí

Semelhante a My mom told me that Git doesn’t scale by Vicent Martí (20)

Mais de Codemotion

Mais de Codemotion (20)

Último

Último (20)

My mom told me that Git doesn’t scale by Vicent Martí