This document discusses using Git to develop code. It begins by motivating the use of Git over the traditional client view by noting that Git allows for code to be backed up, easily shared, and developed in branches without destroying work. It then outlines how Git can provide snapshots of work, create branches for ideas, undo changes, share branches, and commit freely without others seeing. The document briefly introduces the Git object model and suggests a simple Git workflow and using git-p4 to integrate with a Perforce server. It cautions that rebasing rewrites history and can cause issues for downstream branches.
Developer Data Modeling Mistakes: From Postgres to NoSQL
Git censored.key
1. USING GIT TO DEVELOP
1. Motivation.
2. Optional segment: Very brief introduction to the git object model.
3. Optional segment: quickstart guide (requires #2).
4. Suggested git workflow for anet.
5. How to set up automatic backups.
5. MOTIVATION
• Client view is not backed up.
• Not easy to share code.
• If you head down a dead end, you have to manually go back and clean that up.
6. MOTIVATION
• Client view is not backed up.
• Not easy to share code.
• If you head down a dead end, you have to manually go back and clean that up.
• Aborting an idea means its permanent destruction
7. MOTIVATION
• Client view is not backed up.
• Not easy to share code.
• If you head down a dead end, you have to manually go back and clean that up.
• Aborting an idea means its permanent destruction
• Unless you want to keep a client open for it permanently.
8. MOTIVATION
• Client view is not backed up.
• Not easy to share code.
• If you head down a dead end, you have to manually go back and clean that up.
• Aborting an idea means its permanent destruction
• Unless you want to keep a client open for it permanently.
• But really, who does that?
9. MOTIVATION
• Client view is not backed up.
• Not easy to share code.
• If you head down a dead end, you have to manually go back and clean that up.
• Aborting an idea means its permanent destruction
• Unless you want to keep a client open for it permanently.
• But really, who does that?
• PITA if you have to rush a bug fix through on a dirty branch.
11. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
12. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
13. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
• Unwind your personal development history to the last good state with a single
command.
14. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
• Unwind your personal development history to the last good state with a single
command.
• Make it harder to clobber unsaved progress.
15. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
• Unwind your personal development history to the last good state with a single
command.
• Make it harder to clobber unsaved progress.
• Share your speculative branches with other developers.
16. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
• Unwind your personal development history to the last good state with a single
command.
• Make it harder to clobber unsaved progress.
• Share your speculative branches with other developers.
• You can commit at any time. Anything you want. No one will ever see it until you
want them to.
17. GIT CAN
• Provide a way for you to make nightly or hourly snapshots of your work.
• Let you create as many personal branches as you want, one for each idea.
• Unwind your personal development history to the last good state with a single
command.
• Make it harder to clobber unsaved progress.
• Share your speculative branches with other developers.
• You can commit at any time. Anything you want. No one will ever see it until you
want them to.
18. GIT OBJECT MODEL
• 5 slides, 5 minutes. Do you want to continue? [Yes]/[No]
24. 3 Types of Objects
• Blobs
• Trees
• Commits
• Refs
25. 3 Types of Objects
• Blobs
• Trees
• Commits
• Refs
• Tags
26. Blobs
• Blobs == The Contents Of A Regular
System File.
• uniquely identified by a sha-1
• Identical files are only _ever_ stored once
in a git object store.
• Git doesn’t store changes. It stores blobs.
Don’t listen to Joel Spoelsky. [1][2].
27. Trees
• A tree of blobs and trees.
• Not file-system dependent*
• An identically structured tree of identical
blobs with identical permissions always
has the same sha-1 hash.
• *but it does store an internal
representation of POSIX-like file
permissions
28. Commits
• A pointer to a tree
• A pointer to the previous commit on the
branch
• Metadata (commit message, time,
author)
• Most of the time, you interact with
commits
29. Refs ands Tags
• Pointers to a commit
• “All problems in computer science can
be solved by another level of
indirection” - Butler Lampson
31. Using Git
Do you want to hear this? [No]
Git changes your “working directory” to look like the tree
pointed at by a commit. (via git checkout).
32. Using Git
Do you want to hear this? [No]
Git changes your “working directory” to look like the tree
pointed at by a commit. (via git checkout).
This is how you revisit prior states, which is git’s primary
purpose.
33. Using Git
Do you want to hear this? [No]
Git changes your “working directory” to look like the tree
pointed at by a commit. (via git checkout).
This is how you revisit prior states, which is git’s primary
purpose.
All other high-level operations in git are for viewing,
creating, pointing at, and modifying commits.
34. Working Directory
Where you make changes to the code.
In our case, probably ~/p4 and children
35. git add <files>
stage the changes in your working directory for being
committed to the current branch (by updating the index).
intermediate step between working directory and the
bowels of the git object store
important difference from P4
Serves a purpose analogous to grouping files by
changelist in perforce.
Can be hunk-level atomic with --patch
36. git commit
“commit” the contents of the index to the permanent
object history.
Creates a new commit object and updates HEAD to point
at the new commit object.
37. Viewing History
git log - show commit metadata
git show - does a lot of stuff
git status - also does a lot of stuff
git diff - diff blobs, trees, and commits
38. The Docs Are Better
To learn how to use git, you can try the quickstart guides
(linked on the wiki).
It did take me a while to grok this stuff.
The time investment paid off quickly.
39. SIMPLE GIT WORKFLOW
• Initialize your p4 client view as a git repo.
• in bash_profile: source /home/mkramer/.gitstuff
• adds git, git-p4, python (2.6) to your $PATH
• modifies your $PS1 to have git-specific information
• If you define $MANPATH, appends git man pages to it
• for now, building from source
• prodsys will support if enough use it
• cd ~/p4; git init; git add -A; git commit -m “Taking the red pill”;
• Work as normal. Commit your intercolary changes to git as you see fit.
• When you are ready, p4 submit.
• I did this for a few months, and it works great.
40. PULLING FROM UPSTREAM
• You need to periodically integrate changes from the Perforce server.
• The simple approach is to just fullsync as normal. Git registers the changes to the
files just as if you had typed them by hand. Then you commit them as a normal
commit.
• git stash; fullsync; git commit -a -m “Pull down changes from p4”; git stash pop
• stash saves the diffs in your working directory to a temporary stack and makes
your working directory look like HEAD.
• If you have merge conflicts with p4 head, stash pop will fail and you will have to
resolve the changes using the convenient git mergetool.
• You were going to have to codecheck the conflict resolution anyway.
• But you don’t have to resolve if you don’t want to: just accept yours and let the
integrators handle it.
• It’s easier to resolve conflicts when you pull from upstream continuously
41. SIMPLE WORKFLOW REPOS LOOK LIKE THIS
• Pink boxes are commits made to pull p4 changes down.
master
HEAD master master~1 master~2 master~3
refs/heads/topic topic topic~1
43. git-p4
• git-p4 is glue that creates a 1:1 relationship between git commits and p4
changelists.
44. git-p4
• git-p4 is glue that creates a 1:1 relationship between git commits and p4
changelists.
• This is opposed to the naive way, where you just create a new commit every time
you fullsync, and a new changelist every time you p4 submit.
45. git-p4
• git-p4 is glue that creates a 1:1 relationship between git commits and p4
changelists.
• This is opposed to the naive way, where you just create a new commit every time
you fullsync, and a new changelist every time you p4 submit.
• It does this by sucking down p4 changelists and turning them into commits
(metadata and all) when you git-p4 sync,
46. git-p4
• git-p4 is glue that creates a 1:1 relationship between git commits and p4
changelists.
• This is opposed to the naive way, where you just create a new commit every time
you fullsync, and a new changelist every time you p4 submit.
• It does this by sucking down p4 changelists and turning them into commits
(metadata and all) when you git-p4 sync,
• and by turning commits into p4 changelists when you git-p4 submit.
47. git-p4: WHY?
• Inspect perforce history from the ease and comfort of git.
• Your commits are never out of sync with your changelists.
• possible convenience in git-p4 rebase
• git bisect - binary search through commits to find the one that introduced the
regression
48. git-p4 SETUP
• initialize your client view as in simple case
• cd ~/p4 && git init && git commit -a -m “Red Pill”
• git-p4 sync --use-client-spec
• The first time, this pulls down the head revision of the client view of your depot
and creates a single, parentless commit out of it, storing a reference to that
commit in ~/p4/.git/refs/remotes/p4/master
• Subsequent times, it finds changelists that have been added since the last time
you called git-p4 sync, makes a commit out of each changelist, and applies them
in order on top of the commit pointed to by refs/remotes/p4/master
• It’s now up to you to pull the changes from p4 into your local branch.
50. git-p4: REBASING FROM UPSTREAM
• git-p4 provides a convenience wrapper, git-p4 rebase, that does the equivalent of
the following:
• git-p4 sync
• git stash
• git rebase refs/remotes/p4/master
• git stash pop
51. git-p4: REBASING FROM UPSTREAM
• git-p4 provides a convenience wrapper, git-p4 rebase, that does the equivalent of
the following:
• git-p4 sync
• git stash
• git rebase refs/remotes/p4/master
• git stash pop
• I’m not sure that you’d ever want to do this
52. git-p4: REBASING FROM UPSTREAM
• git-p4 provides a convenience wrapper, git-p4 rebase, that does the equivalent of
the following:
• git-p4 sync
• git stash
• git rebase refs/remotes/p4/master
• git stash pop
• I’m not sure that you’d ever want to do this
• It depends on whether your topic branches are all based on the remote ref or
whether they’re based on each other
53. git-p4: REBASING FROM UPSTREAM
• git-p4 provides a convenience wrapper, git-p4 rebase, that does the equivalent of
the following:
• git-p4 sync
• git stash
• git rebase refs/remotes/p4/master
• git stash pop
• I’m not sure that you’d ever want to do this
• It depends on whether your topic branches are all based on the remote ref or
whether they’re based on each other
• You really ought to understand what this is doing before you do it
54. git-p4: REBASING FROM UPSTREAM
• git-p4 provides a convenience wrapper, git-p4 rebase, that does the equivalent of
the following:
• git-p4 sync
• git stash
• git rebase refs/remotes/p4/master
• git stash pop
• I’m not sure that you’d ever want to do this
• It depends on whether your topic branches are all based on the remote ref or
whether they’re based on each other
• You really ought to understand what this is doing before you do it
• Let’s go through it now
57. WHAT REBASING DOES
• git rebase is a very powerful tool
• allows you to rewrite history
58. WHAT REBASING DOES
• git rebase is a very powerful tool
• allows you to rewrite history
• allows you to drop commits, squash commits together, split commits apart,
59. WHAT REBASING DOES
• git rebase is a very powerful tool
• allows you to rewrite history
• allows you to drop commits, squash commits together, split commits apart,
• alter history so that branches are based on other commits (which is where the
command got its name, though it really does way more than that)
60. WHAT REBASING DOES
• git rebase is a very powerful tool
• allows you to rewrite history
• allows you to drop commits, squash commits together, split commits apart,
• alter history so that branches are based on other commits (which is where the
command got its name, though it really does way more than that)
• there’s a catch
61. WHAT REBASING DOES
• git rebase is a very powerful tool
• allows you to rewrite history
• allows you to drop commits, squash commits together, split commits apart,
• alter history so that branches are based on other commits (which is where the
command got its name, though it really does way more than that)
• there’s a catch
• Downstream branches don’t get the memo
62. THE CATCH
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master
refs/heads/topic topic topic~1
63. THE CATCH
• Simple example from the manpage. Your repository looks like this:
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master
refs/heads/topic topic topic~1
64. THE CATCH
• Simple example from the manpage. Your repository looks like this:
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master
refs/heads/topic topic topic~1
• Circular figures are refs. Angular figures are commits.
• A branch is defined as the commit pointed to by a ref, and all of its parents.
• master~3 actually points to the commit pointed to by the ref in this diagram, but
this serves for simplicity. (git documentation equivocates between refs and the
commits that they point at all the time).
65. THEN YOU git-p4 sync
• Again, this sucks changelists down from p4, turns them into commits, then applies
them on top of refs/remotes/p4/master
refs/remotes/p4/master
refs/remotes/p4/master~1
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master~2
refs/heads/topic topic topic~1
66. THEN YOU git-p4 sync
• Again, this sucks changelists down from p4, turns them into commits, then applies
them on top of refs/remotes/p4/master
• So far, so good.
• master~2 and master~3 are actually nameable by that
syntax. ~n means “the nth parent” (following the leftmost
parent, in the case of merges). refs/remotes/p4/master
• master~2 and master~3 are also nameable with topic~2 and
topic~3
refs/remotes/p4/master~1
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master~2
refs/heads/topic topic topic~1
67. THEN YOU REBASE
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
master master~1 topic~2 topic~3 refs/remotes/p4/master~2
68. THEN YOU REBASE
• Rebase master onto the commit now pointed to by refs/remotes/p4/master, and you
get this mess.
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
master master~1 topic~2 topic~3 refs/remotes/p4/master~2
69. THEN YOU REBASE
• Rebase master onto the commit now pointed to by refs/remotes/p4/master, and you
get this mess.
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
master master~1 topic~2 topic~3 refs/remotes/p4/master~2
• Topic is pointing at the wrong place. It does not see the upstream changes
• As a side note, master and master~1 are “dangling commits”, because they aren’t
reachable by any ref. You can’t name them with “master” any more, as that points
at master`.
70. MERGE NOW, THINGS GET UGLY PERMANENTLY
master~1 master~2 master~3 master~4 refs/remotes/p4/master
master
HEAD master
refs/remotes/p4/master~1
refs/heads/topic topic topic~1 topic~2 topic~3 refs/remotes/p4/master~2
71. MERGE NOW, THINGS GET UGLY PERMANENTLY
• Say you merge topic back into master
master~1 master~2 master~3 master~4 refs/remotes/p4/master
master
HEAD master
refs/remotes/p4/master~1
refs/heads/topic topic topic~1 topic~2 topic~3 refs/remotes/p4/master~2
72. MERGE NOW, THINGS GET UGLY PERMANENTLY
• Say you merge topic back into master
master~1 master~2 master~3 master~4 refs/remotes/p4/master
master
HEAD master
refs/remotes/p4/master~1
refs/heads/topic topic topic~1 topic~2 topic~3 refs/remotes/p4/master~2
• topic~2 and 3 shouldn’t be there. We have tried to rewrite history so that they never
existed, but here they are.
• Side note: the merge commit points to two parent commits.
• Also: Yes, refs/heads/topic will still be there. You might want to delete it now (with git
branch -d topic). You typically don’t merge “upwards” until your topic branches are
finished.
73. RECOVERING FROM UPSTREAM REBASE
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
master master~1 topic~2 topic~3 refs/remotes/p4/master~2
74. RECOVERING FROM UPSTREAM REBASE
• Back up. Here’s where we are after git-p4 rebase
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
master master~1 topic~2 topic~3 refs/remotes/p4/master~2
75. REBASE ALL OF YOUR BRANCHES
• An upstream rebase has to cascade down.
• This is what you’d want to do, probably:
• git checkout topic && git rebase --onto master~1
refs/heads/topic topic topic~1
HEAD
master~1` master~2` master~3` refs/remotes/p4/master
master master`
refs/remotes/p4/master~1
refs/remotes/p4/master~2
76. DON’T USE git-p4 rebase
• Else you’ll get this:
refs/heads/topic topic topic~1 topic~2 topic~3
refs/remotes/p4/master
HEAD master` master~1` master~2` master~3`
refs/remotes/p4/master~1
master
• This isn’t what you wanted, is it? refs/remotes/p4/master~2
• Oh, it is?
• Well, if you base all of your topics off of refs/remotes/p4/master, then this
workflow works great!
• People do work this way.
77. REBASE ONTO master
• A lot of people just keep their topics rebased onto master
master
HEAD master` master~1` master~2` master~3` refs/remotes/p4/master
refs/heads/topic topic topic~1 refs/remotes/p4/master~1
• So: git checkout master; git-p4 rebase; git checkout topic; git
rebase --onto master; git checkout master refs/remotes/p4/master~2
• Tragically, have to rebase --onto master for every topic
• You’d have to git-p4 rebase for every topic, anyway. So it’s
really up to you.
• You can mix and match these techniques
78. ALTERNATIVE TO REBASING
• Merging.
• After a git-p4 sync, you could merge instead:
refs/remotes/p4/master
refs/remotes/p4/master~1
master
HEAD master master~1 master~2 master~3 refs/remotes/p4/master~2
refs/heads/topic topic topic~1
79. MERGING cont
• A merging flow looks like this.
refs/remotes/p4/master
refs/remotes/p4/master~1
master master
refs/remotes/p4/master~2
HEAD master~1 master~2 master~3 master~4
refs/heads/topic topic topic~1
80. MERGING contd
• Continually merging “down” from the “upstream” produces a stitching pattern
master
refs/remotes/p4/master
refs/remotes/p4/master~1
HEAD refs/remotes/p4/master~2
master
master~1
refs/remotes/p4/master~3
master~2 master~3 master~4 master~5
refs/heads/topic topic topic~1
81. REBASE YOUR TOPICS
master
refs/remotes/p4/master
refs/remotes/p4/master~1
HEAD refs/remotes/p4/master~2
master
master~1
refs/remotes/p4/master~3
master~2 master~3 master~4 master~5
refs/heads/topic topic topic~1
82. BACKUPS
• Git only ever stores a blob one time.
• We can use this to create a master backup repository where everyone can push all
their branches. Only the files that have changed will be saved again.
• This is already done. The repo lives in /home/mkramer/omni-backup for now.
• It automatically fetches any repo it finds in ~/p4 every night. So if you’ve got your
repo there, you’re done. (You can confirm this by stashing your work and then
checking out /home/mkramer/omni-backup/refs/remotes/$your_user_name/
$your_branch_name. It should reflect the state of that branch from last night.)
• If you want a remote to get sucked in from elsewhere, ask to have the remote set
up.
83. I JUST WANT BACKUPS
• Use the simple workflow. Setup:
• source /home/mkramer/.gitstuff
• cd ~/p4; git init; git commit --all -m “init”
• periodically: git stash; fullsync; git commit --all -m “p4”; git stash pop
• Just don’t branch or use any of git’s features.
• Done.
master
HEAD master master~1 master~2 master~3