<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Version control for fun and profit
Step1:  A repository
Step2:  And this is pretty much the essence of Git!
First
Step3:  Other settings
Change how you will edit text files (it will often ask you to edit messages and other information, and thus wants to know how you like to edit your files)
Step4:  Password memory
Set git to use the credential memory cache so we don't have to retype passwords too frequently. On Linux, you should run the following (note that this requires git version 1.7.10 or newer)
Step5:  Github offers in its help pages instructions on how to configure the credentials helper for Mac OSX and Windows.
Double-checking the result
Step6:  Stage 1
Step7:  git init
Step8:  Note
Step9:  Now let's edit our first file in the test directory with a text editor... I'm doing it programatically here for automation purposes, but you'd normally be editing by hand
Step10:  git add
Step11:  We can now ask git about what happened with status
Step12:  git commit
Step13:  In the commit above, we  used the -m flag to specify a message at the command line.
If we don't do that, git will open the editor we specified in our configuration above and require that we enter a message.
By default, git refuses to record changes that don't have a message to go along with them (though you can obviously 'cheat' by using an empty or meaningless string
Step14:  git diff
Step15:  And now we can ask git what is different
Step16:  The cycle of git virtue
Step17:  git log revisited
First, let's see what the log shows us now
Step18:  Sometimes it's handy to see a very summarized version of the log
Step19:  Defining an alias
Git supports aliases
Step20:  git mv and rm
Step21:  Note that these changes must be committed too, to become permanent!  In git's world, until something hasn't been committed, it isn't permanently recorded anywhere.
Step22:  And git rm works in a similar fashion.
Exercise
Add a new file file2.txt, commit it, make some changes to it, commit them again, and then remove it (and don't forget to commit this last step!).
2. Single Local user, branching
What is a branch?  Simply a label for the 'current' commit in a sequence of ongoing commits
Step23:  We are now going to try two different routes of development
Step24:  3. Using remotes as a single user
We are now going to introduce the concept of a remote repository
Step25:  Since the above cell didn't produce any output after the git remote -v call, it means we have no remote repositories configured.
Configuring a remote
Log into GitHub, go to the new repository page and make a repository called test.
Do not check the box that says Initialize this repository with a README, since we already have an existing repository here.  That option is useful when you're starting first at Github and don't have a repo made already on a local computer.
We can now follow the instructions from the next page
Step26:  Let's see the remote situation again
Step27:  Pushing changes to a remote repository
Now push the master branch to the remote named origin
Step28:  We can now see this repository publicly on github.
Using Git to Sync Work
Let's see how this can be useful for backup and syncing work between two different computers.  I'll simulate a 2nd computer by working in a different directory...
Step29:  Let's now make some changes in one 'computer' and synchronize them on the second.
Step30:  Now we put this new work up on the github server so it's available from the internet
Step31:  Now let's fetch that work from machine #1
Step32:  An important aside
Step33:  And now we go back to the master branch, where we change the same file
Step34:  The conflict...
So now let's see what happens if we try to merge the trouble branch into master
Step35:  Let's see what git has put into our file
Step36:  At this point, we go into the file with a text editor, decide which changes to keep, and make a new commit that records our decision.  I've now made the edits, in this case I decided that both pieces of text were useful, but integrated them with some changes
Step37:  Let's then make our new commit
<ASSISTANT_TASK:>
Python Code:
!ls
Explanation: Version control for fun and profit:
Git: the tool you didn't know you needed
Sources of this material:
This tutorial is adapted from 
"Version Control for Fun and Profit" by Fernando Perez
For an excellent list of Git resources for scientists, see Fernando's Page.
Fernando's original notebook specifically mentions two references he drew from:
"Git for Scientists: A Tutorial" by John McDonnell 
Emanuele Olivetti's lecture notes and exercises from the G-Node summer school on Advanced Scientific Programming in Python.
Via Fernando, some of the images below are copied from the Pro Git book
Also see J.R. Johansson's tutorial on version control, part of his excellent series Lectures on Scientific Computing with Python
What is Version Control?
From Wikipedia:
“Revision control, also known as version control, source control
or software configuration management (SCM), is the
management of changes to documents, programs, and other information stored as computer files.”
Reproducibility?
Tracking and recreating every step of your work
In the software world: it's called Version Control!
What do (good) version control tools give you?
Peace of mind (backups)
Freedom (exploratory branching)
Collaboration (synchronization)
Git is an enabling technology: Use version control for everything
Paper writing (never get paper_v5_jake_final_oct22_9.tex
  by email again!)
Grant writing
Everyday research
Teaching (never accept an emailed homework assignment again!)
Code management
Personal website history tracking
The plan for this tutorial
Overview of Git key concepts
Hands-on work with Git
5 "stages" of using Git:
Local, single-user, linear workflow
Single local user, branching
Using remotes as a single user
Remotes for collaborating in a small team
Full-contact github: distributed collaboration with large teams
High level picture: overview of key concepts
The commit: a snapshot of work at a point in time
Credit: ProGit book, by Scott Chacon, CC License.
Looking at my current directory:
End of explanation
import sha
# Our first commit
data1 = 'This is the start of my paper2.'
meta1 = 'date: 1/1/12'
hash1 = sha.sha(data1 + meta1).hexdigest()
print('Hash:', hash1)
# Our second commit, linked to the first
data2 = 'Some more text in my paper...'
meta2 = 'date: 1/2/12'
# Note we add the parent hash here!
hash2 = sha.sha(data2 + meta2 + hash1).hexdigest()
print('Hash:', hash2)
Explanation: A repository: a group of linked commits
Note: these form a Directed Acyclic Graph (DAG), with nodes identified by their hash.
A hash: a fingerprint of the content of each commit and its parent
End of explanation
%%bash
git config --global user.name "John Doe"
git config --global user.email "johndoe@uw.edu"
Explanation: And this is pretty much the essence of Git!
First: Configuring Git
The minimal amount of configuration for git to work without pestering you is to tell it who you are.  All the commands here modify the .gitconfig file in your home
directory.
Modify these before running them:
End of explanation
%%bash
# Put here your preferred editor. If this is not set, git will honor
# the $EDITOR environment variable
git config --global core.editor /usr/bin/nano  # my preferred editor
# On Windows Notepad will do in a pinch,
#   I recommend Notepad++ as a free alternative
# On the mac, you can set nano or emacs as a basic option
%%bash
# And while we're at it, we also turn on the use of color, which is very useful
git config --global color.ui "auto"
Explanation: Other settings
Change how you will edit text files (it will often ask you to edit messages and other information, and thus wants to know how you like to edit your files):
End of explanation
%%bash 
git config --global credential.helper cache
# Set the cache to timeout after 2 hours (setting is in seconds)
git config --global credential.helper 'cache --timeout=7200'
Explanation: Password memory
Set git to use the credential memory cache so we don't have to retype passwords too frequently. On Linux, you should run the following (note that this requires git version 1.7.10 or newer):
End of explanation
!cat ~/.gitconfig
Explanation: Github offers in its help pages instructions on how to configure the credentials helper for Mac OSX and Windows.
Double-checking the result:
End of explanation
!git
Explanation: Stage 1: Local, single-user, linear workflow
Simply type git to see a full list of all the 'core' commands.  We'll now go through most of these via small practical exercises:
End of explanation
%%bash
rm -rf test
git init test
Explanation: git init: create an empty repository
End of explanation
%%bash
cd test
ls
%%bash
cd test
ls -la
%%bash
cd test
ls -l .git
Explanation: Note: all these cells below are meant to be run by you in a terminal  where  you change once to the test directory and continue working there.
Since we are putting all of them here in a single notebook for the purposes of the tutorial, they will all be prepended with the first two lines:
%%bash
cd test
that tell IPython to do that each time.  But you should ignore those two lines and type the rest of each cell yourself in your terminal.
Let's look at what git did:
End of explanation
%%bash
cd test
echo "My first bit of text" > file1.txt
%%bash
cd test
ls -al
Explanation: Now let's edit our first file in the test directory with a text editor... I'm doing it programatically here for automation purposes, but you'd normally be editing by hand
End of explanation
%%bash
cd test
git add file1.txt
Explanation: git add: tell git about this new file
End of explanation
%%bash
cd test
git status
Explanation: We can now ask git about what happened with status:
End of explanation
%%bash
cd test
git commit -a -m "This is our first commit"
Explanation: git commit: permanently record our changes in git's database
For now, we are always going to call git commit either with the -a option or with specific filenames (git commit file1 file2...).
This delays the discussion of an aspect of git called the index (often referred to also as the 'staging area') that we will cover later.  Most everyday work in regular scientific practice doesn't require understanding the extra moving parts that the index involves, so on a first round we'll bypass it.  Later on we will discuss how to use it to achieve more fine-grained control of what and how git records our actions.
End of explanation
%%bash
cd test
git log
Explanation: In the commit above, we  used the -m flag to specify a message at the command line.
If we don't do that, git will open the editor we specified in our configuration above and require that we enter a message.
By default, git refuses to record changes that don't have a message to go along with them (though you can obviously 'cheat' by using an empty or meaningless string: git only tries to facilitate best practices, it's not your nanny).
git log: what has been committed so far
End of explanation
%%bash
cd test
echo "And now some more text..." >> file1.txt
Explanation: git diff: what have I changed?
Let's do a little bit more work... Again, in practice you'll be editing the files by hand, here we do it via shell commands for the sake of automation (and therefore the reproducibility of this tutorial!)
End of explanation
%%bash
cd test
git diff
Explanation: And now we can ask git what is different:
End of explanation
%%bash
cd test
git commit -a -m "I have made great progress on this critical matter."
Explanation: The cycle of git virtue: work, commit, work, commit, ...
End of explanation
%%bash
cd test
git log
Explanation: git log revisited
First, let's see what the log shows us now:
End of explanation
%%bash
cd test
git log --oneline --topo-order --graph
Explanation: Sometimes it's handy to see a very summarized version of the log:
End of explanation
%%bash
cd test
# We create our alias (this saves it in git's permanent configuration file):
git config --global alias.slog "log --oneline --topo-order --graph"
# And now we can use it
git slog
Explanation: Defining an alias
Git supports aliases: new names given to command combinations. Let's make this handy shortlog an alias, so we only have to type git slog and see this compact log:
End of explanation
%%bash
cd test
git mv file1.txt file-newname.txt
git status
Explanation: git mv and rm: moving and removing files
While git add is used to add fils to the list git tracks, we must also tell it if we want their  names to change or for it to stop tracking them.  In familiar Unix fashion, the mv and rm git commands do precisely this:
End of explanation
%%bash
cd test
git commit -a -m"I like this new name better"
echo "Let's look at the log again:"
git slog
Explanation: Note that these changes must be committed too, to become permanent!  In git's world, until something hasn't been committed, it isn't permanently recorded anywhere.
End of explanation
%%bash
cd test
git status
ls
Explanation: And git rm works in a similar fashion.
Exercise
Add a new file file2.txt, commit it, make some changes to it, commit them again, and then remove it (and don't forget to commit this last step!).
2. Single Local user, branching
What is a branch?  Simply a label for the 'current' commit in a sequence of ongoing commits:
Mulitple Branches
There can be multiple branches alive at any point in time; the working directory is the state of a special pointer called HEAD.  In this example there are two branches, master and testing, and testing is the currently active branch since it's what HEAD points to:
Once new commits are made on a branch, HEAD and the branch label move with the new commits:
This allows the history of both branches to diverge:
But based on this graph structure, git can compute the necessary information to merge the divergent branches back and continue with a unified line of development:
Branching Example
Let's now illustrate all of this with a concrete example.  Let's get our bearings first:
End of explanation
%%bash
cd test
git branch experiment
git checkout experiment
%%bash
cd test
echo "Some crazy idea" > experiment.txt
git add experiment.txt
git commit -a -m"Trying something new"
git slog
%%bash
cd test
git checkout master
git slog
%%bash
cd test
echo "All the while, more work goes on in master..." >> file-newname.txt
git commit -a -m"The mainline keeps moving"
git slog
%%bash
cd test
ls
%%bash
cd test
git merge experiment
git slog
Explanation: We are now going to try two different routes of development: on the master branch we will add one file and on the experiment branch, which we will create, we will add a different one.  We will then merge the experimental branch into master.
End of explanation
%%bash
cd test
ls
echo "Let's see if we have any remote repositories here:"
git remote -v
Explanation: 3. Using remotes as a single user
We are now going to introduce the concept of a remote repository: a pointer to another copy of the repository that lives on a different location.  This can be simply a different path on the filesystem or a server on the internet.
For this discussion, we'll be using remotes hosted on the GitHub.com service, but you can equally use other services like BitBucket or Gitorious as well as host your own.
If you don't have a Github account, take a moment now to sign up
git remote: view/modify remote repositories
End of explanation
%%bash
cd test
git remote add origin https://github.com/jakevdp/test.git
Explanation: Since the above cell didn't produce any output after the git remote -v call, it means we have no remote repositories configured.
Configuring a remote
Log into GitHub, go to the new repository page and make a repository called test.
Do not check the box that says Initialize this repository with a README, since we already have an existing repository here.  That option is useful when you're starting first at Github and don't have a repo made already on a local computer.
We can now follow the instructions from the next page:
End of explanation
%%bash
cd test
git remote -v
Explanation: Let's see the remote situation again:
End of explanation
%%bash
cd test
git push origin master
Explanation: Pushing changes to a remote repository
Now push the master branch to the remote named origin:
End of explanation
%%bash
# Here I clone my 'test' repo but with a different name, test2, to simulate a 2nd computer
git clone https://github.com/jakevdp/test.git test2
cd test2
pwd
git remote -v
Explanation: We can now see this repository publicly on github.
Using Git to Sync Work
Let's see how this can be useful for backup and syncing work between two different computers.  I'll simulate a 2nd computer by working in a different directory...
End of explanation
%%bash
cd test2  # working on computer #2
echo "More new content on my experiment" >> experiment.txt
git commit -a -m"More work, on machine #2"
Explanation: Let's now make some changes in one 'computer' and synchronize them on the second.
End of explanation
%%bash
cd test2
git push origin master
Explanation: Now we put this new work up on the github server so it's available from the internet
End of explanation
%%bash
cd test
git pull origin master
Explanation: Now let's fetch that work from machine #1:
End of explanation
%%bash
cd test
git branch trouble
git checkout trouble
echo "This is going to be a problem..." >> experiment.txt
git commit -a -m"Changes in the trouble branch"
Explanation: An important aside: conflict management
While git is very good at merging, if two different branches modify the same file in the same location, it simply can't decide which change should prevail.  At that point, human intervention is necessary to make the decision.  Git will help you by marking the location in the file that has a problem, but it's up to you to resolve the conflict.  Let's see how that works by intentionally creating a conflict.
We start by creating a branch and making a change to our experiment file:
End of explanation
%%bash
cd test
git checkout master
echo "More work on the master branch..." >> experiment.txt
git commit -a -m"Mainline work"
Explanation: And now we go back to the master branch, where we change the same file:
End of explanation
%%bash
cd test
git merge trouble
Explanation: The conflict...
So now let's see what happens if we try to merge the trouble branch into master:
End of explanation
%%bash
cd test
cat experiment.txt
Explanation: Let's see what git has put into our file:
End of explanation
%%bash
cd test
cat experiment.txt
Explanation: At this point, we go into the file with a text editor, decide which changes to keep, and make a new commit that records our decision.  I've now made the edits, in this case I decided that both pieces of text were useful, but integrated them with some changes:
End of explanation
%%bash
cd test
git commit -a -m"Completed merge of trouble, fixing conflicts along the way"
git slog
Explanation: Let's then make our new commit:
End of explanation
<END_TASK>