Exposing GIT over HTTP in Ubuntu

Continuing my GIT exploration here I will talk about how to expose GIT over HTTP in UBUNTU

Pre-requisites: -

GIT - GIT Core should be installed (Refer to Working with GIT)
Apache2 - Install it by running the following command: -

sudo apt-get install apache2

The above command will install the apache in the following Directories

1. /etc/apache2 - will contain all the configurations like ports, virtual hosts etc
2. /var/www - Contains the HTML files, CGI scripts etc

GITWEB

GITWEB is a package which can be used to expose GIT repositories over HTTP

It is nothing more than few set of files: -

1. gitweb.css
2. git-logo.png
3. git-favicon.png
4. gitweb.cgi
5. gitweb.js
6. gitweb.conf (Configuration File which usually is found at /etc/gitweb.conf)

sudo apt-get install gitweb will download all of the above files

Preparing Apache for Hosting GIT: -

1. Create a directory by name of "gitweb" under "/var/www".
2. Except gitweb.conf, copy all other files downloaded with GITWEB package to "/var/www/gitweb" directory.
3. Create a another folder by name of "/var/www/gitRepo" and configure your git client and also download the code from git Server (refer to Working with GIT) for all the branches.
4. Open /etc/gitweb.conf and modify the following highlighted Attributes: -

$GIT = "/usr/bin/git";
# path to git projects (.git)
# $projectroot = "/var/cache/git";
$projectroot = "/var/www/gitRepo";

# directory to use for temp files
$git_temp = "/tmp";

# target of the home link on top of all pages
#$home_link = $my_uri || "/";

# Description Text for your Repository
$site_name = "Project Trees";

# html text to include at home page
$home_text = "indextext.html";

# file with project list; by default, simply scan the projectroot dir.
$projects_list = $projectroot;

# stylesheet to use
$stylesheet = "/gitweb.css";

# logo to use
$logo = "/git-logo.png";

# the 'favicon'
$favicon = "/git-favicon.png";

5. Now execute the following Commands in "/etc/apache2/mods-enabled" which will enable the specific modules which are needed for this exercise: -

ln -s ../mods-available/dav.load dav.load
ln -s ../mods-available/dav_lock.load DAVLockDB
ln -s ../mods-available/dav_fs.load dev_fs.load
ln -s ../mods-available/dav_fs.conf dev_fs.conf
ln -s ../mods-available/cgid.load cgid.load
ln -s ../mods-available/cgid.conf cgid.conf
ln -s ../mods-available/rewrite.load rewrite.load

sudo a2enmod rewrite

6. Next create a virtual Hosts, that can expose your git repository

Create a git_vhosts.conf file in "/etc/apache2/sites-available" which will contain the details about the virtual Hosts.
The content of the file will look like this: -

#
# Virtual Hosts
#
# If you want to maintain multiple domains/hostnames on your
# machine you can setup VirtualHost containers for them. Most configurations
# use only name-based virtual hosts so the server doesn't need to worry about
# IP addresses. This is indicated by the asterisks in the directives below.
#
# Please see the documentation at
# <URL:http://httpd.apache.org/docs/2.2/vhosts/>
# for further details before you try to setup virtual hosts.
#
# You may use the command line option '-S' to verify your virtual host
# configuration.

#
# Use name-based virtual hosting.
#
NameVirtualHost *:801

#
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for all requests that do not
# match a ServerName or ServerAlias in any <VirtualHost> block.
#

<VirtualHost *:801>
    ServerName git.myrep.net
    DocumentRoot "/var/www/gitweb"
    DirectoryIndex gitweb.cgi
    SetEnv  GITWEB_CONFIG   /etc/gitweb.conf

    <Directory "/var/www/gitweb">
        Options FollowSymlinks ExecCGI
        Allow from all
        AllowOverride all
        Order allow,deny

        <Files gitweb.cgi>
            SetHandler cgi-script
        </Files>

        RewriteEngine on
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteRule ^.* /gitweb.cgi/$0 [L,PT]
    </Directory>

    <Directory "/opt/git/myrepo.git">
        Allow from all
    </Directory>

    # To debug rewrite rules, which were very painful to figure out. Create directories in case they don't exists
    RewriteLog /var/log/httpd/rewrite_log
    RewriteLogLevel 9

    ErrorLog /var/log/httpd/gitweb
</VirtualHost>

You need to focus on the highlighted text in the above configuration as these attributes are very much specific to the paths exposed on the individual boxes.
In case you have followed all the previous instructions line by line than you just need to change "ServerName" attribute, which should be the IP or Domain Name of your machine.

7. Now to enable the Virtual Host and execute the below command from "/etc/apache2/sites-enabled" directory

ln -s /etc/apache2/sites-available/git_vhosts.conf 002-git_vhost.conf

8. Lastly restart your Apache server by using the following set of Commands

sudo /etc/init.d/apache2 stop
sudo /etc/init.d/apache2 start

NOTE: Make sure it is a clean start and there are no Errors shown on console.

9. Now browse the URL (http://:801/gitweb.cgi?p=.git;a=tree) and you will be able to see the GIT repository.

The window which Appears would be something like the below snapshot

Next I leave to you to play around this UI.

Updating Git repository: -

As we are exposing client repository over HTTP, so naturally the question arises: -

How do you make sure that your repository is continuously updated and doesn't require any manual intervention?

There are 2 ways which can be used to do this: -

1. Create a Cron JOB in Ubuntu which updates the repository at every fixed interval
2. enable the "post-update Hook" of your GIT repository and from there execute a Shell Script which does the job for you.

As both the above the options requires a Shell Script which is capable to logging remotely and updating the repository through various SSH and GIT commands.

Normal Shell scripts are nowhere capable to doing this thing....but thanks to Linux which always have a solution to any problem.

"Expect" is a wonderful utility which helped me to develop a short and simple script which does the needful.

sudo apt-get install expect

Here is actual Script which I created and used with #2 to keep my repository in Sync with each and every commit: -

#!/bin/bash

/usr/bin/expect << EOD

spawn  ssh <username>@localhost

expect "password:"

send -- "<password>\r"

expect "$"

send -- "cd /var/www/gitRepo \r"

expect "$"

send -- "unset GIT_DIR \r"

expect "$"

send -- "git pull ssh://<username>@<IP>/<PATH> <BRANCH> \r"

expect "password:"

send -- "<password>\r"

expect "$"

expect eof

EOD

echo "rep updated"

Read More!

Sharing your Git repository

Out of the several configuration parameters provided by Git..."core.sharedrepository" is one of the most important parameter which enables your GIT repository to be shared within/ outside your development team.

By default - GIT repositories are not shareable and can be used only by 1 person.

for e.g. if a developer X add a new file than any other developer apart from X will not be able to update the same file.

The reason behind is that the GIT repositories are created in an non-Shareable mode and the files created in the repository are in read only mode, which can be updated only be the creator and not by anyone else.

Here are the steps which will enable your repository being shared among group of developers (on Unix): -

1. create a group which will be assigned to all developers working on Git

   groupadd {group-name}
  

2. Now assign the primary group of all new or existing users (who needs to work on git) same as what we have created in #1.

   for new users : - useradd -g {group-name} {username}
   for existing users : - usermod -g {group-name} {username}

3. Define GIT_DIR as global variable pointing out to the physical location of the GIT repository

   export GIT_DIR=<Physical Location of your Git Repo>

In order to make this variable available globally you can also define the above parameter in /etc/profile or create a new sh file in /etc/profile.d/*sh

4. Execute the following Command in your Git Repository

   git config --add core.sharedRepository group

The above command will enable your GIT repository to be shared among group of developers.
i.e. Now the repository is in group writable mode, where all users in the same group (as defined
in #1) will be able to read and write, to/ from the repository.

Apart from "group" there are other parameters which are explained very well in the documentation: -

"When group (or true), the repository is made shareable between several users in a group (making sure all the files and objects are group-writable). When all (or world or everybody), the repository will be readable by all users, additionally to being group-shareable. When umask (or false), git will use permissions reported by umask(2). When 0xxx, where 0xxx is an octal number, files in the repository will have this mode value. 0xxx will override user’s umask value (whereas the other options will only override requested parts of the user’s umask value). Examples: 0660 will make the repo read/write-able for the owner and group, but inaccessible to others (equivalent to group unless umask is e.g. 0022). 0640 is a repository that is group-readable but not group-writable. This attribute is false by Default."

Also see git-init to do the same while initializing the repository

5. Lastly fix the permission on the existing directories create in your GIT repository

   sudo chgrp -R coders .
   sudo chmod -R g+ws .

Thats it...from now on any new developer just needs to be added in the same user group (created to git developers) and he/ she will be able to participate in the development.

Read More!

Configuring and Working with GIT in Ubuntu

GIT - A new name in the Arena of SCM's

Though we already have many SCM's available (Perforce,CVS, SVN, VSS etc), than why GIT?

Actually GIT offers much more than just a simple repository.

Apart from all the other standard SCM features, there are few which I liked the most: -

1. Every developer (by default) has its own private repository, which can be synced with the mainstream at periodical intervals.
2. Already Hosted over the internet (GIT-HUB) and anybody can use it, which itself makes the source code sharing such an easy task. Seems like a Social Networking of code :)
3. Local Branching is easy and Cheaper now, which was a pretty heavy and almost difficult in other SCM's
4. Faster than any other SCM's.
5. Over the Network - Works very well and far better than any other SCM, doesn't eat my network resources for each commit and checkouts are also very quick.


Visit the following liks to know more benefits of using GIT over other SCM
https://git.wiki.kernel.org/index.php/GitSvnComparison
http://whygitisbetterthanx.com/


Installing GIT: -

login as root or use sudo to run the below command

apt-get install git-core

Setting up Private repository: -

create and browse the directory which will be used as your local repository
<Path>/<code-dir>

run the following commands

git init

The above command initialize the empty GIT repository

Then clone the empty git repository to create a MATSER REPOSITORY (thats the default Branch for GIT empty repo)

git clone --bare /<Path>/<code-dir>/.git /<PATH>/myrepo.git

myrepo.git - is a Directory which will contain your master repository. All other development folks will be referring to this directory when checking out the code

That's it!!!!You have just completed the client setup and your repository can be used by dev folks

Setting up client or local repository

create and browse the directory which will be used as your local repository
<Path>/<code-dir>

run the following commands

git init

The above command initialize the empty GIT repository

git config user.email <email id>

The above will configure the repository to use the specified the email id for each commit

git remote add origin ssh://<serverIP/ domain-Name>/<Path of your GIT repo>

e.g. in our case it would be something like: -

git remote add origin ssh://<serverIP/ domain-Name>/<PATH>/myrepo.git

The above adds a master server to your repository..This would be used while you are trying to sync your local repository with main repository.

HURRAY----We are done with client too...so easy isn't it

Lets try a simple commit: -

Browse your local directory where you have set up your GIT working repository and create a file by by name "first.txt" and write something in it

Now execute the following commands from the console

git add *

Above will add all the changed or untracked files to your local repository

git commit -m "my First Commit"

Above will commit the code to your local repository

git push -u origin master

Now this will sync your local repo with the master...i.e. you just ready to upload all your changes to the master repository.
As soon as you execute the above command, it will ask for the password and here is what exactly it is doing: -

GIT mainly works on 2 protocols GIT and SSH and while doing an Sync it needs to login to the remote box and connect to the repository.

In case of ssh it asks for the password of the current user (by which you are logged into your local box), assuming that same user is already setup on remote box and have SSH privileges and also have access to git repo.

So make sure that the username (same as your local login) is created on the remote server.

Though it is not necessary that the SSH login of remote box matches your local login (you can use a different username while Syncing your changes) but just to avoid any unnecessary complexities I would have always suggested to use the same Logins.
Also while working on the real networks you will always use your network login, which anyways will be same on the remote box and local box.

Frequently used commands: -

Below is the list of GIT commands which are commonly used in our day to day life

git

Simply shows the available commands and their usage

git add "list of files to be added"

Adds the List of Files to your local repo

git commit –m "comments"

Commits the files to your local repo

git push -u origin "Branch Name"

Pushes the code to upstream branch and now other developers can checkout

git fetch ssh://<username>@&server-IP or domain-Name>/<Path of the repo on remote box>.git <branch-name>

Used to fetch the branches from the upstream branches to your local repo

git pull ssh://<username>@&server-IP or domain-Name>/<Path of the repo on remote box>.git <branch-name>

Same as Fetch but the only difference is that it tries to merge into the current branch

git checkout <branch-name>

By default every user is on "master" branch. The above command will move him to the specified Branch

git log

Shows the History of the branch on which the person is working.

git status -s

Shows the status of your local repo. Status of all committed or un-commited files

git rm -r >dir>

Used to remove the specified directory from your local repo

git reset --hard <branch-name>

Loose all your local changes and make your code in Sync with the upstream Branch.

Get more insight of GIT: -

http://book.git-scm.com
http://git-scm.com
http://gitref.org/

Read More!