Repack of Git repository fails

I have a git repository residing on a server with limited memory. When I try to clone an existing repository from the server I get the following error

hemi@ubuntu:$ git clone ssh://[email protected]/home/hemi/repos/articles
Initialized empty Git repository in /home/hemi/Skrivebord/articles/.git/
[email protected]'s password: 
remote: Counting objects: 666, done.
remote: warning: suboptimal pack - out of memory
remote: fatal: Out of memory, malloc failed
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOF
fatal: index-pack failed
hemi@ubuntu:$ 

To handle this error I have tried to repack the original repository (according to this forum post). But instead of repacking the repository it describes how to use the "git pack-objects" command.

hemi@servername:~/repos/articles$ git repack -a -d --window-memory 10m --max-pack-size 100m
usage: git pack-objects [{ -q | --progress | --all-progress }]
        [--all-progress-implied]
        [--max-pack-size=N] [--local] [--incremental]
        [--window=N] [--window-memory=N] [--depth=N]
        [--no-reuse-delta] [--no-reuse-object] [--delta-base-offset]
        [--threads=N] [--non-empty] [--revs [--unpacked | --all]*]
        [--reflog] [--stdout | base-name] [--include-tag]
        [--keep-unreachable | --unpack-unreachable 
        [<ref-list | <object-list]

Git 1.6.5.7 is installed on the server.

Solution 1:

Your solution has got you a working copy locally and remotely, but will cause problems again when the remote repository decides to repack itself again. Fortunately, you can set config options that will reduce the amount of memory needed for repacking in both repositories -- these essentially make the command line parameters that you added into the default options when repacking. So, you should log in to the remote, change into the repository and do:

git config pack.windowMemory 10m
git config pack.packSizeLimit 20m

You may want to do the same on your local repository. (Incidentally I guess that either your repository is very large or these are machines with little memory - these values seem very low to me.)

For what it's worth, when getting malloc failures on repacking very large repositories in the past, I've also changed the values of core.packedgitwindowsize, core.packedgitlimit, core.deltacachesize, pack.deltacachesize, pack.window and pack.threads but it sounds as if you don't need any further options :)

Solution 2:

With no direct access to repository and hence being unable to perform a repack, performing a shallow clone and then gradually fetching while increasing depth helped for me.

git clone YOUR_REPO --depth=1
git fetch --depth=10
...
git fetch --depth=100
git fetch --unshallow    //Downloads all history allowing to push from repo

Hope it can still help someone.

Solution 3:

I solved the problem using the following steps.

1. Got repository checked out from the server to my local machine (using a raw copy over ssh)
2. Repacked the local repository
   git repack -a -d --window-memory 10m --max-pack-size 20m
3. Created an empty repository on the server
   git init --bare
4. Pushed the local repository to the server
5. Checked that it is possible to clone the server repository

Solution 4:

This does not answer the question, but somebody might run into it: repacking might also fail on the server when pack-objects is terminated by some kind of memory killer (such as the one used on Dreamhost):

$ git clone project-url project-folder
Cloning into project-folder...
remote: Counting objects: 6606, done.
remote: Compressing objects: 100% (2903/2903), done.
error: pack-objects died of signal 9284.51 MiB | 2.15 MiB/s   
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOF
fatal: index-pack failed

On Dreamhost this appears to be caused by mmap. The repack code uses mmap to map some files’ contents into memory, and as the memory killer is not smart enough, it counts the mmapped files as used memory, killing the Git process when it tries to mmap a large file.

The solution is to compile a custom Git binary with mmap support turned off (configure NO_MMAP=1).