Git on Windows, "Out of memory - malloc failed"

Have run into a problem with repository and tried almost every possible config setting found out there eg. pack.WindowMemory etc etc

I believe someone has checked in a large file to remote repository and now each time I try and pull or push to it, GIT tries to pack it and runs out of memory:

Auto packing the repository for optimum performance. You may also
run "git gc" manually. See "git help gc" for more information.
Counting objects: 6279, done.
Compressing objects: 100% (6147/6147), done.
fatal: Out of memory, malloc failed (tried to allocate 1549040327 bytes)
error: failed to run repack

Have tried git gc & git repack with various options but keeps returning same error.

Almost given up and about to just create a new repo but thought I'd ask around first :)

I found a solution Here that worked for me.

In .git/config file (client and/or server) I added the following:

[core]
  packedGitLimit = 128m
  packedGitWindowSize = 128m

[pack]
  deltaCacheSize = 128m
  packSizeLimit = 128m
  windowMemory = 128m

For reference (you might already seen it), the msysgit case dealing with that issue is the ticket 292.

It suggests several workarounds:

• Disable delta compression globally. For this you have to set pack.window to 0. Of course this will make the repository much larger on disc.
• Disable delta compression for some files. Check the delta flag on the manpage to gitattributes.
• git config --global pack.threads 1
• git config --global pack.windowMemory 256m (you already tried that one, but also illustrated in "Error when pulling warning: suboptimal pack - out of memory")
• other settings are mentioned in "git push fatal: unable to create thread: Resource temporarily unavailable" and "Git pull fails with bad pack header error" in case this is pack-related.
• sm4 adds in the comments:

To disable the delta compression for certain files, in .git/info/attributes, add:

*.zip binary -delta

From Gitattributes man page:

Delta compression will not be attempted for blobs for paths with the attribute delta set to false.

Maybe a simpler workaround would be to somehow reset the history before that large file commit, and redo the other commits from there.

EDIT:  Since git-v2.5.0 _^(Aug/2015), git-for-windows (formerly MSysGit)
      provides 64-bits versions as noticed by Pan.student.
      In this answer I was advising to install Cygwin 64-bits (providing 64-bits Git version).

I got a similar Out of memory, malloc failed issue using MSysGit when reaching the 4GB barrier:

> git --version
git version 1.8.3.msysgit.0

> file path/Git/cmd/git
path/Git/cmd/git: PE32 executable for MS Windows (console) Intel 80386 32-bit

> time git clone --bare -v ssh://linuxhost/path/repo.git
Cloning into bare repository 'repo.git'...
remote: Counting objects: 1664490, done.
remote: Compressing objects: 100% (384843/384843), done.
remote: Total 1664490 (delta 1029586), reused 1664490 (delta 1029586)
Receiving objects: 100% (1664490/1664490), 550.96 MiB | 1.55 MiB/s, done.
Resolving deltas: 100% (1029586/1029586), done.
fatal: Out of memory, malloc failed (tried to allocate 4691583 bytes)
fatal: remote did not send all necessary objects

real    13m8.901s
user    0m0.000s
sys     0m0.015s

Finally git 64 bits from Cygwin fix it:

> git --version
git version 1.7.9

> file /usr/bin/git
/usr/bin/git: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows

> time git clone --bare -v ssh://linuxhost/path/repo.git
Cloning into bare repository 'repo.git'...
remote: Counting objects: 1664490, done.
remote: Compressing objects: 100% (384843/384843), done.
remote: Total 1664490 (delta 1029586), reused 1664490 (delta 1029586)
Receiving objects: 100% (1664490/1664490), 550.96 MiB | 9.19 MiB/s, done.
Resolving deltas: 100% (1029586/1029586), done.

real    13m9.451s
user    3m2.488s
sys     3m53.234s

FYI on linuxhost 64 bits:

repo.git> git config -l
[email protected]
core.repositoryformatversion=0
core.filemode=true
core.bare=true

repo.git> git --version
git version 1.8.3.4

repo.git> uname -a
Linux linuxhost 2.6.32-279.19.1.el6.x86_64 #1 SMP Sat Nov 24 14:35:28 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

If my answer does not fix your issue, you may also check these pages:

• git clone out of memory even with 5.6GB RAM free and 50 GB hard disk
• Git clone fails with out of memory error - “fatal: out of memory, malloc failed (tried to allocate 905574791 bytes) / fatal: index-pack failed”
• git-clone memory allocation error
• MSysGit issues tracker