Error while pushing to github repo

Question

I receive the following error when pushing my commits

remote: warning: File var/log/system.log is 57.82 MB; this is larger than recommended maximum file size of 50 MB
remote: error: GH001: Large files detected.
remote: error: Trace: 96d01231dffac3fbc3ba1eb2e9f01a93
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File var/report/752246136671 is 100.86 MB; this exceeds github's file size limit of 100 MB

I tried the following the commands listed step by step below:

git push -u origin master

cant find these in git files to commit when i typed git status.

Could you please let me know how to push my changes to repo without these errors ? I guess these files are in github index . I also tried git rm --cached var/log/system.log. but no results.

hitting my head to wall !

UPDATE 1 Kindly find the Gists here based on the two answers from experts below:

answer#1 - https://gist.github.com/haijerome/9405598
answer#2 -https://gist.github.com/haijerome/9405492

UPDATE 2 Kindly find below the git Log details for the both the files that i tried to remove:

https://gist.github.com/haijerome/9406273
https://gist.github.com/haijerome/9406263

ANSWER THAT WORKED

Please find the gist for the final answer that solved my issue

https://gist.github.com/haijerome/9478989

credits to git experts VonC, Holger Just and all other experts who have provided their inputs and ofcourse to stackoverflow.

VonC 504k 123 1326 1418 · Accepted Answer · Mar 10 '14 at 12:08

up vote 4 down vote accepted

+50

git rm or git rm --cached isn't enough to remove that file fir the history stored in your repo.

You need to:

use BFG Repo Cleaner, as suggested above.

bfg --strip-blobs-bigger-than 1M  my-repo.git

use git gc --agrressive --prune=now (after BFG), as detailed in "Reduce git repository size"
git push -f to force the new history on your remote repo.

answered Mar 10 '14 at 12:08

VonC
504k 123 1326 1418

Is it possible that some other blob > 1MB (say those corresponding to seeds for the db) also get mistakenly deleted while trying to clean up for these 2 files? If yes, how to avoid that using bfg? – mu 無 Mar 10 '14 at 20:03

2

@mu無 Sure. You can add filters (--delete-files) to get a more precise action. Or do it the hard way, with agit filter-branch. I wanted to bring attention to the next step (the git gc) in order to force a more compact repack, and making the git push possible. – VonC Mar 10 '14 at 20:09

@VonC Thanks a lot for your answer. I tried the above GIT commands and it worked as you suggested. I have updated my question with the gist containing the git commands that solved my issue. Waiting for SO to allow me to award the bounty. – Haijerome Mar 11 '14 at 3:42

add a comment

score 1 · Answer 2 · Mar 10 '14 at 13:26

The message contains information about two files. var/log/system.log generates a warning but it would be pushed. var/report/752246136671 is too large and thus prevents the push. You thus have to delete at least the latter file.

Before Github will let you push, you'll have to remove the file from all commits you want to push. It is not enough to just delete the file in a later commit after having it added before.

According to the article linked in the message, you can perform one of the two recommended operations:

If you have added the file in the most recent commit, you can change it to remove the file:

git rm --cached var/report/752246136671
# Stage our giant file for removal, but leave it on disk

git commit --amend -CHEAD
# Amend the previous commit with your change
# Simply making a new commit won't work, as you need
# to remove the file from the unpushed history as well

git push
# Push our rewritten, smaller commit

Or you can use The BFG to filter your repository and remove the file from all commits. This is required if you have added the file in the git history (as opposed to only in your most recent commit), you have to clean your history. Github will not allow to push the large file in any commit even if it is later removed again. This is because in this case, the file will still be part of the history and will thus bloat the repo.

You can install The BFG from https://github.com/rtyley/bfg-repo-cleaner/releases/latest.

The you can remove any indication of any files larger than 100MB by running this command:

cd /path/to/your/git/repo
java -jar bfg.jar --strip-blobs-bigger-than 100M
# Git history will be cleaned - files in your latest commit will *not* be touched

Note that this will change history of your repository, leading to a potential force-push. You might thus have to coordinate with your fellow developers.

Also, if you still need the file, you should make a backup before as you won't be able to restore it from git.

I tried git rm --cached var/report/752246136671 and got the following error "fatal: pathspec 'var/report/752246136671' did not match any files". In fact i have removed the file manually from my server. I feel its somewhere in Git history but am not able to locate it. Kindly find this gistgist.github.com/haijerome/9405492 for more information. — Mar 7 '14 at 5:04
I have updated my answer below with the git log details for both of these files i tried to remove. — Mar 7 '14 at 6:25
As I said in my answer (and is said in the article the error message linked to and I quoted): If you have added the file in your git history, you have to clean it to remove the file completely from all commits. Github recommends the BFG. Please see the update to my answer. — Mar 10 '14 at 13:25
Your answer helped me a lot. Thanks a lot. bfg is the key to solve the major problem. +1 for your answer and for your latest comment. — Mar 11 '14 at 3:43

score 1 · Answer 3 · Mar 10 '14 at 20:05

The problem at hand is that while you have removed the relevant files using new commits, there are blobs corresponding to the older commits for the same files in you repo, and the size of 2 of these objects is throwing you a warning and an error. To be able to push again, you need to remove the same as well.

While bfg-repo should work for most people to rectify the situation here, it requires java to be installed and configured on the system and that is not always available.

You have 2 files var/log/system.log and var/report/752246136671 which exceed the limit, so I would suggest using filter-branch for removing them for a git only solution:

git filter-branch --index-filter 'git rm --cached --ignore-unmatch var/log/system.log var/report/752246136671' --tag-name-filter cat -- --all

You might need to force push the changes later (using git push -f origin).

In general, it is a good practice to ignore the *.log, *.info and other log files from the git repo completely by using a *.log entry in your .gitignore file. You can similarly ignore the var/reportsfolder.

In case there are files which have already been committed to the repo and pushed earlier, you might need to use git rm --cached "*.log" to remove the same and commit those changes to untrack them permanently.

PS: This seems like a magento/joomla installation to me, in that case ignore the var folder completely in your .gitignore, you don't want objects from var/cache or var/session etc to be tracked in the repo.

I had to try git filter-branch if i cant get the bfg working but BFG worked like a cake. and regarding your suggestions to ignore var folder is most welcome. I ignored it. Thanks again. This question gets the attention of good number of experts with different ways to resolve the problem. +1 for your answer. — Mar 11 '14 at 3:48

Agis 12.2k 1 20 43 · Answer 4 · Mar 6 '14 at 14:50

up vote 0 down vote

It's clear that you are trying to push a very big file, as the error is indicating. It is also a log file, therefore I assume it should not be in the repository.

If this is true, do this to fix the error:

$ git rm var/log/system.log
$ git commit -m "Delete system.log file"
$ git push -u origin master

answered Mar 6 '14 at 14:50

Agis
12.2k 1 20 43

I manually removed the file from my FTP but it seems its still in the GIT history somewhere. I tried your input but still the same issue. Kindly find the gist here: gist.github.com/haijerome/9405598 which shows the commands i tried and its output. – Haijerome Mar 7 '14 at 5:13

I have updated my answer below with the git log details for both of these files i tried to remove. – Haijerome Mar 7 '14 at 6:25

使用git push到github时有大文件的错误

Error while pushing to github repo

4 Answers

你可能感兴趣的:(应用,软件,git)