JGIT The clone appears to hang indefinitely

https://www.eclipse.org/forums/index.php/t/261560/

Hello,

We're in the process of converting from SVN to GIT, and are running into a performance issue when cloning the repository with Eclipse/EGit. The clone appears to hang indefinitely

For various research and the below stack trace, the issue is related to large files in the repository. The specific file it is stuck on is a 48M (uncompressed) SQL script (there are subsequent files that are slightly larger, up to 72M uncompressed).

Have tried setting core.streamFileThreshold to various values (100m, 200m and finally 2047m) - in each case the behaviour is unchanged.

Have been unable to locate any information on how to correctly set core.streamFileThreshold through Eclipse; have been testing with it added to Team->Git->Configuration->User Settings. Adding it to the System Settings doesn't seem to take (on redisplay the value is not present).

Current test configuration (after trying many other permutations without success) is Eclipse 3.7.1 w/ EGit 1.1.0.201109151100-r on Windows 7, with Sun 64-bit JVM, 3G heap.


"Worker-8" prio=6 tid=0x000000000963f800 nid=0x12e8 runnable [0x00000000101af000]
java.lang.Thread.State: RUNNABLE
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:238)
- locked <0x0000000755f0a370> (a java.util.zip.ZStreamRef)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:135)
at java.util.zip.InflaterInputStream.skip(InflaterInputStream.java:191)
at java.io.BufferedInputStream.skip(BufferedInputStream.java:349)
- locked <0x00000007ec7b3b98> (a java.io.BufferedInputStream)
at org.eclipse.jgit.lib.ObjectStream$Filter.skip(ObjectStream.java:199)
at org.eclipse.jgit.util.IO.skipFully(IO.java:244)
at org.eclipse.jgit.storage.pack.DeltaStream.seekBase(DeltaStream.java:339)
at org.eclipse.jgit.storage.pack.DeltaStream.read(DeltaStream.java:213)
at org.eclipse.jgit.storage.pack.DeltaStream.read(DeltaStream.java:214)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
- locked <0x000000075657a338> (a java.io.BufferedInputStream)
at org.eclipse.jgit.util.io.TeeInputStream.read(TeeInputStream.java:111)
at org.eclipse.jgit.lib.ObjectStream$Filter.read(ObjectStream.java:209)
at java.io.InputStream.read(InputStream.java:82)
at org.eclipse.jgit.lib.ObjectLoader.copyTo(ObjectLoader.java:254)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkoutEntry(DirCacheCheckout.java:936)
at org.eclipse.jgit.dircache.DirCacheCheckout.doCheckout(DirCacheCheckout.java:447)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:380)
at org.eclipse.jgit.api.CloneCommand.checkout(CloneCommand.java:225)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:120)
at org.eclipse.egit.core.op.CloneOperation.run(CloneOperation.java:142)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard.executeCloneOperation(GitCloneWizard.java:306)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard.access$3(GitCloneWizard.java:299)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard$5.run(GitCloneWizard.java:278)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

ok, you found the best info on this topic: http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg00689.html explains quite well a possible reason for your performance problem. Strange is why it doesn't help to set core.streamFileThreshold to something big. Did you try the repack with -f and have the -delta gitattribute set? This has helped here: http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg00694.html 

Ciao

Chris

Report message to a moderator

   

- Re: Performance issue with large files [message #755454 is a reply to message #755327] Tue, 08 November 2011 14:59 Go to previous message

Chris Lee is currently offline Chris LeeFriend

Messages: 6

Registered: November 2011

Junior Member

That worked.

Added this to <repo>/info/attributes:

Quote:

*.sql -delta

*.zip -delta

*.jpg -delta

*.gif -delta

*.png -delta

*.doc -delta

*.docx -delta

*.pdf -delta

Repacked using:

Quote:

git repack -f -a -d --depth=250 --window=250

...and can now successfully clone from EGit.

Thx.