View Ticket
Not logged in
Ticket UUID: 537da0e3089b0f6a256cd4b0361afde463127d3c
Title: update on large repository takes a long time even when no action needed
Status: Fixed Type: Feature_Request
Severity: Minor Priority:
Subsystem: Resolution: Fixed
Last Modified: 2010-01-24 07:48:48
Version Found In: d2204a00fb6bc71bdd13ffdafc469667b356172d
Description & Comments:
In a very large repository (over 52,000 files), an initial "open" takes 8 minutes. Immediately doing an "update" takes 3 minutes.

It would be very good if there were some way to make this take much less time.


drh added on 2010-01-20 18:40:37:
Is

fossil setting mtime-changes

turned on? If not, Fossil has to compute a SHA1 hash for every file in the project in order to see which ones have changed.

If that isn't the problem, can you run fossil in a profiler to give me a better idea of what the problem might be?


ron added on 2010-01-20 19:09:23:
Well: "fossil set mtime-changes on" reduces the update time to 19.5 seconds (!). So I guess this isn't a real problem, though perhaps it should be documented as an optimization?

With 'mtime-changes' off, gprof indicates:

  %   cumulative   self              self     total
 time   seconds   seconds    calls   s/call   s/call  name
 65.45     79.69    79.69 52540564     0.00     0.00  SHA1ProcessMessageBlock
 26.47    111.92    32.23   362862     0.00     0.00  SHA1Input

Over 90% of the time is in these two routines ...


ron added on 2010-01-20 19:40:22:
Just as another data-point: SVN on the same repository (using svn+ssh) takes 29 seconds to do a do-nothing update.

It would be interesting to see how much additional time it would take were I to host the repo and use http sync ...


ron added on 2010-01-20 20:20:25:
The answer seems to be that adding the sync part (as opposed to the local only upd) adds almost nothing to the time an update takes. That's nice :)


ron added on 2010-01-24 07:48:48:
Setting 'mtime-changes' on fixes the problem. It is also now the default.