Ticket Change Details
Not logged in
Overview

Artifact ID: 8d36528cd742ddbbed516e269993bcbdfbe6b8af
Ticket: 5199df97bc08fff1e5656d165738f0d87f2a3ad1
Alternative to reconstruct for large imports
User & Date: anonymous 2010-10-05 15:40:20
Changes

  1. comment changed to:
    The current "fossil reconstruct" interface works well for small scale operation, but having one file per artifact (even across multiple directories) brings a lot of overhead.
    
    Attached is a patch to implement a variant called reconstruct-sql, which takes the artifacts from a sqlite3 database. This makes it a lot easier to deal with, especially if the processing is already done in a higher level language. Compressing the artifacts helps to cut down required disk space a lot, too.
    
    Related questions would be if explicitly tagging artifacts as manifests or including the hash or temporal order would be used to speed up the import. A lot of time is currently still spend in content_put. I haven't analyzed yet whether it is the work required for delta processing or whether it is the plain issue of having very large manifests (e.g. a working copy of pkgsrc is 60k files).
    
  2. foundin changed to: "aab38ef02f"
  3. private_contact changed to: "63070188043a9522ca95dca0d28c3b9eff7a697b"
  4. severity changed to: "Important"
  5. status changed to: "Open"
  6. title changed to: "Alternative to reconstruct for large imports"
  7. type changed to: "Feature_Request"