Overview
Artifact ID: | 8d36528cd742ddbbed516e269993bcbdfbe6b8af |
---|---|
Ticket: | 5199df97bc08fff1e5656d165738f0d87f2a3ad1
Alternative to reconstruct for large imports |
User & Date: | anonymous 2010-10-05 15:40:20 |
Changes
- comment changed to:
The current "fossil reconstruct" interface works well for small scale operation, but having one file per artifact (even across multiple directories) brings a lot of overhead. Attached is a patch to implement a variant called reconstruct-sql, which takes the artifacts from a sqlite3 database. This makes it a lot easier to deal with, especially if the processing is already done in a higher level language. Compressing the artifacts helps to cut down required disk space a lot, too. Related questions would be if explicitly tagging artifacts as manifests or including the hash or temporal order would be used to speed up the import. A lot of time is currently still spend in content_put. I haven't analyzed yet whether it is the work required for delta processing or whether it is the plain issue of having very large manifests (e.g. a working copy of pkgsrc is 60k files).
- foundin changed to: "aab38ef02f"
- private_contact changed to: "63070188043a9522ca95dca0d28c3b9eff7a697b"
- severity changed to: "Important"
- status changed to: "Open"
- title changed to: "Alternative to reconstruct for large imports"
- type changed to: "Feature_Request"