Artifact
a4c3ed78a947b4c531251a5c502e544df95c74f4:
Ticket change
[a4c3ed78a9]
- Ticket [6c54714451]
fossil clone is slow status still Open with 1 other change
by
drh
2010-10-08 14:31:42.
D 2010-10-08T14:31:42
J +comment \n\n<hr\s/><i>drh\sadded\son\s2010-10-08\s14:31:42:</i><br\s/>\nA\svery\simpressive\srepository\s-\sthanks\sfor\ssharing\sit.\r\n\r\npkgsrc.fossil\sis\sabout\s10x\sor\s100x\slarger\sthan\sany\srepository\swe\shave\sdealt\r\nwith\sbefore.\s\sSQLite\sis\sreally\sthe\sbiggest\srepository\sthat\sFossil\sdeals\swith\r\non\sa\sregular\sbasis.\s\sSQLite\shas\s10.3\syears\sof\shistory\scompared\sto\s13.1\syears\r\nof\shistory\swith\spkgsrc.\s\sBut\sthere\shave\sbeen\san\saverage\sof\sjust\s2.2\scheckins\r\nper\sday\sto\sSQLite\sfor\sa\stotal\sof\s8446\scheckins,\swhereas\spkgsrc\shas\shad\son\r\naverage\salmost\s40\scheckins\sper\sday\sfor\sa\stotal\sof\s190281\scheckins.\s\s\r\nFossil\slikes\sto\sstore\sthe\soriginal\scontent\sof\sthe\slatest\scheckin\sas\r\nfull-text\sand\sthen\sstore\sprior\sversions\sas\sdeltas\sfrom\sthe\slatest.\s\s(Most\r\nother\sVCSes\sdo\sthe\ssame\s-\sRCS\sin\sparticular.)\s\sIn\sSQLite,\sthe\slongest\sdelta\r\nchain\sis\stherefore\s8446\sdeep\s-\swhich\sis\sdeep\sbut\snot\sunmanageable.\s\sWith\r\npkgsrc,\sthe\slongest\schange\sis\sa\swhopping\s190281\sdeep\s-\s22x\sdeeper.\s\sIt\r\nmakes\sme\swonder\sif\swe\sshouldn't\strick\sfossil\sinto\sstoring\sa\sfull-text\r\nversion\sof\seach\sfile\safter\ssome\sfixed\snumber\sof\sdeltas\s-\sfor\sexample\r\nstore\sfull\stext\safter\seach\s500\sor\s1000\sdeltas.\s\sThat\swill\smake\sthe\r\nrespository\slarger,\sbut\sit\swill\salso\smake\sa\s100x\sperformance\simprovement\r\nwhen\strying\sto\saccess\sfiles\sthat\sare\svery\sdeep\sin\sthe\sdelta\sstack.\r\n\r\nSQLite\shas\s1057\sseparate\sfiles.\s\sWe\shost\sa\sfew\sother\srepositories\swith\r\nmore\sthan\sthis,\sbut\snever\smore\sthan\sa\scouple\sthousand.\s\spksrc,\son\sthe\r\nother\shand\shas\sover\s100,000\sseparate\sfiles.\s\sMany\sof\sthe\sfile\sbrowsing\r\nlinks\sscan\sthe\sentire\slist\sof\sfiles.\s\sWith\sonly\sa\sthousand\sor\sso,\sthis\r\nis\snot\sproblem.\s\sBut\swith\s100x\sas\smany,\sthose\spages\sare\sslow.\s\sWe\smight\r\nneed\sto\sreconsider\sthe\sway\ssome\sof\sthe\sfile\sbrowsing\spages\sare\srendered.\r\n\r\nThe\shuge\snumber\sof\sfiles\salso\smakes\sfor\slarge\smanifests.\s\sI\slooked\sat\r\none\sof\sthe\smore\srecent\smanifests\sin\spkgsrc\sand\sit\swas\s60936\slines\slong\r\nand\scontains\snearly\s5MB\sof\stext.\s\sThere\sare\sabout\s340MB\sof\scontent\sin\r\na\scheckout.\s\sFossil\sdoes\slots\sof\sconsistency\schecks\sfor\seach\scheck-in,\r\nwhich\sinvolves\scomputing\sMD5\schecksums\sover\sthe\sentire\scheckout,\stwice.\r\nRunning\sMD5\sover\s720MB\stakes\ssome\stime.\s\sA\ssimple\scheckin\son\smy\s\r\n4-year-old\sdesktop\sused\s18\sseconds\sof\sCPU\stime\sand\s55\sseconds\sof\r\nreal-time.\s\s(Note\sthat\sa\sclone\sof\sthe\srepository\swas\srunning\sin\sparallel\r\nwhile\sdoing\sthe\sperformance\stest\sof\sthe\sprevious\ssentence.)\s\sCheckins\r\nwould\sbe\svery\sfast\sif\swe\sturned\soff\sthe\serror\schecks.\s\sMaybe\sthat\r\nneeds\sto\sbe\san\soption.\r\n\r\nMy\sclone\s(via\slocalhost)\shas\smade\ssubstantial\sprogress\sin\s33\sminutes\r\nof\sCPU\stime.\s\sBut\sit\sseems\sto\sbe\sstuck\snow.\s\s
K 6c5471445173a17fffa6aaf867f2f7d3da8f8151
U drh
Z b87d4954f03a9880406303bcfabeba13