[freenet-dev] More database/datastore problems

[Florent Daignière]

* Chris Carlin <carlin at jlab.org> [2007-05-16 13:42:33]:

> 
> > So our options are:
> > 1) Open the block numbers database with sorted duplicates enabled. Then scan 
> > through it for dupes, keep the correct one in each case, close the database, 
> > and re-open it again with sorted duplicates disabled.
> > 2) Keep the block numbers index open with sorted duplicates enabled. When we 
> > need to look up a block number, deal with the fact that there may be multiple 
> > keys referring to it, and delete as appropriate. Deal with the fact that this 
> > may cause keys in the main database that don't exist in the store.
> > 3) Improve the data stored on disk: Store the LRU access time, and the key, on 
> > disk. Probably we would need migration code.
> > 4) Use a completely different database for the index.
> > 5) Use a completely different database for the whole store, and trust it not 
> > to lose the data. (from our experience with BDB, it is likely that it will!)
> > 6) Roll our own database code.
> 
> It seems to me that BDB has been rejected as the proper solution 
> multiple times on this mailing list. Maybe I misunderstood, but in 
> particular I remember when I was going to follow up with Oracle about 
> memory usage issues but stopped when consensus seemed to find that BDB 
> wasn't worth pursuing for various other reasons.
> 
> ~Chris

We weren't using it properly, hence  the high memory usage we noticed...
Now  toad  is  willing  to  switch to  something  else  because  of  the
"robustness"  criteria  he has  defined  (ability  for the  database  to
auto-repair itself  and perform  catastrophic recovery proceedure  in an
automated  way without  involving  the user).  I do  think  that we  are
looking for a rara avis here :/

NextGen$
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070517/425e0e3d/attachment.pgp