[Tech] Cache and Store

[Matthew Toseland]

On Fri, May 04, 2007 at 08:43:43AM +0100, Bob Ham wrote:
> On Thu, 2007-05-03 at 23:29 +0100, Matthew Toseland wrote:
> > On Thu, May 03, 2007 at 05:43:46PM +0100, Bob Ham wrote:
> > > Secondly, the node should fill up each store regardless of the
> > > configured split point until it reaches the maximum overall store size.
> > > At that time, it should reduce the size of whichever store is over its
> > > allocation as and when the other store needs it.
> > 
> > It's not easy to do online shrinking without ending up losing the most
> > valuable rather than the least valuable data.
> 
> What do you mean by "easy", exactly?

Well... it's possible, but it's probably best to wait until we get
around to doing the drastic datastore changes that we have queued for a
while, here's the wishlist:
- Include the key for SSKs so that we can reconstruct the SSK store as
  well as the other two.
- Histogram of keys stored/cached by key/location.
- An extra layer of encryption: You look up a key by H(H(key+salt)) and
  decrypt it using H(key+salt). (To make it harder to analyse
  confiscated datastores). Optional as incompatible with the previous
  item.
- The "client cache" (ephemerally encrypted cache of only stuff
  requested by the local user).
- Temporary files allocated from chains of blocks in the encrypted
  store.
- No free blocks: just move them to the bottom of the LRU.
- One big file for all stores for a given keysize (e.g. CHKs plus
  tempfiles, store and cache). Or alternatively keep the data in the
  database (would be simpler but less reliable based on past
  experience as no way to recover).

Most of these have bugs filed. At the moment none of them is a priority,
but if you want to work on them that'd be great.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://emu.freenetproject.org/pipermail/tech/attachments/20070504/9b827ca3/attachment.pgp