[Tech] Cache and Store

[Florent Daignière]

* Juiceman <juiceman69 at gmail.com> [2007-05-11 19:32:05]:

> On 5/11/07, Florent Daignière <nextgens at freenetproject.org> wrote:
> >* Bob Ham <rah at bash.sh> [2007-05-11 08:19:49]:
> >
> >> On Thu, 2007-05-10 at 22:47 +0200, Florent Daignière wrote:
> >> > * Bob Ham <rah at bash.sh> [2007-05-10 21:45:59]:
> >> >
> >> > > On Thu, 2007-05-10 at 12:53 +0100, Matthew Toseland wrote:
> >> > > > On Thursday 10 May 2007 11:52, Florent Daignière wrote:
> >> > > > > * Matthew Toseland <toad at amphibian.dyndns.org> [2007-05-10 
> >11:31:22]:
> >> > > > > >
> >> > > > > > As long as it's an expert option, I don't see any reason why 
> >it shouldn't
> >> > > > > > be accepted.
> >> > > > >
> >> > > > > Preventing users from their stupidity ?
> >> > > > >
> >> > > > > We don't want everyone to have a cache but no specialized data 
> >on the
> >> > > > > basis that "with a bigger cache downloads are 'resuming' ; I 
> >don't care
> >> > > > > about others nor the network so I only cache"
> >> > > >
> >> > > > Fair point. If they want to break their node that much they can 
> >maintain a
> >> > > > fork.
> >> > >
> >> > > This is a real cognitive problem showing up right there.  It isn't 
> >your
> >> > > responsibility to second-guess the user.  There are valid reasons for
> >> > > the node to have this functionality.  The only reason for it not to 
> >is
> >> > > to inhibit users.  That's what Microsoft do.
> >> >
> >> > Indeed... and experience has shown that it works.
> >>
> >> What do you mean "it works"?  What does it work to do?
> >>
> >
> >Shall I remind you that most of our users do use Microsoft products and
> >are happy with them ?
> >
> >I'm  against the  introduction of  new useless  (no need  to debate  the
> >meaning of useless this time; show me some stats/simulations if you want
> >to be heard)  settings, especially when they can harm  the network... So
> >far we don't  have content reachability problems and if  we did, I would
> >suggest to fix the datastore code or  to prevent users from nuking it on
> >a regular basis.
> >
> >The  50%-50% limit  is arbitrary,  so  are many  "unknown" and  "hidden"
> >settings in the node; unless you  have strong evidence (with a theorical
> >basis) explaining  why it's bad/wrong  you will  be directed to  the bug
> >tracking system and  the "wishlist" category. I will ask  you to explain
> >the "This  is a problem."  in your  first mail [1]  as well (so  that we
> >could at least agree on the fact  that we don't have the same definition
> >of what a problem is either ;) ).
> >
> >Moreover  some related  work  has  to be  done  on  the shrinking  code,
> >mroggers  has suggested  that  the  LRU policy  might  not  be the  best
> >solution  to  choose which  data  will  be  pruned;  he came  with  some
> >references and  is likely to be  heard... You didn't, did  you ? Anyway,
> >consequently his suggestion is likely to be implemented before any other
> >"related" work is done  in that area of the code.  IMO we shouldn't give
> >users incentives to use a "maybe broken and harmful" schrinker, hence
> >I'm against letting him play with the ratio of the cache/store.
> >
> 
> No, I don't have data and/or references.  As you said yourself "the
> 50%-50% limit  is arbitrary".  Why did you decide on 50%, where is
> your statical results from a simulation?

I am not suggesting to change it, that's the point. Why would your
arbitrary choosen value be any better than the current default ?

> What I do know is my gut
> tells me I probably have one the largest caches in Freenet.  Any
> request that passes thru my node has a good chance of finding the data
> in my node.  Hence, my outbound bandwidth is always pegged at 25KB/sec
> (which I don't have a problem with) I just have a feeling the "deep
> store of data" that I want my node to concentrate on is not being
> served because its answering "random requests."

Replying to requests using the cache doesn't prevent your store from
specializing.

> Is there a histogram of inbound and outbound transfers somewhere I can check?

Not that I'm aware of.

> 
> BTW, I recently "nuked" my datastore so I would stop seeing error
> messages about my store and corruption (and I knew it would fill back
> up quick enough with random requests).  Maybe we should hide error
> messages from users so they think everything is "peachy"?

Thanks for prooving me right... The urgency is to fix the DB code, not
to implemenet a better schrinking mechanism.

NextGen$
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://emu.freenetproject.org/pipermail/tech/attachments/20070512/199674ae/attachment.pgp