[freenet-dev] Why Freenet is so SLOW! / Finding data

[Robert Hailey]

On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote:

> Hi,
>
> I think freenet is slow because its having problems finding data.   
> The hit rates in 0.5 ranged around 5%.
> in 0.7 I see 2% on cache and 0% on store.  We just are not finding  
> data.  My location shifts so fast that
> other nodes have no idea what is in my node's 8G store.  I strongly  
> suggest that we experiment with letting
> the store contents limit how far from the current location we can  
> swap.  This could easily be done with
> a tunable such that the effect could be turned off or could be made  
> very strict.  We could even use
> a feedback loop to find an optimal value with the store/cache hit  
> rate being used to control just how strict
> we are.  The idea would be to minimize the swapping limitation while  
> maximizing the hit rate which
> would decrease the path length and hence the traffic.
>
> In my case the node usually uses a good chunk (80-90%) of the  
> bandwidth I give to it.   I think the
> bandwidth limits work.  I would agree that backout is probably not  
> needed and think it would be a
> good idea to try with it turn off - again this would improve routing  
> which I think is the core of 0.7
> performance problems.
>
> Thanks
> Ed


I have been musing on Ed's post for a while. Along with my  
understandings:

* The contents of the datastore(s) are not given weight when  
considering a swap request.
* For a particular CHK/SSK insert, it finds one home node to be put  
into the store (the rest in only cached).
* If the node targeted by the insert 'drifts' sufficiently far from  
it's origin
** that data in the store be unfindable by the network, and
** is all-but guaranteed to be LOST forever, is it can only survive in  
the networks caches (if it is un-popular).

Consider the following patch (completed to the furthest point that my  
current freenet understand can take it).
- If a CHKBlock is evicted from the datastore (not cache), AND we are  
not (as best we can tell) "close" to where it should be, THEN re- 
insert that block.

This approximates the decision (is this eviction OLD/UNPOPULAR or  
UNFINDABLE), and brings with it a rather odd perk for having small  
datastores (that they would find these misplaced blocks sooner).

The major risk, of course, is renewing old data.

I do not have the means to test this on any sizable network. Would  
this be a candidate for 'testnet' or simulations of which I hear?

Also, I could not figure out how to revive a SSKBlock from the  
datastore (or find out if it is even possible).

--
Robert Hailey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: freenet-reinsert-evicts.diff
Type: application/octet-stream
Size: 10864 bytes
Desc: not available
Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/347a1a78/attachment.obj 
-------------- next part --------------