[freenet-dev] Why Freenet is so SLOW! / r16372

[Matthew Toseland]

On Thursday 06 December 2007 20:16, Robert Hailey wrote:
> 
> On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote:
> 
> > On Friday 30 November 2007 23:03, Robert Hailey wrote:
> >>
> >> A much-more aggressive patch (which makes all RequestHandler sends
> >> Async except the 'last') shows could be 12x; but am concerned over  
> >> the
> >> byte-counter side effects with which I am not familiar.
> >
> > [...]
> > - To get the correct byte counts.
> > This is really important, because it underlies the major load limiting
> > systems. However it is only important for terminal messages: we can
> > reasonably assume the rest will be piggybacked if necessary.
> >
> > In RequestHandler, most of the sendSync()'s are terminal, and  
> > therefore must
> > remain. The one that is the most interesting is the one you  
> > mentioned above:
> > sending the Accepted message.
> 
> 
> Well... I found the time to get the "more aggressive" patch working.
> 
> The general idea is that (after the final request) because the ~only~  
> reason to keep the thread waiting on the final transmission is the  
> byte count, to delegate that to the 'sent()' callback thread.

Makes sense.
> 
> On my machine (which was showing pre-emptive reject cause as  
> ">threadLimit", because of the high bandwidth ceiling I have set), I  
> saw a increase in node bandwidth (in & out), and saw a new pre-emptive  
> rejection cause "insufficient output bandwidth". Realizing then that  
> the bandwidth ceiling (being so high) was the cause for my node  
> behaving differently than others, I changed my config to less-than  
> that observed value, and "output bandwidth liability" became  
> predominate (as Matthew said), and re-ran the tests with the more- 
> expected results:
> 
> For my machine, it appears to save 10 to 49 threads (the JVM-supplied  
> value), varying largely presumably due to variances in the length of  
> send queues. Also, I've seen quite high numbers for 'pooled threads  
> awaiting work'.

Right, no real network-level effects, unless lots of nodes are bottlenecked on 
threads, but a good local optimisation for thread-scarce nodes.
> 
> After cursory inspection, I suspect that the other related handlers  
> (Insert/SSK) would not see the same benefit as RequestHandler; and are  
> not modified.

Well they generally take longer, and use more threads (especially CHK 
inserts).
> 
> It may not therefore show a significant change to the common user (as  
> the ack appears to have), but (at least for me, i.e. ">threadLimit"  
> reject) it appears to give the user the ability to use more bandwidth,  
> or perhaps to have longer send-queues (service slower peers).
> 
> Question: Is it a bug that messages back to the source are not counted  
> as the liability for handling a request, or is this intentional? (e.g.  
> sendSync(dnf, null); )

Probably. If it was intentional there should be a comment explaining it.
> 
> Question: Should there be a note in the config page about bandwidth  
> limits effect on threads? It seems the more bandwidth you allocate,  
> the more requests you serve, the more threads freenet uses.

Isn't that obvious? The more load the node is under, the more resources it's 
likely to use?
> 
> Commited in r16372
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/23261e60/attachment.pgp