[freenet-dev] Why Freenet is so SLOW! / r16372

[Robert Hailey]

On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote:

> On Friday 30 November 2007 23:03, Robert Hailey wrote:
>>
>> A much-more aggressive patch (which makes all RequestHandler sends
>> Async except the 'last') shows could be 12x; but am concerned over  
>> the
>> byte-counter side effects with which I am not familiar.
>
> [...]
> - To get the correct byte counts.
> This is really important, because it underlies the major load limiting
> systems. However it is only important for terminal messages: we can
> reasonably assume the rest will be piggybacked if necessary.
>
> In RequestHandler, most of the sendSync()'s are terminal, and  
> therefore must
> remain. The one that is the most interesting is the one you  
> mentioned above:
> sending the Accepted message.


Well... I found the time to get the "more aggressive" patch working.

The general idea is that (after the final request) because the ~only~  
reason to keep the thread waiting on the final transmission is the  
byte count, to delegate that to the 'sent()' callback thread.

On my machine (which was showing pre-emptive reject cause as  
">threadLimit", because of the high bandwidth ceiling I have set), I  
saw a increase in node bandwidth (in & out), and saw a new pre-emptive  
rejection cause "insufficient output bandwidth". Realizing then that  
the bandwidth ceiling (being so high) was the cause for my node  
behaving differently than others, I changed my config to less-than  
that observed value, and "output bandwidth liability" became  
predominate (as Matthew said), and re-ran the tests with the more- 
expected results:

For my machine, it appears to save 10 to 49 threads (the JVM-supplied  
value), varying largely presumably due to variances in the length of  
send queues. Also, I've seen quite high numbers for 'pooled threads  
awaiting work'.

After cursory inspection, I suspect that the other related handlers  
(Insert/SSK) would not see the same benefit as RequestHandler; and are  
not modified.

It may not therefore show a significant change to the common user (as  
the ack appears to have), but (at least for me, i.e. ">threadLimit"  
reject) it appears to give the user the ability to use more bandwidth,  
or perhaps to have longer send-queues (service slower peers).

Question: Is it a bug that messages back to the source are not counted  
as the liability for handling a request, or is this intentional? (e.g.  
sendSync(dnf, null); )

Question: Should there be a note in the config page about bandwidth  
limits effect on threads? It seems the more bandwidth you allocate,  
the more requests you serve, the more threads freenet uses.

Commited in r16372

--
Robert Hailey