[freenet-dev] Why Freenet is so SLOW! / r16372

[Ed Tomlinson]

Hi,

Some feedback on this.  Before updating to testing I was seeing about 10% of my 
Preemptive Rejection Reasons attributed to freeHeapPercentThreshold now its:

 101639    Output bandwidth liability
   1216    <freeHeapPercentThreshold

I think this is helping.  I have a 50K limit with 256K for the jvm and an 8G store.

Thanks
Ed


On December 6, 2007, Robert Hailey wrote:
> 
> On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote:
> 
> > On Friday 30 November 2007 23:03, Robert Hailey wrote:
> >>
> >> A much-more aggressive patch (which makes all RequestHandler sends
> >> Async except the 'last') shows could be 12x; but am concerned over  
> >> the
> >> byte-counter side effects with which I am not familiar.
> >
> > [...]
> > - To get the correct byte counts.
> > This is really important, because it underlies the major load limiting
> > systems. However it is only important for terminal messages: we can
> > reasonably assume the rest will be piggybacked if necessary.
> >
> > In RequestHandler, most of the sendSync()'s are terminal, and  
> > therefore must
> > remain. The one that is the most interesting is the one you  
> > mentioned above:
> > sending the Accepted message.
> 
> 
> Well... I found the time to get the "more aggressive" patch working.
> 
> The general idea is that (after the final request) because the ~only~  
> reason to keep the thread waiting on the final transmission is the  
> byte count, to delegate that to the 'sent()' callback thread.
> 
> On my machine (which was showing pre-emptive reject cause as  
> ">threadLimit", because of the high bandwidth ceiling I have set), I  
> saw a increase in node bandwidth (in & out), and saw a new pre-emptive  
> rejection cause "insufficient output bandwidth". Realizing then that  
> the bandwidth ceiling (being so high) was the cause for my node  
> behaving differently than others, I changed my config to less-than  
> that observed value, and "output bandwidth liability" became  
> predominate (as Matthew said), and re-ran the tests with the more- 
> expected results:
> 
> For my machine, it appears to save 10 to 49 threads (the JVM-supplied  
> value), varying largely presumably due to variances in the length of  
> send queues. Also, I've seen quite high numbers for 'pooled threads  
> awaiting work'.
> 
> After cursory inspection, I suspect that the other related handlers  
> (Insert/SSK) would not see the same benefit as RequestHandler; and are  
> not modified.
> 
> It may not therefore show a significant change to the common user (as  
> the ack appears to have), but (at least for me, i.e. ">threadLimit"  
> reject) it appears to give the user the ability to use more bandwidth,  
> or perhaps to have longer send-queues (service slower peers).
> 
> Question: Is it a bug that messages back to the source are not counted  
> as the liability for handling a request, or is this intentional? (e.g.  
> sendSync(dnf, null); )
> 
> Question: Should there be a note in the config page about bandwidth  
> limits effect on threads? It seems the more bandwidth you allocate,  
> the more requests you serve, the more threads freenet uses.
> 
> Commited in r16372
> 
> --
> Robert Hailey
> 
> _______________________________________________
> Devl mailing list
> Devl at freenetproject.org
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
> 
>