From nextgens at freenetproject.org Sat Dec 1 00:08:40 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sat, 1 Dec 2007 01:08:40 +0100 Subject: [freenet-dev] Another update to german language In-Reply-To: <4750A3C4.5020009@gmx.de> References: <20071130224522.26781390874@freenetproject.org> <4750A3C4.5020009@gmx.de> Message-ID: <20071201000840.GC10184@freenetproject.org> * Thomas [2007-12-01 00:59:00]: > Added/changed some more strings and reverted 2 not intended changes from last commit Commited in r16172 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/18513a85/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 16:18:22 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 16:18:22 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <47509E27.6090303@cs.ucl.ac.uk> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711301859.58824.toad@amphibian.dyndns.org> <47509E27.6090303@cs.ucl.ac.uk> Message-ID: <200712011618.30874.toad@amphibian.dyndns.org> On Friday 30 November 2007 23:35, Michael Rogers wrote: > Matthew Toseland wrote: > > TCP/IP is end-to-end. It loses packets on each hop. > > There's really no comparison between Freenet requests and IP packets. > Packets don't backtrack. Routers don't set local timeouts for every > packet. There's no flow control or retransmission between routers, only > between endpoints. TCP is *not* a good model for what happens end-to-end > across Freenet. (In fact the term end-to-end doesn't even make sense - a > single insert can reach multiple endpoints!) Agreed, it's not a good analogy. But it's the basis of the current backoff-plus-AIMD load management system. Which is why we need to get rid of it. > > *However* TCP is a good model for what happens between neighbouring > nodes. In fact it's such a good model that I think we should copy it. We do. The link layer uses a protocol which behaves more or less the same as TCP with regard to congestion control. > > > One request and one ack use the same number of bytes in immediate terms of > > just that message, but a request will use a LOT more bytes overall. > > > > Therefore, IMHO it would be inappropriate to limit messages rather than > > requests. We may need to limit messages as well, but requests are the main > > concern. > > You're right, token passing isn't redundant after all. We need a > transport layer that limits messages, and a load balancing layer that > limits requests. They should be separate layers. > > > How exactly do you decide which requests to accept then? > > You don't decide, because there's no rejection any more. You just hand > out tokens and try to forward whatever requests your peers in their > infinite wisdom decide to send you. IMHO we would still need rejection, for several reasons: 1. Reject due to loop. 2. Reject due to overload if something unexpected happens. 3. Reject due to overload as a normal part of the operation of the node because if we just send one token to one node it will not always send a request, so IMHO we have to send tokens to several nodes and then reject some if we get more requests than we'd expected. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/9d72e59b/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 17:51:35 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 17:51:35 +0000 Subject: [freenet-dev] [freenet-cvs] r16168 - trunk/freenet/src/freenet/node/fcp In-Reply-To: <20071130234406.7F295479EF5@freenetproject.org> References: <20071130234406.7F295479EF5@freenetproject.org> Message-ID: <200712011751.41083.toad@amphibian.dyndns.org> IMHO we should allow DDA's to have an Identifier, which if not specified is the directory name given. Then we can easily implement #1753. On Friday 30 November 2007 23:44, you wrote: > Author: nextgens > Date: 2007-11-30 23:44:06 +0000 (Fri, 30 Nov 2007) > New Revision: 16168 > > Modified: > trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > Log: > Resolve #1856: Nested TestDDA fails > > Modified: trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > =================================================================== > --- trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 23:34:04 UTC (rev 16167) > +++ trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 23:44:06 UTC (rev 16168) > @@ -372,8 +372,15 @@ > if(!directory.exists() || !directory.isDirectory()) > throw new IllegalArgumentException("The specified path isn't a directory! or doesn't exist or the node doesn't have access to it!"); > > + // See #1856 > + DDACheckJob job = null; > + synchronized (inTestDirectories) { > + job = (DDACheckJob) inTestDirectories.get(directory); > + } > + if(job != null) > + throw new IllegalArgumentException("There is already a TestDDA going on for that directory!"); > + > File writeFile = (write ? new File(path, "DDACheck-" + new Random().nextInt() + ".tmp") : null); > - > File readFile = null; > if(read) { > try { > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/f1c714d0/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 18:07:57 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 18:07:57 +0000 Subject: [freenet-dev] [freenet-cvs] r16182 - trunk/freenet/src/freenet/node In-Reply-To: <20071201151436.632FC47AE7D@freenetproject.org> References: <20071201151436.632FC47AE7D@freenetproject.org> Message-ID: <200712011807.57341.toad@amphibian.dyndns.org> Wrong. The below will not disconnect if we rekeyed due to transferred data rather than due to time. On Saturday 01 December 2007 15:14, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-01 15:14:36 +0000 (Sat, 01 Dec 2007) > New Revision: 16182 > > Modified: > trunk/freenet/src/freenet/node/PeerNode.java > Log: > JFK: simplify|fix the logic of rekeying > > Modified: trunk/freenet/src/freenet/node/PeerNode.java > =================================================================== > --- trunk/freenet/src/freenet/node/PeerNode.java 2007-12-01 14:50:20 UTC (rev 16181) > +++ trunk/freenet/src/freenet/node/PeerNode.java 2007-12-01 15:14:36 UTC (rev 16182) > @@ -899,30 +899,33 @@ > > protected void maybeRekey() { > long now = System.currentTimeMillis(); > - boolean hasRekeyed = false; > - if(hasLiveHandshake(now)) > - return; > - > + boolean shouldDisconnect = false; > + boolean shouldReturn = false; > + boolean shouldRekey = false; > long timeWhenRekeyingShouldOccur = 0; > - synchronized(this) { > - if(isRekeying || !isConnected) > - return; > + > + synchronized (this) { > timeWhenRekeyingShouldOccur = timeLastRekeyed + FNPPacketMangler.SESSION_KEY_REKEYING_INTERVAL; > - if((timeWhenRekeyingShouldOccur < now) || (totalBytesExchangedWithCurrentTracker > FNPPacketMangler.AMOUNT_OF_BYTES_ALLOWED_BEFORE_WE_REKEY)) { > - hasRekeyed = true; > + shouldDisconnect = (timeWhenRekeyingShouldOccur + FNPPacketMangler.MAX_SESSION_KEY_REKEYING_DELAY < now) && isRekeying; > + shouldReturn = isRekeying || !isConnected; > + shouldRekey = (timeWhenRekeyingShouldOccur < now) || (totalBytesExchangedWithCurrentTracker > FNPPacketMangler.AMOUNT_OF_BYTES_ALLOWED_BEFORE_WE_REKEY); > + } > + > + if(shouldDisconnect) { > + String time = TimeUtil.formatTime(FNPPacketMangler.MAX_SESSION_KEY_REKEYING_DELAY); > + System.err.println("The peer (" + this + ") has been asked to rekey " + time + " ago... force disconnect."); > + Logger.error(this, "The peer (" + this + ") has been asked to rekey " + time + " ago... force disconnect."); > + forceDisconnect(); > + } else if (shouldReturn || hasLiveHandshake(now)) { > + return; > + } else if(shouldRekey) { > + synchronized(this) { > isRekeying = true; > sendHandshakeTime = now; // Immediately > ctx = null; > } > + Logger.normal(this, "We are asking for the key to be renewed (" + this.detectedPeer + ')'); > } > - > - if(timeWhenRekeyingShouldOccur + FNPPacketMangler.MAX_SESSION_KEY_REKEYING_DELAY < now) { > - Logger.error(this, "The peer (" + this + ") has been asked to rekey " + TimeUtil.formatTime(FNPPacketMangler.MAX_SESSION_KEY_REKEYING_DELAY) + " ago... force disconnect."); > - forceDisconnect(); > - } > - > - if(hasRekeyed) > - Logger.normal(this, "We are asking for the key to be renewed (" + this.detectedPeer + ')'); > } > > /** > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/8f3325ab/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 18:17:09 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 18:17:09 +0000 Subject: [freenet-dev] [freenet-cvs] r16125 - trunk/freenet/src/freenet/node In-Reply-To: <20071130131032.8590E47962C@freenetproject.org> References: <20071130131032.8590E47962C@freenetproject.org> Message-ID: <200712011817.10452.toad@amphibian.dyndns.org> Is there any way that this can be verified automatically? If not, I'd like you to revert it. Indenting as in changing whitespace is one thing, indenting as in eliminating the {}'s in { one statement } is quite another thing... On Friday 30 November 2007 13:10, you wrote: > Author: nextgens > Date: 2007-11-30 13:10:32 +0000 (Fri, 30 Nov 2007) > New Revision: 16125 > > Modified: > trunk/freenet/src/freenet/node/PeerNode.java > Log: > Indent properly > > Modified: trunk/freenet/src/freenet/node/PeerNode.java > =================================================================== [ snip almost the entire file ] -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/cf97d6e3/attachment.pgp From nextgens at freenetproject.org Sat Dec 1 19:27:21 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sat, 1 Dec 2007 20:27:21 +0100 Subject: [freenet-dev] [freenet-cvs] r16168 - trunk/freenet/src/freenet/node/fcp In-Reply-To: <200712011751.41083.toad@amphibian.dyndns.org> References: <20071130234406.7F295479EF5@freenetproject.org> <200712011751.41083.toad@amphibian.dyndns.org> Message-ID: <20071201192721.GC4351@freenetproject.org> * Matthew Toseland [2007-12-01 17:51:35]: > IMHO we should allow DDA's to have an Identifier, which if not specified is > the directory name given. Then we can easily implement #1753. > And what would be the point ? how do we decide which one rules if results are different ? > On Friday 30 November 2007 23:44, you wrote: > > Author: nextgens > > Date: 2007-11-30 23:44:06 +0000 (Fri, 30 Nov 2007) > > New Revision: 16168 > > > > Modified: > > trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > > Log: > > Resolve #1856: Nested TestDDA fails > > > > Modified: trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > > =================================================================== > > --- trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 > 23:34:04 UTC (rev 16167) > > +++ trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 > 23:44:06 UTC (rev 16168) > > @@ -372,8 +372,15 @@ > > if(!directory.exists() || !directory.isDirectory()) > > throw new IllegalArgumentException("The specified path isn't a > directory! or doesn't exist or the node doesn't have access to it!"); > > > > + // See #1856 > > + DDACheckJob job = null; > > + synchronized (inTestDirectories) { > > + job = (DDACheckJob) inTestDirectories.get(directory); > > + } > > + if(job != null) > > + throw new IllegalArgumentException("There is already a TestDDA going on > for that directory!"); > > + > > File writeFile = (write ? new File(path, "DDACheck-" + new > Random().nextInt() + ".tmp") : null); > > - > > File readFile = null; > > if(read) { > > try { > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/1c62d709/attachment.pgp From nextgens at freenetproject.org Sat Dec 1 19:30:33 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sat, 1 Dec 2007 20:30:33 +0100 Subject: [freenet-dev] [freenet-cvs] r16125 - trunk/freenet/src/freenet/node In-Reply-To: <200712011817.10452.toad@amphibian.dyndns.org> References: <20071130131032.8590E47962C@freenetproject.org> <200712011817.10452.toad@amphibian.dyndns.org> Message-ID: <20071201193033.GF4351@freenetproject.org> * Matthew Toseland [2007-12-01 18:17:09]: > Is there any way that this can be verified automatically? If not, I'd like you > to revert it. Indenting as in changing whitespace is one thing, indenting as > in eliminating the {}'s in { one statement } is quite another thing... Hmm, sorry about that... doe anyone know how to tell netbeans not to do that when it indents ? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/b83b8f91/attachment.pgp From nextgens at freenetproject.org Sat Dec 1 19:49:17 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sat, 1 Dec 2007 20:49:17 +0100 Subject: [freenet-dev] [freenet-cvs] r16125 - trunk/freenet/src/freenet/node In-Reply-To: <200712011817.10452.toad@amphibian.dyndns.org> References: <20071130131032.8590E47962C@freenetproject.org> <200712011817.10452.toad@amphibian.dyndns.org> Message-ID: <20071201194917.GG4351@freenetproject.org> * Matthew Toseland [2007-12-01 18:17:09]: > Is there any way that this can be verified automatically? I've found http://andrei.gmxhome.de/bytecode/index.html ... maybe you can use it to compare the produced bytecode. > If not, I'd like you to revert it. Well, tell me; if that doesn't work I'll revert it. > Indenting as in changing whitespace is one thing, indenting as > in eliminating the {}'s in { one statement } is quite another thing... Agreed; I haven't done it on purpose : I naively trusted netbeans to "only indent" the code. NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/ddccbe5e/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 22:54:30 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 22:54:30 +0000 Subject: [freenet-dev] Freenet 0.7 build 1084 Message-ID: <200712012254.35799.toad@amphibian.dyndns.org> Freenet 0.7 build 1084 is now available. Please upgrade; hopefully your node will do this for you automatically. This build features some significant bug fixes for recent connectivity problems, some FCP fixes, we drop some obsolete code, and the beginnings of opennet announcement (which isn't used yet). Please report any bugs you find to the bug tracker (https://bugs.freenetproject.org ), via IRC, Frost (the freenet board) or support at freenetproject.org. Thanks for helping build Freenet. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/7653285c/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 23:13:40 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 23:13:40 +0000 Subject: [freenet-dev] =?utf-8?q?=5Bfreenet-cvs=5D_r16168_-=09trunk/freene?= =?utf-8?q?t/src/freenet/node/fcp?= In-Reply-To: <20071201192721.GC4351@freenetproject.org> References: <20071130234406.7F295479EF5@freenetproject.org> <200712011751.41083.toad@amphibian.dyndns.org> <20071201192721.GC4351@freenetproject.org> Message-ID: <200712012313.45234.toad@amphibian.dyndns.org> On Saturday 01 December 2007 19:27, Florent Daigni?re wrote: > * Matthew Toseland [2007-12-01 17:51:35]: > > > IMHO we should allow DDA's to have an Identifier, which if not specified is > > the directory name given. Then we can easily implement #1753. > > And what would be the point ? how do we decide which one rules if > results are different ? The point is it can be tied to a request. A request can then block on the result of the testDDA. > > > On Friday 30 November 2007 23:44, you wrote: > > > Author: nextgens > > > Date: 2007-11-30 23:44:06 +0000 (Fri, 30 Nov 2007) > > > New Revision: 16168 > > > > > > Modified: > > > trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > > > Log: > > > Resolve #1856: Nested TestDDA fails > > > > > > Modified: trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java > > > =================================================================== > > > --- trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 > > 23:34:04 UTC (rev 16167) > > > +++ trunk/freenet/src/freenet/node/fcp/FCPConnectionHandler.java 2007-11-30 > > 23:44:06 UTC (rev 16168) > > > @@ -372,8 +372,15 @@ > > > if(!directory.exists() || !directory.isDirectory()) > > > throw new IllegalArgumentException("The specified path isn't a > > directory! or doesn't exist or the node doesn't have access to it!"); > > > > > > + // See #1856 > > > + DDACheckJob job = null; > > > + synchronized (inTestDirectories) { > > > + job = (DDACheckJob) inTestDirectories.get(directory); > > > + } > > > + if(job != null) > > > + throw new IllegalArgumentException("There is already a TestDDA going on > > for that directory!"); > > > + > > > File writeFile = (write ? new File(path, "DDACheck-" + new > > Random().nextInt() + ".tmp") : null); > > > - > > > File readFile = null; > > > if(read) { > > > try { > > > > > > _______________________________________________ > > > cvs mailing list > > > cvs at freenetproject.org > > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > > > > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/f0759707/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 23:26:50 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 23:26:50 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: References: <200711300126.45810.toad@amphibian.dyndns.org> <200711301732.30968.toad@amphibian.dyndns.org> Message-ID: <200712012326.51301.toad@amphibian.dyndns.org> On Friday 30 November 2007 19:04, Robert Hailey wrote: > > On Nov 30, 2007, at 11:32 AM, Matthew Toseland wrote: > > Not true either, most rejects are due to bandwidth liability > > I'm not disagreeing with you, but at least from my node: > > Preemptive Rejection Reasons > 28543 >threadLimit > 655 >SUB_MAX_PING_TIME > > Local Preemptive Rejection Reasons > 718 >threadLimit > 33 >SUB_MAX_PING_TIME > > These, the only two listed values, I interpret as meaning that my node > refuses requests primarily because it is waiting on everyone else (the > thread; remembering the state information of requests while they are > processed). That's only your node. And is it still doing that, post 1084? There were some wierd bugs going around... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/4691dcda/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 1 23:30:01 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Dec 2007 23:30:01 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: References: <200711300126.45810.toad@amphibian.dyndns.org> <200711301901.22336.toad@amphibian.dyndns.org> Message-ID: <200712012330.01276.toad@amphibian.dyndns.org> On Friday 30 November 2007 19:35, Robert Hailey wrote: > > On Nov 30, 2007, at 1:01 PM, Matthew Toseland wrote: > > > It really isn't THAT slow. It's not a significant factor in the > > network's > > overall performance. > > > > On Friday 30 November 2007 18:49, Robert Hailey wrote: > >> > >> [...] that an SSK signature is verified at every node[...] would > >> yield the same > >> result 100% of the time.[...] > > On Nov 30, 2007, at 1:04 PM, Robert Hailey wrote: > > [...] I interpret as meaning that my node refuses requests primarily > > because it is waiting on everyone else [...] > > To use an analogy from the recent Houston/Katrina-Rita evacuation, in > relating to bandwidth. A single stoplight at the end of a major > freeway is not a significant factor, unless of course there are > 100,000 cars waiting at it. Even worse, make it two distantly-spaced > stoplights, and no effective progress will be made. > > If it is was the case that all rejects are from bandwidth limitations, > it would follow that the network was running at optimal capacity, only > to be made better by re-arranging the utilization of the bandwidth. Or > also if the network was mostly-idle, I suppose. Besides i/o (network/ > disk), the other major latency is processing (crypto). Perhaps. Or perhaps it isn't being used because bandwidth liability is too conservative. Or perhaps it isn't being used because we're not matching demand to supply, as I originally suggested: when there is a gap, we don't get a request, when there isn't a gap, we do. > > My point is that while node X is verifying my request (which I'm going > to be doing anyway), I must wait. And in waiting, requests are being > dropped. I see the major problem at hand is that the time in-handling > of packets must be minimized, as if we are all waiting on the slowest > node, and for a slow-enough node, the cryptographic verification in a > virtual machine would be slow indeed. This might even be an effective > DDoS attack (accept-and-wait --> overload threads). If every node on the network was waiting for one really slow node then yes we'd have a problem. :) > > Or else, the current system in-place may be like ethernet: beyond a > point of congestion, it just doesn't work. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071201/ed35968e/attachment.pgp From toad at amphibian.dyndns.org Sun Dec 2 00:02:02 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sun, 2 Dec 2007 00:02:02 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! In-Reply-To: <8EFC03C4-519E-48A0-8463-EE77E717B9BB@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <8EFC03C4-519E-48A0-8463-EE77E717B9BB@osndok.com> Message-ID: <200712020002.10322.toad@amphibian.dyndns.org> On Friday 30 November 2007 23:03, Robert Hailey wrote: > > On Nov 30, 2007, at 1:35 PM, Robert Hailey wrote: > > [...] I see the major problem at hand is that the time in-handling > > of packets must be minimized[...] > > To illustrate my point the following patch shows an approximate 100% > speedup for freenet, by my crude profiling. Speedup measured how? More comments below. > > Index: freenet/node/RequestHandler.java > =================================================================== > --- freenet/node/RequestHandler.java (revision 16160) > +++ freenet/node/RequestHandler.java (working copy) > @@ -121,7 +121,7 @@ > htl = source.decrementHTL(htl); > > Message accepted = DMT.createFNPAccepted(uid); > - source.sendSync(accepted, null); > + source.sendAsync(accepted, null, 0, null); > > Object o = node.makeRequestSender(key, htl, uid, source, > closestLoc, resetClosestLoc, false, true, false); > if(o instanceof KeyBlock) { > > A much-more aggressive patch (which makes all RequestHandler sends > Async except the 'last') shows could be 12x; but am concerned over the > byte-counter side effects with which I am not familiar. Hmmm. Okay, here's how it works: *Sender is responsible for sending a request out, talking to the nodes doing the request. *Handler is responsible for receiving a request, and keeping the requestor up to date. One Sender may serve many Handlers, or it may serve a local request. One Handler may have no Sender if it is e.g. a request satisfied from the store, or if it ran out of HTL. There are two main reasons for using sendSync(): - To make sure the connection to the requestor is still open, and responsive (won't timeout us and then reroute the request) and kill the request if it isn't. Once a request has started, it is not possible to stop it (on downstream nodes), so opportunities are quite limited; the fact that one Sender can serve many Handlers or none makes it difficult to push the stop button. - To get the correct byte counts. This is really important, because it underlies the major load limiting systems. However it is only important for terminal messages: we can reasonably assume the rest will be piggybacked if necessary. In RequestHandler, most of the sendSync()'s are terminal, and therefore must remain. The one that is the most interesting is the one you mentioned above: sending the Accepted message. One reason to keep it as-is is to deal with nodes connecting, sending a load of requests, and then disconnecting, as a DoS attack. This is implausible as a sensible the load limiting system won't let them start enough requests to be a problem given the relative difficulty of getting new connections. Of course, getting new connections is likely to be rather easy on opennet... Another reason is if a node or a link is hopelessly overloaded, we don't want to run a request which will not serve any purpose because the node will timeout on us and route it to another node. This isn't really dealt with by the current code either. So it looks like your patch is worthwhile... You can use the same logic on all the Handler's and Sender's. Either send me a new patch, or send me your preferred username, password and email address so I can give you an SVN account and you can commit it yourself. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071202/a95cea10/attachment.pgp From m.rogers at cs.ucl.ac.uk Sun Dec 2 00:25:57 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sun, 02 Dec 2007 00:25:57 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <200712011618.30874.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711301859.58824.toad@amphibian.dyndns.org> <47509E27.6090303@cs.ucl.ac.uk> <200712011618.30874.toad@amphibian.dyndns.org> Message-ID: <4751FB95.2030802@cs.ucl.ac.uk> Matthew Toseland wrote: > We do. The link layer uses a protocol which behaves more or less the same as > TCP with regard to congestion control. Maybe this has been fixed in 1084, but a couple of days ago my node was trying to send 107 KB/s through a 16 KB/s pipe for long enough to kill all the TCP connections sharing the pipe. I understand that was due to a bug at the request layer, but it shows that congestion control at the message layer currently isn't working: we're depending on the request layer to do both jobs. > IMHO we would still need rejection, for several reasons: > 1. Reject due to loop. Yeah, of course. :-) > 2. Reject due to overload if something unexpected happens. I'm not sure what you have in mind... > 3. Reject due to overload as a normal part of the operation of the node > because if we just send one token to one node it will not always send a > request, so IMHO we have to send tokens to several nodes and then reject some > if we get more requests than we'd expected. If we still need to reject requests pre-emptively then what's the advantage of tokens over the current system? But on the other hand I see your point, it's possible that several peers will spend their tokens at once. How about this: instead of tokens, we give each peer an enforced delay between requests. If the load is light, we gradually decrease the delay. If the load is heavy, we increase the delay. Like token buckets, we can enforce fairness or other policies by giving different peers different delays. Unlike stop/start signals or tokens it smooths out bursts, because a peer can't save up delay: if there's a long delay between requests A and B, you still have to wait the full period between B and C. What happens if a request arrives early? Maybe the peer's ignoring the delay or maybe it was network jitter. Keep the peer's requests in a queue and process them at the proper time... if the queue gets more than about one second long, something's wrong - drop the connection and warn the user. What do you reckon? Cheers, Michael From nextgens at freenetproject.org Sun Dec 2 14:52:51 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sun, 2 Dec 2007 15:52:51 +0100 Subject: [freenet-dev] Freenet build 1085 Message-ID: <20071202145250.GC4369@freenetproject.org> Freenet build 1085 is out. It fixes an important bug, hence it is self-mandatory tonight at midnight. Your node should auto-update soon, if it doesn't before tomorrow, force a manual update. NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071202/54d0b22b/attachment.pgp From batosai at batosai.net Sun Dec 2 17:26:35 2007 From: batosai at batosai.net (Julien Cornuwel aka batosai) Date: Sun, 02 Dec 2007 18:26:35 +0100 Subject: [freenet-dev] Updated french translation for 1085 Message-ID: <4752EACB.1060004@batosai.net> -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.fr.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071202/f129b444/attachment.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071202/f129b444/attachment.pgp From liquido at mail.com Sun Dec 2 18:55:57 2007 From: liquido at mail.com (Cooo heter) Date: Sun, 2 Dec 2007 13:55:57 -0500 Subject: [freenet-dev] Swedish translation V.7 - Cooo Message-ID: <20071202185557.9F9C916427D@ws1-4.us4.outblaze.com> Swedish translation V.7 - Cooo Another update to the swedish translation. Mostly new translations, some of them might be updates to old ones. HTMLFilter.unknownTag=ok?nd tagg ${tag} QueueToadlet.progress=Utveckling QueueToadlet.insertFileInsertFileLabel=Ladda upp fil QueueToadlet.insertFileLabel=Fil QueueToadlet.insertFileBrowseLabel=Bl?ddra QueueToadlet.insertFileCompressLabel=Komprimera Toadlet.nodeHomepage=Startsidan IPDetectorPluginManager.direct=Du verkar vara direkt ansluten till internet. Anslutningar till andra Freenet noder b?r ske utan n?gra problem. IPDetectorPluginManager.noConnectivity=Din internet anslutning verkar inte st?dja UPD.Om inte detta ?r en felbed?mmning s? ?r det inte troligt att du kan anv?nda Freenet. StatisticsToadlet.fullTitle=Statistik f?r ${name} StatisticsToadlet.statisticGatheringTitle=Statistik samling PluginManager.pluginLoadingFailedWithMessage=${name} kunde inte laddas: ${message} PluginManager.pluginLoadingFailedTitle=Kunde inte ladda plugin! PluginMana ger.pluginLoadingFailed=${name} kunde inte laddas FirstTimeWizardToadlet.connectToStrangers=Anslut till fr?mlingar? FirstTimeWizardToadlet.continue=Forts?tt FetchException.shortError.29=Felaktig MIME typ UserAlertManager.alertsOnHomepage=| Mer information finns p? ${link}startsidan${/link}. N2NTMUserAlert.title=Nod till Nod Text meddelande nummer ${number} fr?n ${peername} (${peer}) N2NTMUserAlert.header=Avs?ndare : ${from} | Skrivet : ${composed} | Skickat : ${sent} | Mottaget : ${received} ConnectivityToadlet.noreply=INGET SVAR ConnectivityToadlet.addressTitle=Adress ConnectivityToadlet.sentReceivedTitle=Skickade / Mottagna paket ConnectivityToadlet.local=LOKAL IPUndetectedUserAlert.suggestForwardTwoPorts=Det ?r ocks? en god ide att vidarebefodra portarna ${port1} och ${port2} (UDP) i din router f?r att g?ra det l?tt att ansluta till din nod. IPUndetectedUserAlert.suggestForwardPort=Det ?r ocks? en god ide att vidarebefodra porten ${port} (UDP) i din router f?r att g?ra det l?tt att ansluta till din nod. NodeUpdateManager.updateCatastropheTitle=Katastrofalt fel vid uppdatering! BookmarkItem.deleteBookmarkUpdateNotification=Radera meddelande BookmarkItem.unnamedBookmark=Namnl?st bokm?rke ConfigToadlet.pluginmanager=Pluginhanterare ConfigToadlet.contributeTranslation=Bidra till ?vers?ttningen ConfigToadlet.pluginmanager2=pluginhanterare2 ConfigToadlet.logger=loggning ConfigToadlet.false=nej ConfigToadlet.true=ja FileOffer.succeededReceiveTitle=Filen togs emot utan problem FileOffer.rejectTransferButton=Avvisa ?verf?ring FProxyToadlet.backToFProxy=${link}Klicka h?r${/link} f?r att g? till FProxys startsida. FProxyToadlet.expectedMimeType=F?rv?ntad MIME typ: ${mime} FProxyToadlet.opennetTitle=Fr?mlingar FProxyToadlet.welcomeTitle=Start FProxyToadlet.welcome=Startsida FProxyToadlet.abortToHomepage=Avbryt och ?terv?nd till FProxys startsida PproxyToadlet.refreshOnStartup=Ladda om fr?n servern vid uppstart PproxyToadlet.startingPluginsTitle=Startar Plugin PproxyToadlet.startingPluginStatus=Status PproxyToadlet.Load=Starta PproxyToadlet.changeReloadOnStartup=?ndra PproxyToadlet.reloadOnStartupShort=Starta varje g?ng Freenet startas? PproxyToadlet.reloadPluginTitle=Ladda om Plugin PproxyToadlet.pluginStopping=Pluginen stoppas PproxyToadlet.startingPluginName=Plugin namn PproxyToadlet.unauthorized=Du har inte beh?righet att bes?ka den h?r sidan. PproxyToadlet.loadOfficialPlugin=Starta en officiell plugin PproxyToadlet.Error=Fel PproxyToadlet.startingPluginStatus.starting=startar PproxyToadlet.startingPluginStatus.downloading=Laddar ner PeerManagerUserAlert.connErrorTitle=Vissa noder kan inte ansluta SimpleToadletServer.cannotChangePortOnTheFly=Kan inte ?ndra FProxys port nummer n?r noden ?r ig?ng Node.maxOpennetPeersMustBeTwentyOrLess=F?r ej vara ?ver 20 Node.maxOpennetPeers=Maximalt antal Opennet noder Node.maxOpennetPeersLong=Maximalt antal Opennet noder (ska vara mellan 0 till och med 20) Node.oneConnectionPerIP=Begr?nsa till en anslutning per adress? BookmarkManager.malformedBookmark=Felaktigt bokm?rke ConnectionsToadlet.nodeStatus.LISTENING=LYSSNAR ConnectionsToadlet.nodeStatus.CLOCK PROBLEM=KLOCK PROBLEM ConnectionsToadlet.nodeStatus.DISCONNECTED=ICKE ANSLUTEN ConnectionsToadlet.nodeStatus.TOO OLD=F?R GAMMAL ConnectionsToadlet.nodeStatus.UNKNOWN STATUS=OK?ND STATUS ConnectionsToadlet.nodeStatus.CONNECTED=ANSLUTEN ConnectionsToadlet.nodeStatus.NEVER CONNECTED=ALDRIG ANSLUTEN ConnectionsToadlet.nodeStatus.CONNECTION ERROR=ANSLUTNINGS FEL ConnectionsToadlet.nodeStatus.LISTEN ONLY=ENDAST LYSSNA ConnectionsToadlet.nodeStatus.BUSY=UPPTAGEN ConnectionsToadlet.nodeStatus.BACKED OFF=UPPTAGEN ConnectionsToadlet.nodeStatus.TOO NEW=F?R NY UpdatedVersionAvailableUserAlert.title=En ny stabil version av Freenet finns tillg?nglig UpdatedVersionAvailableUserAlert.fetchingNewNode=Noden laddar nu ner en ny version av Freenet (nod version ${nodeVersion}). UpdatedVersionAvailableUserAlert.alsoDownloadedNewExtJar=Noden har ocks? laddat ner en ny version av Freenet extra jar, version ${version} WelcomeToadlet.update=Uppdatera WelcomeToadlet.shutdownConfirm=?r det s?kert att du vill st?nga av noden? WelcomeToadlet.privateKeyHeader=Privat nyckel WelcomeToadlet.shutdownNode=St?ng av noden WelcomeToadlet.subjectHeader=?mne WelcomeToadlet.homepageFullTitleWithName=Freenet FProxys startsida f?r ${name} WelcomeToadlet.thanks=Tack f?r att du anv?nder Freenet. WelcomeToadlet.restarting=V?nligen v?nta medans Freenet startas om, detta kan ta upp till 3 minuter.Tack f?r att du anv?nder Freenet. WelcomeToadlet.shutdownConfirmTitle=Nod avst?ngning WelcomeToadlet.keyRequestLabel=Nyckel: WelcomeToadlet.publicKeyHeader=Publik nyckel WelcomeToadlet.versionHeader=Versions information & nod kontrol WelcomeToadlet.shutdownDone=Freenet noden st?ngs ner. WelcomeToadlet.shutdown=St?ng av WelcomeToadlet.restart=Starta om WelcomeToadlet.testnetWarning=K?r i testnet l?ge. Detta kommer allvarligt att ?ventyra din anonymitet! WelcomeToadlet.updatingTitle=Noden uppdateras WelcomeToadlet.restartingTitle=Freenet noden startas om. WelcomeToadlet.restartNode=Starta om noden WelcomeToadlet.restartConfirm=?r det s?kert att du vill starta om noden? WelcomeToadlet.restartConfirmTitle=Nod omstart WelcomeToadlet.nodeUpdateConfirmTitle=Godk?nn nod uppdatering WelcomeToadlet.nodeUpdateConfirm=Vill du uppdatera Freenet noden? WelcomeToadlet.messageHeader=Meddelande WelcomeToadlet.threadDumpTitle=Skapa en tr?d dump PluginToadlet.pluginListTitle=Startade plugins BookmarkEditorToadlet.descLabel=Beskrivning: BookmarkEditorToadlet.invalidKey=Freenet nyckeln ?r ogiltig. End -- Over 2 Million Holiday Gift Ideas - Take a Look! mail.com shopping at http://mail.shopping.com/?linkin_id=8033174 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071202/d2a1b815/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: freenet.l10n.se.override.properties.0.7-Cooo Type: application/octet-stream Size: 6552 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071202/d2a1b815/attachment.obj From nextgens at freenetproject.org Sun Dec 2 19:58:00 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sun, 2 Dec 2007 20:58:00 +0100 Subject: [freenet-dev] Swedish translation V.7 - Cooo In-Reply-To: <20071202185557.9F9C916427D@ws1-4.us4.outblaze.com> References: <20071202185557.9F9C916427D@ws1-4.us4.outblaze.com> Message-ID: <20071202195759.GD4369@freenetproject.org> * Cooo heter [2007-12-02 13:55:57]: > Swedish translation V.7 - Cooo > > Another update to the swedish translation. > Mostly new translations, some of them might be updates to old ones. > Commited in r16208 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071202/cc5e8d86/attachment.pgp From luke771 at gmail.com Mon Dec 3 11:27:26 2007 From: luke771 at gmail.com (luke771) Date: Mon, 03 Dec 2007 12:27:26 +0100 Subject: [freenet-dev] ITA l10n 071203_ph00 Message-ID: <4753E81E.4050102@gmail.com> - Some rewordings to complete and make more accurate some translations, and also make things easier to understand, expecially in long strings with technical explanations. => One mess sorted out: part of the First Time Wizard section was "out of synch" (some strings would show each other's translation in the IT version) - Several typos corrected _ph00 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.it.override.properties_071203_ph00 Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071203/0b9e2b2f/attachment.txt From toad at amphibian.dyndns.org Mon Dec 3 12:17:11 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 12:17:11 +0000 Subject: [freenet-dev] [freenet-cvs] r16205 - trunk/freenet/src/freenet/node In-Reply-To: <20071202141659.AB0494797DF@freenetproject.org> References: <20071202141659.AB0494797DF@freenetproject.org> Message-ID: <200712031217.17983.toad@amphibian.dyndns.org> That's one of nextgens' netbeans auto-indents, are you sure it's real? If it is, we may have a lot more similar problems... On Sunday 02 December 2007 14:16, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-02 14:16:59 +0000 (Sun, 02 Dec 2007) > New Revision: 16205 > > Modified: > trunk/freenet/src/freenet/node/PeerManager.java > trunk/freenet/src/freenet/node/PeerNode.java > Log: > Fix a variable shadowing in PeerManager and a typo preventing version informations from beeing parsed. > > Modified: trunk/freenet/src/freenet/node/PeerNode.java > =================================================================== > --- trunk/freenet/src/freenet/node/PeerNode.java 2007-12-02 11:40:07 UTC (rev 16204) > +++ trunk/freenet/src/freenet/node/PeerNode.java 2007-12-02 14:16:59 UTC (rev 16205) > @@ -1556,7 +1556,7 @@ > node.peers.addConnectedPeer(this); > onConnect(); > } > - > + > setPeerNodeStatus(now); > return true; > } > @@ -1798,18 +1798,16 @@ > throw new FSParseException(err); > } > String newVersion = fs.get("version"); > - if(newVersion == null) > + if(newVersion == null) { > // Version may be ommitted for an ARK. > - > if(!forARK) > throw new FSParseException("No version"); > - else { > + } else { > if(!newVersion.equals(version)) > changedAnything = true; > version = newVersion; > Version.seenVersion(newVersion); > } > - > lastGoodVersion = fs.get("lastGoodVersion"); > > updateShouldDisconnectNow(); > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/06d61553/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 3 12:23:13 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 12:23:13 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <4751FB95.2030802@cs.ucl.ac.uk> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712011618.30874.toad@amphibian.dyndns.org> <4751FB95.2030802@cs.ucl.ac.uk> Message-ID: <200712031223.14490.toad@amphibian.dyndns.org> On Sunday 02 December 2007 00:25, Michael Rogers wrote: > Matthew Toseland wrote: > > We do. The link layer uses a protocol which behaves more or less the same as > > TCP with regard to congestion control. > > Maybe this has been fixed in 1084, but a couple of days ago my node was > trying to send 107 KB/s through a 16 KB/s pipe for long enough to kill > all the TCP connections sharing the pipe. I understand that was due to a > bug at the request layer, but it shows that congestion control at the > message layer currently isn't working: we're depending on the request > layer to do both jobs. Well... yes and no. It was a bug at the LINK layer iirc. Remember the pathetically low payload percentages? > > > IMHO we would still need rejection, for several reasons: > > 1. Reject due to loop. > > Yeah, of course. :-) > > > 2. Reject due to overload if something unexpected happens. > > I'm not sure what you have in mind... > > > 3. Reject due to overload as a normal part of the operation of the node > > because if we just send one token to one node it will not always send a > > request, so IMHO we have to send tokens to several nodes and then reject some > > if we get more requests than we'd expected. > > If we still need to reject requests pre-emptively then what's the > advantage of tokens over the current system? But on the other hand I see > your point, it's possible that several peers will spend their tokens at > once. The point is we can't have large numbers of tokens accumulate on a peer: this is very difficult to manage. Therefore, we must send short-lived tokens to multiple peers whenever there's an opportunity to do a request. > > How about this: instead of tokens, we give each peer an enforced delay > between requests. If the load is light, we gradually decrease the delay. > If the load is heavy, we increase the delay. Like token buckets, we can > enforce fairness or other policies by giving different peers different > delays. We tried this in 0.5... > > Unlike stop/start signals or tokens it smooths out bursts, because a > peer can't save up delay: if there's a long delay between requests A and > B, you still have to wait the full period between B and C. I'm not talking about stop/start signals, nor am I talking about tokens in the sense that you use the word. > > What happens if a request arrives early? If it arrives early, if we've already accepted a different request, etc etc, we simply reject it with overload. Then it remains queued on the sender. > Maybe the peer's ignoring the > delay or maybe it was network jitter. Keep the peer's requests in a > queue and process them at the proper time... if the queue gets more than > about one second long, something's wrong - drop the connection and warn > the user. The proposal already queues the requests, but we need to control the rate at which requests enter the queue. > > What do you reckon? > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/1ef54260/attachment.pgp From nextgens at freenetproject.org Mon Dec 3 13:01:31 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 3 Dec 2007 14:01:31 +0100 Subject: [freenet-dev] ITA l10n 071203_ph00 In-Reply-To: <4753E81E.4050102@gmail.com> References: <4753E81E.4050102@gmail.com> Message-ID: <20071203130130.GD4285@freenetproject.org> * luke771 [2007-12-03 12:27:26]: > - Some rewordings to complete and make more accurate some translations, and > also make things easier to understand, expecially in long strings with > technical explanations. > > => One mess sorted out: part of the First Time Wizard section was "out of > synch" (some strings would show each other's translation in the IT version) > > - Several typos corrected > > _ph00 Commited in r16219 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/fbf52455/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 3 14:24:07 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 14:24:07 +0000 Subject: [freenet-dev] [freenet-cvs] r16220 - trunk/freenet/src/freenet/support In-Reply-To: <20071203141944.0373E479871@freenetproject.org> References: <20071203141944.0373E479871@freenetproject.org> Message-ID: <200712031424.07941.toad@amphibian.dyndns.org> 1L* ? That's also redundant, just put the L on the 1000, or the 7. On Monday 03 December 2007 14:19, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-03 14:19:43 +0000 (Mon, 03 Dec 2007) > New Revision: 16220 > > Modified: > trunk/freenet/src/freenet/support/TimeUtil.java > Log: > TimeUtil: remove all the casting altogether > > Modified: trunk/freenet/src/freenet/support/TimeUtil.java > =================================================================== > --- trunk/freenet/src/freenet/support/TimeUtil.java 2007-12-03 13:08:51 UTC (rev 16219) > +++ trunk/freenet/src/freenet/support/TimeUtil.java 2007-12-03 14:19:43 UTC (rev 16220) > @@ -57,41 +57,41 @@ > return sb.toString(); > } > // > - long weeks = (l / ((long)7*24*60*60*1000)); > + long weeks = (l / (1L*7*24*60*60*1000)); > if (weeks > 0) { > sb.append(weeks).append('w'); > termCount++; > - l = l - (weeks * ((long)7*24*60*60*1000)); > + l = l - (weeks * (1L*7*24*60*60*1000)); > } > if(termCount >= maxTerms) { > return sb.toString(); > } > // > - long days = (l / ((long)24*60*60*1000)); > + long days = (l / (1L*24*60*60*1000)); > if (days > 0) { > sb.append(days).append('d'); > termCount++; > - l = l - (days * ((long)24*60*60*1000)); > + l = l - (days * (1L*24*60*60*1000)); > } > if(termCount >= maxTerms) { > return sb.toString(); > } > // > - long hours = (l / ((long)60*60*1000)); > + long hours = (l / (1L*60*60*1000)); > if (hours > 0) { > sb.append(hours).append('h'); > termCount++; > - l = l - (hours * ((long)60*60*1000)); > + l = l - (hours * (1L*60*60*1000)); > } > if(termCount >= maxTerms) { > return sb.toString(); > } > // > - long minutes = (l / ((long)60*1000)); > + long minutes = (l / (1L*60*1000)); > if (minutes > 0) { > sb.append(minutes).append('m'); > termCount++; > - l = l - (minutes * ((long)60*1000)); > + l = l - (minutes * (1L*60*1000)); > } > if(termCount >= maxTerms) { > return sb.toString(); > @@ -105,7 +105,7 @@ > //l = l - ((long)fractionalSeconds * (long)1000); > } > } else { > - long seconds = (l / (long)1000); > + long seconds = (l / 1L*1000); > if (seconds > 0) { > sb.append(seconds).append('s'); > termCount++; > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/e7b2df52/attachment.pgp From nextgens at freenetproject.org Mon Dec 3 14:26:30 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 3 Dec 2007 15:26:30 +0100 Subject: [freenet-dev] [freenet-cvs] r16220 - trunk/freenet/src/freenet/support In-Reply-To: <200712031424.07941.toad@amphibian.dyndns.org> References: <20071203141944.0373E479871@freenetproject.org> <200712031424.07941.toad@amphibian.dyndns.org> Message-ID: <20071203142630.GF4285@freenetproject.org> * Matthew Toseland [2007-12-03 14:24:07]: > 1L* ? That's also redundant, just put the L on the 1000, or the 7. > Imho it's clearer code-wise... so that if someone changes the value it *will* be clear for him that it's meant to be casted into a long. Arguably noone will ever touch that method again. > On Monday 03 December 2007 14:19, nextgens at freenetproject.org wrote: > > Author: nextgens > > Date: 2007-12-03 14:19:43 +0000 (Mon, 03 Dec 2007) > > New Revision: 16220 > > > > Modified: > > trunk/freenet/src/freenet/support/TimeUtil.java > > Log: > > TimeUtil: remove all the casting altogether > > > > Modified: trunk/freenet/src/freenet/support/TimeUtil.java > > =================================================================== > > --- trunk/freenet/src/freenet/support/TimeUtil.java 2007-12-03 13:08:51 UTC > (rev 16219) > > +++ trunk/freenet/src/freenet/support/TimeUtil.java 2007-12-03 14:19:43 UTC > (rev 16220) > > @@ -57,41 +57,41 @@ > > return sb.toString(); > > } > > // > > - long weeks = (l / ((long)7*24*60*60*1000)); > > + long weeks = (l / (1L*7*24*60*60*1000)); > > if (weeks > 0) { > > sb.append(weeks).append('w'); > > termCount++; > > - l = l - (weeks * ((long)7*24*60*60*1000)); > > + l = l - (weeks * (1L*7*24*60*60*1000)); > > } > > if(termCount >= maxTerms) { > > return sb.toString(); > > } > > // > > - long days = (l / ((long)24*60*60*1000)); > > + long days = (l / (1L*24*60*60*1000)); > > if (days > 0) { > > sb.append(days).append('d'); > > termCount++; > > - l = l - (days * ((long)24*60*60*1000)); > > + l = l - (days * (1L*24*60*60*1000)); > > } > > if(termCount >= maxTerms) { > > return sb.toString(); > > } > > // > > - long hours = (l / ((long)60*60*1000)); > > + long hours = (l / (1L*60*60*1000)); > > if (hours > 0) { > > sb.append(hours).append('h'); > > termCount++; > > - l = l - (hours * ((long)60*60*1000)); > > + l = l - (hours * (1L*60*60*1000)); > > } > > if(termCount >= maxTerms) { > > return sb.toString(); > > } > > // > > - long minutes = (l / ((long)60*1000)); > > + long minutes = (l / (1L*60*1000)); > > if (minutes > 0) { > > sb.append(minutes).append('m'); > > termCount++; > > - l = l - (minutes * ((long)60*1000)); > > + l = l - (minutes * (1L*60*1000)); > > } > > if(termCount >= maxTerms) { > > return sb.toString(); > > @@ -105,7 +105,7 @@ > > //l = l - ((long)fractionalSeconds * (long)1000); > > } > > } else { > > - long seconds = (l / (long)1000); > > + long seconds = (l / 1L*1000); > > if (seconds > 0) { > > sb.append(seconds).append('s'); > > termCount++; > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/3be33f3c/attachment.pgp From nextgens at freenetproject.org Mon Dec 3 16:56:46 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 3 Dec 2007 17:56:46 +0100 Subject: [freenet-dev] [freenet-cvs] r16205 - trunk/freenet/src/freenet/node In-Reply-To: <200712031217.17983.toad@amphibian.dyndns.org> References: <20071202141659.AB0494797DF@freenetproject.org> <200712031217.17983.toad@amphibian.dyndns.org> Message-ID: <20071203165645.GH4285@freenetproject.org> * Matthew Toseland [2007-12-03 12:17:11]: > That's one of nextgens' netbeans auto-indents, are you sure it's real? If it > is, we may have a lot more similar problems... > Yeah, I apologize about that, most of the new-build-spree going on lately could have been avoided... Yes, tt turns out that it's a real IDE problem (I've reported the issue; see : http://www.netbeans.org/issues/show_bug.cgi?id=123258). In the meantime, I've written a script to verify "indent-only" commits so that we can ensure that no more mistakes due to IDE bugs are made. Emu will run that script against any commit which message is "indent" in the 0.7's trunk. According to what I recall and svn log, here is the sum up of indenting commits I've done since I've installed netbeans... I ran the bytecode comparison script I wrote against them and here is the result : r16197 OK r16178 OK r16141 OK r16140 OK r16125 NOK r16090 OK r16077 OK r16072 OK r16069 OK r16067 OK r15884 OK I've fixed the bugs I've introduced in r16224. The morale of the story is "never trust your IDE, even to perform basic tasks like indenting". Shame on me! NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/6cbbc745/attachment.pgp From m.rogers at cs.ucl.ac.uk Mon Dec 3 17:25:52 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Mon, 03 Dec 2007 17:25:52 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <200712031223.14490.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712011618.30874.toad@amphibian.dyndns.org> <4751FB95.2030802@cs.ucl.ac.uk> <200712031223.14490.toad@amphibian.dyndns.org> Message-ID: <47543C20.2070303@cs.ucl.ac.uk> Matthew Toseland wrote: > Well... yes and no. It was a bug at the LINK layer iirc. Remember the > pathetically low payload percentages? What you call the link layer is supposed to be congestion-controlled, right? > I'm not talking about stop/start signals, nor am I talking about tokens in the > sense that you use the word. Then whey did you say your proposal was basically the same as token passing? So just to be clear, you're talking about tokens that expire, backed up by pre-emptive rejection if too many tokens are spent at once? Will the grounds for pre-emptive rejection be the same as they are now (bandwidth liability etc)? If so, how will tokens solve the current problem of too many requests being rejected? Cheers, Michael From toad at amphibian.dyndns.org Mon Dec 3 17:32:36 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 17:32:36 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <47543C20.2070303@cs.ucl.ac.uk> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712031223.14490.toad@amphibian.dyndns.org> <47543C20.2070303@cs.ucl.ac.uk> Message-ID: <200712031732.41169.toad@amphibian.dyndns.org> On Monday 03 December 2007 17:25, Michael Rogers wrote: > Matthew Toseland wrote: > > Well... yes and no. It was a bug at the LINK layer iirc. Remember the > > pathetically low payload percentages? > > What you call the link layer is supposed to be congestion-controlled, right? Yes, at the link layer. Congestion control presently only applies to data packets, and yes I accept that this sucks and we have plans for a new link layer/packet format, which is more like TCP, but they haven't yet been implemented. > > > I'm not talking about stop/start signals, nor am I talking about tokens in the > > sense that you use the word. > > Then whey did you say your proposal was basically the same as token passing? > > So just to be clear, you're talking about tokens that expire, backed up > by pre-emptive rejection if too many tokens are spent at once? Yes. > > Will the grounds for pre-emptive rejection be the same as they are now > (bandwidth liability etc)? If so, how will tokens solve the current > problem of too many requests being rejected? I'm not sure that that is the main problem we have to deal with right now. My suspicion is that pre-emptive rejection is fine, but we're not being sent enough requests in the first place. Output liability limiting *may* cause too many requests to be rejected, but I don't see any evidence for that right now. And anything we could do about that would run the risk of creating timeouts if we have a burst of unusually successful requests. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/3479b431/attachment.pgp From m.rogers at cs.ucl.ac.uk Mon Dec 3 17:44:55 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Mon, 03 Dec 2007 17:44:55 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <200712031732.41169.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712031223.14490.toad@amphibian.dyndns.org> <47543C20.2070303@cs.ucl.ac.uk> <200712031732.41169.toad@amphibian.dyndns.org> Message-ID: <47544097.40104@cs.ucl.ac.uk> Matthew Toseland wrote: > I'm not sure that that is the main problem we have to deal with right now. My > suspicion is that pre-emptive rejection is fine, but we're not being sent > enough requests in the first place. Cool, I see what you mean now - I'd been thinking in terms of replacing both backoff and pre-emptive rejection but I guess it makes sense to replace backoff first and see if that solves the problem. Cheers, Michael From m.rogers at cs.ucl.ac.uk Mon Dec 3 18:52:59 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Mon, 03 Dec 2007 18:52:59 +0000 Subject: [freenet-dev] Short refs was Re: alternative to #freenet-refs In-Reply-To: <474EBFC4.7030201@WhenGendarmeSleeps.org> References: <20071107220045.GA4554@freenetproject.org> <1195995849.10868.24.camel@localhost> <474B09FD.7010906@cs.ucl.ac.uk> <200711271144.47979.toad@amphibian.dyndns.org> <474DD412.307@cs.ucl.ac.uk> <474EBFC4.7030201@WhenGendarmeSleeps.org> Message-ID: <4754508B.7000606@cs.ucl.ac.uk> Thanks! Here's the Java version of the poem generator (same dictionaries as before). Cheers, Michael Volodya wrote: > Michael Rogers wrote: >> Matthew Toseland wrote: >>> Base32 this is 35 chars; the base32 may be easier, and is certainly shorter, >>> but this would be a nice option if you port the code to java. :) >> Yeah, base32 would definitely be more practical but I just love the idea >> of using random poetry to set up a darknet. :-) I'll have a crack at >> porting it to Java when I get a minute. >> >> Cheers, >> Michael > > This is something to start with. It assumes to find the words in the "words-#.dic" files where # represents the > order of the words... so its probably not as clean as you'd want it to be. > > Also i think the problem would be with loading the file into the memory, when it will get large enough to be > representing the true key i think you'll be having some serious out of memory issues. > > - Volodya > > > > ------------------------------------------------------------------------ > > import java.io.BufferedReader; > import java.io.FileReader; > > import java.io.FileNotFoundException; > import java.io.IOException; > > public class PoemKey { > public static int WORDS_PER_KEY = 4; > public static int BYTES_PER_WORD = 2; > > private static String[][] words; > > static > { > words = new String[WORDS_PER_KEY][]; > final int numOfWords = (int)Math.pow(2, BYTES_PER_WORD * 8); > > for(int i=0; i { > words[i] = new String[numOfWords]; > try > { > BufferedReader fin = new BufferedReader(new FileReader("words-"+i+".dic")); > for(int j=0; j < numOfWords; j++) > { > words[i][j] = fin.readLine(); > } > fin.close(); > } > catch(FileNotFoundException e) > { > //Well i don't really know what you'd want to do then. > } > catch(IOException e) > { > //Well i don't really know what you'd want to do then. > } > } > > } > > public static int[] poemToKey(String poem[]) > { > int result[] = new int[WORDS_PER_KEY]; > > return result; > } > > public static String[] keyToPoem(int[] key) > { > String result[] = new String[WORDS_PER_KEY]; > > for(int i=0; i { > result[i] = words[i][key[i]]; > } > > return result; > } > > public static void main(String args[]) > { > int key[] = new int[WORDS_PER_KEY]; > > System.out.print("The random key with "+WORDS_PER_KEY+" blocks "+BYTES_PER_WORD+" bytes each is: "); > > for(int i=0; i { > key[i]=(int)(Math.random()*8*BYTES_PER_WORD); > System.out.print(Integer.toHexString(key[i])+" "); > } > > System.out.println(""); > > System.out.print("The 'Poem' key is: "); > String[] poem = keyToPoem(key); > for(int i=0; i { > System.out.print(poem[i]+" "); > } > System.out.println(""); > } > } > > > ------------------------------------------------------------------------ > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: BasicEnglish.java Type: text/x-java Size: 1845 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/6aedb03b/attachment.java From toad at amphibian.dyndns.org Mon Dec 3 19:26:01 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 19:26:01 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management In-Reply-To: <47544097.40104@cs.ucl.ac.uk> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712031732.41169.toad@amphibian.dyndns.org> <47544097.40104@cs.ucl.ac.uk> Message-ID: <200712031926.05613.toad@amphibian.dyndns.org> On Monday 03 December 2007 17:44, Michael Rogers wrote: > Matthew Toseland wrote: > > I'm not sure that that is the main problem we have to deal with right now. My > > suspicion is that pre-emptive rejection is fine, but we're not being sent > > enough requests in the first place. > > Cool, I see what you mean now - I'd been thinking in terms of replacing > both backoff and pre-emptive rejection but I guess it makes sense to > replace backoff first and see if that solves the problem. Well, the initial idea was for pre-emptive rejection to be *mostly* replaced by a simple limit on the number of running requests (including those queued awaiting a node to be routed to). I would probably add this at least to begin with. However, some other forms of pre-emptive rejection also lend themselves to easily determining that we can accept a few more requests now e.g. bandwidth liability can do this calculation trivially. As you can see there are some details to work out. Lets consider the current pre-emptive rejection code. We currently reject if: - We are over the thread count. We can guesstimate how many threads will be used by a request, and therefore determine how many requests we can accept. - The ping time is too high. Not obvious how we could adapt this, but we can keep it. - The bandwidth-limited-packets average delay is too high. Not obvious how we could adapt this, but we can keep it. This is largely obsoleted by bandwidth liability limiting on most nodes anyway. We could even get rid of it. - Input or output bandwidth liability limits would be exceeded. We only accept any request if we have enough spare bandwidth for one of each type. This is a measure against accidentally favouring specific request types. We can either multiply this, to get a low figure, add an average value based on the average request, to get an average figure, or assume past the first they are all SSK requests, to get a high figure. One of these options should be workable, if not, just go one at a time. - The high level token buckets don't have enough space for the expected bytes transferred. Trivial to adapt, subject to the same worries about request types as above. Again, there may be an argument for getting rid of this. - There isn't enough memory left. Difficult to estimate a number, but not necessary most of the time. Hard resource limits are one obvious reason why we might reject a request even in a "traditional" token passing scheme. Note that while being able to estimate a number greater than one is useful, it's not essential in the new scheme: we expect to send time-limited request solicitations to many of our peers anyway. What happens if we have plenty of capacity? Well, we send out tokens to all our peers with a largish number, and when they make a request, we send another RequestARequest in response. This costs nothing (after packet padding), as we will have to send an Accepted or Reject* in response anyway. But what if there isn't much demand? What if the time-limited tokens expire before the node wants to send a request? Well, we don't specify any specific timeout, we simply reject any requests it sends us if we don't have the capacity to take them. If a node gets a RequestARequest, it updates its token count. If we reject a request due to overload, the token count is reset to zero, and the node won't send us any more requests until we solicit some explicitly by another RequestARequest. Another option would be to issue tokens in terms of bytes rather than requests, but this would likely cause a preference for SSKs, so is probably a bad idea. Does this make sense? However, there is no urgency as the network appears to be behaving at the moment. It would be worth looking into after opennet is fully working (opennet will be fully implemented soon). > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/06b5139b/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 3 19:31:50 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 19:31:50 +0000 Subject: [freenet-dev] Disk space usage choice in the post-setup wizard Message-ID: <200712031931.51440.toad@amphibian.dyndns.org> Would this make the setup process less user-friendly? Is it worth it anyway? ----- Anonymous ----- 2007.10.28 - 23:43:22GMT ----- I suggest setting the default storage to 5GB and putting a real big warning in the installation process telling the user something like "The store is only 5GB by default becuse Freenet needs to be able to run on really OLD machines: you should set this value to 25GB or more, trying to aim for several hundreds GB" Insisting a bit more about this point on the website, wiki, faq, etc., would also be a good idea. This would considerably extend the average life of not-hugely-popular files. ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.12.03 - 15:26:13GMT ----- Would it make it more obvious if we changed it from a dropdown box to a series of radio buttons? So you don't need to click on it to see that you can make it much bigger? ----- cptn_insano at _g2YxqIynCrs2bcwLGQkr+0b544 ----- 2007.12.03 - 17:23:29GMT ----- Just don't give it a default so the user has to pick something. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/225bc51a/attachment.pgp From freenet-devl at osndok.com Mon Dec 3 20:09:40 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 3 Dec 2007 14:09:40 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! In-Reply-To: <200712020002.10322.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <8EFC03C4-519E-48A0-8463-EE77E717B9BB@osndok.com> <200712020002.10322.toad@amphibian.dyndns.org> Message-ID: On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > On Friday 30 November 2007 23:03, Robert Hailey wrote: >> >> To illustrate my point the following patch shows an approximate 100% >> speedup for freenet, by my crude profiling. > > Speedup measured how? More comments below. Measured in average time to handle a request (i.e. the realRun()-time of RequestHandler), which I reason is proportional to the time it takes for a request at-large (propagating through the network). > One reason to keep it as-is is to deal with nodes connecting, > sending a load > of requests, and then disconnecting, as a DoS attack. This is > implausible as > a sensible the load limiting system won't let them start enough > requests to > be a problem given the relative difficulty of getting new > connections. Of > course, getting new connections is likely to be rather easy on > opennet... I agree. Although, whereas sendSync() waits up to one minute for the given request to clear the queue, this attack would be available presently... only offset by a constant amount of time. And the attacker would have to accept enough packets to make the node believe you are 'connected'. > Another reason is if a node or a link is hopelessly overloaded, we > don't want > to run a request which will not serve any purpose because the node > will > timeout on us and route it to another node. This isn't really dealt > with by > the current code either. I'm not sure I understand your point, but (not being a point of contention) you don't have to explain it. It seems to me that this would HELP overloaded nodes, as we process their requests without waiting for the ack which is a mile down there send queue; they might actually get there data (once it arrives and makes it to the front of the send queue). > So it looks like your patch is worthwhile... > > You can use the same logic on all the Handler's and Sender's. Either > send me a > new patch, or send me your preferred username, password and email > address so > I can give you an SVN account and you can commit it yourself. Can do. -- Robert Hailey From gmaxwell at gmail.com Mon Dec 3 20:17:36 2007 From: gmaxwell at gmail.com (Gregory Maxwell) Date: Mon, 3 Dec 2007 15:17:36 -0500 Subject: [freenet-dev] Disk space usage choice in the post-setup wizard In-Reply-To: <200712031931.51440.toad@amphibian.dyndns.org> References: <200712031931.51440.toad@amphibian.dyndns.org> Message-ID: Eh.. The default value is powerful. Why not suggest a default based on free disk space? On Dec 3, 2007 2:31 PM, Matthew Toseland wrote: > Would this make the setup process less user-friendly? Is it worth it anyway? > > ----- Anonymous ----- 2007.10.28 - 23:43:22GMT ----- > > I suggest setting the default storage to 5GB and putting a real big warning in > the installation process telling the user something like "The store is only > 5GB by default becuse Freenet needs to be able to run on really OLD machines: > you should set this value to 25GB or more, trying to aim for several hundreds > GB" > Insisting a bit more about this point on the website, wiki, faq, etc., would > also be a good idea. > This would considerably extend the average life of not-hugely-popular files. > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.12.03 - 15:26:13GMT ----- > > Would it make it more obvious if we changed it from a dropdown box to a series > of radio buttons? So you don't need to click on it to see that you can make > it much bigger? > > ----- cptn_insano at _g2YxqIynCrs2bcwLGQkr+0b544 ----- 2007.12.03 - > 17:23:29GMT ----- > > Just don't give it a default so the user has to pick something. > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > From toad at amphibian.dyndns.org Mon Dec 3 21:42:30 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Dec 2007 21:42:30 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! In-Reply-To: References: <200711300126.45810.toad@amphibian.dyndns.org> <200712020002.10322.toad@amphibian.dyndns.org> Message-ID: <200712032142.35501.toad@amphibian.dyndns.org> On Monday 03 December 2007 20:09, Robert Hailey wrote: > > On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > > > On Friday 30 November 2007 23:03, Robert Hailey wrote: > >> > >> To illustrate my point the following patch shows an approximate 100% > >> speedup for freenet, by my crude profiling. > > > > Speedup measured how? More comments below. > > Measured in average time to handle a request (i.e. the realRun()-time > of RequestHandler), which I reason is proportional to the time it > takes for a request at-large (propagating through the network). Not necessarily, but it looks like the Accepted round-trip is a big part of the total request time yeah. *Very* nice catch! > > > One reason to keep it as-is is to deal with nodes connecting, > > sending a load > > of requests, and then disconnecting, as a DoS attack. This is > > implausible as > > a sensible the load limiting system won't let them start enough > > requests to > > be a problem given the relative difficulty of getting new > > connections. Of > > course, getting new connections is likely to be rather easy on > > opennet... > > I agree. Although, whereas sendSync() waits up to one minute for the > given request to clear the queue, this attack would be available > presently... only offset by a constant amount of time. Well, without the sendSync(), the attacker doesn't have to ack the packet, which saves him some bandwidth. > And the > attacker would have to accept enough packets to make the node believe > you are 'connected'. Okay, on further research... If an attacker connects to a node and starts sending requests, he can stay connected forever simply by keeping on sending packets. He can quickly gag the node simply by never acknowledging any packet, but the node will continue to accept and propagate his requests subject to global limits. Furthermore, the node will leak memory: all the packets that would have been sent to the attacker get queued in RAM. Eventually the node will OOM, and be restarted by the wrapper. Once it loads back up, it will start accepting requests all over again. Ouch. Fixed (or greatly ameliorated) in trunk by amongst other things rejecting requests when the queue to the node is over 4MB, or when it would take over 10 minutes to transfer at current throttle rates. With that fix, the attack we were talking about is very much reduced. > > > Another reason is if a node or a link is hopelessly overloaded, we > > don't want > > to run a request which will not serve any purpose because the node > > will > > timeout on us and route it to another node. This isn't really dealt > > with by > > the current code either. > > I'm not sure I understand your point, but (not being a point of > contention) you don't have to explain it. It seems to me that this > would HELP overloaded nodes, as we process their requests without > waiting for the ack which is a mile down there send queue; they might > actually get there data (once it arrives and makes it to the front of > the send queue). If a request comes in, and the node which sent the request is so hopelessly overloaded that it will miss our Accepted and timeout on us, and send it to another node, then it's not very useful to do the request. Actually that's not true: if it goes to another node, it will get coalesced, or it will get the data that we would have sent to it. So it's not a big deal. > > > So it looks like your patch is worthwhile... > > > > You can use the same logic on all the Handler's and Sender's. Either > > send me a > > new patch, or send me your preferred username, password and email > > address so > > I can give you an SVN account and you can commit it yourself. > > Can do. I look forward to your commit! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071203/f3b567d4/attachment.pgp From m.rogers at cs.ucl.ac.uk Mon Dec 3 23:43:09 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Mon, 03 Dec 2007 23:43:09 +0000 Subject: [freenet-dev] Short refs was Re: alternative to #freenet-refs In-Reply-To: <474EBFC4.7030201@WhenGendarmeSleeps.org> References: <20071107220045.GA4554@freenetproject.org> <1195995849.10868.24.camel@localhost> <474B09FD.7010906@cs.ucl.ac.uk> <200711271144.47979.toad@amphibian.dyndns.org> <474DD412.307@cs.ucl.ac.uk> <474EBFC4.7030201@WhenGendarmeSleeps.org> Message-ID: <4754948D.5060400@cs.ucl.ac.uk> Argh, same mistake again, replace "rand (1)" with "rand (2)" in the Java version. =0) Volodya wrote: > Michael Rogers wrote: >> Matthew Toseland wrote: >>> Base32 this is 35 chars; the base32 may be easier, and is certainly shorter, >>> but this would be a nice option if you port the code to java. :) >> Yeah, base32 would definitely be more practical but I just love the idea >> of using random poetry to set up a darknet. :-) I'll have a crack at >> porting it to Java when I get a minute. >> >> Cheers, >> Michael > > This is something to start with. It assumes to find the words in the "words-#.dic" files where # represents the > order of the words... so its probably not as clean as you'd want it to be. > > Also i think the problem would be with loading the file into the memory, when it will get large enough to be > representing the true key i think you'll be having some serious out of memory issues. > > - Volodya From toad at amphibian.dyndns.org Tue Dec 4 00:00:50 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Dec 2007 00:00:50 +0000 Subject: [freenet-dev] [freenet-cvs] r16249 - trunk/freenet/src/freenet/node In-Reply-To: <20071203234539.1F2F14798BD@freenetproject.org> References: <20071203234539.1F2F14798BD@freenetproject.org> Message-ID: <200712040000.50645.toad@amphibian.dyndns.org> That should probably be two commits, since the two changes are unrelated. On Monday 03 December 2007 23:45, you wrote: > Author: robert > Date: 2007-12-03 23:45:38 +0000 (Mon, 03 Dec 2007) > New Revision: 16249 > > Modified: > trunk/freenet/src/freenet/node/PeerNode.java > trunk/freenet/src/freenet/node/RequestHandler.java > Log: > Fix synchronization target, remove redundant connection check. > > > Modified: trunk/freenet/src/freenet/node/PeerNode.java > =================================================================== > --- trunk/freenet/src/freenet/node/PeerNode.java 2007-12-03 23:17:43 UTC (rev 16248) > +++ trunk/freenet/src/freenet/node/PeerNode.java 2007-12-03 23:45:38 UTC (rev 16249) > @@ -872,8 +872,11 @@ > public void sendAsync(Message msg, AsyncMessageCallback cb, int alreadyReportedBytes, ByteCounter ctr) throws NotConnectedException { > if(logMINOR) > Logger.minor(this, "Sending async: " + msg + " : " + cb + " on " + this); > - if(!isConnected()) > + if(!isConnected()) { > + if (cb instanceof SyncMessageCallback) > + Logger.error(this, "Tried to send " + msg + " but not connected to " + this, new Exception("debug")); > throw new NotConnectedException(); > + } > addToLocalNodeSentMessagesToStatistic(msg); > MessageItem item = new MessageItem(msg, cb == null ? null : new AsyncMessageCallback[]{cb}, alreadyReportedBytes, ctr); > item.getData(this); > @@ -1255,15 +1258,9 @@ > } > > /** > - * Send a message, right now, on this thread, to this node. > + * Enqueue a message to be sent to this node and wait up to a minute for it to be transmitted. > */ > public void sendSync(Message req, ByteCounter ctr) throws NotConnectedException { > - synchronized(this) { > - if(!isConnected()) { > - Logger.error(this, "Tried to send " + req + " but not connected to " + this, new Exception("debug")); > - throw new NotConnectedException(); > - } > - } > SyncMessageCallback cb = new SyncMessageCallback(); > sendAsync(req, cb, 0, ctr); > cb.waitForSend(60 * 1000); > > Modified: trunk/freenet/src/freenet/node/RequestHandler.java > =================================================================== > --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-03 23:17:43 UTC (rev 16248) > +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-03 23:45:38 UTC (rev 16249) > @@ -85,7 +85,7 @@ > if((!finalTransferFailed) && rs != null && status != RequestSender.TIMED_OUT && status != RequestSender.GENERATED_REJECTED_OVERLOAD > && status != RequestSender.INTERNAL_ERROR && !thrown) { > int sent, rcvd; > - synchronized(this) { > + synchronized(bytesSync) { > sent = sentBytes; > rcvd = receivedBytes; > } > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/297bb8ed/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 4 00:05:47 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Dec 2007 00:05:47 +0000 Subject: [freenet-dev] Disk space usage choice in the post-setup wizard In-Reply-To: References: <200712031931.51440.toad@amphibian.dyndns.org> Message-ID: <200712040005.48180.toad@amphibian.dyndns.org> On Monday 03 December 2007 20:17, Gregory Maxwell wrote: > Eh.. The default value is powerful. Why not suggest a default based on > free disk space? It is possible to do this with java 1.6, but not before. Maybe we could use reflection to attempt it. > > On Dec 3, 2007 2:31 PM, Matthew Toseland wrote: > > Would this make the setup process less user-friendly? Is it worth it anyway? > > > > ----- Anonymous ----- 2007.10.28 - 23:43:22GMT ----- > > > > I suggest setting the default storage to 5GB and putting a real big warning in > > the installation process telling the user something like "The store is only > > 5GB by default becuse Freenet needs to be able to run on really OLD machines: > > you should set this value to 25GB or more, trying to aim for several hundreds > > GB" > > Insisting a bit more about this point on the website, wiki, faq, etc., would > > also be a good idea. > > This would considerably extend the average life of not-hugely-popular files. > > > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.12.03 - 15:26:13GMT ----- > > > > Would it make it more obvious if we changed it from a dropdown box to a series > > of radio buttons? So you don't need to click on it to see that you can make > > it much bigger? > > > > ----- cptn_insano at _g2YxqIynCrs2bcwLGQkr+0b544 ----- 2007.12.03 - > > 17:23:29GMT ----- > > > > Just don't give it a default so the user has to pick something. > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/431c26c4/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 4 13:27:59 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Dec 2007 13:27:59 +0000 Subject: [freenet-dev] Freenet 0.7 build 1087 Message-ID: <200712041328.06601.toad@amphibian.dyndns.org> Freenet 0.7 build 1087 is now available. This will be mandatory at 0:00 on Thursday, not because of any critical bugfix but because of a performance enhancement that we need network-wide. Please upgrade. The main changes in this build are: - A significant reduction in per-hop latency found by our new developer, Robert Hailey. Should be around a 50% cut in per hop latency, but I don't expect any long term gain in throughput. - Prevent a nasty memory/bandwidth denial of service attack: Don't let a node make requests faster than it accepts the data resulting from it. - Slightly more accurate request bytes usage statistics, which should mean slightly more accurate load limiting. - Don't keep old queued messages around (dump them after an explicit disconnect, an hour of being disconnected, etc). - Clarifications in the first-time wizard. Please let us know if you find any bugs, via the bug tracker, the mailing lists or Frost. Thanks to: juiceman nextgens robert toad (me) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/a16b7923/attachment.pgp From batosai at batosai.net Tue Dec 4 17:10:05 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Dec 2007 18:10:05 +0100 Subject: [freenet-dev] French translation update for 1087 Message-ID: <475589ED.1060608@batosai.net> Hi, This translation file features several new strings and numerous fixes of chains that had been modified in the english version. Regards. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.fr.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071204/dd96a60b/attachment.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/dd96a60b/attachment.pgp From nextgens at freenetproject.org Tue Dec 4 17:14:19 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 4 Dec 2007 18:14:19 +0100 Subject: [freenet-dev] French translation update for 1087 In-Reply-To: <475589ED.1060608@batosai.net> References: <475589ED.1060608@batosai.net> Message-ID: <20071204171419.GB4322@freenetproject.org> * Julien Cornuwel [2007-12-04 18:10:05]: > Hi, > > This translation file features several new strings and numerous fixes of > chains that had been modified in the english version. > > Regards. Commited in r16265 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/f61d32bf/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 4 17:23:35 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Dec 2007 17:23:35 +0000 Subject: [freenet-dev] Updating the People page In-Reply-To: <4741D958.7090009@batosai.net> References: <200711191729.43587.toad@amphibian.dyndns.org> <4741D958.7090009@batosai.net> Message-ID: <200712041723.40669.toad@amphibian.dyndns.org> Added. Anyone else who has contributed in the last few years who wants to be on the People page, please either add yourself (I will vet outrageous entries!) or ask. On Monday 19 November 2007 18:43, Julien Cornuwel aka batosai wrote: > Matthew Toseland a ?crit : > > Would anyone like to be added to the People page? If you have > contributed code > > or translations either via SVN or via patches, and you want to be > included, > > please reply to this email. Or edit the people page directly in the > website > > SVN (we will edit any outrageous entries!). > > Thanks ! I'd be glad to be added on this page for my work on French > translations (fProxy, Frost and now jSite) and on the documentation. > > Maybe you could add a link to the documentation site ? It's not > finished and french only (ATM, internationalization is planned) but > you can see it at http://www.freenet-doc.fr It's a common work with > Utopiah and Dieppe. I don't know their real names or if they want to > appear on the page. > > My old French documentation site (freenet-fr.info) is no longer > maintained and will disappear as soon as the new one will be validated. > > Regards -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/c5f9a66e/attachment.pgp From batosai at batosai.net Tue Dec 4 17:25:48 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Dec 2007 18:25:48 +0100 Subject: [freenet-dev] Updating the People page In-Reply-To: <200712041723.40669.toad@amphibian.dyndns.org> References: <200711191729.43587.toad@amphibian.dyndns.org> <4741D958.7090009@batosai.net> <200712041723.40669.toad@amphibian.dyndns.org> Message-ID: <47558D9C.8050903@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland a ?crit : > Added. Anyone else who has contributed in the last few years who wants to be > on the People page, please either add yourself (I will vet outrageous > entries!) or ask. Thanks ;) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHVY2co6N05NzaOvURApdwAJ9K9/cGTwcW1v3x09ERNd/TZKjnpgCfQThs svVwkxtI1OMrnyNY5uWJqt0= =6clk -----END PGP SIGNATURE----- From freenet-devl at osndok.com Tue Dec 4 19:58:27 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Tue, 4 Dec 2007 13:58:27 -0600 Subject: [freenet-dev] Updating the People page In-Reply-To: <200712041723.40669.toad@amphibian.dyndns.org> References: <200711191729.43587.toad@amphibian.dyndns.org> <4741D958.7090009@batosai.net> <200712041723.40669.toad@amphibian.dyndns.org> Message-ID: On Dec 4, 2007, at 11:23 AM, Matthew Toseland wrote: > Added. Anyone else who has contributed in the last few years who > wants to be > on the People page, please either add yourself (I will vet outrageous > entries!) or ask. I was going to add:

Robert Hailey
Robert has helped the speed and security of Freenet. Credited with finding the '1010' crypto bug.

But on trying to commit the page I get 403/Forbidden. I would also recommend either bolding the "And hundreds of others" (or a horizontal rule) to separate the postscript from the list. -- Robert Hailey From toad at amphibian.dyndns.org Tue Dec 4 20:07:15 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Dec 2007 20:07:15 +0000 Subject: [freenet-dev] Freenet 0.7 build 1088 Message-ID: <200712042007.21097.toad@amphibian.dyndns.org> Freenet 0.7 build 1088 is now available. Please upgrade. The major changes in this build relate to opennet: We are trying to get full opennet working, so that Freenet will work out of the box with no need for reference exchange on #freenet-refs . Build 1088 includes a hopefully working announcement protocol, which is the second half of the bootstrapping mechanism: once a node has connected to a seednode (a node from a list we provide at install time), it does an announcement to get a bunch of nodes to connect to. After that, opennet path folding will keep the node busy and integrated. Because we need to get full opennet working as soon as possible, this is another mandatory build (at 0:00 on Thursday). Sorry, but the end justifies the means in this case: Getting full opennet working (hopefully in time for Christmas) will greatly improve Freenet's ease of installation. Thank you for using Freenet. If you find any bugs, please report them. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071204/5b890494/attachment.pgp From juiceman69 at gmail.com Wed Dec 5 00:42:17 2007 From: juiceman69 at gmail.com (Juiceman) Date: Tue, 4 Dec 2007 19:42:17 -0500 Subject: [freenet-dev] NPE in trunk (rev ~16295) Message-ID: <8b525dee0712041642j4da0c27kebaafae5e54f4109@mail.gmail.com> INFO | jvm 1 | 2007/12/04 19:38:37 | Error in WrapperListener.start callback. java.lang.NullPointerException INFO | jvm 1 | 2007/12/04 19:38:37 | java.lang.NullPointerException INFO | jvm 1 | 2007/12/04 19:38:37 | at java.lang.System.arraycopy(Native Method) INFO | jvm 1 | 2007/12/04 19:38:37 | at freenet.crypt.ciphers.Rijndael.initialize(Rijndael.java:73) INFO | jvm 1 | 2007/12/04 19:38:37 | at freenet.node.NodeCrypto.(NodeCrypto.java:160) INFO | jvm 1 | 2007/12/04 19:38:37 | at freenet.node.Node.(Node.java:693) INFO | jvm 1 | 2007/12/04 19:38:37 | at freenet.node.NodeStarter.start(NodeStarter.java:152) INFO | jvm 1 | 2007/12/04 19:38:37 | at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) STATUS | wrapper | 2007/12/04 19:38:38 | <-- Wrapper Stopped -- I may disagree with what you have to say, but I shall defend, to the death, your right to say it. - Voltaire Those who would give up Liberty, to purchase temporary Safety, deserve neither Liberty nor Safety. - Ben Franklin From toad at amphibian.dyndns.org Thu Dec 6 02:01:50 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 6 Dec 2007 02:01:50 +0000 Subject: [freenet-dev] Opennet progress report Message-ID: <200712060201.55033.toad@amphibian.dyndns.org> Announcement is working, including connecting to initial seednodes. We don't have any automated way of gathering seednodes yet. There are a couple of largish bugs in the deployed code which cause announcements to bounce around the network indefinitely; fortunately, the deployed code doesn't actually make announcement requests. The trunk code will out of the box announce itself to randomly chosen nodes in its seednodes.fref (if there is one) until it has 10 opennet peers (if opennet is enabled). So, we need: - Yet another mandatory (1089). Sorry folk. - A list of public seednodes. If a few people with static IPs/dyndns and opennet would send me their noderefs, we can build a temporary seednodes.fref from that. In the longer term, the node will detect if you are eligible to be a seednode, ask if you want to be one, and if so upload regularly to a server on emu. - To bundle this list with the node. In conclusion: the end of #freenet-refs is nigh, finally! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071206/04260f06/attachment.pgp From m.rogers at cs.ucl.ac.uk Thu Dec 6 03:25:11 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Thu, 06 Dec 2007 03:25:11 +0000 Subject: [freenet-dev] Opennet progress report In-Reply-To: <200712060201.55033.toad@amphibian.dyndns.org> References: <200712060201.55033.toad@amphibian.dyndns.org> Message-ID: <47576B97.9070407@cs.ucl.ac.uk> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland wrote: > - A list of public seednodes. If a few people with static IPs/dyndns and > opennet would send me their noderefs, we can build a temporary seednodes.fref > from that. In the longer term, the node will detect if you are eligible to be > a seednode, ask if you want to be one, and if so upload regularly to a server > on emu. Attached - but I'll probably be taking my node offline between the 12th and the 18th, so if that's likely to be a critical period then it might be better not to distribute my ref. > In conclusion: the end of #freenet-refs is nigh, finally! Nice work! :-) Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHV2uXyua14OQlJ3sRAr68AJ9Ppw6yUHxs9m4WIXH5eZpPa5g51QCeKOcW d/8GUkxxFr/91u6mp6eULbw= =7gai -----END PGP SIGNATURE----- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: myref.txt Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071206/9c12489f/attachment.txt From m.rogers at cs.ucl.ac.uk Thu Dec 6 03:37:38 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Thu, 06 Dec 2007 03:37:38 +0000 Subject: [freenet-dev] Opennet progress report In-Reply-To: <47576B97.9070407@cs.ucl.ac.uk> References: <200712060201.55033.toad@amphibian.dyndns.org> <47576B97.9070407@cs.ucl.ac.uk> Message-ID: <47576E82.6010409@cs.ucl.ac.uk> Sorry, that was supposed to be off-list. Argh. From Volodya at WhenGendarmeSleeps.org Thu Dec 6 07:45:01 2007 From: Volodya at WhenGendarmeSleeps.org (Volodya) Date: Thu, 06 Dec 2007 10:45:01 +0300 Subject: [freenet-dev] Short refs was Re: alternative to #freenet-refs In-Reply-To: <4754508B.7000606@cs.ucl.ac.uk> References: <20071107220045.GA4554@freenetproject.org> <1195995849.10868.24.camel@localhost> <474B09FD.7010906@cs.ucl.ac.uk> <200711271144.47979.toad@amphibian.dyndns.org> <474DD412.307@cs.ucl.ac.uk> <474EBFC4.7030201@WhenGendarmeSleeps.org> <4754508B.7000606@cs.ucl.ac.uk> Message-ID: <4757A87D.8040201@WhenGendarmeSleeps.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Rogers wrote: > Thanks! Here's the Java version of the poem generator (same dictionaries > as before). > > Cheers, > Michael Have you tried to give it enough words to represent a full key? I have a feeling that what might be needed to not to load the whole file, but rather use it sax style. So you know that you need 1047th line, and that is the only one you actually store in the memory. I'll see if i can improve on your code a bit. I really loved it. q;-) - Volodya - -- http://freedom.libsyn.com/ Voice of Freedom, Radical Podcast http://eng.anarchopedia.org/ Anarchopedia, A Free Knowledge Portal "None of us are free until all of us are free." ~ Mihail Bakunin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHV6h9uWy2EFICg+0RArltAKC2qOsYqxnfmwh1CQeqFlMMN0rLPQCfc2R2 8pvaCGIsHHU/plJbn5OmGgs= =IRo/ -----END PGP SIGNATURE----- From nextgens at freenetproject.org Thu Dec 6 09:53:16 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 6 Dec 2007 10:53:16 +0100 Subject: [freenet-dev] Opennet progress report In-Reply-To: <200712060201.55033.toad@amphibian.dyndns.org> References: <200712060201.55033.toad@amphibian.dyndns.org> Message-ID: <20071206095315.GA4201@freenetproject.org> * Matthew Toseland [2007-12-06 02:01:50]: > Announcement is working, including connecting to initial seednodes. We don't > have any automated way of gathering seednodes yet. There are a couple of > largish bugs in the deployed code which cause announcements to bounce around > the network indefinitely; fortunately, the deployed code doesn't actually > make announcement requests. The trunk code will out of the box announce > itself to randomly chosen nodes in its seednodes.fref (if there is one) until > it has 10 opennet peers (if opennet is enabled). > > So, we need: > - Yet another mandatory (1089). Sorry folk. > - A list of public seednodes. If a few people with static IPs/dyndns and > opennet would send me their noderefs, we can build a temporary seednodes.fref > from that. In the longer term, the node will detect if you are eligible to be > a seednode, ask if you want to be one, and if so upload regularly to a server > on emu. On .5 Emu was polling the data out of some nodes... I would prefer it to remain that way in the longer-term ;) NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071206/36845e22/attachment.pgp From toad at amphibian.dyndns.org Thu Dec 6 14:10:21 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 6 Dec 2007 14:10:21 +0000 Subject: [freenet-dev] Opennet progress report In-Reply-To: <20071206095315.GA4201@freenetproject.org> References: <200712060201.55033.toad@amphibian.dyndns.org> <20071206095315.GA4201@freenetproject.org> Message-ID: <200712061410.27765.toad@amphibian.dyndns.org> On Thursday 06 December 2007 09:53, Florent Daigni?re wrote: > * Matthew Toseland [2007-12-06 02:01:50]: > > On .5 Emu was polling the data out of some nodes... I would prefer it to > remain that way in the longer-term ;) IMHO it's not really feasible as most nodes won't be eligible to be seednodes (generally due to being NATed). > > NextGen$ > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071206/a5a9111c/attachment.pgp From toad at amphibian.dyndns.org Thu Dec 6 16:40:52 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 6 Dec 2007 16:40:52 +0000 Subject: [freenet-dev] Freenet 0.7 build 1089 Message-ID: <200712061640.58234.toad@amphibian.dyndns.org> Freenet 0.7 build 1089 is now available. Please upgrade. This will be mandatory 0:00 Saturday (GMT). The main changes in this build relate to opennet: opennet is now 95% working, if the node has no peers, has opennet enabled, and has a seednodes file, it will try to connect to randomly chosen seednodes and do an announcement to them. Sadly 1088 had some major bugs in the announcement code, so we need 1089 to be deployed as quickly as possible so we can test to see whether they are fixed before actually adding a seednodes file to the installer. But certainly within the coming week #freenet-refs will be no more. So please upgrade! Testing, of both this build, and trunk builds, would be appreciated, and please report any bugs you find to the bug tracker (https://bugs.freenetproject.org/ ), to the mailing lists or to Frost. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071206/58a4c8c4/attachment.pgp From freenet-devl at david.sowder.com Thu Dec 6 17:32:06 2007 From: freenet-devl at david.sowder.com (David Sowder (Zothar)) Date: Thu, 06 Dec 2007 11:32:06 -0600 Subject: [freenet-dev] Opennet progress report In-Reply-To: <200712061410.27765.toad@amphibian.dyndns.org> References: <200712060201.55033.toad@amphibian.dyndns.org> <20071206095315.GA4201@freenetproject.org> <200712061410.27765.toad@amphibian.dyndns.org> Message-ID: <47583216.8030102@david.sowder.com> I believe eligible seednodes pushing voluntarily to emu is much more maintainable than emu needing to be configured to pull from specific volunteer nodes that may or may not still be around in six months. Nodes pushing is more automatic. Matthew Toseland wrote: > On Thursday 06 December 2007 09:53, Florent Daigni?re wrote: > >> * Matthew Toseland [2007-12-06 02:01:50]: >> >> On .5 Emu was polling the data out of some nodes... I would prefer it to >> remain that way in the longer-term ;) >> > > IMHO it's not really feasible as most nodes won't be eligible to be seednodes > (generally due to being NATed). > From ian.clarke at gmail.com Thu Dec 6 17:57:27 2007 From: ian.clarke at gmail.com (Ian Clarke) Date: Thu, 6 Dec 2007 11:57:27 -0600 Subject: [freenet-dev] Opennet progress report In-Reply-To: <47583216.8030102@david.sowder.com> References: <200712060201.55033.toad@amphibian.dyndns.org> <20071206095315.GA4201@freenetproject.org> <200712061410.27765.toad@amphibian.dyndns.org> <47583216.8030102@david.sowder.com> Message-ID: <823242bd0712060957i32c28998ycdc9c3229fd3c1b1@mail.gmail.com> Agreed, I think a push approach will be better, and probably simpler to implement. Ian. On 12/6/07, David Sowder (Zothar) wrote: > I believe eligible seednodes pushing voluntarily to emu is much more > maintainable than emu needing to be configured to pull from specific > volunteer nodes that may or may not still be around in six months. > Nodes pushing is more automatic. > > Matthew Toseland wrote: > > On Thursday 06 December 2007 09:53, Florent Daigni?re wrote: > > > >> * Matthew Toseland [2007-12-06 02:01:50]: > >> > >> On .5 Emu was polling the data out of some nodes... I would prefer it to > >> remain that way in the longer-term ;) > >> > > > > IMHO it's not really feasible as most nodes won't be eligible to be seednodes > > (generally due to being NATed). > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From freenet-devl at osndok.com Thu Dec 6 18:27:21 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Thu, 6 Dec 2007 12:27:21 -0600 Subject: [freenet-dev] NPE in trunk Message-ID: <18A11276-4F8F-4726-B178-8C845E8385E6@osndok.com> Someone's sending packets w/o a source, or something. java.lang.NullPointerException at freenet.io.comm.Message.(Message.java:121) at freenet.io.comm.Message.(Message.java:114) at freenet.io.comm.DMT.createFNPVoid(DMT.java:1198) at freenet.node.PacketSender.realRun(PacketSender.java:308) at freenet.node.PacketSender.run(PacketSender.java:160) at java.lang.Thread.run(Thread.java:613) -- Robert Hailey From freenet-devl at osndok.com Thu Dec 6 20:16:34 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Thu, 6 Dec 2007 14:16:34 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / r16372 In-Reply-To: <200712020002.10322.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <8EFC03C4-519E-48A0-8463-EE77E717B9BB@osndok.com> <200712020002.10322.toad@amphibian.dyndns.org> Message-ID: <15D4A571-25AD-48C4-A12D-8E79E66E2F12@osndok.com> On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > On Friday 30 November 2007 23:03, Robert Hailey wrote: >> >> A much-more aggressive patch (which makes all RequestHandler sends >> Async except the 'last') shows could be 12x; but am concerned over >> the >> byte-counter side effects with which I am not familiar. > > [...] > - To get the correct byte counts. > This is really important, because it underlies the major load limiting > systems. However it is only important for terminal messages: we can > reasonably assume the rest will be piggybacked if necessary. > > In RequestHandler, most of the sendSync()'s are terminal, and > therefore must > remain. The one that is the most interesting is the one you > mentioned above: > sending the Accepted message. Well... I found the time to get the "more aggressive" patch working. The general idea is that (after the final request) because the ~only~ reason to keep the thread waiting on the final transmission is the byte count, to delegate that to the 'sent()' callback thread. On my machine (which was showing pre-emptive reject cause as ">threadLimit", because of the high bandwidth ceiling I have set), I saw a increase in node bandwidth (in & out), and saw a new pre-emptive rejection cause "insufficient output bandwidth". Realizing then that the bandwidth ceiling (being so high) was the cause for my node behaving differently than others, I changed my config to less-than that observed value, and "output bandwidth liability" became predominate (as Matthew said), and re-ran the tests with the more- expected results: For my machine, it appears to save 10 to 49 threads (the JVM-supplied value), varying largely presumably due to variances in the length of send queues. Also, I've seen quite high numbers for 'pooled threads awaiting work'. After cursory inspection, I suspect that the other related handlers (Insert/SSK) would not see the same benefit as RequestHandler; and are not modified. It may not therefore show a significant change to the common user (as the ack appears to have), but (at least for me, i.e. ">threadLimit" reject) it appears to give the user the ability to use more bandwidth, or perhaps to have longer send-queues (service slower peers). Question: Is it a bug that messages back to the source are not counted as the liability for handling a request, or is this intentional? (e.g. sendSync(dnf, null); ) Question: Should there be a note in the config page about bandwidth limits effect on threads? It seems the more bandwidth you allocate, the more requests you serve, the more threads freenet uses. Commited in r16372 -- Robert Hailey From m.rogers at cs.ucl.ac.uk Thu Dec 6 22:03:32 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Thu, 06 Dec 2007 22:03:32 +0000 Subject: [freenet-dev] Short refs was Re: alternative to #freenet-refs In-Reply-To: <4757A87D.8040201@WhenGendarmeSleeps.org> References: <20071107220045.GA4554@freenetproject.org> <1195995849.10868.24.camel@localhost> <474B09FD.7010906@cs.ucl.ac.uk> <200711271144.47979.toad@amphibian.dyndns.org> <474DD412.307@cs.ucl.ac.uk> <474EBFC4.7030201@WhenGendarmeSleeps.org> <4754508B.7000606@cs.ucl.ac.uk> <4757A87D.8040201@WhenGendarmeSleeps.org> Message-ID: <475871B4.1070705@cs.ucl.ac.uk> Volodya wrote: > Have you tried to give it enough words to represent a full key? Yup, it can represent any amount of data as poetry without using any extra memory - try feeding it a very large command line argument. Of course, at the moment it's just representing random data. Where it currently calls rand() to generate array indices it should actually be reading data from stdin, and there should be a companion program that reads poetry from stdin, uses a hashtable to recover the array indices of the words, and prints the original data on stdout. I've done something similar in C before so I know it works in principle, I just need to get round to finishing this version. Cheers, Michael From toad at amphibian.dyndns.org Fri Dec 7 00:00:28 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 7 Dec 2007 00:00:28 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / r16372 In-Reply-To: <15D4A571-25AD-48C4-A12D-8E79E66E2F12@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712020002.10322.toad@amphibian.dyndns.org> <15D4A571-25AD-48C4-A12D-8E79E66E2F12@osndok.com> Message-ID: <200712070000.34903.toad@amphibian.dyndns.org> On Thursday 06 December 2007 20:16, Robert Hailey wrote: > > On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > > > On Friday 30 November 2007 23:03, Robert Hailey wrote: > >> > >> A much-more aggressive patch (which makes all RequestHandler sends > >> Async except the 'last') shows could be 12x; but am concerned over > >> the > >> byte-counter side effects with which I am not familiar. > > > > [...] > > - To get the correct byte counts. > > This is really important, because it underlies the major load limiting > > systems. However it is only important for terminal messages: we can > > reasonably assume the rest will be piggybacked if necessary. > > > > In RequestHandler, most of the sendSync()'s are terminal, and > > therefore must > > remain. The one that is the most interesting is the one you > > mentioned above: > > sending the Accepted message. > > > Well... I found the time to get the "more aggressive" patch working. > > The general idea is that (after the final request) because the ~only~ > reason to keep the thread waiting on the final transmission is the > byte count, to delegate that to the 'sent()' callback thread. Makes sense. > > On my machine (which was showing pre-emptive reject cause as > ">threadLimit", because of the high bandwidth ceiling I have set), I > saw a increase in node bandwidth (in & out), and saw a new pre-emptive > rejection cause "insufficient output bandwidth". Realizing then that > the bandwidth ceiling (being so high) was the cause for my node > behaving differently than others, I changed my config to less-than > that observed value, and "output bandwidth liability" became > predominate (as Matthew said), and re-ran the tests with the more- > expected results: > > For my machine, it appears to save 10 to 49 threads (the JVM-supplied > value), varying largely presumably due to variances in the length of > send queues. Also, I've seen quite high numbers for 'pooled threads > awaiting work'. Right, no real network-level effects, unless lots of nodes are bottlenecked on threads, but a good local optimisation for thread-scarce nodes. > > After cursory inspection, I suspect that the other related handlers > (Insert/SSK) would not see the same benefit as RequestHandler; and are > not modified. Well they generally take longer, and use more threads (especially CHK inserts). > > It may not therefore show a significant change to the common user (as > the ack appears to have), but (at least for me, i.e. ">threadLimit" > reject) it appears to give the user the ability to use more bandwidth, > or perhaps to have longer send-queues (service slower peers). > > Question: Is it a bug that messages back to the source are not counted > as the liability for handling a request, or is this intentional? (e.g. > sendSync(dnf, null); ) Probably. If it was intentional there should be a comment explaining it. > > Question: Should there be a note in the config page about bandwidth > limits effect on threads? It seems the more bandwidth you allocate, > the more requests you serve, the more threads freenet uses. Isn't that obvious? The more load the node is under, the more resources it's likely to use? > > Commited in r16372 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/23261e60/attachment.pgp From jolantern60 at yahoo.com Fri Dec 7 00:27:47 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Thu, 6 Dec 2007 16:27:47 -0800 (PST) Subject: [freenet-dev] 0.5 unstable build.xml Message-ID: <159763.52877.qm@web44812.mail.sp1.yahoo.com> Hi, I've applied the Diffie-Hellman patch to unstable now. Could Nextgens or a different site admin apply the same patch to the unstable build.xml that was applied to the stable one in revs 16002/16003/16113, please? Thank you! Jack ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ From toad at amphibian.dyndns.org Fri Dec 7 13:09:22 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 7 Dec 2007 13:09:22 +0000 Subject: [freenet-dev] 0.5 unstable build.xml In-Reply-To: <159763.52877.qm@web44812.mail.sp1.yahoo.com> References: <159763.52877.qm@web44812.mail.sp1.yahoo.com> Message-ID: <200712071309.29170.toad@amphibian.dyndns.org> On Friday 07 December 2007 00:27, Jack O'Lantern wrote: > Hi, > > I've applied the Diffie-Hellman patch to unstable now. Could Nextgens > or a different site admin apply the same patch to the unstable > build.xml that was applied to the stable one in revs 16002/16003/16113, > please? Thank you! Are you planning on maintaining the unstable branch (0.6) ? If so, why? We deprecated it, is there any good reason to revive it? > > Jack -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/3619feef/attachment.pgp From toad at amphibian.dyndns.org Fri Dec 7 18:34:10 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 7 Dec 2007 18:34:10 +0000 Subject: [freenet-dev] [freenet-cvs] r16372 - trunk/freenet/src/freenet/node In-Reply-To: <20071206202408.D8A18480F1F@freenetproject.org> References: <20071206202408.D8A18480F1F@freenetproject.org> Message-ID: <200712071834.15332.toad@amphibian.dyndns.org> Some issues.. On Thursday 06 December 2007 20:24, you wrote: > Author: robert > Date: 2007-12-06 20:24:08 +0000 (Thu, 06 Dec 2007) > New Revision: 16372 > > Modified: > trunk/freenet/src/freenet/node/MessageItem.java > trunk/freenet/src/freenet/node/RequestHandler.java > Log: > Don't hang onto a RequestHandler thread just to track the bytes. > > @@ -144,11 +148,14 @@ > new BlockTransmitter(node.usm, source, uid, prb, node.outputThrottle, this); > node.addTransferringRequestHandler(uid); > if(bt.send(node.executor)) { > - status = RequestSender.SUCCESS; // for byte logging > + // for byte logging > + status = RequestSender.SUCCESS; > // We've fetched it from our datastore, so there won't be a downstream noderef. > // But we want to send at least an FNPOpennetCompletedAck, otherwise the request source > // may have to timeout waiting for one. > finishOpennetNoRelay(); > + //also for byte logging, since the block is the 'terminal' message. > + applyByteCounts(); Careful here! IMHO we need to include the opennet messages in the bytes total for the request. finishOpennetNoReply() may sendAsync a FNPOpennetCompletedAck or an actual noderef... > @@ -172,13 +179,14 @@ > if((waitStatus & RequestSender.WAIT_REJECTED_OVERLOAD) != 0) { > // Forward RejectedOverload > Message msg = DMT.createFNPRejectedOverload(uid, false); > - source.sendAsync(msg, null, 0, null); > + source.sendAsync(msg, null, 0, this); > } > > if((waitStatus & RequestSender.WAIT_TRANSFERRING_DATA) != 0) { > // Is a CHK. > Message df = DMT.createFNPCHKDataFound(uid, rs.getHeaders()); > - source.sendSync(df, null); > + source.sendAsync(df, null, 0, this); > + > PartiallyReceivedBlock prb = rs.getPRB(); > BlockTransmitter bt = > new BlockTransmitter(node.usm, source, uid, prb, node.outputThrottle, this); > @@ -190,6 +198,8 @@ > finishOpennetChecked(); > } > status = rs.getStatus(); > + //for byte logging, since the block is the 'terminal' message. > + applyByteCounts(); This is fine btw, we wait for an explicit acknowledgement so we know the blocks have been sent. > + private boolean once=true; > + public void sent() { > + //For byte counting, this relies on the fact that the callback will only be excuted once. This check might be paranoid. > + if (once) { > + applyByteCounts(); > + } else { > + Logger.error(this, "terminalMessage sent multiple times? for " + RequestHandler.this); > + } > + } It's a sensible check but afaics you haven't implemented it - where is once set to false?! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/5bd06be6/attachment.pgp From jolantern60 at yahoo.com Fri Dec 7 18:51:46 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Fri, 7 Dec 2007 10:51:46 -0800 (PST) Subject: [freenet-dev] 0.5 unstable build.xml In-Reply-To: <200712071309.29170.toad@amphibian.dyndns.org> Message-ID: <293123.20280.qm@web44816.mail.sp1.yahoo.com> --- Matthew Toseland wrote: > Are you planning on maintaining the unstable branch (0.6) ? If so, > why? We deprecated it, is there any good reason to revive it? The differences between stable and unstable are very small (try diff -ruwB -x .svn stable/ unstable/, that returns only few dozen K of differences, about half of which are unimportant). The thing is that every commit to stable triggers the auto-builder and on the official download page you get something which claims to be 5108 but is really halfway through to 5109. So I thought I'd use unstable for regular committing and stable only when a new build is due. I'm aware that this is not the most efficient way to handle this problem and I'd be glad if you would be willing to offer me a better solution. Jack ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping From freenet-devl at osndok.com Fri Dec 7 21:06:47 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Fri, 7 Dec 2007 15:06:47 -0600 Subject: [freenet-dev] [freenet-cvs] r16372 - trunk/freenet/src/freenet/node In-Reply-To: <200712071834.15332.toad@amphibian.dyndns.org> References: <20071206202408.D8A18480F1F@freenetproject.org> <200712071834.15332.toad@amphibian.dyndns.org> Message-ID: On Dec 7, 2007, at 12:34 PM, Matthew Toseland wrote: > Some issues.. > > Careful here! IMHO we need to include the opennet messages in the > bytes total > for the request. finishOpennetNoReply() may sendAsync a > FNPOpennetCompletedAck or an actual noderef... Ah, yes. I changed the ack in both open net finish..() to sendTerminal(), and moved applyByteCounts() to the error condition (when no ack is sent). > It's a sensible check but afaics you haven't implemented it - where > is once > set to false?! Quite right. Both fixed in r16398. -- Robert Hailey From nextgens at freenetproject.org Fri Dec 7 22:32:10 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Fri, 7 Dec 2007 23:32:10 +0100 Subject: [freenet-dev] 0.5 unstable build.xml In-Reply-To: <293123.20280.qm@web44816.mail.sp1.yahoo.com> References: <200712071309.29170.toad@amphibian.dyndns.org> <293123.20280.qm@web44816.mail.sp1.yahoo.com> Message-ID: <20071207223210.GA3886@freenetproject.org> * Jack O'Lantern [2007-12-07 10:51:46]: > --- Matthew Toseland wrote: > > Are you planning on maintaining the unstable branch (0.6) ? If so, > > why? We deprecated it, is there any good reason to revive it? > > The differences between stable and unstable are very small (try diff > -ruwB -x .svn stable/ unstable/, that returns only few dozen K of > differences, about half of which are unimportant). That's a good reason to focus on .5 ;) > The thing is that > every commit to stable triggers the auto-builder and on the official > download page you get something which claims to be 5108 but is really > halfway through to 5109. I'm not convinced that back-porting my scripts to the .5 tree (auto-release gets triggered only when me or toad change the version string in Version.java) is worth it... So we will KISS: - Forget about .6 - I've disabled the auto-builder so that you can commit/do whatever needs to be done without breaking everyone's node - You will have to ping me (possibly off-list as it doesn't concern the *current* version of freenet) when you want a build to be released. > So I thought I'd use unstable for regular > committing and stable only when a new build is due. I'm aware that this > is not the most efficient way to handle this problem and I'd be glad if > you would be willing to offer me a better solution. > Please don't assume things in the future, just tell me what you want to do and I'll do what is needed server-side :) NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/8b36201b/attachment.pgp From freenet-devl at osndok.com Fri Dec 7 22:47:59 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Fri, 7 Dec 2007 16:47:59 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200711300740.12146.edt@aei.ca> References: <200711300126.45810.toad@amphibian.dyndns.org> <20071130085432.GD4365@freenetproject.org> <200711301121.58041.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> Message-ID: <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > Hi, > > I think freenet is slow because its having problems finding data. > The hit rates in 0.5 ranged around 5%. > in 0.7 I see 2% on cache and 0% on store. We just are not finding > data. My location shifts so fast that > other nodes have no idea what is in my node's 8G store. I strongly > suggest that we experiment with letting > the store contents limit how far from the current location we can > swap. This could easily be done with > a tunable such that the effect could be turned off or could be made > very strict. We could even use > a feedback loop to find an optimal value with the store/cache hit > rate being used to control just how strict > we are. The idea would be to minimize the swapping limitation while > maximizing the hit rate which > would decrease the path length and hence the traffic. > > In my case the node usually uses a good chunk (80-90%) of the > bandwidth I give to it. I think the > bandwidth limits work. I would agree that backout is probably not > needed and think it would be a > good idea to try with it turn off - again this would improve routing > which I think is the core of 0.7 > performance problems. > > Thanks > Ed I have been musing on Ed's post for a while. Along with my understandings: * The contents of the datastore(s) are not given weight when considering a swap request. * For a particular CHK/SSK insert, it finds one home node to be put into the store (the rest in only cached). * If the node targeted by the insert 'drifts' sufficiently far from it's origin ** that data in the store be unfindable by the network, and ** is all-but guaranteed to be LOST forever, is it can only survive in the networks caches (if it is un-popular). Consider the following patch (completed to the furthest point that my current freenet understand can take it). - If a CHKBlock is evicted from the datastore (not cache), AND we are not (as best we can tell) "close" to where it should be, THEN re- insert that block. This approximates the decision (is this eviction OLD/UNPOPULAR or UNFINDABLE), and brings with it a rather odd perk for having small datastores (that they would find these misplaced blocks sooner). The major risk, of course, is renewing old data. I do not have the means to test this on any sizable network. Would this be a candidate for 'testnet' or simulations of which I hear? Also, I could not figure out how to revive a SSKBlock from the datastore (or find out if it is even possible). -- Robert Hailey -------------- next part -------------- A non-text attachment was scrubbed... Name: freenet-reinsert-evicts.diff Type: application/octet-stream Size: 10864 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071207/347a1a78/attachment.obj -------------- next part -------------- From juiceman69 at gmail.com Fri Dec 7 23:05:48 2007 From: juiceman69 at gmail.com (Juiceman) Date: Fri, 7 Dec 2007 18:05:48 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <20071130085432.GD4365@freenetproject.org> <200711301121.58041.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> Message-ID: <8b525dee0712071505n74b1ecb5le9c2af934b04cb36@mail.gmail.com> On Dec 7, 2007 5:47 PM, Robert Hailey wrote: > > On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > > > Hi, > > > > I think freenet is slow because its having problems finding data. > > The hit rates in 0.5 ranged around 5%. > > in 0.7 I see 2% on cache and 0% on store. We just are not finding > > data. My location shifts so fast that > > other nodes have no idea what is in my node's 8G store. I strongly > > suggest that we experiment with letting > > the store contents limit how far from the current location we can > > swap. This could easily be done with > > a tunable such that the effect could be turned off or could be made > > very strict. We could even use > > a feedback loop to find an optimal value with the store/cache hit > > rate being used to control just how strict > > we are. The idea would be to minimize the swapping limitation while > > maximizing the hit rate which > > would decrease the path length and hence the traffic. > > > > In my case the node usually uses a good chunk (80-90%) of the > > bandwidth I give to it. I think the > > bandwidth limits work. I would agree that backout is probably not > > needed and think it would be a > > good idea to try with it turn off - again this would improve routing > > which I think is the core of 0.7 > > performance problems. > > > > Thanks > > Ed > > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible). > > -- > Robert Hailey > Do we wait at all for the node to be up and connected to its usual peers? It looks like we start swapping as soon as we get one peer connected. This IMHO is bad because we could accept a swap to a completely different specialization based on only 1-2 connected peers. If so, I would suggest waiting for 5-10 minutes of uptime to allow our usual connections to be reestablished and thus hopefully keep us near our usual specialization. -- I may disagree with what you have to say, but I shall defend, to the death, your right to say it. - Voltaire Those who would give up Liberty, to purchase temporary Safety, deserve neither Liberty nor Safety. - Ben Franklin From edt at aei.ca Fri Dec 7 23:24:06 2007 From: edt at aei.ca (Ed Tomlinson) Date: Fri, 7 Dec 2007 18:24:06 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> Message-ID: <200712071824.07180.edt@aei.ca> On December 7, 2007, Robert Hailey wrote: > > On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > > > Hi, > > > > I think freenet is slow because its having problems finding data. > > The hit rates in 0.5 ranged around 5%. > > in 0.7 I see 2% on cache and 0% on store. We just are not finding > > data. My location shifts so fast that > > other nodes have no idea what is in my node's 8G store. I strongly > > suggest that we experiment with letting > > the store contents limit how far from the current location we can > > swap. This could easily be done with > > a tunable such that the effect could be turned off or could be made > > very strict. We could even use > > a feedback loop to find an optimal value with the store/cache hit > > rate being used to control just how strict > > we are. The idea would be to minimize the swapping limitation while > > maximizing the hit rate which > > would decrease the path length and hence the traffic. > > > > In my case the node usually uses a good chunk (80-90%) of the > > bandwidth I give to it. I think the > > bandwidth limits work. I would agree that backout is probably not > > needed and think it would be a > > good idea to try with it turn off - again this would improve routing > > which I think is the core of 0.7 > > performance problems. > > > > Thanks > > Ed > > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible). This is an interesting way to tackle the problem. Might I suggest that instead of doing this for data that we are about to discard we do it select a key from the DS at random and reinsert if its far from our current location. One thing that may be happening that somewant offsets this problem could be clustering of locations. e.g. even though my nodes location changes often it could be that it favours some subsets of the total location space. It would be worthing adding a metric to track how long a node spends in a given area of the location space. Ed From freenet-devl at osndok.com Fri Dec 7 23:45:24 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Fri, 7 Dec 2007 17:45:24 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712071824.07180.edt@aei.ca> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <200712071824.07180.edt@aei.ca> Message-ID: <66C9C89D-D06F-4213-AB11-C67458A2A77A@osndok.com> On Dec 7, 2007, at 5:24 PM, Ed Tomlinson wrote: > On December 7, 2007, Robert Hailey wrote: >> [...] >> Consider the following patch (completed to the furthest point that my >> current freenet understand can take it). >> - If a CHKBlock is evicted from the datastore (not cache), AND we are >> not (as best we can tell) "close" to where it should be, THEN re- >> insert that block. >> >> This approximates the decision (is this eviction OLD/UNPOPULAR or >> UNFINDABLE), and brings with it a rather odd perk for having small >> datastores (that they would find these misplaced blocks sooner). >> >> The major risk, of course, is renewing old data. >> [...] > > This is an interesting way to tackle the problem. Might I suggest > that instead of > doing this for data that we are about to discard we do it select a > key from the DS > at random and reinsert if its far from our current location. That may be a good strategy for shrinking the datastore, to slowly reinsert it into the network. But the problem is one of finding the initiative, such a task would yield itself to occurring when we change locations, which (if that is the case; that we have a store full of stuff for a different location) means we probably shouldn't be swapping locations to begin with (your first point). > One thing that may be happening that somewant offsets this problem > could be > clustering of locations. e.g. even though my nodes location changes > often it could > be that it favours some subsets of the total location space. It > would be worthing > adding a metric to track how long a node spends in a given area of > the location > space. Hmm... that may be implementable as a running average of (location over time). i.e. every fixed interval (hour? / 5-minute) average one 'location' into the running average. Which amounts to a location- shifting 'velocity' measurement, or 'average-location'. -- Robert Hailey From edt at aei.ca Sat Dec 8 01:38:55 2007 From: edt at aei.ca (Ed Tomlinson) Date: Fri, 7 Dec 2007 20:38:55 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <66C9C89D-D06F-4213-AB11-C67458A2A77A@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712071824.07180.edt@aei.ca> <66C9C89D-D06F-4213-AB11-C67458A2A77A@osndok.com> Message-ID: <200712072038.55935.edt@aei.ca> On December 7, 2007, Robert Hailey wrote: > > One thing that may be happening that somewant offsets this problem ? > > could be > > clustering of locations. ?e.g. even though my nodes location changes ? > > often it could > > be that it favours some subsets of the total location space. ?It ? > > would be worthing > > adding a metric to track how long a node spends in a given area of ? > > the location > > space. > > Hmm... that may be implementable as a running average of (location ? > over time). i.e. every fixed interval (hour? / 5-minute) average one ? > 'location' into the running average. Which amounts to a location- > shifting 'velocity' measurement, or 'average-location'. > I was thinking more along the lines of a histogram like 0.5 used to use to show specialization. Ed From toad at amphibian.dyndns.org Sat Dec 8 12:48:00 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 12:48:00 +0000 Subject: [freenet-dev] [freenet-cvs] r16388 - trunk/freenet/src/freenet/client/async In-Reply-To: <20071207021337.0937C479706@freenetproject.org> References: <20071207021337.0937C479706@freenetproject.org> Message-ID: <200712081248.05902.toad@amphibian.dyndns.org> On Friday 07 December 2007 02:13, you wrote: > Author: robert > Date: 2007-12-07 02:13:36 +0000 (Fri, 07 Dec 2007) > New Revision: 16388 > > Modified: > trunk/freenet/src/freenet/client/async/USKFetcher.java > Log: > Use one thread for all delayed USK checks With regards to Timer: Why not just use the Ticker ? Admittedly they will sometimes get scheduled in parallel, but iirc it doesn't do that much work on-thread so it's not really a problem... > > Modified: trunk/freenet/src/freenet/client/async/USKFetcher.java > =================================================================== > --- trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 01:41:35 UTC (rev 16387) > +++ trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 02:13:36 UTC (rev 16388) > @@ -252,43 +255,28 @@ > long now = System.currentTimeMillis(); > synchronized(this) { > started = false; // don't finish before have rescheduled > + > + //Find out when we should check next ('end'), in an increasing delay (unless we make progress). > + long newSleepTime = sleepTime * 2; > + if(newSleepTime > maxSleepTime) newSleepTime = maxSleepTime; > + sleepTime = newSleepTime; > + end = now + sleepTime; > + > if(valAtEnd > valueAtSchedule) { > - // Have advanced. > + // We have advanced; keep trying as if we just started. > minFailures = origMinFailures; > sleepTime = origSleepTime; > + end = now; end = now + origSleepTime. What you have there makes it retry *immediately*. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/25b678a7/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 8 13:05:11 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 13:05:11 +0000 Subject: [freenet-dev] [freenet-cvs] r16398 - trunk/freenet/src/freenet/node In-Reply-To: <20071207211120.CF3C44796AA@freenetproject.org> References: <20071207211120.CF3C44796AA@freenetproject.org> Message-ID: <200712081305.18305.toad@amphibian.dyndns.org> IMHO the (once) check should in fact be in applyByteCounts(). On Friday 07 December 2007 21:11, you wrote: > Author: robert > Date: 2007-12-07 21:11:20 +0000 (Fri, 07 Dec 2007) > New Revision: 16398 > > Modified: > trunk/freenet/src/freenet/node/RequestHandler.java > Log: > fixup opennet byte counts and (once) check (errors from r16372) > > > Modified: trunk/freenet/src/freenet/node/RequestHandler.java > =================================================================== > --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-07 18:52:29 UTC (rev 16397) > +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-07 21:11:20 UTC (rev 16398) > @@ -152,8 +152,9 @@ > status = RequestSender.SUCCESS; > // We've fetched it from our datastore, so there won't be a downstream noderef. > // But we want to send at least an FNPOpennetCompletedAck, otherwise the request source > - // may have to timeout waiting for one. > + // may have to timeout waiting for one. That will be the terminal message. > finishOpennetNoRelay(); > + } else { > //also for byte logging, since the block is the 'terminal' message. > applyByteCounts(); > } > @@ -191,15 +192,16 @@ > BlockTransmitter bt = > new BlockTransmitter(node.usm, source, uid, prb, node.outputThrottle, this); > node.addTransferringRequestHandler(uid); > - if(!bt.send(node.executor)){ > - finalTransferFailed = true; > - } else { > + if(bt.send(node.executor)) { > + status = rs.getStatus(); > // Successful CHK transfer, maybe path fold > finishOpennetChecked(); > - } > - status = rs.getStatus(); > - //for byte logging, since the block is the 'terminal' message. > - applyByteCounts(); > + } else { > + finalTransferFailed = true; > + status = rs.getStatus(); > + //for byte logging, since the block is the 'terminal' message. > + applyByteCounts(); > + } > return; > } > > @@ -309,6 +311,7 @@ > //For byte counting, this relies on the fact that the callback will only be excuted once. This check might be paranoid. > if (once) { > applyByteCounts(); > + once=false; > } else { > Logger.error(this, "terminalMessage sent multiple times? for " + RequestHandler.this); > } > @@ -320,39 +323,35 @@ > * or wait for a noderef and relay it and wait for a response and relay that, > * or send our own noderef and wait for a response and add that. > */ > - private void finishOpennetChecked() { > + private void finishOpennetChecked() throws NotConnectedException { > OpennetManager om = node.getOpennet(); > if(om != null && > (node.passOpennetRefsThroughDarknet() || source.isOpennet()) && > - finishOpennetInner(om)) > + finishOpennetInner(om)) { > + applyByteCounts(); > return; > + } > > Message msg = DMT.createFNPOpennetCompletedAck(uid); > - try { > - source.sendAsync(msg, null, 0, this); > - } catch (NotConnectedException e) { > - // Oh well... > - } > + sendTerminal(msg); > } > > /** > * There is no noderef to pass downstream. If we want a connection, send our > * noderef and wait for a reply, otherwise just send an ack. > */ > - private void finishOpennetNoRelay() { > + private void finishOpennetNoRelay() throws NotConnectedException { > OpennetManager om = node.getOpennet(); > > if(om != null && (source.isOpennet() || node.passOpennetRefsThroughDarknet()) && > - finishOpennetNoRelayInner(om)) > + finishOpennetNoRelayInner(om)) { > + applyByteCounts(); > return; > + } > > // Otherwise just ack it. > Message msg = DMT.createFNPOpennetCompletedAck(uid); > - try { > - source.sendAsync(msg, null, 0, this); > - } catch (NotConnectedException e) { > - // Oh well... > - } > + sendTerminal(msg); > } > > private boolean finishOpennetInner(OpennetManager om) { > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/ab59fa3e/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 8 13:14:58 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 13:14:58 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> Message-ID: <200712081314.58744.toad@amphibian.dyndns.org> On Friday 07 December 2007 22:47, Robert Hailey wrote: > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. Indeed. This is entirely deliberate. We have no simulations of what would happen if we did bias by datastore contents, and we are not going to implement unsimulated changes to core routing, certainly not to swapping. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). Not one, average of 3. The nodes which are the closest-so-far to the target. There will typically be 2 or 3. > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). Not true. We have mechanisms to get popular data back into the datastore. The main one is 1 in every 100 successful requests gets turned into an insert post-completion. > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. This is going to generate an insane amount of traffic, and sabotage the natural decay of data that happens to be in the wrong place. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible). An SSK requires the pubkey from one datastore and the actual SSK from another. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/10b6fe4b/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 8 13:18:17 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 13:18:17 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <8b525dee0712071505n74b1ecb5le9c2af934b04cb36@mail.gmail.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <8b525dee0712071505n74b1ecb5le9c2af934b04cb36@mail.gmail.com> Message-ID: <200712081318.18019.toad@amphibian.dyndns.org> On Friday 07 December 2007 23:05, Juiceman wrote: > > Do we wait at all for the node to be up and connected to its usual > peers? It looks like we start swapping as soon as we get one peer > connected. This IMHO is bad because we could accept a swap to a > completely different specialization based on only 1-2 connected peers. > > If so, I would suggest waiting for 5-10 minutes of uptime to allow our > usual connections to be reestablished and thus hopefully keep us near > our usual specialization. Done. 1 minute delay to let our nodes connect, before we startup outgoing swap requests. However we are likely to swap back reasonably quickly if we do mess up for this reason. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/2d8808cf/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 8 13:19:17 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 13:19:17 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712071824.07180.edt@aei.ca> References: <200711300126.45810.toad@amphibian.dyndns.org> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <200712071824.07180.edt@aei.ca> Message-ID: <200712081319.17377.toad@amphibian.dyndns.org> On Friday 07 December 2007 23:24, Ed Tomlinson wrote: > This is an interesting way to tackle the problem. Might I suggest that instead of > doing this for data that we are about to discard we do it select a key from the DS > at random and reinsert if its far from our current location. > > One thing that may be happening that somewant offsets this problem could be > clustering of locations. e.g. even though my nodes location changes often it could > be that it favours some subsets of the total location space. It would be worthing > adding a metric to track how long a node spends in a given area of the location > space. Agreed, a simple record of our last 100 locations and how long (uptime) we spent in each would be interesting. > > Ed -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/f07618a7/attachment.pgp From edt at aei.ca Sat Dec 8 14:17:39 2007 From: edt at aei.ca (Ed Tomlinson) Date: Sat, 8 Dec 2007 09:17:39 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / r16372 In-Reply-To: <15D4A571-25AD-48C4-A12D-8E79E66E2F12@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712020002.10322.toad@amphibian.dyndns.org> <15D4A571-25AD-48C4-A12D-8E79E66E2F12@osndok.com> Message-ID: <200712080917.39599.edt@aei.ca> Hi, Some feedback on this. Before updating to testing I was seeing about 10% of my Preemptive Rejection Reasons attributed to freeHeapPercentThreshold now its: 101639 Output bandwidth liability 1216 > On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > > > On Friday 30 November 2007 23:03, Robert Hailey wrote: > >> > >> A much-more aggressive patch (which makes all RequestHandler sends > >> Async except the 'last') shows could be 12x; but am concerned over > >> the > >> byte-counter side effects with which I am not familiar. > > > > [...] > > - To get the correct byte counts. > > This is really important, because it underlies the major load limiting > > systems. However it is only important for terminal messages: we can > > reasonably assume the rest will be piggybacked if necessary. > > > > In RequestHandler, most of the sendSync()'s are terminal, and > > therefore must > > remain. The one that is the most interesting is the one you > > mentioned above: > > sending the Accepted message. > > > Well... I found the time to get the "more aggressive" patch working. > > The general idea is that (after the final request) because the ~only~ > reason to keep the thread waiting on the final transmission is the > byte count, to delegate that to the 'sent()' callback thread. > > On my machine (which was showing pre-emptive reject cause as > ">threadLimit", because of the high bandwidth ceiling I have set), I > saw a increase in node bandwidth (in & out), and saw a new pre-emptive > rejection cause "insufficient output bandwidth". Realizing then that > the bandwidth ceiling (being so high) was the cause for my node > behaving differently than others, I changed my config to less-than > that observed value, and "output bandwidth liability" became > predominate (as Matthew said), and re-ran the tests with the more- > expected results: > > For my machine, it appears to save 10 to 49 threads (the JVM-supplied > value), varying largely presumably due to variances in the length of > send queues. Also, I've seen quite high numbers for 'pooled threads > awaiting work'. > > After cursory inspection, I suspect that the other related handlers > (Insert/SSK) would not see the same benefit as RequestHandler; and are > not modified. > > It may not therefore show a significant change to the common user (as > the ack appears to have), but (at least for me, i.e. ">threadLimit" > reject) it appears to give the user the ability to use more bandwidth, > or perhaps to have longer send-queues (service slower peers). > > Question: Is it a bug that messages back to the source are not counted > as the liability for handling a request, or is this intentional? (e.g. > sendSync(dnf, null); ) > > Question: Should there be a note in the config page about bandwidth > limits effect on threads? It seems the more bandwidth you allocate, > the more requests you serve, the more threads freenet uses. > > Commited in r16372 > > -- > Robert Hailey > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > From bbackde at googlemail.com Sat Dec 8 14:29:01 2007 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sat, 8 Dec 2007 15:29:01 +0100 Subject: [freenet-dev] Store size not used Message-ID: I run freenet 0.7 for a very long time. My datastore size is 200GB, and my cache is 100% full. But the storage is still only at 10% of its maximum size, and it doesn't grow. There was some discussion about this in the past, but I see no advance. I know this has something to do with specialization, but does this mean that my node specialized to something that is not inserted? It would be nice if the freenet could use the remaining 90GB that I offer. My stats: # Cached keys: 3,222,330 (98.3 GiB) (100%) # Stored keys: 346,172 (10.5 GiB) (10%) # Overall size: 3,568,502 / 6,444,660 (108 GiB / 196 GiB) (55%) -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From toad at amphibian.dyndns.org Sat Dec 8 16:52:37 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 16:52:37 +0000 Subject: [freenet-dev] Major weakness in our current link-level congestion control Message-ID: <200712081652.42068.toad@amphibian.dyndns.org> A user on Frost found despite the recent fixes that he still got a problem resembling the high-bandwidth-usage-way-over-the-limit-no-payload bug. He had the problem only when a specific peer was connected. He sent me logs. I have analysed the logs and it appears that: - While the global bandwidth limit is the dominant factor, the AIMD maximum window increases indefinitely. - Once the external bandwidth pressure (from other peers) abates, we are sending packets at a much faster rate than the link can handle. - If the link then goes down, or has gaps, e.g. because it's over wifi, or severe latency, we may have major problems: we continue to send packets far too quickly, and when they are not acknowledged, although we will slow down the addition of new packets to send, we may have a lot of packets queued by then, and we will have to retransmit them. Long term causes of this are the fact that our link layer code is not all that TCP-like and needs to be rewritten; it does congestion avoidance at a higher level than it needs to, for example, the congestion window is done on a flow rates basis rather than an actual window... The interaction between the bandwidth limiter and the congestion avoidance algorithm is however interesting in itself. For TCP, bandwidth limiting can be done externally (by dropping packets that go over the limit), or internally (by limiting the flow of data before it reaches TCP). It is not clear that TCP (RFC2581) adequately handles the latter case. There is a note in section 4.1 to the effect that if a connection is idle for a long period and then restarted you can get a burst (so reset to slow-start if the connection has been idle for a while), but it appears to me that the same is possible if you have a trickle of data for a long period and then a burst: TCP will send it at something approaching the maximum possible speed, and get into trouble. IMHO the solution for us is to not increase the congestion window size unless we have actually had a full window in flight recently. I'm not sure how this would fit into the NewTransportLayer... The short term solution appears to be the following: - Have an explicit window for each PacketThrottle: be able to count the number of packets in flight to a specific peer. - Enforce the window, rather than enforcing the rate. - Don't increase the window size (in congestion control) unless we have actually used the full window - had the full window in flight - within the last round trip time. The new transport layer would have an explicit window, and would do bandwidth limiting at a much lower level, including retransmits, have a much larger maximum retransmission window, and generally be much closer to TCP and work better on high latency connections - but without the last provision above, it could still run into this kind of problem. New transport layer: http://wiki.freenetproject.org/NewTransportLayer http://wiki.freenetproject.org/NewPacketFormat Thoughts? The short-term fix should be reasonably easy. The new transport layer otoh could take some time. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/eb0a017e/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 8 16:57:41 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 8 Dec 2007 16:57:41 +0000 Subject: [freenet-dev] Store size not used In-Reply-To: References: Message-ID: <200712081657.41575.toad@amphibian.dyndns.org> On Saturday 08 December 2007 14:29, bbackde at googlemail.com wrote: > I run freenet 0.7 for a very long time. My datastore size is 200GB, > and my cache is 100% full. > But the storage is still only at 10% of its maximum size, and it doesn't grow. What do you mean it doesn't grow? It should store more keys over time, even if it's only one key per day. > There was some discussion about this in the past, but I see no advance. > > I know this has something to do with specialization, but does this > mean that my node specialized > to something that is not inserted? It would be nice if the freenet > could use the remaining 90GB that > I offer. Auto-shrinking the cache as the store gets bigger requires an efficient and non-LRU-damaging way to do online shrink. It's possible but it's not a priority right now. > > My stats: > > # Cached keys: 3,222,330 (98.3 GiB) (100%) > # Stored keys: 346,172 (10.5 GiB) (10%) > # Overall size: 3,568,502 / 6,444,660 (108 GiB / 196 GiB) (55%) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/54a22480/attachment.pgp From bbackde at googlemail.com Sat Dec 8 17:10:33 2007 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sat, 8 Dec 2007 18:10:33 +0100 Subject: [freenet-dev] Store size not used In-Reply-To: <200712081657.41575.toad@amphibian.dyndns.org> References: <200712081657.41575.toad@amphibian.dyndns.org> Message-ID: I do not count each key, but its at 10% for months now. On Dec 8, 2007 5:57 PM, Matthew Toseland wrote: > On Saturday 08 December 2007 14:29, bbackde at googlemail.com wrote: > > I run freenet 0.7 for a very long time. My datastore size is 200GB, > > and my cache is 100% full. > > But the storage is still only at 10% of its maximum size, and it doesn't > grow. > > What do you mean it doesn't grow? It should store more keys over time, even if > it's only one key per day. > > > There was some discussion about this in the past, but I see no advance. > > > > I know this has something to do with specialization, but does this > > mean that my node specialized > > to something that is not inserted? It would be nice if the freenet > > could use the remaining 90GB that > > I offer. > > Auto-shrinking the cache as the store gets bigger requires an efficient and > non-LRU-damaging way to do online shrink. It's possible but it's not a > priority right now. > > > > > My stats: > > > > # Cached keys: 3,222,330 (98.3 GiB) (100%) > > # Stored keys: 346,172 (10.5 GiB) (10%) > > # Overall size: 3,568,502 / 6,444,660 (108 GiB / 196 GiB) (55%) > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From bbackde at googlemail.com Sat Dec 8 17:16:47 2007 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sat, 8 Dec 2007 18:16:47 +0100 Subject: [freenet-dev] Store size not used In-Reply-To: References: <200712081657.41575.toad@amphibian.dyndns.org> Message-ID: What I wonder about: after I run freenet for more than a year, why did it only collect 10GB in the store? May others have a larger usage? On Dec 8, 2007 6:10 PM, wrote: > I do not count each key, but its at 10% for months now. > > > On Dec 8, 2007 5:57 PM, Matthew Toseland wrote: > > On Saturday 08 December 2007 14:29, bbackde at googlemail.com wrote: > > > I run freenet 0.7 for a very long time. My datastore size is 200GB, > > > and my cache is 100% full. > > > But the storage is still only at 10% of its maximum size, and it doesn't > > grow. > > > > What do you mean it doesn't grow? It should store more keys over time, even if > > it's only one key per day. > > > > > There was some discussion about this in the past, but I see no advance. > > > > > > I know this has something to do with specialization, but does this > > > mean that my node specialized > > > to something that is not inserted? It would be nice if the freenet > > > could use the remaining 90GB that > > > I offer. > > > > Auto-shrinking the cache as the store gets bigger requires an efficient and > > non-LRU-damaging way to do online shrink. It's possible but it's not a > > priority right now. > > > > > > > > My stats: > > > > > > # Cached keys: 3,222,330 (98.3 GiB) (100%) > > > # Stored keys: 346,172 (10.5 GiB) (10%) > > > # Overall size: 3,568,502 / 6,444,660 (108 GiB / 196 GiB) (55%) > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > > > > -- > __________________________________________________ > GnuPG key: (0x48DBFA8A) > Keyserver: pgpkeys.pca.dfn.de > Fingerprint: > 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A > __________________________________________________ > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From batosai at batosai.net Sat Dec 8 17:40:06 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Sat, 08 Dec 2007 18:40:06 +0100 Subject: [freenet-dev] Store size not used In-Reply-To: References: <200712081657.41575.toad@amphibian.dyndns.org> Message-ID: <475AD6F6.8040909@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 bbackde at googlemail.com a ?crit : > What I wonder about: after I run freenet for more than a year, why did > it only collect 10GB in the store? May others have a larger usage? Yep, here is what I've got : # Cached keys: 3 283 708 (100 GiB) (100%) # Stored keys: 714 862 (21.8 GiB) (21%) It grows very slowly, but it does... -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHWtb2o6N05NzaOvURAgdPAJ49LOsB1u9NG0PLPw0fRV9VivynCwCfafdo sBgpOuyDV1gxTIKoZ7l0k/w= =WjvC -----END PGP SIGNATURE----- From nextgens at freenetproject.org Sat Dec 8 21:48:35 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sat, 8 Dec 2007 22:48:35 +0100 Subject: [freenet-dev] [freenet-cvs] r16403 - trunk/freenet/src/freenet/io In-Reply-To: <20071208171556.BD54E479758@freenetproject.org> References: <20071208171556.BD54E479758@freenetproject.org> Message-ID: <20071208214835.GA4559@freenetproject.org> * toad at freenetproject.org [2007-12-08 17:15:56]: > Author: toad > Date: 2007-12-08 17:15:56 +0000 (Sat, 08 Dec 2007) > New Revision: 16403 > > Modified: > trunk/freenet/src/freenet/io/NetworkInterface.java > Log: > ignoreUnbindable -> ignoreUnbindableIP6, and change it to implement that. > In that patch you are denaturing ignoreUnbindable... That's *bad*. Its purpose was to allow services to spawn up on some interfaces even if binding on others was not possible. How is that related to ipv6 ? NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071208/db6d6a86/attachment.pgp From jolantern60 at yahoo.com Sun Dec 9 13:50:46 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Sun, 9 Dec 2007 05:50:46 -0800 (PST) Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? Message-ID: <347664.3305.qm@web44805.mail.sp1.yahoo.com> Hi, I'm in the process of updating the README of 0.5 and found the section on securing Mozilla in need of a rewrite. I haven't found a similar section in the 0.7 README, so the follwing information might be of interest for 0.7, too. I know of three Mozilla features potentially destroying your anonymity when using FProxy: * GoBrowsing: feeds URLs of failed requests into a search engine. This is a well-known problem but the workaround has changed. Whereas in older versions of Mozilla, the variable "browser.goBrowsing.enabled" had to be set to false, now it is "keyword.enabled". * Prefetching: loads links in a page in the background. If I understand correctly, 0.7 already protects itself against this feature by converting anchors to form submit buttons. This feature may be disabled by setting "network.prefetch-next" to false. * Safebrowsing: communicates the URL (and contents?) of each request to a "safebrowsing provider" (Google is the default). This feature appears to be deactivated in most, if not all, browsers by default. It may be deactivated by setting "browser.safebrowsing.enabled" to false. Are there other funny new Mozilla features I should include in the security cautions section? Jack ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs From freenet-devl at david.sowder.com Sun Dec 9 17:47:31 2007 From: freenet-devl at david.sowder.com (David Sowder (Zothar)) Date: Sun, 09 Dec 2007 11:47:31 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712081314.58744.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <200712081314.58744.toad@amphibian.dyndns.org> Message-ID: <475C2A33.1030403@david.sowder.com> As I see it currently, stuff being kicked out of the datastore isn't happening except when the datastore is being shrunk since AFAIK, nobody's datastore is filling, only their datacache is. Matthew Toseland wrote: > On Friday 07 December 2007 22:47, Robert Hailey wrote: > >> I have been musing on Ed's post for a while. Along with my >> understandings: >> >> * The contents of the datastore(s) are not given weight when >> considering a swap request. >> > > Indeed. This is entirely deliberate. We have no simulations of what would > happen if we did bias by datastore contents, and we are not going to > implement unsimulated changes to core routing, certainly not to swapping. > > >> * For a particular CHK/SSK insert, it finds one home node to be put >> into the store (the rest in only cached). >> > > Not one, average of 3. The nodes which are the closest-so-far to the target. > There will typically be 2 or 3. > > >> * If the node targeted by the insert 'drifts' sufficiently far from >> it's origin >> ** that data in the store be unfindable by the network, and >> ** is all-but guaranteed to be LOST forever, is it can only survive in >> the networks caches (if it is un-popular). >> > > Not true. We have mechanisms to get popular data back into the datastore. The > main one is 1 in every 100 successful requests gets turned into an insert > post-completion. > >> Consider the following patch (completed to the furthest point that my >> current freenet understand can take it). >> - If a CHKBlock is evicted from the datastore (not cache), AND we are >> not (as best we can tell) "close" to where it should be, THEN re- >> insert that block. >> > > This is going to generate an insane amount of traffic, and sabotage the > natural decay of data that happens to be in the wrong place. > >> This approximates the decision (is this eviction OLD/UNPOPULAR or >> UNFINDABLE), and brings with it a rather odd perk for having small >> datastores (that they would find these misplaced blocks sooner). >> >> The major risk, of course, is renewing old data. >> >> I do not have the means to test this on any sizable network. Would >> this be a candidate for 'testnet' or simulations of which I hear? >> >> Also, I could not figure out how to revive a SSKBlock from the >> datastore (or find out if it is even possible). >> > > An SSK requires the pubkey from one datastore and the actual SSK from another. > > ------------------------------------------------------------------------ > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From m.rogers at cs.ucl.ac.uk Sun Dec 9 21:01:40 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sun, 09 Dec 2007 21:01:40 +0000 Subject: [freenet-dev] Major weakness in our current link-level congestion control In-Reply-To: <200712081652.42068.toad@amphibian.dyndns.org> References: <200712081652.42068.toad@amphibian.dyndns.org> Message-ID: <475C57B4.6030807@cs.ucl.ac.uk> Matthew Toseland wrote: > There is a note > in section 4.1 to the effect that if a connection is idle for a long period > and then restarted you can get a burst (so reset to slow-start if the > connection has been idle for a while), but it appears to me that the same is > possible if you have a trickle of data for a long period and then a burst: Wow, nice catch! I googled around for this and it appears to have been spotted (eventually) - the recommended fix is to halve the window for every RTT in which the full window isn't used: http://www.faqs.org/rfcs/rfc2861.html > IMHO the solution for us is to not increase the congestion > window size unless we have actually had a full window in flight recently. I'm > not sure how this would fit into the NewTransportLayer... It should be easy enough to record the time we were last limited by the window, and use the elapsed time to adjust the window when checking how many bytes are available. We might have to be a bit careful with rounding, though, because we send messages rather than bytes, so we're very unlikely to use every last byte in the window even when the connection's congestion-limited. In the longer term we should probably also incorporate a slow start threshold into the NewTransportLayer to make it as TCP-like as possible - I won't be able to work on it before Christmas though... Cheers, Michael From martin.nyhus at sensewave.com Sun Dec 9 21:55:00 2007 From: martin.nyhus at sensewave.com (Martin Nyhus) Date: Sun, 09 Dec 2007 22:55:00 +0100 Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <347664.3305.qm@web44805.mail.sp1.yahoo.com> References: <347664.3305.qm@web44805.mail.sp1.yahoo.com> Message-ID: <1197237300.14150.6.camel@decoy.lan> On Sun, 2007-12-09 at 05:50 -0800, Jack O'Lantern wrote: > * Safebrowsing: communicates the URL (and contents?) of each request to > a "safebrowsing provider" (Google is the default). This feature appears > to be deactivated in most, if not all, browsers by default. It may be > deactivated by setting "browser.safebrowsing.enabled" to false. According to mozillaZine[1], setting "browser.safebrowsing.remoteLookups" to false should be enough to stop remote lookups. [1] http://kb.mozillazine.org/Browser.safebrowsing.remoteLookups -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071209/6a77d524/attachment.pgp From Volodya at WhenGendarmeSleeps.org Thu Dec 6 18:53:46 2007 From: Volodya at WhenGendarmeSleeps.org (Volodya) Date: Thu, 06 Dec 2007 21:53:46 +0300 Subject: [freenet-dev] Short refs was Re: alternative to #freenet-refs In-Reply-To: <4757A87D.8040201@WhenGendarmeSleeps.org> References: <20071107220045.GA4554@freenetproject.org> <1195995849.10868.24.camel@localhost> <474B09FD.7010906@cs.ucl.ac.uk> <200711271144.47979.toad@amphibian.dyndns.org> <474DD412.307@cs.ucl.ac.uk> <474EBFC4.7030201@WhenGendarmeSleeps.org> <4754508B.7000606@cs.ucl.ac.uk> <4757A87D.8040201@WhenGendarmeSleeps.org> Message-ID: <4758453A.3090703@WhenGendarmeSleeps.org> Volodya wrote: > Michael Rogers wrote: >> Thanks! Here's the Java version of the poem generator (same dictionaries >> as before). > >> Cheers, >> Michael > > Have you tried to give it enough words to represent a full key? I have a feeling that what might be needed to > not to load the whole file, but rather use it sax style. So you know that you need 1047th line, and that is the > only one you actually store in the memory. I'll see if i can improve on your code a bit. I really loved it. q;-) > > - Volodya This is basically the same thing but it doesn't load anything in the memory that doesn't need to be there. Do with it as you will. - Volodya -- http://freedom.libsyn.com/ Voice of Freedom, Radical Podcast http://eng.anarchopedia.org/ Anarchopedia, A Free Knowledge Portal "None of us are free until all of us are free." ~ Mihail Bakunin -------------- next part -------------- A non-text attachment was scrubbed... Name: PoemKey.java Type: text/x-java Size: 1739 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071206/73a4abf2/attachment.java From nextgens at freenetproject.org Sun Dec 9 22:40:25 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sun, 9 Dec 2007 23:40:25 +0100 Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <347664.3305.qm@web44805.mail.sp1.yahoo.com> References: <347664.3305.qm@web44805.mail.sp1.yahoo.com> Message-ID: <20071209224024.GA4199@freenetproject.org> * Jack O'Lantern [2007-12-09 05:50:46]: > Hi, > > I'm in the process of updating the README of 0.5 and found the section > on securing Mozilla in need of a rewrite. I haven't found a similar > section in the 0.7 README, so the follwing information might be of > interest for 0.7, too. > > I know of three Mozilla features potentially destroying your anonymity > when using FProxy: > > * GoBrowsing: feeds URLs of failed requests into a search engine. This > is a well-known problem but the workaround has changed. Whereas in > older versions of Mozilla, the variable "browser.goBrowsing.enabled" > had to be set to false, now it is "keyword.enabled". > > * Prefetching: loads links in a page in the background. If I understand > correctly, 0.7 already protects itself against this feature by > converting anchors to form submit buttons. This feature may be disabled > by setting "network.prefetch-next" to false. > > * Safebrowsing: communicates the URL (and contents?) of each request to > a "safebrowsing provider" (Google is the default). This feature appears > to be deactivated in most, if not all, browsers by default. It may be > deactivated by setting "browser.safebrowsing.enabled" to false. > > Are there other funny new Mozilla features I should include in the > security cautions section? > > Jack > There is at least a fourth one we are immune to in .7... The "If I read anything looking like a RSS feed, I decide to ignore the mime-type" one. NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071209/cd2cfa50/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 10 11:18:54 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 11:18:54 +0000 Subject: [freenet-dev] [freenet-cvs] r16403 - trunk/freenet/src/freenet/io In-Reply-To: <20071208214835.GA4559@freenetproject.org> References: <20071208171556.BD54E479758@freenetproject.org> <20071208214835.GA4559@freenetproject.org> Message-ID: <200712101119.02966.toad@amphibian.dyndns.org> On Saturday 08 December 2007 21:48, Florent Daigni?re wrote: > * toad at freenetproject.org [2007-12-08 17:15:56]: > > > Author: toad > > Date: 2007-12-08 17:15:56 +0000 (Sat, 08 Dec 2007) > > New Revision: 16403 > > > > Modified: > > trunk/freenet/src/freenet/io/NetworkInterface.java > > Log: > > ignoreUnbindable -> ignoreUnbindableIP6, and change it to implement that. > > > > In that patch you are denaturing ignoreUnbindable... That's *bad*. Its > purpose was to allow services to spawn up on some interfaces even if > binding on others was not possible. How is that related to ipv6 ? Because that's not what we want. If port 8888 isn't available we don't want fproxy to load on port 8888. IIRC the ONLY purpose of ignoreUnbindable was to workaround the fact that you added IPv6 addresses to the default bind list for all the services. > > NextGen$ > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/c5395879/attachment.pgp From freenet-devl at osndok.com Mon Dec 10 16:34:32 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 10:34:32 -0600 Subject: [freenet-dev] [freenet-cvs] r16388 - trunk/freenet/src/freenet/client/async In-Reply-To: <200712081248.05902.toad@amphibian.dyndns.org> References: <20071207021337.0937C479706@freenetproject.org> <200712081248.05902.toad@amphibian.dyndns.org> Message-ID: On Dec 8, 2007, at 6:48 AM, Matthew Toseland wrote: > On Friday 07 December 2007 02:13, you wrote: > > With regards to Timer: Why not just use the Ticker ? Admittedly they > will > sometimes get scheduled in parallel, but iirc it doesn't do that > much work > on-thread so it's not really a problem... I was not aware of Ticker. It would seem to be more consistent with the code base to use Ticker. >> >> Modified: trunk/freenet/src/freenet/client/async/USKFetcher.java >> =================================================================== >> --- trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 > 01:41:35 UTC (rev 16387) >> +++ trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 > 02:13:36 UTC (rev 16388) >> @@ -252,43 +255,28 @@ >> long now = System.currentTimeMillis(); >> synchronized(this) { >> started = false; // don't finish before have rescheduled >> + >> + //Find out when we should check next ('end'), in an > increasing delay (unless we make progress). >> + long newSleepTime = sleepTime * 2; >> + if(newSleepTime > maxSleepTime) newSleepTime = maxSleepTime; >> + sleepTime = newSleepTime; >> + end = now + sleepTime; >> + >> if(valAtEnd > valueAtSchedule) { >> - // Have advanced. >> + // We have advanced; keep trying as if we just started. >> minFailures = origMinFailures; >> sleepTime = origSleepTime; >> + end = now; > > end = now + origSleepTime. What you have there makes it retry > *immediately*. Correct. If a USKFetch makes progress it must retry with a distinct window-of-five, and the original code appeared to do this immediately, so I maintained that. This code should only be executed when the very last fetch attempt is complete. -- Robert Hailey From freenet-devl at osndok.com Mon Dec 10 17:27:35 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 11:27:35 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712081314.58744.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <200712081314.58744.toad@amphibian.dyndns.org> Message-ID: <1712CC0F-8679-403C-B4CB-4485D96169BC@osndok.com> On Dec 8, 2007, at 7:14 AM, Matthew Toseland wrote: > On Friday 07 December 2007 22:47, Robert Hailey wrote: >> * The contents of the datastore(s) are not given weight when >> considering a swap request. > > Indeed. This is entirely deliberate. We have no simulations of what > would > happen if we did bias by datastore contents, and we are not going to > implement unsimulated changes to core routing, certainly not to > swapping. Yeah, it seems like the swapping/routing is presently the best it has been thus far; but I do think the 0% store hit the Ed mentions may be a factor in freenet's slowness. I see that in my nodes as well. > [...] >> Consider the following patch (completed to the furthest point that my >> current freenet understand can take it). >> - If a CHKBlock is evicted from the datastore (not cache), AND we are >> not (as best we can tell) "close" to where it should be, THEN re- >> insert that block. > > This is going to generate an insane amount of traffic, and sabotage > the > natural decay of data that happens to be in the wrong place. In the present network, it probably would; but in theory I think that the patch is correct (or some variant thereto). The problem is... it presumes that the network location is stable. Now, if a nodes location in the network was presently "stable-enough", this entire issue would not be brought up. Whenever a request would be made, the same node that would receive it as in insert would respond from the datastore, even for potentially really-old data, as there is a lot of storage available on the network. In fact, if the network was as rock-solid, this patch would amount to a no-op, as we are close to anything that we put in our store, then we are close to anything that we evict and the 'closer' condition would always be false. In the worst case, if two nodes have data stores exactly specialized for there present location, and they swap locations, then they would ~eventually~ (one block at at time) swap data stores too. At the end of which, they would be in the same LRU-order too. What's more, in the case where a new node is introduced to a stable network, any errant inserts it stores while it roams the network for it's location would be saved. I imagine that insert traffic is a small percentage of total traffic at large, and the secondary insert could be tuned differently than a standard insert (probably in HTL). As presented, this patch would yield an increasing rate of transfer as more inserts come in (throttled by the thread limit). I suppose the real issue remains network-location being incompatible with store values. >> Also, I could not figure out how to revive a SSKBlock from the >> datastore (or find out if it is even possible). > > An SSK requires the pubkey from one datastore and the actual SSK > from another. So implementing SSK healing-at-eviction-time would require several extra database hits and an extra store-block-read, but would be possible in theory. > Not true. We have mechanisms to get popular data back into the > datastore. The > main one is 1 in every 100 successful requests gets turned into an > insert > post-completion. That is intriguing. I wonder how much of my 0.3% store-hit-successes (my present value) are actually blocks recently-inserted from the 1% healing (i.e. bleed-over from the cache). -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071210/657b66fd/attachment.htm From freenet-devl at osndok.com Mon Dec 10 17:32:20 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 11:32:20 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <475C2A33.1030403@david.sowder.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200711300740.12146.edt@aei.ca> <3D69B822-8CFE-4540-A7BC-DF073ABB698E@osndok.com> <200712081314.58744.toad@amphibian.dyndns.org> <475C2A33.1030403@david.sowder.com> Message-ID: On Dec 9, 2007, at 11:47 AM, David Sowder (Zothar) wrote: > As I see it currently, stuff being kicked out of the datastore isn't > happening except when the datastore is being shrunk since AFAIK, > nobody's datastore is filling, only their datacache is. My datastore is 100% full of data (which according to the logging from my patch, continuously evicts data which cannot be found [> routing miss distance?]), hence the idea and testing for this patch; and (as I said earlier) the interesting incentive for small datastores. -- Robert Hailey From toad at amphibian.dyndns.org Mon Dec 10 17:43:26 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 17:43:26 +0000 Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <347664.3305.qm@web44805.mail.sp1.yahoo.com> References: <347664.3305.qm@web44805.mail.sp1.yahoo.com> Message-ID: <200712101743.32010.toad@amphibian.dyndns.org> On Sunday 09 December 2007 13:50, Jack O'Lantern wrote: > Hi, > > I'm in the process of updating the README of 0.5 and found the section > on securing Mozilla in need of a rewrite. I haven't found a similar > section in the 0.7 README, so the follwing information might be of > interest for 0.7, too. > > I know of three Mozilla features potentially destroying your anonymity > when using FProxy: > > * GoBrowsing: feeds URLs of failed requests into a search engine. This > is a well-known problem but the workaround has changed. Whereas in > older versions of Mozilla, the variable "browser.goBrowsing.enabled" > had to be set to false, now it is "keyword.enabled". Is there anything we can do about this in 0.7 other than adding a note about it to the README? Does anyone read the README? :| Bombe was working on a documentation toadlet, which might help. > > * Prefetching: loads links in a page in the background. If I understand > correctly, 0.7 already protects itself against this feature by > converting anchors to form submit buttons. This feature may be disabled > by setting "network.prefetch-next" to false. I believe this is off by default in most mozilla's/firefox's? It's harmless in 0.7 anyway, but it may cost some performance (due to tying up connections), or gain some (by prefetching high latency pages)... > > * Safebrowsing: communicates the URL (and contents?) of each request to > a "safebrowsing provider" (Google is the default). This feature appears > to be deactivated in most, if not all, browsers by default. It may be > deactivated by setting "browser.safebrowsing.enabled" to false. Would be nice to detect it, I suppose we ought to mention it in the README...? What is its purpose? > > Are there other funny new Mozilla features I should include in the > security cautions section? > > Jack -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/61c56805/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 10 17:44:30 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 17:44:30 +0000 Subject: [freenet-dev] [freenet-cvs] r16388 - trunk/freenet/src/freenet/client/async In-Reply-To: References: <20071207021337.0937C479706@freenetproject.org> <200712081248.05902.toad@amphibian.dyndns.org> Message-ID: <200712101744.30749.toad@amphibian.dyndns.org> On Monday 10 December 2007 16:34, Robert Hailey wrote: > > On Dec 8, 2007, at 6:48 AM, Matthew Toseland wrote: > > > On Friday 07 December 2007 02:13, you wrote: > > > > With regards to Timer: Why not just use the Ticker ? Admittedly they > > will > > sometimes get scheduled in parallel, but iirc it doesn't do that > > much work > > on-thread so it's not really a problem... > > I was not aware of Ticker. It would seem to be more consistent with > the code base to use Ticker. Have you implemented this? > > >> > >> Modified: trunk/freenet/src/freenet/client/async/USKFetcher.java > >> =================================================================== > >> --- trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 > > 01:41:35 UTC (rev 16387) > >> +++ trunk/freenet/src/freenet/client/async/USKFetcher.java 2007-12-07 > > 02:13:36 UTC (rev 16388) > >> @@ -252,43 +255,28 @@ > >> long now = System.currentTimeMillis(); > >> synchronized(this) { > >> started = false; // don't finish before have rescheduled > >> + > >> + //Find out when we should check next ('end'), in an > > increasing delay (unless we make progress). > >> + long newSleepTime = sleepTime * 2; > >> + if(newSleepTime > maxSleepTime) newSleepTime = maxSleepTime; > >> + sleepTime = newSleepTime; > >> + end = now + sleepTime; > >> + > >> if(valAtEnd > valueAtSchedule) { > >> - // Have advanced. > >> + // We have advanced; keep trying as if we just started. > >> minFailures = origMinFailures; > >> sleepTime = origSleepTime; > >> + end = now; > > > > end = now + origSleepTime. What you have there makes it retry > > *immediately*. > > Correct. If a USKFetch makes progress it must retry with a distinct > window-of-five, and the original code appeared to do this immediately, > so I maintained that. This code should only be executed when the very > last fetch attempt is complete. I thought it slept for origSleepTime in that case? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/05473d5d/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 10 17:45:13 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 17:45:13 +0000 Subject: [freenet-dev] Store size not used In-Reply-To: References: Message-ID: <200712101745.13624.toad@amphibian.dyndns.org> On Saturday 08 December 2007 17:16, bbackde at googlemail.com wrote: > What I wonder about: after I run freenet for more than a year, why did > it only collect 10GB in the store? May others have a larger usage? With similar transfer volumes? > > On Dec 8, 2007 6:10 PM, wrote: > > I do not count each key, but its at 10% for months now. > > > > > > On Dec 8, 2007 5:57 PM, Matthew Toseland wrote: > > > On Saturday 08 December 2007 14:29, bbackde at googlemail.com wrote: > > > > I run freenet 0.7 for a very long time. My datastore size is 200GB, > > > > and my cache is 100% full. > > > > But the storage is still only at 10% of its maximum size, and it doesn't > > > grow. > > > > > > What do you mean it doesn't grow? It should store more keys over time, even if > > > it's only one key per day. > > > > > > > There was some discussion about this in the past, but I see no advance. > > > > > > > > I know this has something to do with specialization, but does this > > > > mean that my node specialized > > > > to something that is not inserted? It would be nice if the freenet > > > > could use the remaining 90GB that > > > > I offer. > > > > > > Auto-shrinking the cache as the store gets bigger requires an efficient and > > > non-LRU-damaging way to do online shrink. It's possible but it's not a > > > priority right now. > > > > > > > > > > > My stats: > > > > > > > > # Cached keys: 3,222,330 (98.3 GiB) (100%) > > > > # Stored keys: 346,172 (10.5 GiB) (10%) > > > > # Overall size: 3,568,502 / 6,444,660 (108 GiB / 196 GiB) (55%) > > > > > > _______________________________________________ > > > Devl mailing list > > > Devl at freenetproject.org > > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > > > > > > > > > > -- > > __________________________________________________ > > GnuPG key: (0x48DBFA8A) > > Keyserver: pgpkeys.pca.dfn.de > > Fingerprint: > > 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A > > __________________________________________________ > > > > > > -- > __________________________________________________ > GnuPG key: (0x48DBFA8A) > Keyserver: pgpkeys.pca.dfn.de > Fingerprint: > 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A > __________________________________________________ > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/d5f260aa/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 10 17:49:35 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 17:49:35 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <1712CC0F-8679-403C-B4CB-4485D96169BC@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712081314.58744.toad@amphibian.dyndns.org> <1712CC0F-8679-403C-B4CB-4485D96169BC@osndok.com> Message-ID: <200712101749.35126.toad@amphibian.dyndns.org> On Monday 10 December 2007 17:27, Robert Hailey wrote: > > On Dec 8, 2007, at 7:14 AM, Matthew Toseland wrote: > > > On Friday 07 December 2007 22:47, Robert Hailey wrote: > >> * The contents of the datastore(s) are not given weight when > >> considering a swap request. > > > > Indeed. This is entirely deliberate. We have no simulations of what > > would > > happen if we did bias by datastore contents, and we are not going to > > implement unsimulated changes to core routing, certainly not to > > swapping. > > Yeah, it seems like the swapping/routing is presently the best it has > been thus far; but I do think the 0% store hit the Ed mentions may be > a factor in freenet's slowness. I see that in my nodes as well. Don't you think it simply means that Freenet is mostly being used for short-term storage i.e. fetching popular/recently inserted content? > > > [...] > >> Consider the following patch (completed to the furthest point that my > >> current freenet understand can take it). > >> - If a CHKBlock is evicted from the datastore (not cache), AND we are > >> not (as best we can tell) "close" to where it should be, THEN re- > >> insert that block. > > > > This is going to generate an insane amount of traffic, and sabotage > > the > > natural decay of data that happens to be in the wrong place. > > In the present network, it probably would; but in theory I think that > the patch is correct (or some variant thereto). Nothing would ever be dropped from the network, because when it's considered for dropping, it would get reinserted to 20 other nodes! > > The problem is... it presumes that the network location is stable. > Now, if a nodes location in the network was presently "stable-enough", > this entire issue would not be brought up. Whenever a request would be > made, the same node that would receive it as in insert would respond > from the datastore, even for potentially really-old data, as there is > a lot of storage available on the network. > > In fact, if the network was as rock-solid, this patch would amount to > a no-op, as we are close to anything that we put in our store, then we > are close to anything that we evict and the 'closer' condition would > always be false. > > In the worst case, if two nodes have data stores exactly specialized > for there present location, and they swap locations, then they would > ~eventually~ (one block at at time) swap data stores too. At the end > of which, they would be in the same LRU-order too. > > What's more, in the case where a new node is introduced to a stable > network, any errant inserts it stores while it roams the network for > it's location would be saved. > > I imagine that insert traffic is a small percentage of total traffic > at large, and the secondary insert could be tuned differently than a > standard insert (probably in HTL). As presented, this patch would > yield an increasing rate of transfer as more inserts come in > (throttled by the thread limit). Inserts cost a lot more than requests, but yes there are fewer of them. However what you are talking about here would make inserts the overwhelming bulk of data transferred, in the long term. > > I suppose the real issue remains network-location being incompatible > with store values. > > >> Also, I could not figure out how to revive a SSKBlock from the > >> datastore (or find out if it is even possible). > > > > An SSK requires the pubkey from one datastore and the actual SSK > > from another. > > So implementing SSK healing-at-eviction-time would require several > extra database hits and an extra store-block-read, but would be > possible in theory. Yes. > > > Not true. We have mechanisms to get popular data back into the > > datastore. The > > main one is 1 in every 100 successful requests gets turned into an > > insert > > post-completion. > > > That is intriguing. I wonder how much of my 0.3% store-hit-successes > (my present value) are actually blocks recently-inserted from the 1% > healing (i.e. bleed-over from the cache). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/fd5196b6/attachment.pgp From freenet-devl at osndok.com Mon Dec 10 18:01:48 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 12:01:48 -0600 Subject: [freenet-dev] [freenet-cvs] r16388 - trunk/freenet/src/freenet/client/async In-Reply-To: <200712101744.30749.toad@amphibian.dyndns.org> References: <20071207021337.0937C479706@freenetproject.org> <200712081248.05902.toad@amphibian.dyndns.org> <200712101744.30749.toad@amphibian.dyndns.org> Message-ID: <90C4E0C2-D560-437E-AFFB-078FF53094BE@osndok.com> On Dec 10, 2007, at 11:44 AM, Matthew Toseland wrote: > On Monday 10 December 2007 16:34, Robert Hailey wrote: >> >> I was not aware of Ticker. It would seem to be more consistent with >> the code base to use Ticker. > > Have you implemented this? Yes, just now fixed in r16445. -- Robert Hailey From freenet-devl at osndok.com Mon Dec 10 18:08:14 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 12:08:14 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712101749.35126.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712081314.58744.toad@amphibian.dyndns.org> <1712CC0F-8679-403C-B4CB-4485D96169BC@osndok.com> <200712101749.35126.toad@amphibian.dyndns.org> Message-ID: <5D1831D3-2FAE-4E23-A118-64D4C45AF185@osndok.com> On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: >> Yeah, it seems like the swapping/routing is presently the best it has >> been thus far; but I do think the 0% store hit the Ed mentions may be >> a factor in freenet's slowness. I see that in my nodes as well. > > Don't you think it simply means that Freenet is mostly being used for > short-term storage i.e. fetching popular/recently inserted content? No, and that I am only able to fetch content with some degree of recentness (e.g. looking up an old ssk-version of a site fails), to me indicates that at present freenet can *only* be used for short-term storage at present. Or more specifically, that the store implementation (most likely with regards to routing) is broken. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071210/a9feb6cd/attachment.htm From toad at amphibian.dyndns.org Mon Dec 10 18:11:17 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 18:11:17 +0000 Subject: [freenet-dev] [freenet-cvs] r16444 - trunk/freenet/src/freenet/node In-Reply-To: <20071209224042.D3A1347971F@freenetproject.org> References: <20071209224042.D3A1347971F@freenetproject.org> Message-ID: <200712101811.22037.toad@amphibian.dyndns.org> Why does this please netbeans? Is there any good reason to make these classes public? On Sunday 09 December 2007 22:40, you wrote: > Author: nextgens > Date: 2007-12-09 22:40:42 +0000 (Sun, 09 Dec 2007) > New Revision: 16444 > > Modified: > trunk/freenet/src/freenet/node/NodeCrypto.java > trunk/freenet/src/freenet/node/ResendPacketItem.java > Log: > Make some classes public to please netbeans > > Modified: trunk/freenet/src/freenet/node/NodeCrypto.java > =================================================================== > --- trunk/freenet/src/freenet/node/NodeCrypto.java 2007-12-09 22:38:45 UTC (rev 16443) > +++ trunk/freenet/src/freenet/node/NodeCrypto.java 2007-12-09 22:40:42 UTC (rev 16444) > @@ -41,7 +41,7 @@ > * Cryptographic and transport level node identity. > * @author toad > */ > -class NodeCrypto { > +public class NodeCrypto { > > /** Length of a node identity */ > public static final int IDENTITY_LENGTH = 32; > > Modified: trunk/freenet/src/freenet/node/ResendPacketItem.java > =================================================================== > --- trunk/freenet/src/freenet/node/ResendPacketItem.java 2007-12-09 22:38:45 UTC (rev 16443) > +++ trunk/freenet/src/freenet/node/ResendPacketItem.java 2007-12-09 22:40:42 UTC (rev 16444) > @@ -9,7 +9,7 @@ > * A packet to be resent. Includes a packet number, and the > * message as byte[]. > */ > -class ResendPacketItem { > +public class ResendPacketItem { > public ResendPacketItem(byte[] payload, int packetNumber, KeyTracker k, AsyncMessageCallback[] callbacks) { > pn = k.pn; > kt = k; > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/58da0094/attachment.pgp From tommy100 at gmx.de Mon Dec 10 18:26:37 2007 From: tommy100 at gmx.de (Tommy[D]) Date: Mon, 10 Dec 2007 19:26:37 +0100 Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <200712101743.32010.toad@amphibian.dyndns.org> References: <347664.3305.qm@web44805.mail.sp1.yahoo.com> <200712101743.32010.toad@amphibian.dyndns.org> Message-ID: <475D84DD.4090208@gmx.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Matthew Toseland schrieb: > On Sunday 09 December 2007 13:50, Jack O'Lantern wrote: >> * Prefetching: loads links in a page in the background. If I understand >> correctly, 0.7 already protects itself against this feature by >> converting anchors to form submit buttons. This feature may be disabled >> by setting "network.prefetch-next" to false. > > I believe this is off by default in most mozilla's/firefox's? It's harmless in > 0.7 anyway, but it may cost some performance (due to tying up connections), > or gain some (by prefetching high latency pages)... network.prefetch-next is default true for me. >> * Safebrowsing: communicates the URL (and contents?) of each request to >> a "safebrowsing provider" (Google is the default). This feature appears >> to be deactivated in most, if not all, browsers by default. It may be >> deactivated by setting "browser.safebrowsing.enabled" to false. > > Would be nice to detect it, I suppose we ought to mention it in the README...? > What is its purpose? It is per default active, but uses per default a local list and no external provider for me. Its goal is to check for fishing sites or other sites which try to harm you. The URL is checked against a local list (or the providers list) and after that, the browser does give out a warning, if the site is on the list. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR12E3Ru5KnE1iZBnAQqtSQP+LaynGHKuN63ZtR5Y0wwnfKbX/2ZbxOS/ 4ZNVNOal2OxkdkPm464O3GK1PB3WdL5EGUlRVJnzVdNMwuV63s50FHVuMykex8ov 36hNylV7cnOTKpCvSJLLC0jXIe+ISTcKFNlyrGtH5p8NSb/GEcMEMDYY6cbkdiKM 3q73qFtsZis= =RGuK -----END PGP SIGNATURE----- From freenet-devl at osndok.com Mon Dec 10 18:52:05 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 12:52:05 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712101749.35126.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712081314.58744.toad@amphibian.dyndns.org> <1712CC0F-8679-403C-B4CB-4485D96169BC@osndok.com> <200712101749.35126.toad@amphibian.dyndns.org> Message-ID: <6E9A61F5-FB81-4AD3-BEDA-24FC66F130A9@osndok.com> On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: >> In the present network, it probably would; but in theory I think that >> the patch is correct (or some variant thereto). > > Nothing would ever be dropped from the network, because when it's > considered > for dropping, it would get reinserted to 20 other nodes! I am not recommending that this patch be applied... yet. Every point that you have raised against it is perfectly valid. In the present network, because the nodes drift locations soo much, this patch (even if perfectly tuned; maybe re-insert with HTL=1) would cause data blocks to "chase" the nodes around the network. Resulting in massive network traffic increases, as you said. *IF* it helped access of data, it would only be due to the renewed data being passed through the node caches (which would probably be overflowed with old insert data). My suggestion at present is to: (1) stabalize node locations enough that data stores come alive, or (2) bias/soft-anchor towards what is in the datastore (or perhaps what has most-recently been put in the data store?). I agree that either of which would require simulations. #1 would be a statistical solution (network drift < datastore utility-threshold) and may be presently attainable with tuning, whereas #2 would be more pragmatic (and tend to specialize nodes further). #1 may already be the case if the network size was large enough, but an algorithmically correct freenet should support any size network (as math scales very well). As an example of the general problem (although it seems to have helped get a routable network); even the theory of a node randomizing it's location totally obsoletes it's datastore. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071210/a4a32033/attachment.htm From toad at amphibian.dyndns.org Mon Dec 10 19:03:34 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 19:03:34 +0000 Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <475D84DD.4090208@gmx.de> References: <347664.3305.qm@web44805.mail.sp1.yahoo.com> <200712101743.32010.toad@amphibian.dyndns.org> <475D84DD.4090208@gmx.de> Message-ID: <200712101903.43461.toad@amphibian.dyndns.org> On Monday 10 December 2007 18:26, Tommy[D] wrote: > Matthew Toseland schrieb: > > On Sunday 09 December 2007 13:50, Jack O'Lantern wrote: > >> * Prefetching: loads links in a page in the background. If I understand > >> correctly, 0.7 already protects itself against this feature by > >> converting anchors to form submit buttons. This feature may be disabled > >> by setting "network.prefetch-next" to false. > > > > I believe this is off by default in most mozilla's/firefox's? It's harmless in > > 0.7 anyway, but it may cost some performance (due to tying up connections), > > or gain some (by prefetching high latency pages)... > > network.prefetch-next is default true for me. Interesting. With the default max connections setting that could cost us more than it gains us.. there isn't any obvious way to fix it though.. > > >> * Safebrowsing: communicates the URL (and contents?) of each request to > >> a "safebrowsing provider" (Google is the default). This feature appears > >> to be deactivated in most, if not all, browsers by default. It may be > >> deactivated by setting "browser.safebrowsing.enabled" to false. > > > > Would be nice to detect it, I suppose we ought to mention it in the README...? > > What is its purpose? > > It is per default active, but uses per default a local list and no external provider for me. Its > goal is to check for fishing sites or other sites which try to harm you. The URL is checked against > a local list (or the providers list) and after that, the browser does give out a warning, if the > site is on the list. Hmmm I still don't understand - show an error message if the user types in a domain? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/94208fb8/attachment.pgp From toad at amphibian.dyndns.org Mon Dec 10 19:10:33 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 19:10:33 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <6E9A61F5-FB81-4AD3-BEDA-24FC66F130A9@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101749.35126.toad@amphibian.dyndns.org> <6E9A61F5-FB81-4AD3-BEDA-24FC66F130A9@osndok.com> Message-ID: <200712101910.34172.toad@amphibian.dyndns.org> On Monday 10 December 2007 18:52, Robert Hailey wrote: > > On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: > > >> In the present network, it probably would; but in theory I think that > >> the patch is correct (or some variant thereto). > > > > Nothing would ever be dropped from the network, because when it's > > considered > > for dropping, it would get reinserted to 20 other nodes! > > I am not recommending that this patch be applied... yet. Every point > that you have raised against it is perfectly valid. In the present > network, because the nodes drift locations soo much, this patch (even > if perfectly tuned; maybe re-insert with HTL=1) would cause data > blocks to "chase" the nodes around the network. Resulting in massive > network traffic increases, as you said. *IF* it helped access of data, > it would only be due to the renewed data being passed through the node > caches (which would probably be overflowed with old insert data). > > My suggestion at present is to: > (1) stabalize node locations enough that data stores come alive, or Dependant on topology (which we can control), node uptimes (which we can't control), ... > (2) bias/soft-anchor towards what is in the datastore (or perhaps what > has most-recently been put in the data store?). Will not happen without major simulations. > > I agree that either of which would require simulations. Right. And in the latter case, simulating it would be slow, as we'd have to maintain fairly large virtual datastores in the simulation. > #1 would be a > statistical solution (network drift < datastore utility-threshold) and > may be presently attainable with tuning, whereas #2 would be more > pragmatic (and tend to specialize nodes further). #1 may already be > the case if the network size was large enough, but an algorithmically > correct freenet should support any size network (as math scales very > well). IMHO the next step forward is simply to log location changes and display them either on the location page or on a subpage, or as CSV data (or perhaps through SNMP) so it can be graphed externally. Maybe for the node's peers as well as itself. Are you interested in doing some data collection code? Lets discover whether there actually is a problem with location drift before we try to solve it ... Another thing you could do would be to implement a datastore histogram generator. We had one in 0.5. > > As an example of the general problem (although it seems to have helped > get a routable network); even the theory of a node randomizing it's > location totally obsoletes it's datastore. Usually the node will swap back to where it should be within a fairly short period. However, again, we need some hard data from the real network before we even implement simulations. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/e1d86024/attachment.pgp From freenet-devl at osndok.com Mon Dec 10 20:00:24 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 14:00:24 -0600 Subject: [freenet-dev] [freenet-cvs] r16398 - trunk/freenet/src/freenet/node In-Reply-To: <200712081305.18305.toad@amphibian.dyndns.org> References: <20071207211120.CF3C44796AA@freenetproject.org> <200712081305.18305.toad@amphibian.dyndns.org> Message-ID: <434C8574-0609-4F56-BE09-B6D4944C7EA4@osndok.com> On Dec 8, 2007, at 7:05 AM, Matthew Toseland wrote: > IMHO the (once) check should in fact be in applyByteCounts(). Fixed in r16455, makes it cleaner too. -- Robert Hailey From freenet-devl at david.sowder.com Mon Dec 10 20:37:33 2007 From: freenet-devl at david.sowder.com (David Sowder) Date: Mon, 10 Dec 2007 14:37:33 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712101910.34172.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101749.35126.toad@amphibian.dyndns.org> <6E9A61F5-FB81-4AD3-BEDA-24FC66F130A9@osndok.com> <200712101910.34172.toad@amphibian.dyndns.org> Message-ID: <475DA38D.5080708@david.sowder.com> I have location data in an RRD for each of my two, currently up nodes, as obtained via FCP using pyFreenet's utility for such. My graphs from the last week has had the location all over the place for one node, while the other node is more calm, but still not appearing to truly favor any particular spot on the location circle. Matthew Toseland wrote: > On Monday 10 December 2007 18:52, Robert Hailey wrote: > >> On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: >> >> >>>> In the present network, it probably would; but in theory I think that >>>> the patch is correct (or some variant thereto). >>>> >>> Nothing would ever be dropped from the network, because when it's >>> considered >>> for dropping, it would get reinserted to 20 other nodes! >>> >> I am not recommending that this patch be applied... yet. Every point >> that you have raised against it is perfectly valid. In the present >> network, because the nodes drift locations soo much, this patch (even >> if perfectly tuned; maybe re-insert with HTL=1) would cause data >> blocks to "chase" the nodes around the network. Resulting in massive >> network traffic increases, as you said. *IF* it helped access of data, >> it would only be due to the renewed data being passed through the node >> caches (which would probably be overflowed with old insert data). >> >> My suggestion at present is to: >> (1) stabalize node locations enough that data stores come alive, or >> > > Dependant on topology (which we can control), node uptimes (which we can't > control), ... > > >> (2) bias/soft-anchor towards what is in the datastore (or perhaps what >> has most-recently been put in the data store?). >> > > Will not happen without major simulations. > >> I agree that either of which would require simulations. >> > > Right. And in the latter case, simulating it would be slow, as we'd have to > maintain fairly large virtual datastores in the simulation. > > >> #1 would be a >> statistical solution (network drift < datastore utility-threshold) and >> may be presently attainable with tuning, whereas #2 would be more >> pragmatic (and tend to specialize nodes further). #1 may already be >> the case if the network size was large enough, but an algorithmically >> correct freenet should support any size network (as math scales very >> well). >> > > IMHO the next step forward is simply to log location changes and display them > either on the location page or on a subpage, or as CSV data (or perhaps > through SNMP) so it can be graphed externally. Maybe for the node's peers as > well as itself. Are you interested in doing some data collection code? Lets > discover whether there actually is a problem with location drift before we > try to solve it ... > > Another thing you could do would be to implement a datastore histogram > generator. We had one in 0.5. > >> As an example of the general problem (although it seems to have helped >> get a routable network); even the theory of a node randomizing it's >> location totally obsoletes it's datastore. >> > > Usually the node will swap back to where it should be within a fairly short > period. However, again, we need some hard data from the real network before > we even implement simulations. > > ------------------------------------------------------------------------ > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From nextgens at freenetproject.org Mon Dec 10 20:55:30 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 10 Dec 2007 21:55:30 +0100 Subject: [freenet-dev] [freenet-cvs] r16444 - trunk/freenet/src/freenet/node In-Reply-To: <200712101811.22037.toad@amphibian.dyndns.org> References: <20071209224042.D3A1347971F@freenetproject.org> <200712101811.22037.toad@amphibian.dyndns.org> Message-ID: <20071210205530.GB4191@freenetproject.org> * Matthew Toseland [2007-12-10 18:11:17]: > Why does this please netbeans? Is there any good reason to make these classes > public? > It was complaining about "private members are beeing exported through the public API" > On Sunday 09 December 2007 22:40, you wrote: > > Author: nextgens > > Date: 2007-12-09 22:40:42 +0000 (Sun, 09 Dec 2007) > > New Revision: 16444 > > > > Modified: > > trunk/freenet/src/freenet/node/NodeCrypto.java > > trunk/freenet/src/freenet/node/ResendPacketItem.java > > Log: > > Make some classes public to please netbeans > > > > Modified: trunk/freenet/src/freenet/node/NodeCrypto.java > > =================================================================== > > --- trunk/freenet/src/freenet/node/NodeCrypto.java 2007-12-09 22:38:45 UTC > (rev 16443) > > +++ trunk/freenet/src/freenet/node/NodeCrypto.java 2007-12-09 22:40:42 UTC > (rev 16444) > > @@ -41,7 +41,7 @@ > > * Cryptographic and transport level node identity. > > * @author toad > > */ > > -class NodeCrypto { > > +public class NodeCrypto { > > > > /** Length of a node identity */ > > public static final int IDENTITY_LENGTH = 32; > > > > Modified: trunk/freenet/src/freenet/node/ResendPacketItem.java > > =================================================================== > > --- trunk/freenet/src/freenet/node/ResendPacketItem.java 2007-12-09 22:38:45 > UTC (rev 16443) > > +++ trunk/freenet/src/freenet/node/ResendPacketItem.java 2007-12-09 22:40:42 > UTC (rev 16444) > > @@ -9,7 +9,7 @@ > > * A packet to be resent. Includes a packet number, and the > > * message as byte[]. > > */ > > -class ResendPacketItem { > > +public class ResendPacketItem { > > public ResendPacketItem(byte[] payload, int packetNumber, KeyTracker k, > AsyncMessageCallback[] callbacks) { > > pn = k.pn; > > kt = k; > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/73b55425/attachment.pgp From freenet-devl at osndok.com Mon Dec 10 23:40:14 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Mon, 10 Dec 2007 17:40:14 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712101910.34172.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101749.35126.toad@amphibian.dyndns.org> <6E9A61F5-FB81-4AD3-BEDA-24FC66F130A9@osndok.com> <200712101910.34172.toad@amphibian.dyndns.org> Message-ID: <479F9263-FE2A-4C39-8DB7-3123A79F7924@osndok.com> On Dec 10, 2007, at 1:10 PM, Matthew Toseland wrote: > IMHO the next step forward is simply to log location changes and > display them > either on the location page or on a subpage, or as CSV data (or > perhaps > through SNMP) so it can be graphed externally. Maybe for the node's > peers as > well as itself. Are you interested in doing some data collection > code? Lets > discover whether there actually is a problem with location drift > before we > try to solve it ... I think that it should be observable with a few more datastore stats. Specifically the datastore location, and the location of succeeding requests. A quick implementation is attached. It only accounts for what is in the datastore since-launch (but shows a percent thereto), and takes up several megabytes more of memory to track the location of the datastore/cache in a running average. If we are leaving our stores behind, I would predict that for many nodes (running suffecient to have meaningful stats) that the 'storeDist' value may become arbitrary. And if it approaches the 'furthestSuccess' value the store is "left-behind"; whereas the cacheDist will maintain approx. the same (once reliable). Do you want this in the trunk? I don't think that I have the time to do much data collection. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071210/61f86662/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: freenet-datastore-location-stats.diff Type: application/octet-stream Size: 9200 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/61f86662/attachment.obj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071210/61f86662/attachment-0001.htm From toad at amphibian.dyndns.org Mon Dec 10 23:57:46 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Dec 2007 23:57:46 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <475DA38D.5080708@david.sowder.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101910.34172.toad@amphibian.dyndns.org> <475DA38D.5080708@david.sowder.com> Message-ID: <200712102357.51735.toad@amphibian.dyndns.org> On Monday 10 December 2007 20:37, David Sowder wrote: > I have location data in an RRD for each of my two, currently up nodes, > as obtained via FCP using pyFreenet's utility for such. My graphs from > the last week has had the location all over the place for one node, > while the other node is more calm, but still not appearing to truly > favor any particular spot on the location circle. So the second node is wandering about a lot? This is probably an artefact of poor topology... > > Matthew Toseland wrote: > > On Monday 10 December 2007 18:52, Robert Hailey wrote: > > > >> On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: > >> > >> > >>>> In the present network, it probably would; but in theory I think that > >>>> the patch is correct (or some variant thereto). > >>>> > >>> Nothing would ever be dropped from the network, because when it's > >>> considered > >>> for dropping, it would get reinserted to 20 other nodes! > >>> > >> I am not recommending that this patch be applied... yet. Every point > >> that you have raised against it is perfectly valid. In the present > >> network, because the nodes drift locations soo much, this patch (even > >> if perfectly tuned; maybe re-insert with HTL=1) would cause data > >> blocks to "chase" the nodes around the network. Resulting in massive > >> network traffic increases, as you said. *IF* it helped access of data, > >> it would only be due to the renewed data being passed through the node > >> caches (which would probably be overflowed with old insert data). > >> > >> My suggestion at present is to: > >> (1) stabalize node locations enough that data stores come alive, or > >> > > > > Dependant on topology (which we can control), node uptimes (which we can't > > control), ... > > > > > >> (2) bias/soft-anchor towards what is in the datastore (or perhaps what > >> has most-recently been put in the data store?). > >> > > > > Will not happen without major simulations. > > > >> I agree that either of which would require simulations. > >> > > > > Right. And in the latter case, simulating it would be slow, as we'd have to > > maintain fairly large virtual datastores in the simulation. > > > > > >> #1 would be a > >> statistical solution (network drift < datastore utility-threshold) and > >> may be presently attainable with tuning, whereas #2 would be more > >> pragmatic (and tend to specialize nodes further). #1 may already be > >> the case if the network size was large enough, but an algorithmically > >> correct freenet should support any size network (as math scales very > >> well). > >> > > > > IMHO the next step forward is simply to log location changes and display them > > either on the location page or on a subpage, or as CSV data (or perhaps > > through SNMP) so it can be graphed externally. Maybe for the node's peers as > > well as itself. Are you interested in doing some data collection code? Lets > > discover whether there actually is a problem with location drift before we > > try to solve it ... > > > > Another thing you could do would be to implement a datastore histogram > > generator. We had one in 0.5. > > > >> As an example of the general problem (although it seems to have helped > >> get a routable network); even the theory of a node randomizing it's > >> location totally obsoletes it's datastore. > >> > > > > Usually the node will swap back to where it should be within a fairly short > > period. However, again, we need some hard data from the real network before > > we even implement simulations. > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071210/4475f0b2/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 00:00:19 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 00:00:19 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <479F9263-FE2A-4C39-8DB7-3123A79F7924@osndok.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101910.34172.toad@amphibian.dyndns.org> <479F9263-FE2A-4C39-8DB7-3123A79F7924@osndok.com> Message-ID: <200712110000.19289.toad@amphibian.dyndns.org> On Monday 10 December 2007 23:40, Robert Hailey wrote: > > On Dec 10, 2007, at 1:10 PM, Matthew Toseland wrote: > > > IMHO the next step forward is simply to log location changes and > > display them > > either on the location page or on a subpage, or as CSV data (or > > perhaps > > through SNMP) so it can be graphed externally. Maybe for the node's > > peers as > > well as itself. Are you interested in doing some data collection > > code? Lets > > discover whether there actually is a problem with location drift > > before we > > try to solve it ... > > I think that it should be observable with a few more datastore stats. > Specifically the datastore location, and the location of succeeding > requests. Not enough on its own, IMHO that doesn't give us proof of cause, only proof that something is broken. > > A quick implementation is attached. It only accounts for what is in > the datastore since-launch (but shows a percent thereto), and takes up > several megabytes more of memory to track the location of the > datastore/cache in a running average. It is possible to iterate the keys and build a histogram. It'll take a while but you don't have to do it often. > > If we are leaving our stores behind, I would predict that for many > nodes (running suffecient to have meaningful stats) that the > 'storeDist' value may become arbitrary. And if it approaches the > 'furthestSuccess' value the store is "left-behind"; whereas the > cacheDist will maintain approx. the same (once reliable). IMHO we need the histogram, or at least a measure of spread, not just an average key. > > Do you want this in the trunk? I don't think that I have the time to > do much data collection. Your diff is empty. Might be a problem on my end though. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/64be9c04/attachment.pgp From freenet-devl at david.sowder.com Tue Dec 11 00:12:22 2007 From: freenet-devl at david.sowder.com (David Sowder (Zothar)) Date: Mon, 10 Dec 2007 18:12:22 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712102357.51735.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101910.34172.toad@amphibian.dyndns.org> <475DA38D.5080708@david.sowder.com> <200712102357.51735.toad@amphibian.dyndns.org> Message-ID: <475DD5E6.3020405@david.sowder.com> The most wandering node is darknet only and you know it as Zothar70. The more calm node is a hybrid darknet/opennet node. That being said, neither node appears to have a favorite location it comes back to. OTOH, if I only look at the 48 hour graph rather than the week graph, the darknet only node appears to be favoring a particular location more than the hybrid node. In any case, I've got the data in an RRD for each node, if it's useful. Perhaps if we had a "location of the datastore" metric formed by an average or something, I could export that via FCP and track that in the RRDs as well. Matthew Toseland wrote: > On Monday 10 December 2007 20:37, David Sowder wrote: > >> I have location data in an RRD for each of my two, currently up nodes, >> as obtained via FCP using pyFreenet's utility for such. My graphs from >> the last week has had the location all over the place for one node, >> while the other node is more calm, but still not appearing to truly >> favor any particular spot on the location circle. >> > > So the second node is wandering about a lot? > > This is probably an artefact of poor topology... > >> Matthew Toseland wrote: >> >>> On Monday 10 December 2007 18:52, Robert Hailey wrote: >>> >>> >>>> On Dec 10, 2007, at 11:49 AM, Matthew Toseland wrote: >>>> >>>> >>>> >>>>>> In the present network, it probably would; but in theory I think that >>>>>> the patch is correct (or some variant thereto). >>>>>> >>>>>> >>>>> Nothing would ever be dropped from the network, because when it's >>>>> considered >>>>> for dropping, it would get reinserted to 20 other nodes! >>>>> >>>>> >>>> I am not recommending that this patch be applied... yet. Every point >>>> that you have raised against it is perfectly valid. In the present >>>> network, because the nodes drift locations soo much, this patch (even >>>> if perfectly tuned; maybe re-insert with HTL=1) would cause data >>>> blocks to "chase" the nodes around the network. Resulting in massive >>>> network traffic increases, as you said. *IF* it helped access of data, >>>> it would only be due to the renewed data being passed through the node >>>> caches (which would probably be overflowed with old insert data). >>>> >>>> My suggestion at present is to: >>>> (1) stabalize node locations enough that data stores come alive, or >>>> >>>> >>> Dependant on topology (which we can control), node uptimes (which we can't >>> control), ... >>> >>> >>> >>>> (2) bias/soft-anchor towards what is in the datastore (or perhaps what >>>> has most-recently been put in the data store?). >>>> >>>> >>> Will not happen without major simulations. >>> >>> >>>> I agree that either of which would require simulations. >>>> >>>> >>> Right. And in the latter case, simulating it would be slow, as we'd have >>> > to > >>> maintain fairly large virtual datastores in the simulation. >>> >>> >>> >>>> #1 would be a >>>> statistical solution (network drift < datastore utility-threshold) and >>>> may be presently attainable with tuning, whereas #2 would be more >>>> pragmatic (and tend to specialize nodes further). #1 may already be >>>> the case if the network size was large enough, but an algorithmically >>>> correct freenet should support any size network (as math scales very >>>> well). >>>> >>>> >>> IMHO the next step forward is simply to log location changes and display >>> > them > >>> either on the location page or on a subpage, or as CSV data (or perhaps >>> through SNMP) so it can be graphed externally. Maybe for the node's peers >>> > as > >>> well as itself. Are you interested in doing some data collection code? >>> > Lets > >>> discover whether there actually is a problem with location drift before we >>> try to solve it ... >>> >>> Another thing you could do would be to implement a datastore histogram >>> generator. We had one in 0.5. >>> >>> >>>> As an example of the general problem (although it seems to have helped >>>> get a routable network); even the theory of a node randomizing it's >>>> location totally obsoletes it's datastore. >>>> >>>> >>> Usually the node will swap back to where it should be within a fairly >>> > short > >>> period. However, again, we need some hard data from the real network >>> > before > >>> we even implement simulations. >>> From toad at amphibian.dyndns.org Tue Dec 11 00:26:41 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 00:26:41 +0000 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <475DD5E6.3020405@david.sowder.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712102357.51735.toad@amphibian.dyndns.org> <475DD5E6.3020405@david.sowder.com> Message-ID: <200712110026.45758.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 00:12, David Sowder (Zothar) wrote: > The most wandering node is darknet only and you know it as Zothar70. > The more calm node is a hybrid darknet/opennet node. That being said, > neither node appears to have a favorite location it comes back to. > OTOH, if I only look at the 48 hour graph rather than the week graph, > the darknet only node appears to be favoring a particular location more > than the hybrid node. > > In any case, I've got the data in an RRD for each node, if it's useful. > Perhaps if we had a "location of the datastore" metric formed by an > average or something, I could export that via FCP and track that in the > RRDs as well. URL for current graphs? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/50f70629/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 01:21:51 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 01:21:51 +0000 Subject: [freenet-dev] Usability testing results Message-ID: <200712110121.57813.toad@amphibian.dyndns.org> I've done some usability testing with a local newbie volunteer. The main conclusions: 1. The website: the download link is not obvious enough! There is a lot of text on the right, and the Download link is lost admid the menu on the left. 2. If java is not installed, the installer pops up a non-fatal error message relating to the registry. http://amphibian.dyndns.org/registry-error.png (I also fixed an issue, which is that we were pointing to the without-a-jvm installer, which was then sending the user to the sun webpage... :( ). 3. The whether-to-enable-opennet decision page: the question at the end of the page is not clear. Perhaps "Do you have at least 5 friends already using Freenet?" 4. Opennet-based Freenet is slow to start with. People may pick this up on the welcome.html, but it's towards the end, and the "next" link is at the top. It would probably be best to mention it on the web interface somehow. 5. Firefox is not optimised for Freenet: it has too few parallel connections, this significantly slows down browsing Freenet, especially if a smart user starts opening stuff in new tabs... This is a major problem, but I'm not sure what we can do about it. It's mentioned in the README (well the 0.5 readme at least), but who reads it? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/40de2e0b/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 01:22:29 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 01:22:29 +0000 Subject: [freenet-dev] Opennet is working; immediate plans Message-ID: <200712110122.29637.toad@amphibian.dyndns.org> Opennet is now working: you can now plug in a node and turn it on, and provided you enable opennet ("connect to untrusted peers") in the wizard, within minutes it will bootstrap itself onto the network. Note that we only have a few hardcoded seednode references at the moment. We will set up an automated harvesting system soon. Also note that we don't want a slashdot right now! We will release a second alpha soon, but we need to make sure opennet is working first, and probably get automatic seednode collection sorted. Hopefully we can do this within the coming weeks, but NOT TODAY! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/97b4a231/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 01:22:33 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 01:22:33 +0000 Subject: [freenet-dev] Freenet 0.7 build 1090 Message-ID: <200712110122.33431.toad@amphibian.dyndns.org> Freenet 0.7 build 1090 is now available. This build is not mandatory, at least not for now, but you should upgrade soon. The main changes in this build include significant improvements to the announcement code (see my other announcement), working multi-homing (i.e. if you have multiple addresses due to e.g. enabling LAN connections), and various optimisations and bugfixes. Please upgrade. If you find any bugs, please report them to the mailing list, the bug tracker or Frost. All testing is appreciated, especially as we may want to release a second alpha in the not too distant future (see the other mail). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/f0bf7c80/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 01:29:09 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 01:29:09 +0000 Subject: [freenet-dev] Auto-detecting connection-limited browsers? was Re: Usability testing results In-Reply-To: <200712110121.57813.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> Message-ID: <200712110129.12827.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 01:21, Matthew Toseland wrote: > I've done some usability testing with a local newbie volunteer. The main > conclusions: > ... > > 5. Firefox is not optimised for Freenet: it has too few parallel connections, > this significantly slows down browsing Freenet, especially if a smart user > starts opening stuff in new tabs... This is a major problem, but I'm not sure > what we can do about it. It's mentioned in the README (well the 0.5 readme at > least), but who reads it? > I wonder if it would be possible to auto-detect this... We could detect how many parallel connections the browser is using, but the problem is an HTTP compliant browser should pipeline requests, (which few do), and use very few connections per server (which firefox does)... ideally we'd have a plugin, which could certainly be detected, but then people start exchanging raw freenet: url's and compatibility gets to be a pain... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/9b8bbe53/attachment.pgp From freenet-devl at david.sowder.com Tue Dec 11 01:58:23 2007 From: freenet-devl at david.sowder.com (David Sowder (Zothar)) Date: Mon, 10 Dec 2007 19:58:23 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712110026.45758.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712102357.51735.toad@amphibian.dyndns.org> <475DD5E6.3020405@david.sowder.com> <200712110026.45758.toad@amphibian.dyndns.org> Message-ID: <475DEEBF.1030200@david.sowder.com> Matthew Toseland wrote: > On Tuesday 11 December 2007 00:12, David Sowder (Zothar) wrote: > >> The most wandering node is darknet only and you know it as Zothar70. >> The more calm node is a hybrid darknet/opennet node. That being said, >> neither node appears to have a favorite location it comes back to. >> OTOH, if I only look at the 48 hour graph rather than the week graph, >> the darknet only node appears to be favoring a particular location more >> than the hybrid node. >> >> In any case, I've got the data in an RRD for each node, if it's useful. >> Perhaps if we had a "location of the datastore" metric formed by an >> average or something, I could export that via FCP and track that in the >> RRDs as well. >> > > URL for current graphs? > Graphs as generated in the last few minutes at http://freenet.xzwq.net/tmp593872/ The "1w" files are for a week, while the non-"1w" files are for 48 hours. From vive at freenetproject.org Tue Dec 11 07:15:17 2007 From: vive at freenetproject.org (vive) Date: Tue, 11 Dec 2007 08:15:17 +0100 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <475DEEBF.1030200@david.sowder.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712102357.51735.toad@amphibian.dyndns.org> <475DD5E6.3020405@david.sowder.com> <200712110026.45758.toad@amphibian.dyndns.org> <475DEEBF.1030200@david.sowder.com> Message-ID: <20071211071517.GA35342@tim.hack.org> On Mon, Dec 10, 2007 at 07:58:23PM -0600, David Sowder (Zothar) wrote: > Matthew Toseland wrote: > > On Tuesday 11 December 2007 00:12, David Sowder (Zothar) wrote: > > > >> The most wandering node is darknet only and you know it as Zothar70. > >> The more calm node is a hybrid darknet/opennet node. That being said, > >> neither node appears to have a favorite location it comes back to. > >> OTOH, if I only look at the 48 hour graph rather than the week graph, > >> the darknet only node appears to be favoring a particular location more > >> than the hybrid node. > >> > >> In any case, I've got the data in an RRD for each node, if it's useful. > >> Perhaps if we had a "location of the datastore" metric formed by an > >> average or something, I could export that via FCP and track that in the > >> RRDs as well. > >> > > > > URL for current graphs? > > > Graphs as generated in the last few minutes at > http://freenet.xzwq.net/tmp593872/ > > The "1w" files are for a week, while the non-"1w" files are for 48 hours. Something interesting to plot could also be the average (sum/neighbors) distance to your peers (on the 0-1 circle). This will show how well the node is adjusted with respect to the position of its neighbors in the network. It would indicate whether the node swaps to a new location due to randomness (many unlikely swap props but finally finding one which works) or due to that peers have also moved. /vive -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/05f2e5b4/attachment.pgp From nextgens at freenetproject.org Tue Dec 11 08:46:05 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 11 Dec 2007 09:46:05 +0100 Subject: [freenet-dev] Auto-detecting connection-limited browsers? was Re: Usability testing results In-Reply-To: <200712110129.12827.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712110129.12827.toad@amphibian.dyndns.org> Message-ID: <20071211084604.GA4233@freenetproject.org> * Matthew Toseland [2007-12-11 01:29:09]: > On Tuesday 11 December 2007 01:21, Matthew Toseland wrote: > > I've done some usability testing with a local newbie volunteer. The main > > conclusions: > > > ... > > > > 5. Firefox is not optimised for Freenet: it has too few parallel > connections, > > this significantly slows down browsing Freenet, especially if a smart user > > starts opening stuff in new tabs... This is a major problem, but I'm not > sure > > what we can do about it. It's mentioned in the README (well the 0.5 readme > at > > least), but who reads it? > > > I wonder if it would be possible to auto-detect this... We could detect how > many parallel connections the browser is using, but the problem is an HTTP > compliant browser should pipeline requests, (which few do), and use very few > connections per server (which firefox does)... ideally we'd have a plugin, > which could certainly be detected, but then people start exchanging raw > freenet: url's and compatibility gets to be a pain... What about finishing my test toadlet ? http://localhost:8888/test/ NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/55ad72f7/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 13:08:24 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 13:08:24 +0000 Subject: [freenet-dev] =?utf-8?q?Auto-detecting_connection-limited_browser?= =?utf-8?q?s=3F_was=09Re=3A_Usability_testing_results?= In-Reply-To: <20071211084604.GA4233@freenetproject.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712110129.12827.toad@amphibian.dyndns.org> <20071211084604.GA4233@freenetproject.org> Message-ID: <200712111308.34130.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 08:46, Florent Daigni?re wrote: > * Matthew Toseland [2007-12-11 01:29:09]: > > > On Tuesday 11 December 2007 01:21, Matthew Toseland wrote: > > > I've done some usability testing with a local newbie volunteer. The main > > > conclusions: > > > > > ... > > > > > > 5. Firefox is not optimised for Freenet: it has too few parallel > > connections, > > > this significantly slows down browsing Freenet, especially if a smart user > > > starts opening stuff in new tabs... This is a major problem, but I'm not > > sure > > > what we can do about it. It's mentioned in the README (well the 0.5 readme > > at > > > least), but who reads it? > > > > > I wonder if it would be possible to auto-detect this... We could detect how > > many parallel connections the browser is using, but the problem is an HTTP > > compliant browser should pipeline requests, (which few do), and use very few > > connections per server (which firefox does)... ideally we'd have a plugin, > > which could certainly be detected, but then people start exchanging raw > > freenet: url's and compatibility gets to be a pain... > > What about finishing my test toadlet ? Good idea. We could maybe include this in the installation wizard? If you can see the green icon, click on it, otherwise we will explain how to reconfigure your browser and retest ? > > http://localhost:8888/test/ > > NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/6a4c8579/attachment.pgp From batosai at batosai.net Tue Dec 11 17:05:56 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 11 Dec 2007 18:05:56 +0100 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712110121.57813.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> Message-ID: <475EC374.6020308@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland a ?crit : > 3. The whether-to-enable-opennet decision page: the question at the end of the > page is not clear. Perhaps "Do you have at least 5 friends already using > Freenet?" I've had exactly the same problem during my own noob-tests of the documentation : the person understood the opennet as being less secure and didn't activate it. He ended up with an unusable node and had to search the documentation why, and then to activate it on the config page. He only did it because I pushed him this way. If not, he would have stopped here and uninstalled Freenet. The question you proposed sounds good to me. I think it should be perfectly clear for the user who has no friends using Freenet (the most common case) that his only way to use it is to enable the Opennet. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHXsN0o6N05NzaOvURAj9zAKCndvE5DGDJ4jNVAZVseDJcMXWT0wCghvcP gfgC4fzgDFIS6xbq0XS+kM8= =B8rQ -----END PGP SIGNATURE----- From Sven-Ola at gmx.de Tue Dec 11 17:10:40 2007 From: Sven-Ola at gmx.de (Sven-Ola =?iso-8859-1?q?T=FCcke?=) Date: Tue, 11 Dec 2007 18:10:40 +0100 Subject: [freenet-dev] Embedded Java Message-ID: <200712111810.41003.Sven-Ola@gmx.de> Hey, I'm an active member of Freifunk Berlin. I'd like to implement a local freenet on embedded devices (mood-300, cheap, 40Gb Disk, MIPS CPU). The vm on the target boxes is named CEE-J which has some compatiblity to the old sun java-1.4 vm. When starting, I got a "Notfound" from the wrapper thingy. Do you think it's worth the time to investiage further? Please mind the attached trace, generated with "java -v -cp freenet.jar:freenet-ext.jar freenet.node.Node > trace.txt 2>&1" Thank you, // Sven-Ola -------------- next part -------------- [loaded java/lang/Object from System.car] [loaded java/lang/Class from System.car] [loaded java/io/Serializable from System.car] [loaded java/lang/String from System.car] [loaded java/lang/Comparable from System.car] [loaded java/lang/Cloneable from System.car] [loaded java/lang/ref/SoftReference from System.car] [loaded java/lang/ref/Reference from System.car] [loaded java/lang/ref/WeakReference from System.car] [loaded java/lang/ref/PhantomReference from System.car] [loaded java/io/Externalizable from System.car] [loaded ceej/jdirect/SharedLibrary from System.car] [loaded java/lang/Throwable from System.car] [loaded java/lang/Error from System.car] [loaded java/lang/OutOfMemoryError from System.car] [loaded java/lang/VirtualMachineError from System.car] [loaded java/lang/NullPointerException from System.car] [loaded java/lang/RuntimeException from System.car] [loaded java/lang/Exception from System.car] [loaded java/lang/NoClassDefFoundError from System.car] [loaded java/lang/LinkageError from System.car] [loaded java/lang/InternalError from System.car] [loaded java/lang/StackOverflowError from System.car] [loaded java/lang/ClassFormatError from System.car] [loaded java/lang/ExceptionInInitializerError from System.car] [loaded java/lang/NoSuchMethodError from System.car] [loaded java/lang/IncompatibleClassChangeError from System.car] [loaded java/lang/System from System.car] [loaded java/io/PrintStream from System.car] [loaded java/io/FilterOutputStream from System.car] [loaded java/io/OutputStream from System.car] [loaded java/io/InputStream from System.car] [loaded java/util/Properties from System.car] [loaded java/util/Hashtable from System.car] [loaded java/util/Dictionary from System.car] [loaded java/util/Map from System.car] [loaded java/util/HashtableEntry from System.car] [loaded java/util/Map$Entry from System.car] [loaded java/io/File from System.car] [loaded ceej/lang/VMSupport from System.car] [loaded java/util/Vector from System.car] [loaded java/util/AbstractList from System.car] [loaded java/util/AbstractCollection from System.car] [loaded java/util/Collection from System.car] [loaded java/util/List from System.car] [loaded java/util/RandomAccess from System.car] [loaded java/lang/StringBuffer from System.car] [loaded java/lang/Boolean from System.car] [loaded java/io/FileInputStream from System.car] [loaded java/io/FileDescriptor from System.car] [loaded java/io/FileOutputStream from System.car] [loaded java/io/PrintWriter from System.car] [loaded java/io/Writer from System.car] [loaded java/io/OutputStreamWriter from System.car] [loaded ceej/io/TextEncodingConverter from System.car] [loaded java/util/Locale from System.car] [loaded java/lang/Character from System.car] [loaded java/lang/ThreadGroup from System.car] [loaded java/lang/Thread from System.car] [loaded java/lang/Runnable from System.car] [loaded java/lang/SecurityManager from System.car] [loaded freenet/node/Node from freenet.jar] [loaded freenet/node/TimeSkewDetectorCallback from freenet.jar] [loaded freenet/node/NodeStarter from freenet.jar] [loaded org/tanukisoftware/wrapper/WrapperListener from freenet-ext.jar] [loaded org/tanukisoftware/wrapper/WrapperManager from freenet-ext.jar] [loaded java/util/ArrayList from System.car] [loaded org/tanukisoftware/wrapper/resources/ResourceManager from freenet-ext.jar] [loaded java/util/ResourceBundle from System.car] [loaded java/lang/ClassNotFoundException from System.car] [loaded java/lang/ClassLoader from System.car] [loaded java/lang/SystemClassLoader from System.car] [loaded java/io/ByteArrayInputStream from System.car] [loaded java/util/PropertyResourceBundle from System.car] [loaded java/io/BufferedInputStream from System.car] [loaded java/io/FilterInputStream from System.car] [loaded org/tanukisoftware/wrapper/WrapperManager$1 from freenet-ext.jar] Error running freenet/node/Node, java/lang/NoClassDefFoundError. [loaded java/lang/Math from System.car] [loaded java/lang/Double from System.car] [loaded java/lang/Number from System.car] java.lang.NoClassDefFoundError: java/security/PrivilegedAction at org/tanukisoftware/wrapper/WrapperManager.()V (WrapperManager.java:325:151) at freenet/node/NodeStarter.main([Ljava/lang/String;)V (NodeStarter.java:261:8) at freenet/node/Node.main([Ljava/lang/String;)V (Node.java:515:4) [loaded java/lang/Runtime from System.car] From batosai at batosai.net Tue Dec 11 17:42:56 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 11 Dec 2007 18:42:56 +0100 Subject: [freenet-dev] French translation update for 1090 Message-ID: <475ECC20.8020205@batosai.net> -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.fr.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071211/dcb84d21/attachment.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/dcb84d21/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 17:49:56 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 17:49:56 +0000 Subject: [freenet-dev] Embedded Java In-Reply-To: <200712111810.41003.Sven-Ola@gmx.de> References: <200712111810.41003.Sven-Ola@gmx.de> Message-ID: <200712111750.02090.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 17:10, Sven-Ola T?cke wrote: > Hey, > > I'm an active member of Freifunk Berlin. I'd like to implement a local freenet > on embedded devices (mood-300, cheap, 40Gb Disk, MIPS CPU). The vm on the > target boxes is named CEE-J which has some compatiblity to the old sun > java-1.4 vm. > > When starting, I got a "Notfound" from the wrapper thingy. Do you think it's > worth the time to investiage further? Please mind the attached trace, > generated with "java -v -cp freenet.jar:freenet-ext.jar freenet.node.Node > > trace.txt 2>&1" For some reason it can't find the node. Are you sure you have the freenet.jar in the classpath? > > Thank you, > // Sven-Ola > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/a74414a2/attachment.pgp From freenet-devl at osndok.com Tue Dec 11 16:52:16 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Tue, 11 Dec 2007 10:52:16 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712110000.19289.toad@amphibian.dyndns.org> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101910.34172.toad@amphibian.dyndns.org> <479F9263-FE2A-4C39-8DB7-3123A79F7924@osndok.com> <200712110000.19289.toad@amphibian.dyndns.org> Message-ID: <9E1801DD-8C2F-494E-AF22-E29191552F27@osndok.com> On Dec 10, 2007, at 6:00 PM, Matthew Toseland wrote: > [...] >> If we are leaving our stores behind, I would predict that for many >> nodes (running suffecient to have meaningful stats) that the >> 'storeDist' value may become arbitrary. And if it approaches the >> 'furthestSuccess' value the store is "left-behind"; whereas the >> cacheDist will maintain approx. the same (once reliable). > > IMHO we need the histogram, or at least a measure of spread, not > just an > average key. >> Do you want this in the trunk? I don't think that I have the time to >> do much data collection. > > Your diff is empty. Might be a problem on my end though. It came up alright in the message coming back to me through the list, but I've pasted it on the end of the message to avoid any attachment trouble. On Dec 9, 2007, at 11:47 AM, David Sowder (Zothar) wrote: > As I see it currently, stuff being kicked out of the datastore isn't > happening except when the datastore is being shrunk since AFAIK, > nobody's datastore is filling, only their datacache is. On Dec 10, 2007, at 11:27 AM, Robert Hailey wrote: >> Not true. We have mechanisms to get popular data back into the >> datastore. The >> main one is 1 in every 100 successful requests gets turned into an >> insert >> post-completion. > > > That is intriguing. I wonder how much of my 0.3% store-hit-successes > (my present value) are actually blocks recently-inserted from the 1% > healing (i.e. bleed-over from the cache). I've got another idea from the code I saw while forming this stats diff... Whereas: (1) presently we always check the datastore before the datacache, (2) the datacache has a 8-9x better hit rate, (3) I suspect that the hits from the store may actually be in the cache as well I think that we should reverse the fetch()es to try the cache first (since it is more likely anyway). Certainly we are running on caches alone, no? If my suspicion is correct, the false-hits for the data store will be removed, and the store-hit-rate may tank to 0.000%. Arguably it would not promote the data in the store, but presently it doesn't promote it in the cache. -- Robert Hailey Index: src/freenet/node/NodeStats.java =================================================================== --- src/freenet/node/NodeStats.java (revision 16462) +++ src/freenet/node/NodeStats.java (working copy) @@ -22,6 +22,7 @@ import freenet.support.api.IntCallback; import freenet.support.api.LongCallback; import freenet.support.math.RunningAverage; +import freenet.support.math.SimpleRunningAverage; import freenet.support.math.TimeDecayingRunningAverage; import freenet.support.math.TrivialRunningAverage; @@ -144,8 +145,17 @@ // various metrics public final RunningAverage routingMissDistance; public final RunningAverage backedOffPercent; + public final RunningAverage avgCacheLocation; + public final RunningAverage avgStoreLocation; + public final RunningAverage avgCacheSuccess; + public final RunningAverage avgStoreSuccess; + // FIXME: does furthest{Store,Cache}Success need to be synchronized? + public double furthestCacheSuccess=0.0; + public double furthestStoreSuccess=0.0; protected final Persister persister; + protected final RunningAverage avgRequestLocation; + // ThreadCounting stuffs public final ThreadGroup rootThreadGroup; private int threadLimit; @@ -176,6 +186,14 @@ this.hardRandom = node.random; this.routingMissDistance = new TimeDecayingRunningAverage(0.0, 180000, 0.0, 1.0, node); this.backedOffPercent = new TimeDecayingRunningAverage(0.0, 180000, 0.0, 1.0, node); + // FIXME PLEASE remove (int) casts + double nodeLoc=node.lm.getLocation(); + this.avgCacheLocation=new SimpleRunningAverage((int)node.maxCacheKeys, nodeLoc); + this.avgStoreLocation=new SimpleRunningAverage((int)node.maxStoreKeys, nodeLoc); + // FIXME average for success-location may not need to be so large as the store. + this.avgCacheSuccess=new SimpleRunningAverage(10000, nodeLoc); + this.avgStoreSuccess=new SimpleRunningAverage(10000, nodeLoc); + this.avgRequestLocation=new SimpleRunningAverage(10000, nodeLoc); preemptiveRejectReasons = new StringCounter(); localPreemptiveRejectReasons = new StringCounter(); pInstantRejectIncoming = new TimeDecayingRunningAverage(0, 60000, 0.0, 1.0, node); @@ -186,6 +204,18 @@ new TimeDecayingRunningAverage(1, 10*60*1000 /* should be significantly longer than a typical transfer */, 0, Long.MAX_VALUE, node); nodePinger = new NodePinger(node); + // FIXME: data-store/cache averages need to be persisted to be valuable (or scanned at every launch). + /* + if (node.isAdvancedModeEnabled()) { + //Uggghh.... + System.err.println("Scanning datastore/cache for location values"); + chkDatastore.kludgeScan(avgStoreLocation); + sskDatastore.kludgeScan(avgStoreLocation); + chkDatacache.kludgeScan(avgCacheLocation); + sskDatacache.kludgeScan(avgCacheLocation); + } + */ + previous_input_stat = 0; previous_output_stat = 0; previous_io_stat_time = 1; Index: src/freenet/node/Node.java =================================================================== --- src/freenet/node/Node.java (revision 16462) +++ src/freenet/node/Node.java (working copy) @@ -247,8 +247,8 @@ /** The maximum number of keys stored in each of the datastores, cache and store combined. */ private long maxTotalKeys; - private long maxCacheKeys; - private long maxStoreKeys; + long maxCacheKeys; + long maxStoreKeys; /** The maximum size of the datastore. Kept to avoid rounding turning 5G into 5368698672 */ private long maxTotalDatastoreSize; /** If true, store shrinks occur immediately even if they are over 10% of the store size. If false, @@ -1767,11 +1767,23 @@ public SSKBlock fetch(NodeSSK key, boolean dontPromote) { if(logMINOR) dumpStoreHits(); try { + double loc=key.toNormalizedDouble(); + double dist=Location.distance(lm.getLocation(), loc); + nodeStats.avgRequestLocation.report(loc); SSKBlock block = sskDatastore.fetch(key, dontPromote); if(block != null) { + nodeStats.avgStoreSuccess.report(loc); + if (dist > nodeStats.furthestStoreSuccess) + nodeStats.furthestStoreSuccess=dist; return block; } - return sskDatacache.fetch(key, dontPromote); + block=sskDatacache.fetch(key, dontPromote); + if (block != null) { + nodeStats.avgCacheSuccess.report(loc); + if (dist > nodeStats.furthestCacheSuccess) + nodeStats.furthestCacheSuccess=dist; + } + return block; } catch (IOException e) { Logger.error(this, "Cannot fetch data: "+e, e); return null; @@ -1781,9 +1793,23 @@ public CHKBlock fetch(NodeCHK key, boolean dontPromote) { if(logMINOR) dumpStoreHits(); try { + double loc=key.toNormalizedDouble(); + double dist=Location.distance(lm.getLocation(), loc); + nodeStats.avgRequestLocation.report(loc); CHKBlock block = chkDatastore.fetch(key, dontPromote); - if(block != null) return block; - return chkDatacache.fetch(key, dontPromote); + if (block != null) { + nodeStats.avgStoreSuccess.report(loc); + if (dist > nodeStats.furthestStoreSuccess) + nodeStats.furthestStoreSuccess=dist; + return block; + } + block=chkDatacache.fetch(key, dontPromote); + if (block != null) { + nodeStats.avgCacheSuccess.report(loc); + if (dist > nodeStats.furthestCacheSuccess) + nodeStats.furthestCacheSuccess=dist; + } + return block; } catch (IOException e) { Logger.error(this, "Cannot fetch data: "+e, e); return null; @@ -1835,10 +1861,13 @@ private void store(CHKBlock block, boolean deep) { try { + double loc=block.getKey().toNormalizedDouble(); if(deep) { chkDatastore.put(block); + nodeStats.avgStoreLocation.report(loc); } chkDatacache.put(block); + nodeStats.avgCacheLocation.report(loc); if(clientCore != null && clientCore.requestStarters != null) clientCore.requestStarters.chkFetchScheduler.tripPendingKey(block); } catch (IOException e) { @@ -2498,6 +2527,10 @@ return usm; } + public double getNodeLocation() { + return lm.getLocation(); + } + public int getSwaps() { return LocationManager.swaps; } Index: src/freenet/clients/http/StatisticsToadlet.java =================================================================== --- src/freenet/clients/http/StatisticsToadlet.java (revision 16462) +++ src/freenet/clients/http/StatisticsToadlet.java (working copy) @@ -29,6 +29,7 @@ import freenet.support.SizeUtil; import freenet.support.TimeUtil; import freenet.support.api.HTTPRequest; +import freenet.support.math.SimpleRunningAverage; public class StatisticsToadlet extends Toadlet { @@ -422,8 +423,47 @@ "\u00a0(" + ((storeHits*100) / (storeAccesses)) + "%)"); storeSizeList.addChild("li", - "Avg. access rate:\u00a0" + thousendPoint.format(overallAccesses/ nodeUptimeSeconds) + "/sec"); + "Avg. access rate:\u00a0" + thousendPoint.format(cacheAccesses/ nodeUptimeSeconds) + "/sec, "+thousendPoint.format(storeAccesses/ nodeUptimeSeconds)+"/sec"); + // location-based stats + boolean hasLoc=true; + double nodeLoc=0.0; + try { + nodeLoc=node.getNodeLocation(); + } catch (Error e) { + //FIXME: PLEASE, how do we get the node location on the stats page? + //Logger.error(this, "why?", e); + e.printStackTrace(); + hasLoc=false; + } + double avgCacheLocation=node.nodeStats.avgCacheLocation.currentValue(); + double avgStoreLocation=node.nodeStats.avgStoreLocation.currentValue(); + double avgCacheSuccess=node.nodeStats.avgCacheSuccess.currentValue(); + double avgStoreSuccess=node.nodeStats.avgStoreSuccess.currentValue(); + double furthestCacheSuccess=node.nodeStats.furthestCacheSuccess; + double furthestStoreSuccess=node.nodeStats.furthestStoreSuccess; + double storeDist=Location.distance(nodeLoc, avgStoreLocation); + double cacheDist=Location.distance(nodeLoc, avgCacheLocation); + + storeSizeList.addChild("li", "avgCacheLocation:\u00a0" + thousendPoint.format(avgCacheLocation)); + storeSizeList.addChild("li", "avgStoreLocation:\u00a0" + thousendPoint.format(avgStoreLocation)); + + storeSizeList.addChild("li", "avgCacheSuccess:\u00a0" + thousendPoint.format(avgCacheSuccess)); + storeSizeList.addChild("li", "avgStoreSuccess:\u00a0" + thousendPoint.format(avgStoreSuccess)); + + storeSizeList.addChild("li", "furthestCacheSuccess:\u00a0" + thousendPoint.format(furthestCacheSuccess)); + storeSizeList.addChild("li", "furthestStoreSuccess:\u00a0" + thousendPoint.format(furthestStoreSuccess)); + + if (hasLoc) { + storeSizeList.addChild("li", "cacheDist:\u00a0" + thousendPoint.format(cacheDist)); + storeSizeList.addChild("li", "storeDist:\u00a0" + thousendPoint.format(storeDist)); + long cacheLocationReports =((SimpleRunningAverage)node.nodeStats.avgCacheLocation).countReports(); + long storeLocationReports =((SimpleRunningAverage)node.nodeStats.avgStoreLocation).countReports(); + double cachePrimePercent=((1.0*cacheLocationReports)/cachedKeys); + double storePrimePercent=((1.0*storeLocationReports)/storeKeys); + storeSizeList.addChild("li", "locStatsReliability: \u00a0"+fix3p1pct.format(cachePrimePercent)+" / "+fix3p1pct.format(storePrimePercent)); + } + } private void drawUnclaimedFIFOMessageCountsBox(HTMLNode unclaimedFIFOMessageCountsInfobox) { ------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071211/ce2e454a/attachment.htm From nextgens at freenetproject.org Tue Dec 11 17:52:21 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 11 Dec 2007 18:52:21 +0100 Subject: [freenet-dev] Embedded Java In-Reply-To: <200712111750.02090.toad@amphibian.dyndns.org> References: <200712111810.41003.Sven-Ola@gmx.de> <200712111750.02090.toad@amphibian.dyndns.org> Message-ID: <20071211175220.GB4233@freenetproject.org> * Matthew Toseland [2007-12-11 17:49:56]: > On Tuesday 11 December 2007 17:10, Sven-Ola T?cke wrote: > > Hey, > > > > I'm an active member of Freifunk Berlin. I'd like to implement a local > freenet > > on embedded devices (mood-300, cheap, 40Gb Disk, MIPS CPU). The vm on the > > target boxes is named CEE-J which has some compatiblity to the old sun > > java-1.4 vm. > > > > When starting, I got a "Notfound" from the wrapper thingy. Do you think it's > > worth the time to investiage further? Please mind the attached trace, > > generated with "java -v -cp freenet.jar:freenet-ext.jar freenet.node.Node > > > trace.txt 2>&1" > > For some reason it can't find the node. Are you sure you have the freenet.jar > in the classpath? You didn't look at the trace, did you ? :) For some reason it can't find the security classes of the jvm... Implementing them (or some mock classes) would be the solution to that problem... It's unlikely that it will be the only problem though :| NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/8dc1fece/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 11 19:02:12 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 19:02:12 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <475EC374.6020308@batosai.net> References: <200712110121.57813.toad@amphibian.dyndns.org> <475EC374.6020308@batosai.net> Message-ID: <200712111902.17457.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 17:05, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > > 3. The whether-to-enable-opennet decision page: the question at the end of the > > page is not clear. Perhaps "Do you have at least 5 friends already using > > Freenet?" > > I've had exactly the same problem during my own noob-tests of the > documentation : the person understood the opennet as being less secure > and didn't activate it. He ended up with an unusable node and had to > search the documentation why, and then to activate it on the config > page. He only did it because I pushed him this way. If not, he would > have stopped here and uninstalled Freenet. > > The question you proposed sounds good to me. I think it should be > perfectly clear for the user who has no friends using Freenet (the most > common case) that his only way to use it is to enable the Opennet. How about this?: Connect to strangers? Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for an attacker to find your node. However, if you don't know at least 5 people already running Freenet nodes, you should turn on Promiscuous Mode so that the node will automatically connect to the network using Strangers. Note that you can always turn off Promiscuous Mode later. Do you know anyone already using Freenet? Activate Promiscuous Mode: I want the node to automatically find nodes to connect to. Do not activate promiscuous mode: I have at least 5 friends already using Freenet and I will add them on the Friends page. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/306401e6/attachment.pgp From freenet-devl at osndok.com Tue Dec 11 21:22:03 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Tue, 11 Dec 2007 15:22:03 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <475DD5E6.3020405@david.sowder.com> References: <200711300126.45810.toad@amphibian.dyndns.org> <200712101910.34172.toad@amphibian.dyndns.org> <475DA38D.5080708@david.sowder.com> <200712102357.51735.toad@amphibian.dyndns.org> <475DD5E6.3020405@david.sowder.com> Message-ID: <5F250A97-0602-4CD9-A63D-0732DCD7ABCF@osndok.com> On Dec 10, 2007, at 6:12 PM, David Sowder (Zothar) wrote: > The most wandering node is darknet only and you know it as Zothar70. > The more calm node is a hybrid darknet/opennet node. That being said, > neither node appears to have a favorite location it comes back to. Presumably, the darknet-only node has fewer total connections. Being that all edges are counted equally, the location swapping implementation appears to favor those nodes with more connections. That is, it tries to reduce 'total' edge stress, not edge-stress-per- node. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071211/6da50649/attachment.htm From jolantern60 at yahoo.com Tue Dec 11 23:24:12 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Tue, 11 Dec 2007 15:24:12 -0800 (PST) Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <20071209224024.GA4199@freenetproject.org> Message-ID: <128302.13989.qm@web44812.mail.sp1.yahoo.com> --- Florent Daigni?re wrote: > There is at least a fourth one we are immune to in .7... The "If I > read anything looking like a RSS feed, I decide to ignore the > mime-type" one. Thanks for this information. I take it this is true for ATOM feeds to? Jack ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ From jolantern60 at yahoo.com Tue Dec 11 23:25:21 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Tue, 11 Dec 2007 15:25:21 -0800 (PST) Subject: [freenet-dev] Usability testing results In-Reply-To: <200712110121.57813.toad@amphibian.dyndns.org> Message-ID: <931525.79026.qm@web44804.mail.sp1.yahoo.com> --- Matthew Toseland wrote: > is a major problem, but I'm not sure what we can do about it. It's > mentioned in the README (well the 0.5 readme at least), but who > reads it? The web interface of 0.5 has a prominent link to the README in the Node Information box on the left. That should sufficiently entice users to read it. Jack ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping From jolantern60 at yahoo.com Tue Dec 11 23:27:28 2007 From: jolantern60 at yahoo.com (Jack O'Lantern) Date: Tue, 11 Dec 2007 15:27:28 -0800 (PST) Subject: [freenet-dev] Mozilla/Gecko browser features destroying your anonymity on freenet? In-Reply-To: <200712101903.43461.toad@amphibian.dyndns.org> Message-ID: <74785.25564.qm@web44814.mail.sp1.yahoo.com> --- Matthew Toseland wrote: > Hmmm I still don't understand - show an error message if the user > types in a domain? There are fraudsters out there who set up sites which look exactly like some legitimate site. If you try to log on, they grab your credentials and perform actions on the legitimate site in your name (such as transferring money from your bank account to . These people use domain names which look very similar or, with some fonts, even equal to the domain names of the legitimate sites. If a safebrowsing provider is activated, it receives every URL you type in and tells your browser whether the site is dangerous. The browser may then show a warning message. Jack ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping From toad at amphibian.dyndns.org Tue Dec 11 23:57:08 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Dec 2007 23:57:08 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <931525.79026.qm@web44804.mail.sp1.yahoo.com> References: <931525.79026.qm@web44804.mail.sp1.yahoo.com> Message-ID: <200712112357.13938.toad@amphibian.dyndns.org> On Tuesday 11 December 2007 23:25, Jack O'Lantern wrote: > --- Matthew Toseland wrote: > > is a major problem, but I'm not sure what we can do about it. It's > > mentioned in the README (well the 0.5 readme at least), but who > > reads it? > > The web interface of 0.5 has a prominent link to the README in the Node > Information box on the left. That should sufficiently entice users to > read it. Does it work? > > Jack -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071211/503973ff/attachment.pgp From m.rogers at cs.ucl.ac.uk Wed Dec 12 00:06:19 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Wed, 12 Dec 2007 00:06:19 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712111902.17457.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <475EC374.6020308@batosai.net> <200712111902.17457.toad@amphibian.dyndns.org> Message-ID: <475F25FB.6050909@cs.ucl.ac.uk> Matthew Toseland wrote: > Activate Promiscuous Mode: I want the node to automatically find nodes to > connect to. > > Do not activate promiscuous mode: I have at least 5 friends already using > Freenet and I will add them on the Friends page. We seem to have three sets of terminology: darknet/opennet, friends/strangers, and promiscuous mode. Perhaps we should pick one set and use it everywhere? (I'd vote for friends/strangers as it's self-explanatory.) Cheers, Michael From toad at amphibian.dyndns.org Wed Dec 12 00:15:59 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 12 Dec 2007 00:15:59 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <475F25FB.6050909@cs.ucl.ac.uk> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> Message-ID: <200712120016.04506.toad@amphibian.dyndns.org> On Wednesday 12 December 2007 00:06, Michael Rogers wrote: > Matthew Toseland wrote: > > Activate Promiscuous Mode: I want the node to automatically find nodes to > > connect to. > > > > Do not activate promiscuous mode: I have at least 5 friends already using > > Freenet and I will add them on the Friends page. > > We seem to have three sets of terminology: darknet/opennet, > friends/strangers, and promiscuous mode. Perhaps we should pick one set > and use it everywhere? (I'd vote for friends/strangers as it's > self-explanatory.) Agreed, we should be more consistent. But friends/strangers makes for awkward wording here, doesn't it? And on the homepage opennet-is-enabled warning too? Hmmm, lets see... "Connect to strangers? Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for an attacker to find your node. However, if you don't know at least 5 people already running Freenet nodes, you can instruct the node to connect to nodes run by people you don't know (Strangers). Note that you can always turn this off later. Do you know anyone already using Freenet? Yes, I have at least 5 friends already running Freenet and I will add their nodes on the Friends page. No, I want the node to automatically find untrusted nodes (aka Strangers) to connect to." Is this clearer than the previous wording? > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071212/0547a057/attachment.pgp From m.rogers at cs.ucl.ac.uk Wed Dec 12 00:30:22 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Wed, 12 Dec 2007 00:30:22 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712120016.04506.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> Message-ID: <475F2B9E.5040608@cs.ucl.ac.uk> Matthew Toseland wrote: > "Connect to strangers? > > Ideally, all Freenet users would connect only to people they know. This is far > more secure, making it very difficult for an attacker to find your node. > However, if you don't know at least 5 people already running Freenet nodes, > you can instruct the node to connect to nodes run by people you don't know > (Strangers). Note that you can always turn this off later. > > Do you know anyone already using Freenet? > > Yes, I have at least 5 friends already running Freenet and I will add their > nodes on the Friends page. > No, I want the node to automatically find untrusted nodes (aka Strangers) to > connect to." > > Is this clearer than the previous wording? Sounds good to me. Maybe we can even get rid of "node" for the most part, since that's a technical term? How about this: ... This is far more secure, making it very difficult for others to tell that you are using Freenet. However, if you don't know at least 5 people already running Freenet, you can choose to connect to strangers... ...Yes, I have at least 5 friends already running Freenet and I will enter their details on the Friends page. No, I want Freenet to automatically find strangers to connect to. Cheers, Michael From batosai at batosai.net Wed Dec 12 10:15:28 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Wed, 12 Dec 2007 11:15:28 +0100 Subject: [freenet-dev] Usability testing results In-Reply-To: <475F2B9E.5040608@cs.ucl.ac.uk> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> Message-ID: <475FB4C0.2010101@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Rogers a ?crit : > Matthew Toseland wrote: >> "Connect to strangers? >> >> Ideally, all Freenet users would connect only to people they know. This is far >> more secure, making it very difficult for an attacker to find your node. >> However, if you don't know at least 5 people already running Freenet nodes, >> you can instruct the node to connect to nodes run by people you don't know >> (Strangers). Note that you can always turn this off later. >> >> Do you know anyone already using Freenet? >> >> Yes, I have at least 5 friends already running Freenet and I will add their >> nodes on the Friends page. >> No, I want the node to automatically find untrusted nodes (aka Strangers) to >> connect to." >> >> Is this clearer than the previous wording? > > Sounds good to me. Maybe we can even get rid of "node" for the most > part, since that's a technical term? How about this: > > ... This is far more secure, making it very difficult for others to tell > that you are using Freenet. However, if you don't know at least 5 people > already running Freenet, you can choose to connect to strangers... > ...Yes, I have at least 5 friends already running Freenet and I will > enter their details on the Friends page. > No, I want Freenet to automatically find strangers to connect to. I like that. Even if we are using "node" in several other messages, that version is more "understandable". I would just make things a bit clearer with : ...However, if you don't know at least 5 people already running Freenet, your only way to use Freenet is to connect to strangers... I would also add a warning for those living in countries where Freenet is illegal : Note : If Freenet is illegal in your country, then connecting to strangers could permit your government to know that you are using it. You should try to convince your friends to install Freenet and only connect to them. We would end up with someting like that : "Connect to strangers? Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for others to tell that you are using Freenet. However, if you don't know at least 5 people already running Freenet, your only way to use Freenet is to connect to strangers. Note that you can always turn this off later. Do you know anyone already using Freenet? Yes, I have at least 5 friends already running Freenet and I will enter their details on the Friends page. No, I want Freenet to automatically find strangers to connect to. Note : If Freenet is illegal in your country, then connecting to strangers could permit your government to know that you are using it. You should try to convince your friends to install Freenet and only connect to them." -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHX7TAo6N05NzaOvURAohvAJ4urjO4O7czXJ29+2vmm7K+1NA13ACdGZAg ytcdSPe+eiI9k/TdLFYMoxA= =MatD -----END PGP SIGNATURE----- From freenetwork at web.de Wed Dec 12 10:41:44 2007 From: freenetwork at web.de (freenetwork at web.de) Date: Wed, 12 Dec 2007 11:41:44 +0100 Subject: [freenet-dev] Usability testing results In-Reply-To: <475F2B9E.5040608@cs.ucl.ac.uk> Message-ID: On Wed, 12 Dec 2007 00:30:22 +0000, Michael Rogers wrote: >Matthew Toseland wrote: >> "Connect to strangers? >> >> Ideally, all Freenet users would connect only to people they know. This is far >> more secure, making it very difficult for an attacker to find your node. >> However, if you don't know at least 5 people already running Freenet nodes, >> you can instruct the node to connect to nodes run by people you don't know >> (Strangers). Note that you can always turn this off later. >> >> Do you know anyone already using Freenet? >> >> Yes, I have at least 5 friends already running Freenet and I will add their >> nodes on the Friends page. >> No, I want the node to automatically find untrusted nodes (aka Strangers) to >> connect to." >> >> Is this clearer than the previous wording? >Sounds good to me. Maybe we can even get rid of "node" for the most >part, since that's a technical term? How about this: >... This is far more secure, making it very difficult for others to tell >that you are using Freenet. However, if you don't know at least 5 people >already running Freenet, you can choose to connect to strangers... >...Yes, I have at least 5 friends already running Freenet and I will >enter their details on the Friends page. >No, I want Freenet to automatically find strangers to connect to. Well. For my personal taste it is necessary to be technical to a degree. To propose to use "Freenet" for everything is not correct. There is - the "Freenet Project" - the overall project, just like "Mozilla Firefox", "Microsoft Windows", etc. - the "Freenet" - the network and it's structure - like "the Internet" - the "Freenet", again - as the complete concept how the network shall work - the "node" - a more theoretical descrition of a participant, a singe "point", of and within the network - the node's implementation "FRED" ("Freenet REference Daemon") - the java program within the .jar, that, when run, provides functionalies of a node - the node's HTTP-user interface "FProxy" - to browse freesites, enter other node/FRED references, see statistics, etc. and you want to smush everything together to a single term "Freenet" ?!! Like "I downloaded teh Internet" ;) From toad at amphibian.dyndns.org Wed Dec 12 13:08:03 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 12 Dec 2007 13:08:03 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: References: Message-ID: <200712121308.11218.toad@amphibian.dyndns.org> On Wednesday 12 December 2007 10:41, freenetwork at web.de wrote: > Well. For my personal taste it is necessary to be technical to a degree. > To propose to use "Freenet" for everything is not correct. > > There is > - the "Freenet Project" - the overall project, just like "Mozilla Firefox", "Microsoft Windows", etc. > - the "Freenet" - the network and it's structure - like "the Internet" > - the "Freenet", again - as the complete concept how the network shall work > - the "node" - a more theoretical descrition of a participant, a singe "point", of and within the network > - the node's implementation "FRED" ("Freenet REference Daemon") - the java program within the .jar, that, when run, provides functionalies of a node > - the node's HTTP-user interface "FProxy" - to browse freesites, enter other node/FRED references, see statistics, etc. > > and you want to smush everything together to a single term "Freenet" ?!! No, in this case it is being used as "the application Freenet", which is quite specific and reasonably obvious to the user. > > Like "I downloaded teh Internet" ;) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071212/e074fefb/attachment.pgp From batosai at batosai.net Wed Dec 12 13:41:52 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Wed, 12 Dec 2007 14:41:52 +0100 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712121308.11218.toad@amphibian.dyndns.org> References: <200712121308.11218.toad@amphibian.dyndns.org> Message-ID: <475FE520.60304@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland a ?crit : > On Wednesday 12 December 2007 10:41, freenetwork at web.de wrote: >> Well. For my personal taste it is necessary to be technical to a degree. >> To propose to use "Freenet" for everything is not correct. >> >> There is >> - the "Freenet Project" - the overall project, just like "Mozilla > Firefox", "Microsoft Windows", etc. >> - the "Freenet" - the network and it's structure - like "the Internet" >> - the "Freenet", again - as the complete concept how the network shall work >> - the "node" - a more theoretical descrition of a participant, a > singe "point", of and within the network >> - the node's implementation "FRED" ("Freenet REference Daemon") - the java > program within the .jar, that, when run, provides functionalies of a node >> - the node's HTTP-user interface "FProxy" - to browse freesites, enter other > node/FRED references, see statistics, etc. >> and you want to smush everything together to a single term "Freenet" ?!! > > No, in this case it is being used as "the application Freenet", which is quite > specific and reasonably obvious to the user. Agreed. We can be as technical as we want in the documentation, but I think the main interface should be as simple as possible. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHX+Ugo6N05NzaOvURAi3GAJ96fVpZtgLO/DJAACnnLGBK5IQuIgCfcO0x 8T90A+BhqgYMp4KDFLMvbJA= =S4u9 -----END PGP SIGNATURE----- From freenet-devl at osndok.com Wed Dec 12 21:31:06 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Wed, 12 Dec 2007 15:31:06 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data References: Message-ID: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > Ok that could be interesting. Although ideally we'd have a > circular-keyspace-aware averager. >> ... > Ok. I suggest you commit, I will review post-commit. Committed it r16508, though it has changed slightly according to my experiments with swap-biasing (e.g. shows number of cache/store writes as well). From what I have discovered (or theorized) thus far, using the average location of the entire store to bias against is way too much of an anchor. This is good, because it takes up way to much memory to remember such a running average anyway. If we end up implementing such a bias, in the end, it will likely just take into account the last few inserts or successes (a small constant amount). In this way, the pressure of your peers can still pull you into a new location (dragging the weight of the last few inserts with you; which will be updated). Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), after running with a recently-stored-location bias, I must have pulled my peers closer to me as well, as now even with the patch off, I do not see the storeDistance go nearly so high (staying around 0.004). Or, since the network has no anchor, maybe I've pressured the whole wheel to stop turning. For at least my node; this (valuing the store as one peer) has not shown any increase in store hit rates, biasing against the succeeding requests, however, has in the past raised it to about 1%; but I would be even more concerning with implementing that. With these stats I have noticed what may be an odd constant-location- shift between what the node is requested to insert and retrieve, like the network does not look for data in the exact same place it stores it?!?!?! Suspect, but still can't confirm anything yet. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071212/a6612ca8/attachment.htm From juiceman69 at gmail.com Wed Dec 12 22:51:04 2007 From: juiceman69 at gmail.com (Juiceman) Date: Wed, 12 Dec 2007 17:51:04 -0500 Subject: [freenet-dev] Usability testing results In-Reply-To: <475FE520.60304@batosai.net> References: <200712121308.11218.toad@amphibian.dyndns.org> <475FE520.60304@batosai.net> Message-ID: <8b525dee0712121451s5f5e7ac1q9842e9693fbe7f00@mail.gmail.com> On Dec 12, 2007 8:41 AM, Julien Cornuwel wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Matthew Toseland a ?crit : > > > On Wednesday 12 December 2007 10:41, freenetwork at web.de wrote: > >> Well. For my personal taste it is necessary to be technical to a degree. > >> To propose to use "Freenet" for everything is not correct. > >> > >> There is > >> - the "Freenet Project" - the overall project, just like "Mozilla > > Firefox", "Microsoft Windows", etc. > >> - the "Freenet" - the network and it's structure - like "the Internet" > >> - the "Freenet", again - as the complete concept how the network shall work > >> - the "node" - a more theoretical descrition of a participant, a > > singe "point", of and within the network > >> - the node's implementation "FRED" ("Freenet REference Daemon") - the java > > program within the .jar, that, when run, provides functionalies of a node > >> - the node's HTTP-user interface "FProxy" - to browse freesites, enter other > > node/FRED references, see statistics, etc. > >> and you want to smush everything together to a single term "Freenet" ?!! > > > > No, in this case it is being used as "the application Freenet", which is quite > > specific and reasonably obvious to the user. I agree. Remember - This is possibly a users first contact with Freenet. As users get deeper in and more exposed they will learn the technical terminology. This needs to be as simple and clear as possible. -- I may disagree with what you have to say, but I shall defend, to the death, your right to say it. - Voltaire Those who would give up Liberty, to purchase temporary Safety, deserve neither Liberty nor Safety. - Ben Franklin From juiceman69 at gmail.com Wed Dec 12 23:20:50 2007 From: juiceman69 at gmail.com (Juiceman) Date: Wed, 12 Dec 2007 18:20:50 -0500 Subject: [freenet-dev] Usability testing results In-Reply-To: <475FB4C0.2010101@batosai.net> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <475FB4C0.2010101@batosai.net> Message-ID: <8b525dee0712121520s7f0581bfpcd8cf4b7e59e052b@mail.gmail.com> On Dec 12, 2007 5:15 AM, Julien Cornuwel wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Michael Rogers a ?crit : > > Matthew Toseland wrote: > >> "Connect to strangers? > >> > >> Ideally, all Freenet users would connect only to people they know. This is far > >> more secure, making it very difficult for an attacker to find your node. > >> However, if you don't know at least 5 people already running Freenet nodes, > >> you can instruct the node to connect to nodes run by people you don't know > >> (Strangers). Note that you can always turn this off later. > >> > >> Do you know anyone already using Freenet? > >> > >> Yes, I have at least 5 friends already running Freenet and I will add their > >> nodes on the Friends page. > >> No, I want the node to automatically find untrusted nodes (aka Strangers) to > >> connect to." > >> > >> Is this clearer than the previous wording? > > > > Sounds good to me. Maybe we can even get rid of "node" for the most > > part, since that's a technical term? How about this: > > > > ... This is far more secure, making it very difficult for others to tell > > that you are using Freenet. However, if you don't know at least 5 people > > already running Freenet, you can choose to connect to strangers... > > ...Yes, I have at least 5 friends already running Freenet and I will > > enter their details on the Friends page. > > No, I want Freenet to automatically find strangers to connect to. > > I like that. Even if we are using "node" in several other messages, that > version is more "understandable". I would just make things a bit clearer > with : > > ...However, if you don't know at least 5 people already running Freenet, > your only way to use Freenet is to connect to strangers... > > > I would also add a warning for those living in countries where Freenet > is illegal : > > Note : If Freenet is illegal in your country, then connecting to > strangers could permit your government to know that you are using it. > You should try to convince your friends to install Freenet and only > connect to them. > > > We would end up with someting like that : > > "Connect to strangers? > > Ideally, all Freenet users would connect only to people they know. This > is far more secure, making it very difficult for others to tell that you > are using Freenet. However, if you don't know at least 5 people already > running Freenet, your only way to use Freenet is to connect to > strangers. Note that you can always turn this off later. > > Do you know anyone already using Freenet? > > Yes, I have at least 5 friends already running Freenet and I will enter > their details on the Friends page. > No, I want Freenet to automatically find strangers to connect to. > > Note : If Freenet is illegal in your country, then connecting to > strangers could permit your government to know that you are using it. > You should try to convince your friends to install Freenet and only > connect to them." I would add right at the end of this "Technically you can start by only connecting to one friend, but your node won't work at all when they are not online and even when they are online it won't perform well until you get more friends. -- I may disagree with what you have to say, but I shall defend, to the death, your right to say it. - Voltaire Those who would give up Liberty, to purchase temporary Safety, deserve neither Liberty nor Safety. - Ben Franklin From edt at aei.ca Thu Dec 13 01:58:26 2007 From: edt at aei.ca (Ed Tomlinson) Date: Wed, 12 Dec 2007 20:58:26 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> References: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> Message-ID: <200712122058.27221.edt@aei.ca> On December 12, 2007, Robert Hailey wrote: > > On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > > Ok that could be interesting. Although ideally we'd have a > > circular-keyspace-aware averager. > >> ... > > Ok. I suggest you commit, I will review post-commit. > > Committed it r16508, though it has changed slightly according to my > experiments with swap-biasing (e.g. shows number of cache/store writes > as well). Were should this appear? Does it take a while to show up? I have r16509 running and am not seeing anything new (yet). > From what I have discovered (or theorized) thus far, using the > average location of the entire store to bias against is way too much > of an anchor. This is good, because it takes up way to much memory to > remember such a running average anyway. If we end up implementing such > a bias, in the end, it will likely just take into account the last few > inserts or successes (a small constant amount). In this way, the > pressure of your peers can still pull you into a new location > (dragging the weight of the last few inserts with you; which will be > updated). Keeping the average key location and standard deviation just requires a few counters along hooks to update them when adding or deleting the store... It also requires a single pass to get the initial numbers. Should not be expensive except for that first pass. I agree that using the average would be very restrictive. Average plus or minus standard deviation would give a much better 'limit'. I suspect that It would take a fair length of time for it to start limiting at all (e.g. 2 x standard deviation would be greater than 1). > Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), > after running with a recently-stored-location bias, I must have pulled > my peers closer to me as well, as now even with the patch off, I do > not see the storeDistance go nearly so high (staying around 0.004). > Or, since the network has no anchor, maybe I've pressured the whole > wheel to stop turning. For at least my node; this (valuing the store > as one peer) has not shown any increase in store hit rates, biasing > against the succeeding requests, however, has in the past raised it to > about 1%; but I would be even more concerning with implementing that. > > With these stats I have noticed what may be an odd constant-location- > shift between what the node is requested to insert and retrieve, like > the network does not look for data in the exact same place it stores > it?!?!?! Suspect, but still can't confirm anything yet. Ed From edt at aei.ca Thu Dec 13 01:58:26 2007 From: edt at aei.ca (Ed Tomlinson) Date: Wed, 12 Dec 2007 20:58:26 -0500 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> References: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> Message-ID: <200712122058.27221.edt@aei.ca> On December 12, 2007, Robert Hailey wrote: > > On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > > Ok that could be interesting. Although ideally we'd have a > > circular-keyspace-aware averager. > >> ... > > Ok. I suggest you commit, I will review post-commit. > > Committed it r16508, though it has changed slightly according to my > experiments with swap-biasing (e.g. shows number of cache/store writes > as well). Were should this appear? Does it take a while to show up? I have r16509 running and am not seeing anything new (yet). > From what I have discovered (or theorized) thus far, using the > average location of the entire store to bias against is way too much > of an anchor. This is good, because it takes up way to much memory to > remember such a running average anyway. If we end up implementing such > a bias, in the end, it will likely just take into account the last few > inserts or successes (a small constant amount). In this way, the > pressure of your peers can still pull you into a new location > (dragging the weight of the last few inserts with you; which will be > updated). Keeping the average key location and standard deviation just requires a few counters along hooks to update them when adding or deleting the store... It also requires a single pass to get the initial numbers. Should not be expensive except for that first pass. I agree that using the average would be very restrictive. Average plus or minus standard deviation would give a much better 'limit'. I suspect that It would take a fair length of time for it to start limiting at all (e.g. 2 x standard deviation would be greater than 1). > Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), > after running with a recently-stored-location bias, I must have pulled > my peers closer to me as well, as now even with the patch off, I do > not see the storeDistance go nearly so high (staying around 0.004). > Or, since the network has no anchor, maybe I've pressured the whole > wheel to stop turning. For at least my node; this (valuing the store > as one peer) has not shown any increase in store hit rates, biasing > against the succeeding requests, however, has in the past raised it to > about 1%; but I would be even more concerning with implementing that. > > With these stats I have noticed what may be an odd constant-location- > shift between what the node is requested to insert and retrieve, like > the network does not look for data in the exact same place it stores > it?!?!?! Suspect, but still can't confirm anything yet. Ed From Volodya at WhenGendarmeSleeps.org Thu Dec 13 06:24:38 2007 From: Volodya at WhenGendarmeSleeps.org (Volodya) Date: Thu, 13 Dec 2007 09:24:38 +0300 Subject: [freenet-dev] Usability testing results In-Reply-To: <475F2B9E.5040608@cs.ucl.ac.uk> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> Message-ID: <4760D026.7070801@WhenGendarmeSleeps.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Rogers wrote: > Matthew Toseland wrote: >> "Connect to strangers? >> >> Ideally, all Freenet users would connect only to people they know. This is far >> more secure, making it very difficult for an attacker to find your node. >> However, if you don't know at least 5 people already running Freenet nodes, >> you can instruct the node to connect to nodes run by people you don't know >> (Strangers). Note that you can always turn this off later. >> >> Do you know anyone already using Freenet? >> >> Yes, I have at least 5 friends already running Freenet and I will add their >> nodes on the Friends page. >> No, I want the node to automatically find untrusted nodes (aka Strangers) to >> connect to." >> >> Is this clearer than the previous wording? > > Sounds good to me. Maybe we can even get rid of "node" for the most > part, since that's a technical term? How about this: > > ... This is far more secure, making it very difficult for others to tell > that you are using Freenet. However, if you don't know at least 5 people > already running Freenet, you can choose to connect to strangers... > ...Yes, I have at least 5 friends already running Freenet and I will > enter their details on the Friends page. > No, I want Freenet to automatically find strangers to connect to. Keep in mind that one of the reasons for the friends-only (aka darknet) design was so that the node must operate in the environment where running Freenet itself is illegal. So i would argue that there should be perhaps a post-scriptum: "P.S. If you live in China or other country where Freenet itself is illegal, you should not allow connections to strangers." Simply saying that it's "less secure" might not be clear to people. - Volodya - -- http://freedom.libsyn.com/ Voice of Freedom, Radical Podcast http://eng.anarchopedia.org/ Anarchopedia, A Free Knowledge Portal "None of us are free until all of us are free." ~ Mihail Bakunin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHYNAmuWy2EFICg+0RAqDJAKCLtgDSf+sVWyFawmaMsWRNGYn7jwCgvvUb MvNlPJ63hj1i+mI/a7egYoY= =0o5+ -----END PGP SIGNATURE----- From tommy100 at gmx.de Thu Dec 13 07:56:25 2007 From: tommy100 at gmx.de (Tommy[D]) Date: Thu, 13 Dec 2007 08:56:25 +0100 Subject: [freenet-dev] Usability testing results In-Reply-To: <8b525dee0712121520s7f0581bfpcd8cf4b7e59e052b@mail.gmail.com> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <475FB4C0.2010101@batosai.net> <8b525dee0712121520s7f0581bfpcd8cf4b7e59e052b@mail.gmail.com> Message-ID: <4760E5A9.2090006@gmx.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Juiceman schrieb: > > I would add right at the end of this "Technically you can start by > only connecting to one friend, but your node won't work at all when > they are not online and even when they are online it won't perform > well until you get more friends. > I would say, this is too much detailed information for the beginning and beginner with freenet. If someone is interested in technical details (e.g. how much friends needed to start or anything else), he will search and/or ask for more information (-> wiki.freenetproject.org/#freenet etc). -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR2DlqRu5KnE1iZBnAQr++AP/S8xSDWAxDPB5BJBJ+cVEISgxx+Or0gIz JUjDpMo6an+U1GzmYb9tqRJPfshlqoiVw2rK9Y16suJKO70yg08HoOad6Or+rjc2 sQ1jma9wS5AYtgx6Sp+9YaNN6N5hBgPv7qK/gtd7yCAjBROXolW4HHpJ3THrrX/H vR3r55cFFyo= =FSMp -----END PGP SIGNATURE----- From freenet-devl at osndok.com Thu Dec 13 15:57:36 2007 From: freenet-devl at osndok.com (Robert Hailey) Date: Thu, 13 Dec 2007 09:57:36 -0600 Subject: [freenet-dev] Why Freenet is so SLOW! / Finding data In-Reply-To: <200712122058.27221.edt@aei.ca> References: <5D71A07A-C228-4512-9AF9-A0F1715E18F2@osndok.com> <200712122058.27221.edt@aei.ca> Message-ID: On Dec 12, 2007, at 7:58 PM, Ed Tomlinson wrote: > On December 12, 2007, Robert Hailey wrote: >> >> On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: >>> Ok that could be interesting. Although ideally we'd have a >>> circular-keyspace-aware averager. >>>> ... >>> Ok. I suggest you commit, I will review post-commit. >> >> Committed it r16508, though it has changed slightly according to my >> experiments with swap-biasing (e.g. shows number of cache/store >> writes >> as well). > > Were should this appear? Does it take a while to show up? I have > r16509 running and am not seeing anything new (yet). Sorry for the confusion, I thought the last message was on the list. The commit (r16509) is only for the statistics, not any routing change. >> From what I have discovered (or theorized) thus far, using the >> average location of the entire store to bias against is way too much >> of an anchor. This is good, because it takes up way to much memory to >> remember such a running average anyway. If we end up implementing >> such >> a bias, in the end, it will likely just take into account the last >> few >> inserts or successes (a small constant amount). In this way, the >> pressure of your peers can still pull you into a new location >> (dragging the weight of the last few inserts with you; which will be >> updated). > > Keeping the average key location and standard deviation just requires > a few counters along hooks to update them when adding or deleting the > store... It also requires a single pass to get the initial > numbers. Should > not be expensive except for that first pass. Yes, but again, I suspect that using the entire store would not allow the node to change locations. Also, varying store sizes would make the network that much harder to derive solid math for. > I agree that using the average would be very restrictive. Average > plus > or minus standard deviation would give a much better 'limit'. I > suspect > that It would take a fair length of time for it to start limiting at > all (e.g. > 2 x standard deviation would be greater than 1). It appears that you are interested in the routing change portion of the patch, so I've pasted it below (It will probably be wrapped terribly). I'm not sure I understand your point with regards to standard deviations, so I'll let you compare that logic to how I use the average in the not-even-proposed-yet routing change. Note that (with the recent commit) the below uses the average of all recently stored keys, and I have already said that probably wont work. -- Robert Hailey =================================================================== --- src/freenet/node/LocationManager.java (revision 16462) +++ src/freenet/node/LocationManager.java (working copy) @@ -378,9 +378,7 @@ // Randomise our location every 2*SWAP_RESET swap attempts, whichever way it went. if(node.random.nextInt(SWAP_RESET) == 0) { - setLocation(node.random.nextDouble()); - announceLocChange(); - node.writeNodeFile(); + randomizeLocation(); } SHA256.returnMessageDigest(md); @@ -394,6 +392,12 @@ } + public void randomizeLocation() { + setLocation(node.random.nextDouble()); + announceLocChange(); + node.writeNodeFile(); + } + /** * Locks the LocationManager. * Sends an FNPSwapRequest out into the network. @@ -646,7 +650,7 @@ * Anyway: * Two nodes choose each other and decide to attempt a switch. They * calculate the distance of all their edges currently (that is the - * distance between their currend ID and that of their neighbors), and + * distance between their current ID and that of their neighbors), and * multiply up all these values to get A. Then they calculate the * distance to all their neighbors as it would be if they switched * IDs, and multiply up these values to get B. @@ -692,6 +696,10 @@ if(logMINOR) Logger.minor(this, sb.toString()); + //If storeBias is true, count the location of the store as one friend (on our side). + boolean storeBias=(node.nodeStats.avgStoreLocation.countReports()>50); + double storeLocation=node.nodeStats.avgStoreLocation.currentValue(); + double A = 1.0; for(int i=0;i References: <200711300126.45810.toad@amphibian.dyndns.org> <200712112359.40871.toad@amphibian.dyndns.org> Message-ID: <200712132024.39708.toad@amphibian.dyndns.org> On Wednesday 12 December 2007 21:14, you wrote: > > On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > > Ok that could be interesting. Although ideally we'd have a > > circular-keyspace-aware averager. > >> ... > > Ok. I suggest you commit, I will review post-commit. > > Committed it r16508, though it has changed slightly according to my > experiments with swap-biasing (e.g. shows number of cache/store writes > as well). Ok, I will review the commit. > > From what I have discovered (or theorized) thus far, using the > average location of the entire store to bias against is way too much > of an anchor. This is good, because it takes up way to much memory to > remember such a running average anyway. It is rarely useful to implement averages that way. Most real world applications use klein filters (we use freenet.support.math.BootstrappingDecayingRunningAverage, which is pretty much the same thing). > If we end up implementing such > a bias, in the end, it will likely just take into account the last few > inserts or successes (a small constant amount). In this way, the > pressure of your peers can still pull you into a new location > (dragging the weight of the last few inserts with you; which will be > updated). Won't happen without convincing simulations. The current network has a lot of #freenet-refs connections (which are random, by no means small-world), and therefore has quite poor topology. A longer term problem is that many nodes don't run 24x7. > > Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), > after running with a recently-stored-location bias, I must have pulled > my peers closer to me as well, as now even with the patch off, I do > not see the storeDistance go nearly so high (staying around 0.004). > Or, since the network has no anchor, maybe I've pressured the whole > wheel to stop turning. This, valuing the store as one peer. Valuing the store as one peer does indeed make a lot of sense. However, we can't just implement something like that. > > With these stats I have noticed what may be an odd constant-location- > shift between what the node is requested to insert and retrieve, like > the network does not look for data in the exact same place it stores > it?!?!?! Suspect, but still can't confirm anything yet. Odd, it really should be the same, except for the known issue that inserts don't stop when they find the data, so they go significantly further than the average (CHK) request. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071213/3d82ac97/attachment.pgp From toad at amphibian.dyndns.org Thu Dec 13 22:25:05 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 13 Dec 2007 22:25:05 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <4760D026.7070801@WhenGendarmeSleeps.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <4760D026.7070801@WhenGendarmeSleeps.org> Message-ID: <200712132225.14284.toad@amphibian.dyndns.org> On Thursday 13 December 2007 06:24, Volodya wrote: > > > > ... This is far more secure, making it very difficult for others to tell > > that you are using Freenet. However, if you don't know at least 5 people > > already running Freenet, you can choose to connect to strangers... > > ...Yes, I have at least 5 friends already running Freenet and I will > > enter their details on the Friends page. > > No, I want Freenet to automatically find strangers to connect to. > > Keep in mind that one of the reasons for the friends-only (aka darknet) design was so that the node must > operate in the environment where running Freenet itself is illegal. So i would argue that there should be > perhaps a post-scriptum: > > "P.S. If you live in China or other country where Freenet itself is illegal, you should not allow connections > to strangers." > > Simply saying that it's "less secure" might not be clear to people. Current text (maybe a bit too detailed/paranoid): Connect to strangers? Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for others to tell that you are using Freenet. However, if you don't know at least 5 people already running Freenet, you can choose to connect to strangers (Promiscuous Mode). Note that you can always turn this off later. Do you know anyone already using Freenet? No, I want the node to automatically find strangers to connect to. Yes, I have at least 5 friends already using Freenet and I will add their details on the Friends page. Warning! Running Freenet may be illegal where you live, as it enables unrestricted, anonymous free speech. If so, telling the node to automatically connect to strangers may make it easy for your government to discover that you are running Freenet, so you should only connect to people you know (and ideally trust). Freenet is still beta software, we make no guarantees about security! In particular, those you connect to may be able to spy on your requests. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071213/37cb85fb/attachment.pgp From toad at amphibian.dyndns.org Thu Dec 13 22:31:34 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 13 Dec 2007 22:31:34 +0000 Subject: [freenet-dev] [freenet-cvs] r16508 - in trunk/freenet/src/freenet: clients/http node In-Reply-To: <20071212210027.C330047B78B@freenetproject.org> References: <20071212210027.C330047B78B@freenetproject.org> Message-ID: <200712132231.38027.toad@amphibian.dyndns.org> Okay, this is good on the whole, but you are right, the memory usage is unacceptable. You should use a cheaper running average; BootstrappingDecayingRunningAverage is the obvious thing (a klein filter whose parameter starts off at 1.0 and decays until it reaches a certain point), but if you want something more conventional, make something that just keeps counters; you know what value you are removing when a key is removed from the store. On Wednesday 12 December 2007 21:00, you wrote: > Author: robert > Date: 2007-12-12 21:00:27 +0000 (Wed, 12 Dec 2007) > New Revision: 16508 > > Modified: > trunk/freenet/src/freenet/clients/http/StatisticsToadlet.java > trunk/freenet/src/freenet/node/Node.java > trunk/freenet/src/freenet/node/NodeStats.java > Log: > Rough implementation of cache & store location statistics (uses a lot of memory) > > > Modified: trunk/freenet/src/freenet/clients/http/StatisticsToadlet.java > =================================================================== > --- trunk/freenet/src/freenet/clients/http/StatisticsToadlet.java 2007-12-12 18:16:09 UTC (rev 16507) > +++ trunk/freenet/src/freenet/clients/http/StatisticsToadlet.java 2007-12-12 21:00:27 UTC (rev 16508) > @@ -29,6 +29,7 @@ > import freenet.support.SizeUtil; > import freenet.support.TimeUtil; > import freenet.support.api.HTTPRequest; > +import freenet.support.math.SimpleRunningAverage; > > public class StatisticsToadlet extends Toadlet { > > @@ -422,8 +423,52 @@ > "\u00a0(" + ((storeHits*100) / (storeAccesses)) + "%)"); > > storeSizeList.addChild("li", > - "Avg. access rate:\u00a0" + thousendPoint.format(overallAccesses/nodeUptimeSeconds) + "/sec"); > + "Avg. access rate:\u00a0" + thousendPoint.format(cacheAccesses/nodeUptimeSeconds) + "/sec, "+thousendPoint.format(storeAccesses/nodeUptimeSeconds)+"/sec"); > > + // location-based stats > + boolean hasLoc=true; > + double nodeLoc=0.0; > + try { > + nodeLoc=node.getLocationManager().getLocation(); > + } catch (Error e) { > + //FIXME: PLEASE, how do we get the node location on the stats page? > + //Logger.error(this, "why?", e); > + e.printStackTrace(); > + hasLoc=false; > + } > + double avgCacheLocation=node.nodeStats.avgCacheLocation.currentValue(); > + double avgStoreLocation=node.nodeStats.avgStoreLocation.currentValue(); > + long cacheWrites=node.nodeStats.avgCacheLocation.countReports(); > + long storeWrites=node.nodeStats.avgStoreLocation.countReports(); > + double avgCacheSuccess=node.nodeStats.avgCacheSuccess.currentValue(); > + double avgStoreSuccess=node.nodeStats.avgStoreSuccess.currentValue(); > + double furthestCacheSuccess=node.nodeStats.furthestCacheSuccess; > + double furthestStoreSuccess=node.nodeStats.furthestStoreSuccess; > + double storeDist=Location.distance(nodeLoc, avgStoreLocation); > + double cacheDist=Location.distance(nodeLoc, avgCacheLocation); > + > + storeSizeList.addChild("li", "avgCacheLocation:\u00a0" + thousendPoint.format(avgCacheLocation)); > + storeSizeList.addChild("li", "avgStoreLocation:\u00a0" + thousendPoint.format(avgStoreLocation)); > + > + storeSizeList.addChild("li", "avgCacheSuccess:\u00a0" + thousendPoint.format(avgCacheSuccess)); > + storeSizeList.addChild("li", "avgStoreSuccess:\u00a0" + thousendPoint.format(avgStoreSuccess)); > + > + storeSizeList.addChild("li", "furthestCacheSuccess:\u00a0" + thousendPoint.format(furthestCacheSuccess)); > + storeSizeList.addChild("li", "furthestStoreSuccess:\u00a0" + thousendPoint.format(furthestStoreSuccess)); > + > + storeSizeList.addChild("li", "cacheWrites:\u00a0" + cacheWrites); > + storeSizeList.addChild("li", "storeWrites:\u00a0" + storeWrites); > + > + if (hasLoc) { > + storeSizeList.addChild("li", "cacheDist:\u00a0" + thousendPoint.format(cacheDist)); > + storeSizeList.addChild("li", "storeDist:\u00a0" + thousendPoint.format(storeDist)); > + long cacheLocationReports=((SimpleRunningAverage)node.nodeStats.avgCacheLocation).countReports(); > + long storeLocationReports=((SimpleRunningAverage)node.nodeStats.avgStoreLocation).countReports(); > + double cachePrimePercent=((1.0*cacheLocationReports)/cachedKeys); > + double storePrimePercent=((1.0*storeLocationReports)/storeKeys); > + storeSizeList.addChild("li", "locStatsReliability: \u00a0"+fix3p1pct.format(cachePrimePercent)+" / "+fix3p1pct.format(storePrimePercent)); > + } > + > } > > private void drawUnclaimedFIFOMessageCountsBox(HTMLNode unclaimedFIFOMessageCountsInfobox) { > > Modified: trunk/freenet/src/freenet/node/Node.java > =================================================================== > --- trunk/freenet/src/freenet/node/Node.java 2007-12-12 18:16:09 UTC (rev 16507) > +++ trunk/freenet/src/freenet/node/Node.java 2007-12-12 21:00:27 UTC (rev 16508) > @@ -247,8 +247,8 @@ > > /** The maximum number of keys stored in each of the datastores, cache and store combined. */ > private long maxTotalKeys; > - private long maxCacheKeys; > - private long maxStoreKeys; > + long maxCacheKeys; > + long maxStoreKeys; > /** The maximum size of the datastore. Kept to avoid rounding turning 5G into 5368698672 */ > private long maxTotalDatastoreSize; > /** If true, store shrinks occur immediately even if they are over 10% of the store size. If false, > @@ -1767,11 +1767,23 @@ > public SSKBlock fetch(NodeSSK key, boolean dontPromote) { > if(logMINOR) dumpStoreHits(); > try { > + double loc=key.toNormalizedDouble(); > + double dist=Location.distance(lm.getLocation(), loc); > + nodeStats.avgRequestLocation.report(loc); > SSKBlock block = sskDatastore.fetch(key, dontPromote); > if(block != null) { > + nodeStats.avgStoreSuccess.report(loc); > + if (dist > nodeStats.furthestStoreSuccess) > + nodeStats.furthestStoreSuccess=dist; > return block; > } > - return sskDatacache.fetch(key, dontPromote); > + block=sskDatacache.fetch(key, dontPromote); > + if (block != null) { > + nodeStats.avgCacheSuccess.report(loc); > + if (dist > nodeStats.furthestCacheSuccess) > + nodeStats.furthestCacheSuccess=dist; > + } > + return block; > } catch (IOException e) { > Logger.error(this, "Cannot fetch data: "+e, e); > return null; > @@ -1781,9 +1793,23 @@ > public CHKBlock fetch(NodeCHK key, boolean dontPromote) { > if(logMINOR) dumpStoreHits(); > try { > + double loc=key.toNormalizedDouble(); > + double dist=Location.distance(lm.getLocation(), loc); > + nodeStats.avgRequestLocation.report(loc); > CHKBlock block = chkDatastore.fetch(key, dontPromote); > - if(block != null) return block; > - return chkDatacache.fetch(key, dontPromote); > + if (block != null) { > + nodeStats.avgStoreSuccess.report(loc); > + if (dist > nodeStats.furthestStoreSuccess) > + nodeStats.furthestStoreSuccess=dist; > + return block; > + } > + block=chkDatacache.fetch(key, dontPromote); > + if (block != null) { > + nodeStats.avgCacheSuccess.report(loc); > + if (dist > nodeStats.furthestCacheSuccess) > + nodeStats.furthestCacheSuccess=dist; > + } > + return block; > } catch (IOException e) { > Logger.error(this, "Cannot fetch data: "+e, e); > return null; > @@ -1835,10 +1861,13 @@ > > private void store(CHKBlock block, boolean deep) { > try { > + double loc=block.getKey().toNormalizedDouble(); > if(deep) { > chkDatastore.put(block); > + nodeStats.avgStoreLocation.report(loc); > } > chkDatacache.put(block); > + nodeStats.avgCacheLocation.report(loc); > if(clientCore != null && clientCore.requestStarters != null) > clientCore.requestStarters.chkFetchScheduler.tripPendingKey(block); > } catch (IOException e) { > @@ -2498,6 +2527,10 @@ > return usm; > } > > + public LocationManager getLocationManager() { > + return lm; > + } > + > public int getSwaps() { > return LocationManager.swaps; > } > > Modified: trunk/freenet/src/freenet/node/NodeStats.java > =================================================================== > --- trunk/freenet/src/freenet/node/NodeStats.java 2007-12-12 18:16:09 UTC (rev 16507) > +++ trunk/freenet/src/freenet/node/NodeStats.java 2007-12-12 21:00:27 UTC (rev 16508) > @@ -22,6 +22,7 @@ > import freenet.support.api.IntCallback; > import freenet.support.api.LongCallback; > import freenet.support.math.RunningAverage; > +import freenet.support.math.SimpleRunningAverage; > import freenet.support.math.TimeDecayingRunningAverage; > import freenet.support.math.TrivialRunningAverage; > > @@ -144,8 +145,17 @@ > // various metrics > public final RunningAverage routingMissDistance; > public final RunningAverage backedOffPercent; > + public final RunningAverage avgCacheLocation; > + public final RunningAverage avgStoreLocation; > + public final RunningAverage avgCacheSuccess; > + public final RunningAverage avgStoreSuccess; > + // FIXME: does furthest{Store,Cache}Success need to be synchronized? > + public double furthestCacheSuccess=0.0; > + public double furthestStoreSuccess=0.0; > protected final Persister persister; > > + protected final RunningAverage avgRequestLocation; > + > // ThreadCounting stuffs > public final ThreadGroup rootThreadGroup; > private int threadLimit; > @@ -176,6 +186,14 @@ > this.hardRandom = node.random; > this.routingMissDistance = new TimeDecayingRunningAverage(0.0, 180000, 0.0, 1.0, node); > this.backedOffPercent = new TimeDecayingRunningAverage(0.0, 180000, 0.0, 1.0, node); > + // FIXME PLEASE remove (int) casts > + double nodeLoc=node.lm.getLocation(); > + this.avgCacheLocation=new SimpleRunningAverage((int)node.maxCacheKeys, nodeLoc); > + this.avgStoreLocation=new SimpleRunningAverage((int)node.maxStoreKeys, nodeLoc); > + // FIXME average for success-location may not need to be so large as the store. > + this.avgCacheSuccess=new SimpleRunningAverage(10000, nodeLoc); > + this.avgStoreSuccess=new SimpleRunningAverage(10000, nodeLoc); > + this.avgRequestLocation=new SimpleRunningAverage(10000, nodeLoc); > preemptiveRejectReasons = new StringCounter(); > localPreemptiveRejectReasons = new StringCounter(); > pInstantRejectIncoming = new TimeDecayingRunningAverage(0, 60000, 0.0, 1.0, node); > @@ -186,6 +204,18 @@ > new TimeDecayingRunningAverage(1, 10*60*1000 /* should be significantly longer than a typical transfer */, 0, Long.MAX_VALUE, node); > nodePinger = new NodePinger(node); > > + // FIXME: data-store/cache averages need to be persisted to be valuable (or scanned at every launch). > + /* > + if (node.isAdvancedModeEnabled()) { > + //Uggghh.... > + System.err.println("Scanning datastore/cache for location values"); > + chkDatastore.kludgeScan(avgStoreLocation); > + sskDatastore.kludgeScan(avgStoreLocation); > + chkDatacache.kludgeScan(avgCacheLocation); > + sskDatacache.kludgeScan(avgCacheLocation); > + } > + */ > + > previous_input_stat = 0; > previous_output_stat = 0; > previous_io_stat_time = 1; > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071213/51c98613/attachment.pgp From toad at amphibian.dyndns.org Fri Dec 14 00:24:22 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 00:24:22 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet Message-ID: <200712140024.29607.toad@amphibian.dyndns.org> The opennet warning is way too big. Currently it is: Warning: Promiscuous Mode Enabled: Your node will connect to Strangers Your node is currently running in promiscuous mode. It will connect to Strangers, and this means that anyone can find out that you are running a node. Most attacks are easier, blocking your node (for example at a national firewall) is much easier, and you have no control over who your node connects to. We strongly recommend you get some connections to Friends (trusted nodes run by people you already know); promiscuous mode is only intended as a temporary measure until you are able to just connect to your friends. If you only connect to your friends, while it may be possible for them to attack you, it is less likely than if your node is exposed to any oppressive government agency/other bad guy who wants to connect to it. Note that adding a peer in the Friends section does not help much unless that peer belongs to somebody you actually know (both for routing and security reasons)! Suggested improvement: Warning: Insecure Mode Enabled: Your node will connect to strangers Your node is currently running in insecure mode. You have no control over who it connects to, which could very well be the Bad Guys. Further, it is trivial for Them to find your node. Please get some connections to people you trust and add them to the Friends page as soon as possible, you will be much less vulnerable. When you have at least 10 Friends, turn off insecure mode and your node will become invisible. Several issues here: 1. Promiscuous Mode -> Insecure Mode. 2. "Bad Guys", "Them". 3. Shorter is better, right? 4. Should this be completely hideable? Or maybe we should have a really short warning after the first time and have it link to a more detailed discussion? We need a documentation toadlet for that to work though... Bombe was working on that. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/62d59436/attachment.pgp From toad at amphibian.dyndns.org Fri Dec 14 00:30:34 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 00:30:34 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712140024.29607.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> Message-ID: <200712140030.35073.toad@amphibian.dyndns.org> On Friday 14 December 2007 00:24, Matthew Toseland wrote: > The opennet warning is way too big. Currently it is: > > Warning: Promiscuous Mode Enabled: Your node will connect to Strangers > > Your node is currently running in promiscuous mode. It will connect to > Strangers, and this means that anyone can find out that you are running a > node. Most attacks are easier, blocking your node (for example at a national > firewall) is much easier, and you have no control over who your node connects > to. We strongly recommend you get some connections to Friends (trusted nodes > run by people you already know); promiscuous mode is only intended as a > temporary measure until you are able to just connect to your friends. If you > only connect to your friends, while it may be possible for them to attack > you, it is less likely than if your node is exposed to any oppressive > government agency/other bad guy who wants to connect to it. Note that adding > a peer in the Friends section does not help much unless that peer belongs to > somebody you actually know (both for routing and security reasons)! > > Suggested improvement: > > Warning: Insecure Mode Enabled: Your node will connect to strangers > > Your node is currently running in insecure mode. You have no control over who > it connects to, which could very well be the Bad Guys. Further, it is trivial > for Them to find your node. Please get some connections to people you trust > and add them to the Friends page as soon as possible, you will be much less > vulnerable. When you have at least 10 Friends, turn off insecure mode and > your node will become invisible. Changed to: Your node is currently running in insecure mode. It is trivial for an attacker to find your node, and you could be connected to the Bad Guys right now. Please get some connections to people you trust and add them to the Friends page as soon as possible, you will be much less vulnerable. When you have at least 10 Friends, turn off insecure mode and your node will become invisible to the outside world. > > > Several issues here: > 1. Promiscuous Mode -> Insecure Mode. > 2. "Bad Guys", "Them". > 3. Shorter is better, right? > 4. Should this be completely hideable? Or maybe we should have a really short > warning after the first time and have it link to a more detailed discussion? > We need a documentation toadlet for that to work though... Bombe was working > on that. > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/c6ea50e1/attachment.pgp From ian.clarke at gmail.com Fri Dec 14 05:16:20 2007 From: ian.clarke at gmail.com (Ian Clarke) Date: Thu, 13 Dec 2007 23:16:20 -0600 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712140024.29607.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> Message-ID: <823242bd0712132116ucd09669q8e09ff09a958de7f@mail.gmail.com> Definitely an improvement. The shorter something is, the more likely people will bother to read it. Ian. On 12/13/07, Matthew Toseland wrote: > The opennet warning is way too big. Currently it is: > > Warning: Promiscuous Mode Enabled: Your node will connect to Strangers > > Your node is currently running in promiscuous mode. It will connect to > Strangers, and this means that anyone can find out that you are running a > node. Most attacks are easier, blocking your node (for example at a national > firewall) is much easier, and you have no control over who your node connects > to. We strongly recommend you get some connections to Friends (trusted nodes > run by people you already know); promiscuous mode is only intended as a > temporary measure until you are able to just connect to your friends. If you > only connect to your friends, while it may be possible for them to attack > you, it is less likely than if your node is exposed to any oppressive > government agency/other bad guy who wants to connect to it. Note that adding > a peer in the Friends section does not help much unless that peer belongs to > somebody you actually know (both for routing and security reasons)! > > Suggested improvement: > > Warning: Insecure Mode Enabled: Your node will connect to strangers > > Your node is currently running in insecure mode. You have no control over who > it connects to, which could very well be the Bad Guys. Further, it is trivial > for Them to find your node. Please get some connections to people you trust > and add them to the Friends page as soon as possible, you will be much less > vulnerable. When you have at least 10 Friends, turn off insecure mode and > your node will become invisible. > > > Several issues here: > 1. Promiscuous Mode -> Insecure Mode. > 2. "Bad Guys", "Them". > 3. Shorter is better, right? > 4. Should this be completely hideable? Or maybe we should have a really short > warning after the first time and have it link to a more detailed discussion? > We need a documentation toadlet for that to work though... Bombe was working > on that. > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From batosai at batosai.net Fri Dec 14 09:12:30 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Fri, 14 Dec 2007 10:12:30 +0100 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712140030.35073.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> Message-ID: <476248FE.3040200@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland a ?crit : > Your node is currently running in insecure mode. It is trivial for an attacker > to find your node, and you could be connected to the Bad Guys right now. > Please get some connections to people you trust and add them to the Friends > page as soon as possible, you will be much less vulnerable. When you have at > least 10 Friends, turn off insecure mode and your node will become invisible > to the outside world. I'm not sure using the words "Insecure" and "Please get some connections" are a good thing. Opennet is not that insecure. Of course it is if compared to pure-darknet, but it's still far more secure than the internet. And most users just can't get connections to people they know and trust because no one they know uses freenet. Just yesterday, I met someone on #freenet-fr who knew nobody in the network. But after a few readings, he decided that opennet was insecure, refused to enable it and came to IRC to get Friends ! He thought we used Friends like in the friend-slots of eMule, not like in real-life-friends. I think we should stress that point. What about : Your node is not running at its maximal security level. It is trivial for an attacker to find your node, and you could be connected to the Bad Guys right now. If people you actually know and trust are using Freenet, you can connect to them using the Friends page to increase your security level. When you have at least 10 Friends, turn off insecure mode and your node will become invisible to the outside world. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHYkj+o6N05NzaOvURAnqEAJ92Y4zamFcL8W3DV4Kq/1bWSZjjPACdEYsS 8zp4UrgNWYzUvhf9AcplpDE= =Q16f -----END PGP SIGNATURE----- From toad at amphibian.dyndns.org Fri Dec 14 13:37:28 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 13:37:28 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <476248FE.3040200@batosai.net> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> Message-ID: <200712141337.33668.toad@amphibian.dyndns.org> On Friday 14 December 2007 09:12, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > > Your node is currently running in insecure mode. It is trivial for an attacker > > to find your node, and you could be connected to the Bad Guys right now. > > Please get some connections to people you trust and add them to the Friends > > page as soon as possible, you will be much less vulnerable. When you have at > > least 10 Friends, turn off insecure mode and your node will become invisible > > to the outside world. > > I'm not sure using the words "Insecure" and "Please get some > connections" are a good thing. Opennet is not that insecure. It isn't? IMHO opennet is fairly insecure. Not only can you do local correlation to determine whether your peers are the source of a specific request, you can do network-level attacks based on the location of requests to gradually locate the originator. > Of course > it is if compared to pure-darknet, but it's still far more secure than > the internet. And most users just can't get connections to people they > know and trust because no one they know uses freenet. "Please get some connections to people you trust" isn't clear enough? It's even clearer on the previous warning, but the previous warning is too big. > > Just yesterday, I met someone on #freenet-fr who knew nobody in the > network. But after a few readings, he decided that opennet was insecure, > refused to enable it and came to IRC to get Friends ! He thought we used > Friends like in the friend-slots of eMule, not like in > real-life-friends. I think we should stress that point. LOL. > > What about : > Your node is not running at its maximal security level. It is trivial > for an attacker to find your node, and you could be connected to the Bad > Guys right now. If people you actually know and trust are using Freenet, > you can connect to them using the Friends page to increase your security > level. When you have at least 10 Friends, turn off insecure mode and > your node will become invisible to the outside world. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/4b2aa6c7/attachment.pgp From colin at sq7.org Fri Dec 14 17:37:54 2007 From: colin at sq7.org (Colin Davis) Date: Fri, 14 Dec 2007 12:37:54 -0500 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712141337.33668.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141337.33668.toad@amphibian.dyndns.org> Message-ID: <4762BF72.5000803@sq7.org> Your Freenet node is only minimally protected. Your machine is currently hiding from most people, but it's doing so by connecting to strangers. If you connect to friends you already know and trust, Freenet will be better able to protect you. Please get some connections to people you trust, and add them to your Friends page as soon as possible. Once you have at least 10 Friends, turn on Secure mode and your node will become invisible to the outside world. ---- This does a few things- We want to acknowledge that using Freenet, even in opennet, is more secure than the public internet. Using words like Insecure makes them think that they are now MORE vulnerable, which isn't the right message for someone who just downloaded and installed this software out of paranoia. We also want to encourage people to get their friends to install Freenet, Social Networking style. This is the best way to increase adoption of the network. Essentially, drive home the message that 'If you get your friends to install freenet, you will be more safe'. It also has the side-effect of being true ;) Additionally, I think it's best to "turn on" Secure mode, rather than turning off opennet. It makes it seem like they are getting ADDITIONAL security, not running dangerously unsafely now. Again, keep in mind the user case. "I want to publish this photograph that proves the Government of my city is corrupt. How can I do it safely? " "TRIVIAL for an attacker? That defeats the entire purpose!" Versus "I want to publish this photograph that proves the Government of my city is corrupt. How can I do it safely? " "Hrmm.. Well, this is better than nothing. I better start small, and get some of my other democracy-loving friends to join too." "Once enough of us do, then it'll be safe enough to release my photo showing the coverup." Matthew Toseland wrote: > On Friday 14 December 2007 09:12, Julien Cornuwel wrote: > >> Matthew Toseland a ?crit : >> >> >>> Your node is currently running in insecure mode. It is trivial for an >>> > attacker > >>> to find your node, and you could be connected to the Bad Guys right now. >>> Please get some connections to people you trust and add them to the >>> > Friends > >>> page as soon as possible, you will be much less vulnerable. When you have >>> > at > >>> least 10 Friends, turn off insecure mode and your node will become >>> > invisible > >>> to the outside world. >>> >> I'm not sure using the words "Insecure" and "Please get some >> connections" are a good thing. Opennet is not that insecure. >> > > It isn't? IMHO opennet is fairly insecure. Not only can you do local > correlation to determine whether your peers are the source of a specific > request, you can do network-level attacks based on the location of requests > to gradually locate the originator. > > >> Of course >> it is if compared to pure-darknet, but it's still far more secure than >> the internet. And most users just can't get connections to people they >> know and trust because no one they know uses freenet. >> > > "Please get some connections to people you trust" isn't clear enough? It's > even clearer on the previous warning, but the previous warning is too big. > >> Just yesterday, I met someone on #freenet-fr who knew nobody in the >> network. But after a few readings, he decided that opennet was insecure, >> refused to enable it and came to IRC to get Friends ! He thought we used >> Friends like in the friend-slots of eMule, not like in >> real-life-friends. I think we should stress that point. >> > > LOL. > >> What about : >> Your node is not running at its maximal security level. It is trivial >> for an attacker to find your node, and you could be connected to the Bad >> Guys right now. If people you actually know and trust are using Freenet, >> you can connect to them using the Friends page to increase your security >> level. When you have at least 10 Friends, turn off insecure mode and >> your node will become invisible to the outside world. >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Devl mailing list >> Devl at freenetproject.org >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From batosai at batosai.net Fri Dec 14 18:04:52 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Fri, 14 Dec 2007 19:04:52 +0100 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762BF72.5000803@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141337.33668.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> Message-ID: <4762C5C4.8060305@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Colin Davis a ?crit : > Your Freenet node is only minimally protected. Your machine is currently > hiding from most people, but it's doing so by connecting to strangers. > If you connect to friends you already know and trust, Freenet will be > better able to protect you. > > Please get some connections to people you trust, and add them to your > Friends page as soon as possible. > Once you have at least 10 Friends, turn on Secure mode and your node > will become invisible to the outside world. Seems to me that "Secure mode" still implies that opennet is insecure. What about "Invisible mode" ? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHYsXEo6N05NzaOvURAiW9AJ4oSg3gr9sw35ZS9UQbnRqL9hjmtQCfcISA 3u1ETv0sCpgaWms0yWpcZuQ= =96+p -----END PGP SIGNATURE----- From colin at sq7.org Fri Dec 14 18:12:51 2007 From: colin at sq7.org (Colin Davis) Date: Fri, 14 Dec 2007 13:12:51 -0500 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762C5C4.8060305@batosai.net> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141337.33668.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> <4762C5C4.8060305@batosai.net> Message-ID: <4762C7A3.5060902@sq7.org> I think that we do want to imply that opennet is insecure. It IS insecure. Just less insecure than the public internet. Keep in mind, we strongly want to encourage people to connect to the darknet, but because it -is- more secure, and because it makes the network stronger/more resilient. I think that "minimally protected" and "turn on Secure mode", emphasize that they are better off than the public internet, but not yet where they should be. Julien Cornuwel wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Colin Davis a ?crit : > >> Your Freenet node is only minimally protected. Your machine is currently >> hiding from most people, but it's doing so by connecting to strangers. >> If you connect to friends you already know and trust, Freenet will be >> better able to protect you. >> >> Please get some connections to people you trust, and add them to your >> Friends page as soon as possible. >> Once you have at least 10 Friends, turn on Secure mode and your node >> will become invisible to the outside world. >> > > Seems to me that "Secure mode" still implies that opennet is insecure. > What about "Invisible mode" ? > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFHYsXEo6N05NzaOvURAiW9AJ4oSg3gr9sw35ZS9UQbnRqL9hjmtQCfcISA > 3u1ETv0sCpgaWms0yWpcZuQ= > =96+p > -----END PGP SIGNATURE----- > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > From batosai at batosai.net Fri Dec 14 18:20:20 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Fri, 14 Dec 2007 19:20:20 +0100 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762C7A3.5060902@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141337.33668.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> <4762C5C4.8060305@batosai.net> <4762C7A3.5060902@sq7.org> Message-ID: <4762C964.4070205@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Right. I made my noob-test again and these sentences are better. The subject understood that he already had some security and he could improve it later. So, it's OK for me. Colin Davis a ?crit : > I think that we do want to imply that opennet is insecure. It IS > insecure. Just less insecure than the public internet. > Keep in mind, we strongly want to encourage people to connect to the > darknet, but because it -is- more secure, and because it makes the > network stronger/more resilient. > > I think that "minimally protected" and "turn on Secure mode", emphasize > that they are better off than the public internet, but not yet where > they should be. > > > Julien Cornuwel wrote: > Colin Davis a ?crit : > >>>> Your Freenet node is only minimally protected. Your machine is currently >>>> hiding from most people, but it's doing so by connecting to strangers. >>>> If you connect to friends you already know and trust, Freenet will be >>>> better able to protect you. >>>> >>>> Please get some connections to people you trust, and add them to your >>>> Friends page as soon as possible. >>>> Once you have at least 10 Friends, turn on Secure mode and your node >>>> will become invisible to the outside world. >>>> > Seems to me that "Secure mode" still implies that opennet is insecure. > What about "Invisible mode" ? > _______________________________________________ Devl mailing list Devl at freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHYslko6N05NzaOvURApFsAJ4rZAIoICrfDB+xr0R+3KUPPDk+UgCfZXZY UYk3Oi/zLtobir2SE/1nFtA= =VhVx -----END PGP SIGNATURE----- From toad at amphibian.dyndns.org Fri Dec 14 19:14:32 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 19:14:32 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762BF72.5000803@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712141337.33668.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> Message-ID: <200712141914.40417.toad@amphibian.dyndns.org> On Friday 14 December 2007 17:37, Colin Davis wrote: > Your Freenet node is only minimally protected. Your machine is currently > hiding from most people, but it's doing so by connecting to strangers. No it isn't. > If you connect to friends you already know and trust, Freenet will be > better able to protect you. > > Please get some connections to people you trust, and add them to your > Friends page as soon as possible. > Once you have at least 10 Friends, turn on Secure mode and your node > will become invisible to the outside world. > > ---- > > This does a few things- We want to acknowledge that using Freenet, even > in opennet, is more secure than the public internet. Marginally! > Using words like > Insecure makes them think that they are now MORE vulnerable, which isn't > the right message for someone who just downloaded and installed this > software out of paranoia. Opennet is insecure. A false sense of security results in people doing things they wouldn't do otherwise because they think they ARE secure. > > We also want to encourage people to get their friends to install > Freenet, Social Networking style. This is the best way to increase > adoption of the network. Essentially, drive home the message that 'If > you get your friends to install freenet, you will be more safe'. It also > has the side-effect of being true ;) Of course. > > Additionally, I think it's best to "turn on" Secure mode, rather than > turning off opennet. It makes it seem like they are getting ADDITIONAL > security, not running dangerously unsafely now. Again, keep in mind the > user case. I had it like that before, I got complaints from some people that it implies that secure mode is an option, and the default is insecure mode. The counter-argument is that practically speaking darknet is the optional mode since it requires the user to *do something*. Input? > > "I want to publish this photograph that proves the Government of my city > is corrupt. How can I do it safely? " > > "TRIVIAL for an attacker? That defeats the entire purpose!" > Or he might connect to some Friends. Okay, maybe "trivial for an attacker to find your node" is a bit strong. Maybe "trivial for an attacker to find out that you are running Freenet" ? But once they've found that out, it IS fairly easy for them to connect to you and attack you. What is harder is to find the origin of specific content - it takes time, it requires that the originator keeps inserting it. But we want to convey this in as few words as possible, and as clearly as possible... > > Versus > > "I want to publish this photograph that proves the Government of my city > is corrupt. How can I do it safely? " > > "Hrmm.. Well, this is better than nothing. I better start small, and get > some of my other democracy-loving friends to join too." > "Once enough of us do, then it'll be safe enough to release my photo > showing the coverup." > Seems unlikely on the basis of the current take-up of freenet. :| -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/12592ac4/attachment.pgp From toad at amphibian.dyndns.org Fri Dec 14 19:23:21 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 19:23:21 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762C5C4.8060305@batosai.net> References: <200712140024.29607.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> <4762C5C4.8060305@batosai.net> Message-ID: <200712141923.21545.toad@amphibian.dyndns.org> On Friday 14 December 2007 18:04, Julien Cornuwel wrote: > Colin Davis a ?crit : > > Your Freenet node is only minimally protected. Your machine is currently > > hiding from most people, but it's doing so by connecting to strangers. > > If you connect to friends you already know and trust, Freenet will be > > better able to protect you. > > > > Please get some connections to people you trust, and add them to your > > Friends page as soon as possible. > > Once you have at least 10 Friends, turn on Secure mode and your node > > will become invisible to the outside world. > > Seems to me that "Secure mode" still implies that opennet is insecure. Opennet IS insecure! > What about "Invisible mode" ? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/f5221cab/attachment.pgp From toad at amphibian.dyndns.org Fri Dec 14 19:27:30 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 14 Dec 2007 19:27:30 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <476248FE.3040200@batosai.net> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> Message-ID: <200712141927.31009.toad@amphibian.dyndns.org> On Friday 14 December 2007 09:12, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > > Your node is currently running in insecure mode. It is trivial for an attacker > > to find your node, and you could be connected to the Bad Guys right now. > > Please get some connections to people you trust and add them to the Friends > > page as soon as possible, you will be much less vulnerable. When you have at > > least 10 Friends, turn off insecure mode and your node will become invisible > > to the outside world. > > I'm not sure using the words "Insecure" and "Please get some > connections" are a good thing. Opennet is not that insecure. Yes it is. It is very easy to find all nodes. It is relatively easy to surveil those nodes that you are connected to and determine what (of big files) is local and what is remote. And it is relatively easy to search the entire network for the originator of specific content by starting off far from them, and using what you know from the locations of those requests that reach you, to gradually narrow down the possible authors. > Of course > it is if compared to pure-darknet, Whereas these attacks are MUCH more difficult on darknet. > but it's still far more secure than > the internet. It may be more secure than the internet, but so what? > And most users just can't get connections to people they > know and trust because no one they know uses freenet. This is why we need to build a big Freenet. This is why we need opennet. And we need to build it not only from those people who absolutely must use Freenet, but also from those who are merely interested. If you have a secret source in the KGB, you probably don't want to directly connect to his node - you want to connect to your friends who eventually indirectly get to him, so that when they bust or surveil either of you, they don't get the other. > > Just yesterday, I met someone on #freenet-fr who knew nobody in the > network. But after a few readings, he decided that opennet was insecure, > refused to enable it and came to IRC to get Friends ! Isn't this just the result of the disclaimer being far too long? > He thought we used > Friends like in the friend-slots of eMule, not like in > real-life-friends. I think we should stress that point. > > What about : > Your node is not running at its maximal security level. It is trivial > for an attacker to find your node, and you could be connected to the Bad > Guys right now. If people you actually know and trust are using Freenet, > you can connect to them using the Friends page to increase your security > level. When you have at least 10 Friends, turn off insecure mode and > your node will become invisible to the outside world. What about: Freenet is running in insecure mode: It is (relatively) easy for an attacker to find out that you are running a node, connect to it, or even trace your requests. You can dramatically improve your security by adding connections to people you trust through the Friends page. When you have at least 10 Friends, please turn off insecure mode and your node will become invisible to the outside world. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071214/9c2e6838/attachment.pgp From colin at sq7.org Fri Dec 14 19:54:01 2007 From: colin at sq7.org (Colin Davis) Date: Fri, 14 Dec 2007 14:54:01 -0500 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712141927.31009.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141927.31009.toad@amphibian.dyndns.org> Message-ID: <4762DF59.90109@sq7.org> You arguments about not providing a false-sense of security are well taken, and I concede. I think that the text that you provided is certainly clear, and should get the job done. What about taking a page from LinkedIn and Facebook? Imply that running in opennet is half-way installed.. You're almost there! Your Freenet installation is almost complete. Freenet is currently running in insecure mode, which allows you to get up and running on the Freenet network, but isn't very secure. It is (relatively) easy for an attacker to find out that you are running a node, connect to it, or even trace your requests. You can dramatically improve your security by adding connections to people you trust through the Friends page. When you have at least 10 Friends, please turn off insecure mode and your node will become invisible to the outside world. > What about: > Freenet is running in insecure mode: It is (relatively) easy for an attacker > to find out that you are running a node, connect to it, or even trace your > requests. You can dramatically improve your security by adding connections to > people you trust through the Friends page. When you have at least 10 Friends, > please turn off insecure mode and your node will become invisible to the > outside world. > > ------------------------------------------------------------------------ > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From colin at sq7.org Fri Dec 14 19:58:06 2007 From: colin at sq7.org (Colin Davis) Date: Fri, 14 Dec 2007 14:58:06 -0500 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712141914.40417.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712141337.33668.toad@amphibian.dyndns.org> <4762BF72.5000803@sq7.org> <200712141914.40417.toad@amphibian.dyndns.org> Message-ID: <4762E04E.8090708@sq7.org> >> This does a few things- We want to acknowledge that using Freenet, even >> in opennet, is more secure than the public internet. >> > > Marginally! > > Point taken. In some ways the public internet is more secure.. For instance, if you use https, only your destination point, and your ISP have any resonable chance of pinpointing what you are uploading/downloading.. In Opennet, an attacker could connect to your node and others that you are likely to be connected to, and try to start correlating traffic, etc. From batosai at batosai.net Fri Dec 14 21:02:01 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Fri, 14 Dec 2007 22:02:01 +0100 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712141927.31009.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141927.31009.toad@amphibian.dyndns.org> Message-ID: <4762EF49.2080207@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthew Toseland a ?crit : >> I'm not sure using the words "Insecure" and "Please get some >> connections" are a good thing. Opennet is not that insecure. > > Yes it is. It is very easy to find all nodes. It is relatively easy to surveil > those nodes that you are connected to and determine what (of big files) is > local and what is remote. And it is relatively easy to search the entire > network for the originator of specific content by starting off far from them, > and using what you know from the locations of those requests that reach you, > to gradually narrow down the possible authors. Maybe my opinion about opennet was a bit too high : I thought it was as secure as Freenet 0.5. Which, while not being perfect, was quite good. > What about: > > Freenet is running in insecure mode: It is (relatively) easy for an attacker > to find out that you are running a node, connect to it, or even trace your > requests. You can dramatically improve your security by adding connections to > people you trust through the Friends page. When you have at least 10 Friends, > please turn off insecure mode and your node will become invisible to the > outside world. OK for me. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHYu9Jo6N05NzaOvURAg5GAKCVZZk65TpL3fz8AxOjuFNRPyWn/gCgx7s0 wz8DPfhuCYbXYYr1zwQILlo= =TS7m -----END PGP SIGNATURE----- From ian.clarke at gmail.com Fri Dec 14 23:30:18 2007 From: ian.clarke at gmail.com (Ian Clarke) Date: Fri, 14 Dec 2007 17:30:18 -0600 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712132225.14284.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <4760D026.7070801@WhenGendarmeSleeps.org> <200712132225.14284.toad@amphibian.dyndns.org> Message-ID: <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> I think its great that we are really aiming for simplicity here, and doing usability testing, there is hope for us yet :-) On 12/13/07, Matthew Toseland wrote: > Do you know anyone already using Freenet? > No, I want the node to automatically find strangers to connect to. > Yes, I have at least 5 friends already using Freenet and I will add their > details on the Friends page. The above is good, clear, concise, and written in plain English. We should ensure that the messaging within FProxy mirrors this language when it displays its "not enough peers" error message. > Warning! Running Freenet may be illegal where you live, as it enables > unrestricted, anonymous free speech. If so, telling the node to automatically > connect to strangers may make it easy for your government to discover that > you are running Freenet, so you should only connect to people you know (and > ideally trust). Freenet is still beta software, we make no guarantees about > security! In particular, those you connect to may be able to spy on your > requests. I don't know about this, too many exclamation points, too wordy. How about: "Warning: Freenet may be illegal in some countries. If you allow you to connect to strangers, those strangers will know that you are running Freenet, and this could be dangerous. Freenet is still beta software, and we offer no guarantees as to its security." The shorter this is, the easier for people to read and absorb it. 3 clear and calm sentences are far more likely to be read than 6 hyperbolic ones. We need to keep up this usability testing, its incredibly valuable. Ian. -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From toad at amphibian.dyndns.org Sat Dec 15 00:42:59 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 00:42:59 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762DF59.90109@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712141927.31009.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> Message-ID: <200712150043.05291.toad@amphibian.dyndns.org> On Friday 14 December 2007 19:54, Colin Davis wrote: > You arguments about not providing a false-sense of security are well > taken, and I concede. I think that the text that you provided is > certainly clear, and should get the job done. > > What about taking a page from LinkedIn and Facebook? Imply that running > in opennet is half-way installed.. > > > You're almost there! Your Freenet installation is almost complete. > Freenet is currently running in insecure mode, which allows you to get > up and running on the Freenet network, but isn't very secure. > It is (relatively) easy for an attacker to find out that you are running > a node, connect to it, or even trace your requests. > > You can dramatically improve your security by adding connections to > people you trust through the Friends page. > When you have at least 10 Friends, please turn off insecure mode and > your node will become invisible to the outside world. That's not bad, although obviously it's longer. Ian? Anyone? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/8919fe77/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 15 00:44:39 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 00:44:39 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762EF49.2080207@batosai.net> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712141927.31009.toad@amphibian.dyndns.org> <4762EF49.2080207@batosai.net> Message-ID: <200712150044.40272.toad@amphibian.dyndns.org> On Friday 14 December 2007 21:02, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > >> I'm not sure using the words "Insecure" and "Please get some > >> connections" are a good thing. Opennet is not that insecure. > > > > Yes it is. It is very easy to find all nodes. It is relatively easy to surveil > > those nodes that you are connected to and determine what (of big files) is > > local and what is remote. And it is relatively easy to search the entire > > network for the originator of specific content by starting off far from them, > > and using what you know from the locations of those requests that reach you, > > to gradually narrow down the possible authors. > > Maybe my opinion about opennet was a bit too high : I thought it was as > secure as Freenet 0.5. Which, while not being perfect, was quite good. No it wasn't. Its routing didn't work, so the global searches we've been talking about would be harder, but otoh its routing breakage happened in predictable ways so presumably other attacks would be possible. > > > What about: > > > > Freenet is running in insecure mode: It is (relatively) easy for an attacker > > to find out that you are running a node, connect to it, or even trace your > > requests. You can dramatically improve your security by adding connections to > > people you trust through the Friends page. When you have at least 10 Friends, > > please turn off insecure mode and your node will become invisible to the > > outside world. > > OK for me. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/89e839f4/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 15 01:02:41 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 01:02:41 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712132225.14284.toad@amphibian.dyndns.org> <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> Message-ID: <200712150102.50055.toad@amphibian.dyndns.org> On Friday 14 December 2007 23:30, Ian Clarke wrote: > I think its great that we are really aiming for simplicity here, and > doing usability testing, there is hope for us yet :-) > > On 12/13/07, Matthew Toseland wrote: > > Do you know anyone already using Freenet? > > No, I want the node to automatically find strangers to connect to. > > Yes, I have at least 5 friends already using Freenet and I will add their > > details on the Friends page. > > The above is good, clear, concise, and written in plain English. We > should ensure that the messaging within FProxy mirrors this language > when it displays its "not enough peers" error message. > > > Warning! Running Freenet may be illegal where you live, as it enables > > unrestricted, anonymous free speech. If so, telling the node to automatically > > connect to strangers may make it easy for your government to discover that > > you are running Freenet, so you should only connect to people you know (and > > ideally trust). Freenet is still beta software, we make no guarantees about > > security! In particular, those you connect to may be able to spy on your > > requests. > > I don't know about this, too many exclamation points, too wordy. How about: > > "Warning: Freenet may be illegal in some countries. If you allow you > to connect to strangers, those strangers will know that you are > running Freenet, and this could be dangerous. Freenet is still beta > software, and we offer no guarantees as to its security." > > The shorter this is, the easier for people to read and absorb it. 3 > clear and calm sentences are far more likely to be read than 6 > hyperbolic ones. Unfortunately it's not hyperbolic (even the last bit about "may be able to spy on your requests"). But it could be seen as such, so lets work on this... > If you allow you > to connect to strangers, those strangers will know that you are > running Freenet, and this could be dangerous." This implies to me that it's a matter of luck, that only those specific strangers that you randomly got connected to will be able to attack you. Whereas the reality is a powerful attacker can target a specific node, cycle through blocks of nodes, or progressively eliminate whole swathes of the network by a location-based requestor search. Also it's not strictly true: your noderef is sent to a lot of nodes which don't actually accept the invitation to connect. In the current code, we've already told the user in the introductory sentence that darknet is more secure: "Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for others to tell that you are using Freenet. However, if you don't know at least 5 people already running Freenet, you can choose to connect to strangers. Note that you can always turn this off later." So the warning is the mirror-image, and can therefore be brief. Maybe: If Freenet is illegal where you live, or if you are using it to access materials that may get you into trouble, telling the node to automatically connect to strangers may be dangerous as it makes life much easier for an attacker. Freenet is still alpha software, and we offer no guarantees about security. (Alpha not beta: we're missing critical functionality i.e. premix routing, betas are supposed to be more or less feature-complete). Putting this into context: "Ideally, all Freenet users would connect only to people they know. This is far more secure, making it very difficult for others to tell that you are using Freenet. However, if you don't know at least 5 people already running Freenet, you can choose to connect to strangers. Note that you can always turn this off later. Do you know anyone already using Freenet? - No, I want the node to automatically find strangers to connect to. - Yes, I have at least 5 friends already using Freenet and I will add their details on the Friends page. Warning: If Freenet is illegal where you live, or if you are using it to access materials that may get you into trouble, telling the node to automatically connect to strangers may be dangerous as it makes life much easier for an attacker. Freenet is still alpha software, and we offer no guarantees about security." > > We need to keep up this usability testing, its incredibly valuable. Agreed. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/9d0475e0/attachment.pgp From NEOatNHNG at users.sourceforge.net Sat Dec 15 02:37:04 2007 From: NEOatNHNG at users.sourceforge.net (=?ISO-8859-15?Q?Michael_T=E4nzer?=) Date: Sat, 15 Dec 2007 03:37:04 +0100 Subject: [freenet-dev] German Translation Update (freenet.l10n.de.v1090) Message-ID: <47633DD0.8030307@users.sourceforge.net> Here's an Update for the German language file. The translations from Ratchet and Anonymous were (I'm sorry to have to say so) crappy. Regards Michael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.de.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071215/19d3e72b/attachment.txt From ian at locut.us Sat Dec 15 06:03:27 2007 From: ian at locut.us (Ian Clarke) Date: Sat, 15 Dec 2007 00:03:27 -0600 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712150043.05291.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712141927.31009.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> <200712150043.05291.toad@amphibian.dyndns.org> Message-ID: <823242bd0712142203l777fdf91r201169ca2cf1acb0@mail.gmail.com> On 12/14/07, Matthew Toseland wrote: > > You're almost there! Your Freenet installation is almost complete. > > Freenet is currently running in insecure mode, which allows you to get > > up and running on the Freenet network, but isn't very secure. > > It is (relatively) easy for an attacker to find out that you are running > > a node, connect to it, or even trace your requests. > > > > You can dramatically improve your security by adding connections to > > people you trust through the Friends page. > > When you have at least 10 Friends, please turn off insecure mode and > > your node will become invisible to the outside world. > > That's not bad, although obviously it's longer. Ian? Anyone? I agree with the intent, but I'm just not sure this will achieve anything in practice. "When you have at least 10 friends"? I'm sure they probably already have 10 friends, the problem is that they aren't Freenet users. This could simply lead to a repeat of #freenet-refs, where the definition of "friend" becomes someone, anyone, not found through the opennet mechanism. We can't ask people to make new friends just so that they can have friends running Freenet nodes, because such convenience friends are unlikely to be much more trustworthy than opennet strangers. I think all we can do is ask users, that don't currently have (genuine) Freenet-using friends, to persuade some of their existing friends to use Freenet. Even in this case, it won't really improve connectivity to the rest of the Freenet network. In general, I'm really not sure how we can make it easier for people to go from opennet to darknet apart from making Freenet as pervasive as possible (and opennet is essential for this). Ian. -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From toad at amphibian.dyndns.org Sat Dec 15 12:44:08 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 12:44:08 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <823242bd0712142203l777fdf91r201169ca2cf1acb0@mail.gmail.com> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712150043.05291.toad@amphibian.dyndns.org> <823242bd0712142203l777fdf91r201169ca2cf1acb0@mail.gmail.com> Message-ID: <200712151244.14831.toad@amphibian.dyndns.org> On Saturday 15 December 2007 06:03, you wrote: > On 12/14/07, Matthew Toseland wrote: > > > You're almost there! Your Freenet installation is almost complete. > > > Freenet is currently running in insecure mode, which allows you to get > > > up and running on the Freenet network, but isn't very secure. > > > It is (relatively) easy for an attacker to find out that you are running > > > a node, connect to it, or even trace your requests. > > > > > > You can dramatically improve your security by adding connections to > > > people you trust through the Friends page. > > > When you have at least 10 Friends, please turn off insecure mode and > > > your node will become invisible to the outside world. Re the below, we're using Friends with a capital F as a technical term here while trying to maintain the implication from the previous sentence that they're actually non-capital friends too ... and that's likely to lead to confusion. So how about: "You can dramatically improve your security by adding connections to people you know through the Friends page. When you have connected to at least 10 friends, please turn off insecure mode and your node will become invisible to the outside world." (Keep the first paragraph). > > > > That's not bad, although obviously it's longer. Ian? Anyone? > > I agree with the intent, but I'm just not sure this will achieve > anything in practice. "When you have at least 10 friends"? I'm sure > they probably already have 10 friends, the problem is that they aren't > Freenet users. This could simply lead to a repeat of #freenet-refs, > where the definition of "friend" becomes someone, anyone, not found > through the opennet mechanism. > > We can't ask people to make new friends just so that they can have > friends running Freenet nodes, because such convenience friends are > unlikely to be much more trustworthy than opennet strangers. > IMHO we can address this to a degree with better wording. See above. > I think all we can do is ask users, that don't currently have > (genuine) Freenet-using friends, to persuade some of their existing > friends to use Freenet. Even in this case, it won't really improve > connectivity to the rest of the Freenet network. That is exactly what we are doing here. And the objective is not connectivity, it is security - robust connectivity. > > In general, I'm really not sure how we can make it easier for people > to go from opennet to darknet apart from making Freenet as pervasive > as possible (and opennet is essential for this). There are lots of things we can do e.g. much shorter noderefs. However, having lots of users is essential. Given that opennet provides a false sense of security, IMHO we have to have some indication on the homepage, but it should be as clear and concise as possible. Hence the above debate. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/de643774/attachment.pgp From Volodya at WhenGendarmeSleeps.org Sat Dec 15 13:51:20 2007 From: Volodya at WhenGendarmeSleeps.org (Volodya) Date: Sat, 15 Dec 2007 16:51:20 +0300 Subject: [freenet-dev] Usability testing results In-Reply-To: <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> References: <200712110121.57813.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <4760D026.7070801@WhenGendarmeSleeps.org> <200712132225.14284.toad@amphibian.dyndns.org> <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> Message-ID: <4763DBD8.20708@WhenGendarmeSleeps.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > "Warning: Freenet may be illegal in some countries. If you allow you > to connect to strangers, those strangers will know that you are > running Freenet, and this could be dangerous. Freenet is still beta > software, and we offer no guarantees as to its security." Some ISPs have it written in the contract that you are not to run p2p software, but that doesn't make it "illegal", maybe "disallowed" would be a better word. - Voldoya - -- http://freedom.libsyn.com/ Voice of Freedom, Radical Podcast http://eng.anarchopedia.org/ Anarchopedia, A Free Knowledge Portal "None of us are free until all of us are free." ~ Mihail Bakunin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHY9vYuWy2EFICg+0RAkG2AKC9iYrnpgDYuQYCT5S4RpUYXf9pvACfU5Mp K5gpqQmz3nA+MVEPrCqXhEs= =Tp00 -----END PGP SIGNATURE----- From toad at amphibian.dyndns.org Sat Dec 15 14:04:32 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 14:04:32 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <4763DBD8.20708@WhenGendarmeSleeps.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> <4763DBD8.20708@WhenGendarmeSleeps.org> Message-ID: <200712151404.37515.toad@amphibian.dyndns.org> On Saturday 15 December 2007 13:51, Volodya wrote: > > "Warning: Freenet may be illegal in some countries. If you allow you > > to connect to strangers, those strangers will know that you are > > running Freenet, and this could be dangerous. Freenet is still beta > > software, and we offer no guarantees as to its security." > > Some ISPs have it written in the contract that you are not to run p2p software, but that doesn't make it > "illegal", maybe "disallowed" would be a better word. And some major governments block it on their national firewalls, and have the death penalty for "revealing state secrets" and similar speech crimes. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/0bc1e040/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 15 14:57:28 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 14:57:28 +0000 Subject: [freenet-dev] [freenet-cvs] r16548 - in trunk/plugins: . KeyExplorer In-Reply-To: <20071214205613.4B4113C073B@freenetproject.org> References: <20071214205613.4B4113C073B@freenetproject.org> Message-ID: <200712151457.33790.toad@amphibian.dyndns.org> Please don't use realGetKey(). That's an internal method, used by the request starters. Using it means your request isn't queued and is sent immediately, outside all the rate limiting etc code. There are other possibilities, such as subclassing SendableGet and queueing yourself. On Friday 14 December 2007 20:56, you wrote: > Author: saces > Date: 2007-12-14 20:56:13 +0000 (Fri, 14 Dec 2007) > New Revision: 16548 > > Added: > trunk/plugins/KeyExplorer/ > trunk/plugins/KeyExplorer/KeyExplorer.java > Log: > new plugin: key info and debug tool > > Added: trunk/plugins/KeyExplorer/KeyExplorer.java > =================================================================== > --- trunk/plugins/KeyExplorer/KeyExplorer.java (rev 0) > +++ trunk/plugins/KeyExplorer/KeyExplorer.java 2007-12-14 20:56:13 UTC (rev 16548) > @@ -0,0 +1,243 @@ > +package plugins.KeyExplorer; > + > +import java.io.IOException; > +import java.net.MalformedURLException; > + > +import freenet.client.Metadata; > +import freenet.client.MetadataParseException; > +import freenet.clients.http.PageMaker; > +import freenet.config.Config; > +import freenet.config.SubConfig; > +import freenet.keys.BaseClientKey; > +import freenet.keys.ClientKey; > +import freenet.keys.ClientKeyBlock; > +import freenet.keys.FreenetURI; > +import freenet.keys.KeyDecodeException; > +import freenet.node.LowLevelGetException; > +import freenet.pluginmanager.FredPlugin; > +import freenet.pluginmanager.FredPluginHTTP; > +import freenet.pluginmanager.FredPluginThreadless; > +import freenet.pluginmanager.PluginHTTPException; > +import freenet.pluginmanager.PluginRespirator; > +import freenet.support.HTMLNode; > +import freenet.support.HexUtil; > +import freenet.support.api.HTTPRequest; > +import freenet.support.io.ArrayBucket; > +import freenet.support.io.ArrayBucketFactory; > +import freenet.support.io.BucketTools; > + > +public class KeyExplorer implements FredPlugin, FredPluginHTTP, FredPluginThreadless { > + > + private PluginRespirator m_pr; > + private PageMaker m_pm; > + > + public String handleHTTPGet(HTTPRequest request) throws PluginHTTPException { > + String uri = request.getParam("key"); > + return makeMainPage(uri); > + } > + > + public String handleHTTPPost(HTTPRequest request) throws PluginHTTPException { > + String uri = request.getPartAsString("key", 1024); > + return makeMainPage(uri); > + } > + > + public String handleHTTPPut(HTTPRequest request) throws PluginHTTPException { > + return makeMainPage(); > + } > + > + public void runPlugin(PluginRespirator pr) { > + Config nc = pr.getNode().config; > + SubConfig fc = nc.get("fproxy"); > + String cssName = fc.getString("css"); > + > + m_pm = new PageMaker(cssName); > + m_pr = pr; > + } > + > + public void terminate() { > + } > + > + private String makeMainPage() { > + return makeMainPage(null); > + } > + > + private String makeMainPage(String key) { > + HTMLNode pageNode = m_pm.getPageNode("KeyExplorer", null); > + HTMLNode contentNode = m_pm.getContentNode(pageNode); > + > + String error = null; > + byte[] data = null; > + ClientKeyBlock ckb = null; > + > + try { > + if (key != null && (key.trim().length() > 0)) { > + FreenetURI furi = new FreenetURI(key); > + ClientKey ck = (ClientKey) BaseClientKey.getBaseKey(furi); > + ckb = m_pr.getNode().clientCore.realGetKey(ck, false, true, false); > + ArrayBucket a = (ArrayBucket) ckb.decode(new ArrayBucketFactory(), 32 * 1024, false); > + data = BucketTools.toByteArray(a); > + } > + } catch (MalformedURLException e) { > + error = "MalformedURL"; > + } catch (LowLevelGetException e) { > + error = "get failed"; > + } catch (KeyDecodeException e) { > + error = "decode error"; > + } catch (IOException e) { > + error = "io error"; > + } > + > + if (error != null) { > + contentNode.addChild(createErrorBox(error)); > + } > + > + contentNode.addChild(createUriBox()); > + > + if (data != null) { > + String title = "Key: " + key; > + if (ckb.isMetadata()) > + title = title + "\u00a0(MetaData)"; > + HTMLNode dataBox2 = m_pm.getInfobox(title); > + > + char[] asciibuf = new char[16]; > + > + for (int j = 0; j < 16; j++) > + asciibuf[j] = ' '; > + > + dataBox2.addChild("%", "
\n");
> +			StringBuffer sb = new StringBuffer();
> +			int offset = 0;
> +
> +			for (int i = 0; i < data.length; i++) {
> +				offset = (i) % 16;
> +				HexUtil.bytesToHexAppend(data, i, 1, sb);
> +				sb.append(' ');
> +				if ((data[i] > 31) && (data[i] < 127)) {
> +					//int j = data[i];
> +					//sb.append((char) j);
> +					asciibuf[offset] =(char) data[i];
> +				} 
> +				
> +				if ((i > 1) && ((i + 1) % 16 == 0)) {
> +					sb.append(' ');
> +					sb.append(asciibuf);
> +					sb.append('\n');
> +					for (int k = 0; k < 16; k++) 
> +						asciibuf[k] = ' ';
> +				}
> +			}
> +			if (offset > 0) {
> +				int n = (15-offset)*3;
> +				for (int m = 0; m < n; m++) 
> +					sb.append(' ');
> +				sb.append(' ');
> +				sb.append(asciibuf);
> +			}
> +
> +			dataBox2.addChild("#", sb.toString());
> +			dataBox2.addChild("%", "\n
"); > + > + contentNode.addChild(dataBox2); > + > + error = null; > + > + if (ckb.isMetadata()) { > + > + Metadata md = null; > + try { > + md = Metadata.construct(data); > + } catch (MetadataParseException e) { > + error = "Metadata parse error"; > + } > + > + if (error != null) > + contentNode.addChild(createErrorBox(error)); > + > + if (md != null) { > + HTMLNode metaBox = m_pm.getInfobox("Decomposed metadata"); > + > + metaBox.addChild("#", "Document type:"); > + metaBox.addChild("%", "
"); > + > + if (md.isSimpleManifest()) { > + metaBox.addChild("#", "Document type: SimpleManifest"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSplitfile()) { > + metaBox.addChild("#", "Document type: Splitfile"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSimpleSplitfile()) { > + metaBox.addChild("#", "Document type: SimpleSplitfile"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSingleFileRedirect()) { > + metaBox.addChild("#", "Document type: SingleFileRedirect"); > + metaBox.addChild("%", "
"); > + String sfrUri = md.getSingleTarget().toString(); > + metaBox.addChild("#", sfrUri); > + metaBox.addChild("#", "\u00a0"); > + metaBox.addChild(new HTMLNode("a", "href", "/?key="+sfrUri, "open")); > + metaBox.addChild("#", "\u00a0"); > + metaBox.addChild(new HTMLNode("a", "href", "/plugins/plugins.KeyExplorer.KeyExplorer/?key="+sfrUri, "explore")); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isArchiveInternalRedirect()) { > + metaBox.addChild("#", "Document type: ArchiveInternalRedirect"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isArchiveManifest()) { > + metaBox.addChild("#", "Document type: ArchiveManifest"); > + metaBox.addChild("%", "
"); > + } > + > + if (!md.isCompressed()) { > + metaBox.addChild("#", "Uncompressed"); > + } else { > + metaBox.addChild("#", "Compressed (codec " + md.getCompressionCodec() + ")"); > + metaBox.addChild("%", "
"); > + metaBox.addChild("#", "Decompressed size: " + md.uncompressedDataLength() + " bytes"); > + } > + metaBox.addChild("%", "
"); > + > + metaBox.addChild("#", "Data size\u00a0=\u00a0" + md.dataLength()); > + metaBox.addChild("%", "
"); > + > + if (md.isResolved()) { > + metaBox.addChild("#", "Resolved URI:\u00a0=\u00a0" + md.getResolvedURI()); > + metaBox.addChild("%", "
"); > + } > + contentNode.addChild(metaBox); > + } > + > + } > + > + } > + > + return pageNode.generate(); > + } > + > + private HTMLNode createUriBox() { > + HTMLNode browseBox = m_pm.getInfobox("Explore a freenet key"); > + HTMLNode browseContent = m_pm.getContentNode(browseBox); > + browseContent.addChild("#", "Display the top level chunk as hexprint"); > + HTMLNode browseForm = m_pr.addFormChild(browseContent, "/plugins/plugins.KeyExplorer.KeyExplorer/", "uriForm"); > + browseForm.addChild("#", "Freenetkey to explore: \u00a0 "); > + browseForm.addChild("input", new String[] { "type", "name", "size" }, new String[] { "text", "key", "70" }); > + browseForm.addChild("#", "\u00a0"); > + browseForm.addChild("input", new String[] { "type", "name", "value" }, new String[] { "submit", "debug", "Explore!" }); > + return browseBox; > + } > + > + private HTMLNode createErrorBox(String errmsg) { > + HTMLNode errorBox = m_pm.getInfobox("infobox-alert", "ERROR"); > + errorBox.addChild("#", errmsg); > + return errorBox; > + } > + > +} > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/cd24f1f1/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 15 14:59:56 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 14:59:56 +0000 Subject: [freenet-dev] [freenet-cvs] r16548 - in trunk/plugins: . KeyExplorer In-Reply-To: <20071214205613.4B4113C073B@freenetproject.org> References: <20071214205613.4B4113C073B@freenetproject.org> Message-ID: <200712151459.56725.toad@amphibian.dyndns.org> Also, metadata is recursive: you probably want to decompose it recursively, maybe as a tree. E.g. simple manifest -> simple manifest -> redirect. On Friday 14 December 2007 20:56, you wrote: > Author: saces > Date: 2007-12-14 20:56:13 +0000 (Fri, 14 Dec 2007) > New Revision: 16548 > > Added: > trunk/plugins/KeyExplorer/ > trunk/plugins/KeyExplorer/KeyExplorer.java > Log: > new plugin: key info and debug tool > > Added: trunk/plugins/KeyExplorer/KeyExplorer.java > =================================================================== > --- trunk/plugins/KeyExplorer/KeyExplorer.java (rev 0) > +++ trunk/plugins/KeyExplorer/KeyExplorer.java 2007-12-14 20:56:13 UTC (rev 16548) > @@ -0,0 +1,243 @@ > +package plugins.KeyExplorer; > + > +import java.io.IOException; > +import java.net.MalformedURLException; > + > +import freenet.client.Metadata; > +import freenet.client.MetadataParseException; > +import freenet.clients.http.PageMaker; > +import freenet.config.Config; > +import freenet.config.SubConfig; > +import freenet.keys.BaseClientKey; > +import freenet.keys.ClientKey; > +import freenet.keys.ClientKeyBlock; > +import freenet.keys.FreenetURI; > +import freenet.keys.KeyDecodeException; > +import freenet.node.LowLevelGetException; > +import freenet.pluginmanager.FredPlugin; > +import freenet.pluginmanager.FredPluginHTTP; > +import freenet.pluginmanager.FredPluginThreadless; > +import freenet.pluginmanager.PluginHTTPException; > +import freenet.pluginmanager.PluginRespirator; > +import freenet.support.HTMLNode; > +import freenet.support.HexUtil; > +import freenet.support.api.HTTPRequest; > +import freenet.support.io.ArrayBucket; > +import freenet.support.io.ArrayBucketFactory; > +import freenet.support.io.BucketTools; > + > +public class KeyExplorer implements FredPlugin, FredPluginHTTP, FredPluginThreadless { > + > + private PluginRespirator m_pr; > + private PageMaker m_pm; > + > + public String handleHTTPGet(HTTPRequest request) throws PluginHTTPException { > + String uri = request.getParam("key"); > + return makeMainPage(uri); > + } > + > + public String handleHTTPPost(HTTPRequest request) throws PluginHTTPException { > + String uri = request.getPartAsString("key", 1024); > + return makeMainPage(uri); > + } > + > + public String handleHTTPPut(HTTPRequest request) throws PluginHTTPException { > + return makeMainPage(); > + } > + > + public void runPlugin(PluginRespirator pr) { > + Config nc = pr.getNode().config; > + SubConfig fc = nc.get("fproxy"); > + String cssName = fc.getString("css"); > + > + m_pm = new PageMaker(cssName); > + m_pr = pr; > + } > + > + public void terminate() { > + } > + > + private String makeMainPage() { > + return makeMainPage(null); > + } > + > + private String makeMainPage(String key) { > + HTMLNode pageNode = m_pm.getPageNode("KeyExplorer", null); > + HTMLNode contentNode = m_pm.getContentNode(pageNode); > + > + String error = null; > + byte[] data = null; > + ClientKeyBlock ckb = null; > + > + try { > + if (key != null && (key.trim().length() > 0)) { > + FreenetURI furi = new FreenetURI(key); > + ClientKey ck = (ClientKey) BaseClientKey.getBaseKey(furi); > + ckb = m_pr.getNode().clientCore.realGetKey(ck, false, true, false); > + ArrayBucket a = (ArrayBucket) ckb.decode(new ArrayBucketFactory(), 32 * 1024, false); > + data = BucketTools.toByteArray(a); > + } > + } catch (MalformedURLException e) { > + error = "MalformedURL"; > + } catch (LowLevelGetException e) { > + error = "get failed"; > + } catch (KeyDecodeException e) { > + error = "decode error"; > + } catch (IOException e) { > + error = "io error"; > + } > + > + if (error != null) { > + contentNode.addChild(createErrorBox(error)); > + } > + > + contentNode.addChild(createUriBox()); > + > + if (data != null) { > + String title = "Key: " + key; > + if (ckb.isMetadata()) > + title = title + "\u00a0(MetaData)"; > + HTMLNode dataBox2 = m_pm.getInfobox(title); > + > + char[] asciibuf = new char[16]; > + > + for (int j = 0; j < 16; j++) > + asciibuf[j] = ' '; > + > + dataBox2.addChild("%", "
\n");
> +			StringBuffer sb = new StringBuffer();
> +			int offset = 0;
> +
> +			for (int i = 0; i < data.length; i++) {
> +				offset = (i) % 16;
> +				HexUtil.bytesToHexAppend(data, i, 1, sb);
> +				sb.append(' ');
> +				if ((data[i] > 31) && (data[i] < 127)) {
> +					//int j = data[i];
> +					//sb.append((char) j);
> +					asciibuf[offset] =(char) data[i];
> +				} 
> +				
> +				if ((i > 1) && ((i + 1) % 16 == 0)) {
> +					sb.append(' ');
> +					sb.append(asciibuf);
> +					sb.append('\n');
> +					for (int k = 0; k < 16; k++) 
> +						asciibuf[k] = ' ';
> +				}
> +			}
> +			if (offset > 0) {
> +				int n = (15-offset)*3;
> +				for (int m = 0; m < n; m++) 
> +					sb.append(' ');
> +				sb.append(' ');
> +				sb.append(asciibuf);
> +			}
> +
> +			dataBox2.addChild("#", sb.toString());
> +			dataBox2.addChild("%", "\n
"); > + > + contentNode.addChild(dataBox2); > + > + error = null; > + > + if (ckb.isMetadata()) { > + > + Metadata md = null; > + try { > + md = Metadata.construct(data); > + } catch (MetadataParseException e) { > + error = "Metadata parse error"; > + } > + > + if (error != null) > + contentNode.addChild(createErrorBox(error)); > + > + if (md != null) { > + HTMLNode metaBox = m_pm.getInfobox("Decomposed metadata"); > + > + metaBox.addChild("#", "Document type:"); > + metaBox.addChild("%", "
"); > + > + if (md.isSimpleManifest()) { > + metaBox.addChild("#", "Document type: SimpleManifest"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSplitfile()) { > + metaBox.addChild("#", "Document type: Splitfile"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSimpleSplitfile()) { > + metaBox.addChild("#", "Document type: SimpleSplitfile"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isSingleFileRedirect()) { > + metaBox.addChild("#", "Document type: SingleFileRedirect"); > + metaBox.addChild("%", "
"); > + String sfrUri = md.getSingleTarget().toString(); > + metaBox.addChild("#", sfrUri); > + metaBox.addChild("#", "\u00a0"); > + metaBox.addChild(new HTMLNode("a", "href", "/?key="+sfrUri, "open")); > + metaBox.addChild("#", "\u00a0"); > + metaBox.addChild(new HTMLNode("a", "href", "/plugins/plugins.KeyExplorer.KeyExplorer/?key="+sfrUri, "explore")); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isArchiveInternalRedirect()) { > + metaBox.addChild("#", "Document type: ArchiveInternalRedirect"); > + metaBox.addChild("%", "
"); > + } > + > + if (md.isArchiveManifest()) { > + metaBox.addChild("#", "Document type: ArchiveManifest"); > + metaBox.addChild("%", "
"); > + } > + > + if (!md.isCompressed()) { > + metaBox.addChild("#", "Uncompressed"); > + } else { > + metaBox.addChild("#", "Compressed (codec " + md.getCompressionCodec() + ")"); > + metaBox.addChild("%", "
"); > + metaBox.addChild("#", "Decompressed size: " + md.uncompressedDataLength() + " bytes"); > + } > + metaBox.addChild("%", "
"); > + > + metaBox.addChild("#", "Data size\u00a0=\u00a0" + md.dataLength()); > + metaBox.addChild("%", "
"); > + > + if (md.isResolved()) { > + metaBox.addChild("#", "Resolved URI:\u00a0=\u00a0" + md.getResolvedURI()); > + metaBox.addChild("%", "
"); > + } > + contentNode.addChild(metaBox); > + } > + > + } > + > + } > + > + return pageNode.generate(); > + } > + > + private HTMLNode createUriBox() { > + HTMLNode browseBox = m_pm.getInfobox("Explore a freenet key"); > + HTMLNode browseContent = m_pm.getContentNode(browseBox); > + browseContent.addChild("#", "Display the top level chunk as hexprint"); > + HTMLNode browseForm = m_pr.addFormChild(browseContent, "/plugins/plugins.KeyExplorer.KeyExplorer/", "uriForm"); > + browseForm.addChild("#", "Freenetkey to explore: \u00a0 "); > + browseForm.addChild("input", new String[] { "type", "name", "size" }, new String[] { "text", "key", "70" }); > + browseForm.addChild("#", "\u00a0"); > + browseForm.addChild("input", new String[] { "type", "name", "value" }, new String[] { "submit", "debug", "Explore!" }); > + return browseBox; > + } > + > + private HTMLNode createErrorBox(String errmsg) { > + HTMLNode errorBox = m_pm.getInfobox("infobox-alert", "ERROR"); > + errorBox.addChild("#", errmsg); > + return errorBox; > + } > + > +} > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/766e0a97/attachment.pgp From toad at amphibian.dyndns.org Sat Dec 15 15:03:18 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 15:03:18 +0000 Subject: [freenet-dev] [freenet-cvs] r16554 - trunk/freenet/src/freenet/node In-Reply-To: <20071214235336.60C9147B7CD@freenetproject.org> References: <20071214235336.60C9147B7CD@freenetproject.org> Message-ID: <200712151503.19077.toad@amphibian.dyndns.org> Why display the countReports() as a double? It's just a long, isn't it? There's no such thing as half a report! On Friday 14 December 2007 23:53, robert at freenetproject.org wrote: > Author: robert > Date: 2007-12-14 23:53:36 +0000 (Fri, 14 Dec 2007) > New Revision: 16554 > > Modified: > trunk/freenet/src/freenet/node/NodeStats.java > Log: > Divide-by-zero/NaN (or the like) > > > Modified: trunk/freenet/src/freenet/node/NodeStats.java > =================================================================== > --- trunk/freenet/src/freenet/node/NodeStats.java 2007-12-14 23:39:56 UTC (rev 16553) > +++ trunk/freenet/src/freenet/node/NodeStats.java 2007-12-14 23:53:36 UTC (rev 16554) > @@ -1016,8 +1016,13 @@ > for(int i=0;i row = list.addChild("tr"); > row.addChild("td", names[i]); > + if (averages[i].countReports()==0) { > + row.addChild("td", "-"); > + row.addChild("td", "0.0"); > + } else { > row.addChild("td", fix3p3pct.format(averages[i].currentValue())); > row.addChild("td", Double.toString(averages[i].countReports())); > + } > } > } > > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/d5fb8e1a/attachment.pgp From thelema314 at gmail.com Sat Dec 15 16:01:15 2007 From: thelema314 at gmail.com (Edgar Friendly) Date: Sat, 15 Dec 2007 10:01:15 -0600 Subject: [freenet-dev] Usability testing results In-Reply-To: <475FB4C0.2010101@batosai.net> References: <200712110121.57813.toad@amphibian.dyndns.org> <200712111902.17457.toad@amphibian.dyndns.org> <475F25FB.6050909@cs.ucl.ac.uk> <200712120016.04506.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <475FB4C0.2010101@batosai.net> Message-ID: <4763FA4B.5080303@gmail.com> Julien Cornuwel wrote: > I would also add a warning for those living in countries where Freenet > is illegal : > > Note : If Freenet is illegal in your country, then connecting to > strangers could permit your government to know that you are using it. > You should try to convince your friends to install Freenet and only > connect to them. > In places where using Freenet is illegal, it's probably illegal for us to tell people "convince your friends to install Freenet". The idea has merit, but we probably can't speak this directly. E. _______________________________________________ Devl mailing list Devl at freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From toad at amphibian.dyndns.org Sat Dec 15 17:00:29 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 15 Dec 2007 17:00:29 +0000 Subject: [freenet-dev] Usability testing results In-Reply-To: <4763FA4B.5080303@gmail.com> References: <200712110121.57813.toad@amphibian.dyndns.org> <475FB4C0.2010101@batosai.net> <4763FA4B.5080303@gmail.com> Message-ID: <200712151700.33366.toad@amphibian.dyndns.org> On Saturday 15 December 2007 16:01, Edgar Friendly wrote: > Julien Cornuwel wrote: > > I would also add a warning for those living in countries where Freenet > > is illegal : > > > > Note : If Freenet is illegal in your country, then connecting to > > strangers could permit your government to know that you are using it. > > You should try to convince your friends to install Freenet and only > > connect to them. > > In places where using Freenet is illegal, it's probably illegal for us > to tell people "convince your friends to install Freenet". The idea has > merit, but we probably can't speak this directly. Not our problem. It's not illegal *where we are*, therefore we can happily incite freedom-loving dissidents to break their local laws.. provided we don't act irresponsibly (we need to be honest as to the risks involved). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071215/96ecf6ef/attachment.pgp From ian.clarke at gmail.com Sat Dec 15 19:10:48 2007 From: ian.clarke at gmail.com (Ian Clarke) Date: Sat, 15 Dec 2007 13:10:48 -0600 Subject: [freenet-dev] Usability testing results In-Reply-To: <4763DBD8.20708@WhenGendarmeSleeps.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <475F2B9E.5040608@cs.ucl.ac.uk> <4760D026.7070801@WhenGendarmeSleeps.org> <200712132225.14284.toad@amphibian.dyndns.org> <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> <4763DBD8.20708@WhenGendarmeSleeps.org> Message-ID: <823242bd0712151110s6c126a5bpdfe611c437438db6@mail.gmail.com> On 12/15/07, Volodya wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > "Warning: Freenet may be illegal in some countries. If you allow you > > to connect to strangers, those strangers will know that you are > > running Freenet, and this could be dangerous. Freenet is still beta > > software, and we offer no guarantees as to its security." > > Some ISPs have it written in the contract that you are not to run p2p > software, but that doesn't make it "illegal", maybe "disallowed" would > be a better word. Well, I believe it is illegal to violate a contract, and "disallowed" is a bit of an aesthetically awkward word in this context. Ian. -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From ian.clarke at gmail.com Sat Dec 15 19:12:46 2007 From: ian.clarke at gmail.com (Ian Clarke) Date: Sat, 15 Dec 2007 13:12:46 -0600 Subject: [freenet-dev] Usability testing results In-Reply-To: <200712151404.37515.toad@amphibian.dyndns.org> References: <200712110121.57813.toad@amphibian.dyndns.org> <823242bd0712141530g367f1cc2yf44ae8a16ae697a1@mail.gmail.com> <4763DBD8.20708@WhenGendarmeSleeps.org> <200712151404.37515.toad@amphibian.dyndns.org> Message-ID: <823242bd0712151112j113c59f2k9113e636b4e86a0c@mail.gmail.com> On 12/15/07, Matthew Toseland wrote: > On Saturday 15 December 2007 13:51, Volodya wrote: > > > "Warning: Freenet may be illegal in some countries. If you allow you > > > to connect to strangers, those strangers will know that you are > > > running Freenet, and this could be dangerous. Freenet is still beta > > > software, and we offer no guarantees as to its security." > > > > Some ISPs have it written in the contract that you are not to run p2p > software, but that doesn't make it > > "illegal", maybe "disallowed" would be a better word. > > And some major governments block it on their national firewalls, and have the > death penalty for "revealing state secrets" and similar speech crimes. We really need to avoid the temptation to get carried away trying to make what needs to be a simple piece of explanatory text somehow cover every eventuality as if it was a legal document. If its too long or two complicated then nobody will read it, and then all of this wordsmithing will be moot anyway. Ian. -- Email: ian.clarke at gmail.com Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From nextgens at freenetproject.org Sat Dec 15 23:20:45 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Sun, 16 Dec 2007 00:20:45 +0100 Subject: [freenet-dev] Auto-assigned IPv6 addresses in Vista In-Reply-To: <735027.16478.qm@web81103.mail.mud.yahoo.com> References: <735027.16478.qm@web81103.mail.mud.yahoo.com> Message-ID: <20071215232044.GA12452@freenetproject.org> * Thomas Young [2007-11-29 14:09:55]: > > 3. When IP addresses are detected by > > IPAddressDetector, there is no checking to see > whether > > this interface is actually connected to another > > computer. In my case, > IPAddressDetector.getAddress() > > returns an IPv6 address that is assigned to one of > my > > Ethernet ports. I am not sure how this address was > > assigned, since that port is not connected to > anything > > at the moment. I am using a wireless adapter. I am > > guessing that other people like me who don't know > how > > to flush old IPv6 addresses will have this problem. > > The consequence of this is that IPDetectorPlugins > will > > not be used because the plugin manager thinks I have > a > > public IP address and am just firewalled at the > > moment. The only way I see to fix this is to either > > ignore all IPv6 addresses or find a way to detect > > whether this interface is connected to another > > computer. > > Ugh. Well if they are auto-assigned they should show > up as link-local, > so it > would still run the detection, no? > > > > Thank you for reading this. > > > > -Thomas Young > > I have done a little more research and it seems that > Vista creates virtual adapters that have IPv6 > addresses. These addresses are not link local since > they start with 2001:0/32 for me. In any case, > IPAddressDetector.getAddress() happily returns this as > a public IPv6 address. I've commited a fix in r16588, please try it out and report back if it doesn't work :) NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071216/24ecd53a/attachment.pgp From toad at amphibian.dyndns.org Sun Dec 16 00:04:31 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sun, 16 Dec 2007 00:04:31 +0000 Subject: [freenet-dev] Freenet 0.7 build 1091 Message-ID: <200712160004.36504.toad@amphibian.dyndns.org> Freenet 0.7 build 1091 is now available. Please upgrade. This build contains many fixes and improvements to the announcement code in various areas (UI, too big packets, ...), some improvements to port forwarding detection and IP detection, fixes related to running out of disk space, some changes to shrink noderefs slightly, and more. It also turns off the location swapping debugging messages for now, as they were causing problems and not yielding that much useful data. Please report any bugs you find ASAP through the bug tracker, mailing lists, IRC, Frost etc. We are trying to get ready for another alpha release of 0.7, in the near future. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071216/e01d944e/attachment.pgp From batosai at batosai.net Sun Dec 16 10:24:14 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Sun, 16 Dec 2007 11:24:14 +0100 Subject: [freenet-dev] Updated French translation for #1091 Message-ID: <4764FCCE.60405@batosai.net> Hi, Here is an updated translation file with several changes to the opennet warnings (as they've been changed in the english version). Regards -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.fr.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071216/de6aee82/attachment.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071216/de6aee82/attachment.pgp From batosai at batosai.net Sun Dec 16 11:10:30 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Sun, 16 Dec 2007 12:10:30 +0100 Subject: [freenet-dev] Missing seednodes.fref Message-ID: <476507A6.60606@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Could we modify the message about missing seednodes.fref so it gives the URL where it can be downloaded ? Of course, it would be better if the node would automagically try to download it but giving the URL would be a good start. Regards -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHZQemo6N05NzaOvURApEcAKCXzO/pOJNKiT4PbVLg0+YhA5XufgCfYwhV Qn3NXAc+8MTFx44mpIM9PKQ= =GpbO -----END PGP SIGNATURE----- From robert at emu.freenetproject.org Mon Dec 17 16:41:03 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Mon, 17 Dec 2007 10:41:03 -0600 Subject: [freenet-dev] r16554 - trunk/freenet/src/freenet/node In-Reply-To: <200712151503.19077.toad@amphibian.dyndns.org> References: <20071214235336.60C9147B7CD@freenetproject.org> <200712151503.19077.toad@amphibian.dyndns.org> Message-ID: <128BCC75-FAB4-45C5-9639-1EB21FE37C6B@emu.freenetproject.org> On Dec 15, 2007, at 9:03 AM, Matthew Toseland wrote: > Why display the countReports() as a double? It's just a long, isn't > it? > There's no such thing as half a report! I noticed that too. Although odd, I thought I wouldn't change it, as after a few million requests it would start representing it in scientific notation. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071217/b348936b/attachment.htm From robert at emu.freenetproject.org Mon Dec 17 17:07:55 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Mon, 17 Dec 2007 11:07:55 -0600 Subject: [freenet-dev] [freenet-cvs] r16635 - trunk/freenet/src/freenet/node In-Reply-To: <20071217001157.7592447A061@freenetproject.org> References: <20071217001157.7592447A061@freenetproject.org> Message-ID: <42922DAA-7A20-4141-B3E4-D3EA9ED2ED8F@emu.freenetproject.org> On Dec 16, 2007, at 6:11 PM, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-17 00:11:57 +0000 (Mon, 17 Dec 2007) > New Revision: 16635 > > Modified: > trunk/freenet/src/freenet/node/RequestHandler.java > Log: > Fix a silly bug which might have weird effects > > Modified: trunk/freenet/src/freenet/node/RequestHandler.java > =================================================================== > --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 > 00:11:18 UTC (rev 16634) > +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 > 00:11:57 UTC (rev 16635) > @@ -246,13 +246,13 @@ > } else { > sendTerminal(df); > } > - return; > } else { > if(!rs.transferStarted()) { > Logger.error(this, "Status is SUCCESS but we never started a > transfer on "+uid); > } > // Wait for transfer to start > } > + return; > case RequestSender.VERIFY_FAILURE: > if(key instanceof NodeCHK) { > if(shouldHaveStartedTransfer) > I'm not so sure... when I was reviewing the code earlier the comment "wait for transfer to start" made me think this was intentional. Specifically, that the code will loop and consider it again? Certainly returning immediately is not waiting for the transfer to start, although... as written it would simply run on into the VERIFY_FAILURE action, and I wouldn't think that would be desired. -- Robert Hailey From nextgens at freenetproject.org Mon Dec 17 17:35:07 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 17 Dec 2007 18:35:07 +0100 Subject: [freenet-dev] [freenet-cvs] r16635 - trunk/freenet/src/freenet/node In-Reply-To: <42922DAA-7A20-4141-B3E4-D3EA9ED2ED8F@emu.freenetproject.org> References: <20071217001157.7592447A061@freenetproject.org> <42922DAA-7A20-4141-B3E4-D3EA9ED2ED8F@emu.freenetproject.org> Message-ID: <20071217173506.GB4388@freenetproject.org> * Robert Hailey [2007-12-17 11:07:55]: > > On Dec 16, 2007, at 6:11 PM, nextgens at freenetproject.org wrote: > > > Author: nextgens > > Date: 2007-12-17 00:11:57 +0000 (Mon, 17 Dec 2007) > > New Revision: 16635 > > > > Modified: > > trunk/freenet/src/freenet/node/RequestHandler.java > > Log: > > Fix a silly bug which might have weird effects > > > > Modified: trunk/freenet/src/freenet/node/RequestHandler.java > > =================================================================== > > --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 > > 00:11:18 UTC (rev 16634) > > +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 > > 00:11:57 UTC (rev 16635) > > @@ -246,13 +246,13 @@ > > } else { > > sendTerminal(df); > > } > > - return; > > } else { > > if(!rs.transferStarted()) { > > Logger.error(this, "Status is SUCCESS but we never started a > > transfer on "+uid); > > } > > // Wait for transfer to start > > } > > + return; > > case RequestSender.VERIFY_FAILURE: > > if(key instanceof NodeCHK) { > > if(shouldHaveStartedTransfer) > > > > I'm not so sure... when I was reviewing the code earlier the comment > "wait for transfer to start" made me think this was intentional. > Specifically, that the code will loop and consider it again? > > Certainly returning immediately is not waiting for the transfer to > start, although... as written it would simply run on into the > VERIFY_FAILURE action ... Which is sending an FNPRejectedOverload message... That's definitly not what we want to do in that case. Let's "continue" instead... see r16659 NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071217/5bd0fa48/attachment.pgp From robert at emu.freenetproject.org Mon Dec 17 17:44:57 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Mon, 17 Dec 2007 11:44:57 -0600 Subject: [freenet-dev] [freenet-cvs] r16635 - trunk/freenet/src/freenet/node In-Reply-To: <20071217173506.GB4388@freenetproject.org> References: <20071217001157.7592447A061@freenetproject.org> <42922DAA-7A20-4141-B3E4-D3EA9ED2ED8F@emu.freenetproject.org> <20071217173506.GB4388@freenetproject.org> Message-ID: On Dec 17, 2007, at 11:35 AM, Florent Daigni?re wrote: > * Robert Hailey [2007-12-17 11:07:55]: > >> >> On Dec 16, 2007, at 6:11 PM, nextgens at freenetproject.org wrote: >> >>> Author: nextgens >>> Date: 2007-12-17 00:11:57 +0000 (Mon, 17 Dec 2007) >>> New Revision: 16635 >>> >>> Modified: >>> trunk/freenet/src/freenet/node/RequestHandler.java >>> Log: >>> Fix a silly bug which might have weird effects >>> >>> Modified: trunk/freenet/src/freenet/node/RequestHandler.java >>> =================================================================== >>> --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 >>> 00:11:18 UTC (rev 16634) >>> +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-12-17 >>> 00:11:57 UTC (rev 16635) >>> @@ -246,13 +246,13 @@ >>> } else { >>> sendTerminal(df); >>> } >>> - return; >>> } else { >>> if(!rs.transferStarted()) { >>> Logger.error(this, "Status is SUCCESS but we never started a >>> transfer on "+uid); >>> } >>> // Wait for transfer to start >>> } >>> + return; >>> case RequestSender.VERIFY_FAILURE: >>> if(key instanceof NodeCHK) { >>> if(shouldHaveStartedTransfer) >>> >> >> I'm not so sure... when I was reviewing the code earlier the comment >> "wait for transfer to start" made me think this was intentional. >> Specifically, that the code will loop and consider it again? >> >> Certainly returning immediately is not waiting for the transfer to >> start, although... as written it would simply run on into the >> VERIFY_FAILURE action > ... > > Which is sending an FNPRejectedOverload message... That's definitly > not > what we want to do in that case. > > Let's "continue" instead... see r16659 > > NextGen$ I'm still confused by that comment. If we continue & loop, will the status ever change from SUCCESS? And why is this behavior so much different from a SSK block? -- Robert Hailey From robert at emu.freenetproject.org Mon Dec 17 20:27:27 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Mon, 17 Dec 2007 14:27:27 -0600 Subject: [freenet-dev] [freenet-cvs] r16635 - trunk/freenet/src/freenet/node In-Reply-To: References: <20071217001157.7592447A061@freenetproject.org> <42922DAA-7A20-4141-B3E4-D3EA9ED2ED8F@emu.freenetproject.org> <20071217173506.GB4388@freenetproject.org> Message-ID: <4FC6C7F5-6432-4F48-BA3E-4C5E6D9013C1@emu.freenetproject.org> On Dec 17, 2007, at 11:44 AM, Robert Hailey wrote: > > On Dec 17, 2007, at 11:35 AM, Florent Daigni?re wrote: > >> * Robert Hailey [2007-12-17 >> 11:07:55]: >> >> Which is sending an FNPRejectedOverload message... That's definitly >> not >> what we want to do in that case. >> >> Let's "continue" instead... see r16659 >> >> NextGen$ > > I'm still confused by that comment. If we continue & loop, will the > status ever change from SUCCESS? And why is this behavior so much > different from a SSK block? > > -- > Robert Hailey Ok... I've convinced myself that simply looping won't do anything useful (as best I can tell it will just get caught on the waitUntilStatusChange();). Apparently (in r14068) the return statement was purposefully dropped/ moved, and the comment in question was added. I'd let Matthew decide. Perhaps sending a failure notify is what is wanted? -- Robert Hailey r14068-20070713-toad-"Fix opennet allow-re-add logic, drop unused variable completedHandshake from PeerNode" [...] Modified: trunk/freenet/src/freenet/node/RequestHandler.java =================================================================== --- trunk/freenet/src/freenet/node/RequestHandler.java 2007-07-13 13:21:16 UTC (rev 14067) +++ trunk/freenet/src/freenet/node/RequestHandler.java 2007-07-13 13:38:13 UTC (rev 14068) @@ -184,12 +184,13 @@ Message pk = DMT.createFNPSSKPubKey(uid, ((NodeSSK)rs.getSSKBlock().getKey()).getPubKey()); source.sendSync(pk, this); } + return; } else { if(!rs.transferStarted()) { Logger.error(this, "Status is SUCCESS but we never started a transfer on "+uid); } + // Wait for transfer to start } - return; case RequestSender.VERIFY_FAILURE: if(key instanceof NodeCHK) { if(shouldHaveStartedTransfer) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20071217/a2c291cc/attachment.htm From toad at amphibian.dyndns.org Tue Dec 18 13:57:28 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 13:57:28 +0000 Subject: [freenet-dev] r16554 - trunk/freenet/src/freenet/node In-Reply-To: <128BCC75-FAB4-45C5-9639-1EB21FE37C6B@emu.freenetproject.org> References: <20071214235336.60C9147B7CD@freenetproject.org> <200712151503.19077.toad@amphibian.dyndns.org> <128BCC75-FAB4-45C5-9639-1EB21FE37C6B@emu.freenetproject.org> Message-ID: <200712181357.32941.toad@amphibian.dyndns.org> On Monday 17 December 2007 16:41, Robert Hailey wrote: > > On Dec 15, 2007, at 9:03 AM, Matthew Toseland wrote: > > > Why display the countReports() as a double? It's just a long, isn't > > it? > > There's no such thing as half a report! > > I noticed that too. Although odd, I thought I wouldn't change it, as > after a few million requests it would start representing it in > scientific notation. That's not necessarily more readable. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/3cfcae7b/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 18 14:53:50 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 14:53:50 +0000 Subject: [freenet-dev] =?utf-8?q?=5Bfreenet-cvs=5D_r16635_-=09trunk/freene?= =?utf-8?q?t/src/freenet/node?= In-Reply-To: <4FC6C7F5-6432-4F48-BA3E-4C5E6D9013C1@emu.freenetproject.org> References: <20071217001157.7592447A061@freenetproject.org> <4FC6C7F5-6432-4F48-BA3E-4C5E6D9013C1@emu.freenetproject.org> Message-ID: <200712181454.00486.toad@amphibian.dyndns.org> On Monday 17 December 2007 20:27, Robert Hailey wrote: > > On Dec 17, 2007, at 11:44 AM, Robert Hailey wrote: > > > > On Dec 17, 2007, at 11:35 AM, Florent Daigni?re wrote: > > > >> * Robert Hailey [2007-12-17 > >> 11:07:55]: > >> > >> Which is sending an FNPRejectedOverload message... That's definitly > >> not > >> what we want to do in that case. > >> > >> Let's "continue" instead... see r16659 This is the correct fix. But it's not obvious why. I've put comments in to explain the situation. > >> > >> NextGen$ > > > > I'm still confused by that comment. If we continue & loop, will the > > status ever change from SUCCESS? And why is this behavior so much > > different from a SSK block? > > Ok... I've convinced myself that simply looping won't do anything > useful (as best I can tell it will just get caught on the > waitUntilStatusChange();). That's the point! There are two cases: If a transfer HAS started, we didn't pick it up in waitUntilStatusChange() due to a race condition. The correct behaviour is to go back around the loop and pick it up, do the transfer and exit. On the other hand, if a transfer has not started, we've got a bug. The correct behaviour is unclear: if it's a synchronization issue, then going round the loop again may help, but otoh if the sender has actually exited, we may get stuck forever... so I have changed it to send a RejectedOverload. So the net effect of all of this is to not fail with RejectedOverload when we get a race condition on the status vs transferStarted(). > > Apparently (in r14068) the return statement was purposefully dropped/ > moved, and the comment in question was added. I'd let Matthew decide. > Perhaps sending a failure notify is what is wanted? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/4a4eec63/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 18 15:19:59 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 15:19:59 +0000 Subject: [freenet-dev] [freenet-cvs] r16638 - trunk/freenet/src/freenet/client In-Reply-To: <20071217002438.7D67E47BBD0@freenetproject.org> References: <20071217002438.7D67E47BBD0@freenetproject.org> Message-ID: <200712181519.59932.toad@amphibian.dyndns.org> I don't get it. On Monday 17 December 2007 00:24, you wrote: > Author: nextgens > Date: 2007-12-17 00:24:38 +0000 (Mon, 17 Dec 2007) > New Revision: 16638 > > Modified: > trunk/freenet/src/freenet/client/FECCodec.java > Log: > FECCodec: fix the logic > > Modified: trunk/freenet/src/freenet/client/FECCodec.java > =================================================================== > --- trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:23:06 UTC (rev 16637) > +++ trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:24:38 UTC (rev 16638) > @@ -323,14 +323,13 @@ > public static void addToQueue(FECJob job, FECCodec codec) { > synchronized(_awaitingJobs) { > if(fecRunnerThread == null) { > - if(fecRunnerThread != null) > - Logger.error(FECCodec.class, "The callback died!! restarting a new one, please report that error."); > fecRunnerThread = new Thread(fecRunner, "FEC Pool " + (fecPoolCounter++)); > fecRunnerThread.setDaemon(true); > fecRunnerThread.setPriority(Thread.MIN_PRIORITY); > > fecRunnerThread.start(); > - } > + }else > + Logger.error(FECCodec.class, "The callback died!! restarting a new one, please report that error."); > > _awaitingJobs.addFirst(job); > } > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/e43dba89/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 18 15:21:28 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 15:21:28 +0000 Subject: [freenet-dev] [freenet-cvs] r16639 - trunk/freenet/src/freenet/support/math In-Reply-To: <20071217003923.F3A0D478187@freenetproject.org> References: <20071217003923.F3A0D478187@freenetproject.org> Message-ID: <200712181521.28280.toad@amphibian.dyndns.org> On Monday 17 December 2007 00:39, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-17 00:39:23 +0000 (Mon, 17 Dec 2007) > New Revision: 16639 > > Modified: > trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > Log: > TimeDecayingRunningAverage: we were loosing in precision here (float => int) No we're not, it's a long! > > Modified: trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > =================================================================== > --- trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java 2007-12-17 00:24:38 UTC (rev 16638) > +++ trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java 2007-12-17 00:39:23 UTC (rev 16639) > @@ -184,7 +184,8 @@ > timeSkewCallback.setTimeSkewDetectedUserAlert(); > return; > } else { > - if((uptime / 4) < thisHalfLife) thisHalfLife = (uptime / 4); > + double oneFourthOfUptime = uptime / 4D; > + if(oneFourthOfUptime < thisHalfLife) thisHalfLife = oneFourthOfUptime; > } > > if(thisHalfLife == 0) thisHalfLife = 1; > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/3012f38a/attachment.pgp From nextgens at freenetproject.org Tue Dec 18 15:25:56 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 18 Dec 2007 16:25:56 +0100 Subject: [freenet-dev] [freenet-cvs] r16638 - trunk/freenet/src/freenet/client In-Reply-To: <200712181519.59932.toad@amphibian.dyndns.org> References: <20071217002438.7D67E47BBD0@freenetproject.org> <200712181519.59932.toad@amphibian.dyndns.org> Message-ID: <20071218152556.GB4205@freenetproject.org> * Matthew Toseland [2007-12-18 15:19:59]: > I don't get it. I don't get what you don't get :) What it was doing was : 1) take the mutex 2) test if fecRunnerThread is null 3) if so, *re*test if it's not (always false) 4) then log something... What it should do is : 1) take the mutex 2) test if fecRunnerThread is null 3) if it's not, log something > > On Monday 17 December 2007 00:24, you wrote: > > Author: nextgens > > Date: 2007-12-17 00:24:38 +0000 (Mon, 17 Dec 2007) > > New Revision: 16638 > > > > Modified: > > trunk/freenet/src/freenet/client/FECCodec.java > > Log: > > FECCodec: fix the logic > > > > Modified: trunk/freenet/src/freenet/client/FECCodec.java > > =================================================================== > > --- trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:23:06 UTC > (rev 16637) > > +++ trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:24:38 UTC > (rev 16638) > > @@ -323,14 +323,13 @@ > > public static void addToQueue(FECJob job, FECCodec codec) { > > synchronized(_awaitingJobs) { > > if(fecRunnerThread == null) { > > - if(fecRunnerThread != null) > > - Logger.error(FECCodec.class, "The callback died!! restarting a new > one, please report that error."); > > fecRunnerThread = new Thread(fecRunner, "FEC Pool " + > (fecPoolCounter++)); > > fecRunnerThread.setDaemon(true); > > fecRunnerThread.setPriority(Thread.MIN_PRIORITY); > > > > fecRunnerThread.start(); > > - } > > + }else > > + Logger.error(FECCodec.class, "The callback died!! restarting a new one, > please report that error."); > > > > _awaitingJobs.addFirst(job); > > } > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/69dbcdcd/attachment.pgp From nextgens at freenetproject.org Tue Dec 18 15:27:39 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 18 Dec 2007 16:27:39 +0100 Subject: [freenet-dev] [freenet-cvs] r16639 - trunk/freenet/src/freenet/support/math In-Reply-To: <200712181521.28280.toad@amphibian.dyndns.org> References: <20071217003923.F3A0D478187@freenetproject.org> <200712181521.28280.toad@amphibian.dyndns.org> Message-ID: <20071218152738.GC4205@freenetproject.org> * Matthew Toseland [2007-12-18 15:21:28]: > On Monday 17 December 2007 00:39, nextgens at freenetproject.org wrote: > > Author: nextgens > > Date: 2007-12-17 00:39:23 +0000 (Mon, 17 Dec 2007) > > New Revision: 16639 > > > > Modified: > > trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > Log: > > TimeDecayingRunningAverage: we were loosing in precision here (float => int) > > No we're not, it's a long! The commit message is wrong indeed, we were dividing a long by an int and casting it to a double. > > > > Modified: > trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > =================================================================== > > --- trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > 2007-12-17 00:24:38 UTC (rev 16638) > > +++ trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > 2007-12-17 00:39:23 UTC (rev 16639) > > @@ -184,7 +184,8 @@ > > timeSkewCallback.setTimeSkewDetectedUserAlert(); > > return; > > } else { > > - if((uptime / 4) < thisHalfLife) thisHalfLife = (uptime / 4); > > + double oneFourthOfUptime = uptime / 4D; > > + if(oneFourthOfUptime < thisHalfLife) thisHalfLife = oneFourthOfUptime; > > } > > > > if(thisHalfLife == 0) thisHalfLife = 1; > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/e005d7c9/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 18 15:37:29 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 15:37:29 +0000 Subject: [freenet-dev] =?utf-8?q?=5Bfreenet-cvs=5D_r16639_-=09trunk/freene?= =?utf-8?q?t/src/freenet/support/math?= In-Reply-To: <20071218152738.GC4205@freenetproject.org> References: <20071217003923.F3A0D478187@freenetproject.org> <200712181521.28280.toad@amphibian.dyndns.org> <20071218152738.GC4205@freenetproject.org> Message-ID: <200712181537.35550.toad@amphibian.dyndns.org> On Tuesday 18 December 2007 15:27, Florent Daigni?re wrote: > * Matthew Toseland [2007-12-18 15:21:28]: > > > On Monday 17 December 2007 00:39, nextgens at freenetproject.org wrote: > > > Author: nextgens > > > Date: 2007-12-17 00:39:23 +0000 (Mon, 17 Dec 2007) > > > New Revision: 16639 > > > > > > Modified: > > > trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > > Log: > > > TimeDecayingRunningAverage: we were loosing in precision here (float => int) > > > > No we're not, it's a long! > > The commit message is wrong indeed, we were dividing a long by an int > and casting it to a double. The fix is okay though. > > > > > > > Modified: > > trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > > =================================================================== > > > --- trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > 2007-12-17 00:24:38 UTC (rev 16638) > > > +++ trunk/freenet/src/freenet/support/math/TimeDecayingRunningAverage.java > > 2007-12-17 00:39:23 UTC (rev 16639) > > > @@ -184,7 +184,8 @@ > > > timeSkewCallback.setTimeSkewDetectedUserAlert(); > > > return; > > > } else { > > > - if((uptime / 4) < thisHalfLife) thisHalfLife = (uptime / 4); > > > + double oneFourthOfUptime = uptime / 4D; > > > + if(oneFourthOfUptime < thisHalfLife) thisHalfLife = oneFourthOfUptime; > > > } > > > > > > if(thisHalfLife == 0) thisHalfLife = 1; > > > > > > _______________________________________________ > > > cvs mailing list > > > cvs at freenetproject.org > > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > > > > > > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/19871b7c/attachment.pgp From nextgens at freenetproject.org Tue Dec 18 15:39:13 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 18 Dec 2007 16:39:13 +0100 Subject: [freenet-dev] [freenet-cvs] r16672 - trunk/freenet/src/freenet/client/async In-Reply-To: <20071218153211.C21E948106A@freenetproject.org> References: <20071218153211.C21E948106A@freenetproject.org> Message-ID: <20071218153912.GD4205@freenetproject.org> * toad at freenetproject.org [2007-12-18 15:32:11]: > Author: toad > Date: 2007-12-18 15:32:11 +0000 (Tue, 18 Dec 2007) > New Revision: 16672 > > Modified: > trunk/freenet/src/freenet/client/async/BinaryBlobInserter.java > Log: > Not necessary > That's everything *but* clear without the 'super' prefix... We are in a nested class here and the "main" class has a method called schedule() too! > Modified: trunk/freenet/src/freenet/client/async/BinaryBlobInserter.java > =================================================================== > --- trunk/freenet/src/freenet/client/async/BinaryBlobInserter.java 2007-12-18 15:00:45 UTC (rev 16671) > +++ trunk/freenet/src/freenet/client/async/BinaryBlobInserter.java 2007-12-18 15:32:11 UTC (rev 16672) > @@ -170,7 +170,7 @@ > fail(InsertException.construct(errors), false); > return; > } > - super.schedule(); > + schedule(); > } > > private void fail(InsertException e, boolean fatal) { -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/26effad1/attachment.pgp From toad at amphibian.dyndns.org Tue Dec 18 15:40:03 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 15:40:03 +0000 Subject: [freenet-dev] =?utf-8?q?=5Bfreenet-cvs=5D_r16638_-=09trunk/freene?= =?utf-8?q?t/src/freenet/client?= In-Reply-To: <20071218152556.GB4205@freenetproject.org> References: <20071217002438.7D67E47BBD0@freenetproject.org> <200712181519.59932.toad@amphibian.dyndns.org> <20071218152556.GB4205@freenetproject.org> Message-ID: <200712181540.03923.toad@amphibian.dyndns.org> On Tuesday 18 December 2007 15:25, Florent Daigni?re wrote: > * Matthew Toseland [2007-12-18 15:19:59]: > > > I don't get it. > > I don't get what you don't get :) > > What it was doing was : > 1) take the mutex > 2) test if fecRunnerThread is null > 3) if so, *re*test if it's not (always false) > 4) then log something... > > What it should do is : > 1) take the mutex > 2) test if fecRunnerThread is null > 3) if it's not, log something Yes but we log an error if it's NOT null ... what's going on here? > > > > > On Monday 17 December 2007 00:24, you wrote: > > > Author: nextgens > > > Date: 2007-12-17 00:24:38 +0000 (Mon, 17 Dec 2007) > > > New Revision: 16638 > > > > > > Modified: > > > trunk/freenet/src/freenet/client/FECCodec.java > > > Log: > > > FECCodec: fix the logic > > > > > > Modified: trunk/freenet/src/freenet/client/FECCodec.java > > > =================================================================== > > > --- trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:23:06 UTC > > (rev 16637) > > > +++ trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:24:38 UTC > > (rev 16638) > > > @@ -323,14 +323,13 @@ > > > public static void addToQueue(FECJob job, FECCodec codec) { > > > synchronized(_awaitingJobs) { > > > if(fecRunnerThread == null) { > > > - if(fecRunnerThread != null) > > > - Logger.error(FECCodec.class, "The callback died!! restarting a new > > one, please report that error."); > > > fecRunnerThread = new Thread(fecRunner, "FEC Pool " + > > (fecPoolCounter++)); > > > fecRunnerThread.setDaemon(true); > > > fecRunnerThread.setPriority(Thread.MIN_PRIORITY); > > > > > > fecRunnerThread.start(); > > > - } > > > + }else > > > + Logger.error(FECCodec.class, "The callback died!! restarting a new one, > > please report that error."); > > > > > > _awaitingJobs.addFirst(job); > > > } -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/11aa7e9d/attachment.pgp From nextgens at freenetproject.org Tue Dec 18 15:52:59 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Tue, 18 Dec 2007 16:52:59 +0100 Subject: [freenet-dev] [freenet-cvs] r16638 -?trunk/freenet/src/freenet/client In-Reply-To: <200712181540.03923.toad@amphibian.dyndns.org> References: <20071217002438.7D67E47BBD0@freenetproject.org> <200712181519.59932.toad@amphibian.dyndns.org> <20071218152556.GB4205@freenetproject.org> <200712181540.03923.toad@amphibian.dyndns.org> Message-ID: <20071218155259.GE4205@freenetproject.org> * Matthew Toseland [2007-12-18 15:40:03]: > On Tuesday 18 December 2007 15:25, Florent Daigni?re wrote: > > * Matthew Toseland [2007-12-18 15:19:59]: > > > > > I don't get it. > > > > I don't get what you don't get :) > > > > What it was doing was : > > 1) take the mutex > > 2) test if fecRunnerThread is null > > 3) if so, *re*test if it's not (always false) > > 4) then log something... > > > > What it should do is : > > 1) take the mutex > > 2) test if fecRunnerThread is null > > 3) if it's not, log something > > Yes but we log an error if it's NOT null ... what's going on here? fixed in r16675-r16676 > > > > > > > > On Monday 17 December 2007 00:24, you wrote: > > > > Author: nextgens > > > > Date: 2007-12-17 00:24:38 +0000 (Mon, 17 Dec 2007) > > > > New Revision: 16638 > > > > > > > > Modified: > > > > trunk/freenet/src/freenet/client/FECCodec.java > > > > Log: > > > > FECCodec: fix the logic > > > > > > > > Modified: trunk/freenet/src/freenet/client/FECCodec.java > > > > =================================================================== > > > > --- trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:23:06 > UTC > > > (rev 16637) > > > > +++ trunk/freenet/src/freenet/client/FECCodec.java 2007-12-17 00:24:38 > UTC > > > (rev 16638) > > > > @@ -323,14 +323,13 @@ > > > > public static void addToQueue(FECJob job, FECCodec codec) { > > > > synchronized(_awaitingJobs) { > > > > if(fecRunnerThread == null) { > > > > - if(fecRunnerThread != null) > > > > - Logger.error(FECCodec.class, "The callback died!! restarting a new > > > one, please report that error."); > > > > fecRunnerThread = new Thread(fecRunner, "FEC Pool " + > > > (fecPoolCounter++)); > > > > fecRunnerThread.setDaemon(true); > > > > fecRunnerThread.setPriority(Thread.MIN_PRIORITY); > > > > > > > > fecRunnerThread.start(); > > > > - } > > > > + }else > > > > + Logger.error(FECCodec.class, "The callback died!! restarting a new > one, > > > please report that error."); > > > > > > > > _awaitingJobs.addFirst(job); > > > > } -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/9297c385/attachment.pgp From robert at emu.freenetproject.org Tue Dec 18 16:16:02 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Tue, 18 Dec 2007 10:16:02 -0600 Subject: [freenet-dev] r16554 - trunk/freenet/src/freenet/node In-Reply-To: <200712181357.32941.toad@amphibian.dyndns.org> References: <20071214235336.60C9147B7CD@freenetproject.org> <200712151503.19077.toad@amphibian.dyndns.org> <128BCC75-FAB4-45C5-9639-1EB21FE37C6B@emu.freenetproject.org> <200712181357.32941.toad@amphibian.dyndns.org> Message-ID: <6BAF0C0B-4E91-4386-88AC-2B6F1CEA7E9E@emu.freenetproject.org> On Dec 18, 2007, at 7:57 AM, Matthew Toseland wrote: > On Monday 17 December 2007 16:41, Robert Hailey wrote: >> >> On Dec 15, 2007, at 9:03 AM, Matthew Toseland wrote: >> >>> Why display the countReports() as a double? It's just a long, isn't >>> it? >>> There's no such thing as half a report! >> >> I noticed that too. Although odd, I thought I wouldn't change it, as >> after a few million requests it would start representing it in >> scientific notation. > > That's not necessarily more readable. Agreed, changed to comma-separated thousands in r16684. -- Robert Hailey From toad at amphibian.dyndns.org Tue Dec 18 18:13:47 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 18:13:47 +0000 Subject: [freenet-dev] Freenet 0.7 build 1092 Message-ID: <200712181813.48991.toad@amphibian.dyndns.org> Freenet 0.7 build 1092 is now available. Please upgrade. Changes include: - Fixed a major bug which caused us to lose various critical files when we ran out of disk space. - More improvements to the announcement code. - Various minor optimisations and bugfixes. There has also been significant work done recently on the download webpage. Please upgrade! This build is mandatory on Thursday. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/645dc57f/attachment.pgp From robert at emu.freenetproject.org Tue Dec 18 20:23:20 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Tue, 18 Dec 2007 14:23:20 -0600 Subject: [freenet-dev] [freenet-cvs] r16700 - trunk/freenet/src/freenet/node In-Reply-To: <20071218201610.EC0E0480F52@freenetproject.org> References: <20071218201610.EC0E0480F52@freenetproject.org> Message-ID: Argghh, sorry. Bad commit-log comment... On Dec 18, 2007, at 2:16 PM, robert at freenetproject.org wrote: > @@ -725,7 +729,8 @@ > } > } > > - if(mf == null) { > + if (mf==null) { > + if (logMINOR) Logger.minor(this, "Done waiting, no more > completion listeners"); > return; > } else { > Message m; I thought that I had added this return statement, but instead committed some minor cleanups/comments and added more log noise. -- Robert Hailey From robert at emu.freenetproject.org Tue Dec 18 21:46:29 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Tue, 18 Dec 2007 15:46:29 -0600 Subject: [freenet-dev] [freenet-cvs] r16700 - trunk/freenet/src/freenet/node In-Reply-To: References: <20071218201610.EC0E0480F52@freenetproject.org> Message-ID: <0D9F82D2-387D-457C-972A-BF4A843160E7@emu.freenetproject.org> On Dec 18, 2007, at 2:23 PM, Robert Hailey wrote: > > Argghh, sorry. Bad commit-log comment... > > [...] > > I thought that I had added this return statement, but instead > committed some minor cleanups/comments and added more log noise. > > -- > Robert Hailey I think I found the bug I was tracking BTW, 'receivedCompletionNotice' was not set in one of the two completed() routines of CHKInsertSender. In fact, since it was this broken function that was being called at the end of the block transfer, I suspect that we *never* would correctly wait on such an insert. I'm not sure how this affects inserts at large, I would think that such a bug would make all inserts timeout... Since applying this patch I've seen a huge spike in inserts (at the moment inserts outweigh requests 3-to-1), I would appreciate critical review of the patches (I know they may be hard to read). -- Robert Hailey From toad at amphibian.dyndns.org Tue Dec 18 22:12:48 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 18 Dec 2007 22:12:48 +0000 Subject: [freenet-dev] [freenet-cvs] r16701 - trunk/freenet/src/freenet/node In-Reply-To: <20071218212213.65926481000@freenetproject.org> References: <20071218212213.65926481000@freenetproject.org> Message-ID: <200712182212.53971.toad@amphibian.dyndns.org> Smaller chunks would be nice, I appreciate that isn't always feasible. Code review follows... On Tuesday 18 December 2007 21:22, robert at freenetproject.org wrote: > Author: robert > Date: 2007-12-18 21:22:13 +0000 (Tue, 18 Dec 2007) > New Revision: 16701 > > Modified: > trunk/freenet/src/freenet/node/CHKInsertSender.java > Log: > simplify, factor out Sender, receivedCompletionNotice was not being set > > > Modified: trunk/freenet/src/freenet/node/CHKInsertSender.java > =================================================================== > --- trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 20:16:10 UTC (rev 16700) > +++ trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 21:22:13 UTC (rev 16701) > @@ -24,36 +24,7 @@ > > public final class CHKInsertSender implements Runnable, AnyInsertSender, ByteCounter { > > - private static class Sender implements Runnable { > - > - final AwaitingCompletion completion; > - final BlockTransmitter bt; > - final Executor executor; > - > - public Sender(AwaitingCompletion ac, Executor executor) { > - this.bt = ac.bt; > - this.completion = ac; > - this.executor = executor; > - } > - > - public void run() { > - freenet.support.Logger.OSThread.logPID(this); > - try { > - bt.send(executor); > - if(bt.failedDueToOverload()) { > - completion.completedTransfer(false); > - } else { > - completion.completedTransfer(true); > - } > - } catch (Throwable t) { > - completion.completedTransfer(false); > - Logger.error(this, "Caught "+t, t); > - } > - } > - } > - > - private class AwaitingCompletion { > - > + private class AwaitingCompletion implements Runnable { > /** Node we are waiting for response from */ > final PeerNode pn; > /** We may be sending data to that node */ > @@ -79,34 +50,32 @@ > } > > void start() { > - Sender s = new Sender(this, node.executor); > - node.executor.execute(s, "Sender for "+uid+" to "+pn.getPeer()); > + node.executor.execute(this, "CHKInsert-AwaitingCompletion for "+uid+" to "+pn.getPeer()); > } > > - void completed(boolean timeout, boolean success) { > - synchronized(this) { > - if(timeout) > - completionTimedOut = true; > - else > - completionSucceeded = success; > - receivedCompletionNotice = true; > - notifyAll(); > - } > - synchronized(nodesWaitingForCompletion) { > - nodesWaitingForCompletion.notifyAll(); > - } > - if(!success) { > - synchronized(CHKInsertSender.this) { > - transferTimedOut = true; > - CHKInsertSender.this.notifyAll(); > + public void run() { > + freenet.support.Logger.OSThread.logPID(this); > + try { > + bt.send(node.executor); > + if(bt.failedDueToOverload()) { > + this.completed(false, false); > + } else { > + this.completed(false, true); > } > + } catch (Throwable t) { > + this.completed(false, false); > + Logger.error(this, "Caught "+t, t); > } > } Okay so here you have moved Sender.run() into AwaitingCompletion, and deleted Sender as it's now redundant. Nice simplification but ideally it should be separate. > > - void completedTransfer(boolean success) { > + void completed(boolean timeout, boolean success) { > + if (logMINOR) Logger.minor(this, "CHKInsert-AwaitingCompletion complete (timeout="+timeout+", success="+success); > + if (success && timeout) > + throw new IllegalArgumentException("how can a request successfully timeout?"); This is just logging. > synchronized(this) { > - transferSucceeded = success; > - completedTransfer = true; > + completionTimedOut = timeout; > + completionSucceeded = success; > + receivedCompletionNotice = true; > notifyAll(); > } Okay, there's something interesting happening here... receivedCompletionNotice means we have not only finished the transfer but also received a message indicating that downstream has also finished transferring. AFAICS you are setting it here to indicate we have finished the transfer ??? There is a reason for this behaviour: We need a realistic round-trip time for the AIMD that controls the number of inserts that we start. If we only take into account our own transfer time that isn't very realistic, as downstream can take much longer to complete sometimes. Obviously AIMD will often run many requests at once, but it needs a correct RTT. > synchronized(nodesWaitingForCompletion) { > @@ -444,7 +413,6 @@ > if (msg.getSpec() == DMT.FNPRouteNotFound) { > if(logMINOR) Logger.minor(this, "Rejected: RNF"); > short newHtl = msg.getShort(DMT.HTL); > - Logger.error(this, "CHKInsert-RNF: htl="+htl+", msg.htl="+newHtl); > synchronized (this) { > if (htl > newHtl) > htl = newHtl; > @@ -697,8 +665,9 @@ > timeout = (int)Math.min(Integer.MAX_VALUE, (transfersCompletedTime + TRANSFER_COMPLETION_ACK_TIMEOUT) - now); > if(timeout <= 0) { > synchronized(CHKInsertSender.this) { > - if(logMINOR) Logger.minor(this, "Timed out waiting for transfers to complete on "+uid); > + Logger.error(this, "Timed out waiting for transfers to complete on "+uid); > transferTimedOut = true; > + //CHKInsertSender.this.notifyAll(); Why? > } > return; > } > @@ -743,9 +712,9 @@ > } > if(m == null) { > Logger.error(this, "Timed out waiting for a final ack from any nodes."); > - //Would looping again help? We will either: > - // (1) time out again (and be right back here if there is more time left), or > - // (2) notice that the nodes we are waiting on are down and exit immediately. > + //Would looping again help? We could either: > + // (1) loop and notice that there is no more time left (handling the timeout), or > + // (2) notice that the nodes we are waiting on are down and possibly exit gracefully (not implemented). > continue; > } else { > // Process message Comments fix... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071218/455f1972/attachment.pgp From jflesch at freenetproject.org Sun Dec 16 17:31:10 2007 From: jflesch at freenetproject.org (jflesch at freenetproject.org) Date: Sun, 16 Dec 2007 17:31:10 +0000 (UTC) Subject: [freenet-dev] r16603 - tags/Thaw Message-ID: <20071216173110.1C6C23C072D@freenetproject.org> Author: jflesch Date: 2007-12-16 17:31:09 +0000 (Sun, 16 Dec 2007) New Revision: 16603 Added: tags/Thaw/0.8/ Log: Thaw 0.8: Changelog since Thaw 0.7.10: - Signatures: Public keys of developpers are now hard-coded - MiniFrost: It's now possible to attach files and boards to a message - Add a new plugin 'index web grapher' : It makes a graph of the web of indexes and tries to optimises the display (tries ...) - Doesn't reload anymore all the plugins each time a change is done : it only reloads the affected plugin(s) - IndexBrowser : Should warn the users when a new stable version is available (thanks to the Thaw index) - MiniFrost: Highlight keys in frost messages (copy keys ; download keys ; add as indexes in the index browser) - Minifrost can be configured to provide an "outlook-like" view instead of an "gmail-like" view - IndexBrowser : The unknown index list in the index browser displays now all the unknown indexes instead of randomly choosen ones - TrayIcon : Tray icon is used to notify when a transfer starts and when it ends - LogConsole : Logs level can now be changed on-the-fly - A new plugin 'IndexTreeRebuilder': Allows to rebuild a broken index tree (should be useless most of the time) - QueueWatcher : It tries to compute and display average speed and ETA of each transfer (not really efficient ; would require some improvements) - IndexBrowser : It allows now to blacklist all the indexes in a same folder all at once - MiniFrost : It's now possible to send private messages - MiniFrost : It's now possible to add a new board from a list of known boards - A lot of bug fixes Copied: tags/Thaw/0.8 (from rev 16602, trunk/apps/Thaw) From robert at emu.freenetproject.org Tue Dec 18 22:49:37 2007 From: robert at emu.freenetproject.org (Robert Hailey) Date: Tue, 18 Dec 2007 16:49:37 -0600 Subject: [freenet-dev] [freenet-cvs] r16701 - trunk/freenet/src/freenet/node In-Reply-To: <200712182212.53971.toad@amphibian.dyndns.org> References: <20071218212213.65926481000@freenetproject.org> <200712182212.53971.toad@amphibian.dyndns.org> Message-ID: On Dec 18, 2007, at 4:12 PM, Matthew Toseland wrote: > Smaller chunks would be nice, I appreciate that isn't always > feasible. Code > review follows... > > On Tuesday 18 December 2007 21:22, robert at freenetproject.org wrote: >> synchronized(this) { >> - transferSucceeded = success; >> - completedTransfer = true; >> + completionTimedOut = timeout; >> + completionSucceeded = success; >> + receivedCompletionNotice = true; >> notifyAll(); >> } > > Okay, there's something interesting happening here... > > receivedCompletionNotice means we have not only finished the > transfer but also > received a message indicating that downstream has also finished > transferring. > AFAICS you are setting it here to indicate we have finished the > transfer ??? > > There is a reason for this behaviour: We need a realistic round-trip > time for > the AIMD that controls the number of inserts that we start. If we > only take > into account our own transfer time that isn't very realistic, as > downstream > can take much longer to complete sometimes. Obviously AIMD will > often run > many requests at once, but it needs a correct RTT. Ok, I think that I have restored this behavior with r16706, but I'm out of time for today. -- Robert Hailey From m.rogers at cs.ucl.ac.uk Wed Dec 19 12:55:05 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Wed, 19 Dec 2007 12:55:05 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <4762DF59.90109@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <200712140030.35073.toad@amphibian.dyndns.org> <476248FE.3040200@batosai.net> <200712141927.31009.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> Message-ID: <476914A9.10106@cs.ucl.ac.uk> Colin Davis wrote: > When you have at least 10 Friends, please turn off insecure mode and > your node will become invisible to the outside world. "Invisible to the outside world" is a great phrase, it really captures the advantages of darknet. Maybe "When at least 10 of your friends have started using Freenet" rather than "When you have at least 10 Friends", to make it clear that you should invite your real friends to use Freenet rather than looking for 'friends' who already use it? And should it be 5 rather than 10, for consistency with the installer? Cheers, Michael From toad at amphibian.dyndns.org Wed Dec 19 14:44:55 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 14:44:55 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <476914A9.10106@cs.ucl.ac.uk> References: <200712140024.29607.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> <476914A9.10106@cs.ucl.ac.uk> Message-ID: <200712191445.00996.toad@amphibian.dyndns.org> On Wednesday 19 December 2007 12:55, Michael Rogers wrote: > Colin Davis wrote: > > When you have at least 10 Friends, please turn off insecure mode and > > your node will become invisible to the outside world. > > "Invisible to the outside world" is a great phrase, it really captures > the advantages of darknet. :) > > Maybe "When at least 10 of your friends have started using Freenet" > rather than "When you have at least 10 Friends", to make it clear that > you should invite your real friends to use Freenet rather than looking > for 'friends' who already use it? > > And should it be 5 rather than 10, for consistency with the installer? Well... 5 gives pretty poor performance, if you've been using opennet since day one and turn it off when you have 5 peers who won't even be on 24x7... Maybe we should change it in the installer, but I'm not sure about that.. hmmm. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/4e38c023/attachment.pgp From toad at amphibian.dyndns.org Wed Dec 19 14:47:16 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 14:47:16 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <476914A9.10106@cs.ucl.ac.uk> References: <200712140024.29607.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> <476914A9.10106@cs.ucl.ac.uk> Message-ID: <200712191447.16838.toad@amphibian.dyndns.org> On Wednesday 19 December 2007 12:55, Michael Rogers wrote: > Colin Davis wrote: > > When you have at least 10 Friends, please turn off insecure mode and > > your node will become invisible to the outside world. > > "Invisible to the outside world" is a great phrase, it really captures > the advantages of darknet. > > Maybe "When at least 10 of your friends have started using Freenet" > rather than "When you have at least 10 Friends", to make it clear that > you should invite your real friends to use Freenet rather than looking > for 'friends' who already use it? Your node is currently running in insecure mode. It is trivial for an attacker to find your node, and you could be connected to the Bad Guys right now. Please get some connections to people you trust and add them to the Friends page as soon as possible, you will be much less vulnerable. When you have added at least 10, turn off insecure mode and your node will become invisible to the outside world. ???? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/e3336319/attachment.pgp From batosai at batosai.net Wed Dec 19 17:40:52 2007 From: batosai at batosai.net (Julien Cornuwel) Date: Wed, 19 Dec 2007 18:40:52 +0100 Subject: [freenet-dev] Updated French translation for #1092 Message-ID: <476957A4.2040209@batosai.net> Hi, Here is an updated translation file fixing the "missing seednodes.fref file" message. Regards -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.fr.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20071219/4d7d2224/attachment.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/4d7d2224/attachment.pgp From m.rogers at cs.ucl.ac.uk Wed Dec 19 18:08:32 2007 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Wed, 19 Dec 2007 18:08:32 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <200712191447.16838.toad@amphibian.dyndns.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> <476914A9.10106@cs.ucl.ac.uk> <200712191447.16838.toad@amphibian.dyndns.org> Message-ID: <47695E20.3040100@cs.ucl.ac.uk> Matthew Toseland wrote: > Your node is currently running in insecure mode. It is trivial for an attacker > to find your node, and you could be connected to the Bad Guys right now. > Please get some connections to people you trust and add them to the Friends > page as soon as possible, you will be much less vulnerable. When you have > added at least 10, turn off insecure mode and your node will become invisible > to the outside world. > > ???? Sounds good - can we get rid of "node" where possible, or is that unnecessary in post-installation messages? "Freenet is currently running in insecure mode. It is trivial for anyone, including attackers, to discover that you are running Freenet and connect to you. Please get some connections to people you trust and add them to the Friends page as soon as possible. (These must be people you know in real life - adding connections to people you don't trust will not increase your security.) When you have added at least 10 friends, turn off insecure mode and your Freenet node will become invisible to the outside world." Cheers, Michael From toad at amphibian.dyndns.org Wed Dec 19 18:15:53 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 18:15:53 +0000 Subject: [freenet-dev] [freenet-cvs] r16706 - trunk/freenet/src/freenet/node In-Reply-To: <20071218224749.8F1E93C072D@freenetproject.org> References: <20071218224749.8F1E93C072D@freenetproject.org> Message-ID: <200712191815.58895.toad@amphibian.dyndns.org> On Tuesday 18 December 2007 22:47, robert at freenetproject.org wrote: > Author: robert > Date: 2007-12-18 22:47:49 +0000 (Tue, 18 Dec 2007) > New Revision: 16706 > > Modified: > trunk/freenet/src/freenet/node/CHKInsertSender.java > Log: > Restore previous behavior for AIMD (total RTT) Okay, this commit: - Reinstates completedTransfer(), exactly as it was. - Deletes the unused variable completionTimedOut. - Drops the timeout parameter from completed(timeout, success) as it is never used. - And renames it from completed(success) to receivedNotice(success). Now, combining the two: - A few trivial code simplifications: if(x) blah(true) else blah(false) -> blah(x). - Code simplifications in completed*() which don't have any practical impact although they change the logic: if(blah) blahInner = true -> blahInner = blah; - Rename some threads. - Delete Sender and merge it into AwaitingCompletion. - Delete unused timeout parameter from completed() and rename to receivedNotice(). - Logging and comments. > > Modified: trunk/freenet/src/freenet/node/CHKInsertSender.java > =================================================================== > --- trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 22:15:14 UTC (rev 16705) > +++ trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 22:47:49 UTC (rev 16706) > @@ -33,9 +33,7 @@ > * or failure of dependant transfers from that node? > * Includes timing out. */ > boolean receivedCompletionNotice; > - /** Timed out - didn't receive completion notice in > - * the allotted time?? */ > - boolean completionTimedOut; > + > /** Was the notification of successful transfer? */ > boolean completionSucceeded; > > @@ -57,23 +55,32 @@ > freenet.support.Logger.OSThread.logPID(this); > try { > bt.send(node.executor); > - if(bt.failedDueToOverload()) { > - this.completed(false, false); > - } else { > - this.completed(false, true); > - } > + this.completedTransfer(bt.failedDueToOverload()); > } catch (Throwable t) { > - this.completed(false, false); > + this.completedTransfer(false); > Logger.error(this, "Caught "+t, t); > } > } > > - void completed(boolean timeout, boolean success) { > - if (logMINOR) Logger.minor(this, "CHKInsert-BackgroundTransfer complete (timeout="+timeout+", success="+success); > - if (success && timeout) > - throw new IllegalArgumentException("how can a request successfully timeout?"); > + void completedTransfer(boolean success) { > synchronized(this) { > - completionTimedOut = timeout; > + transferSucceeded = success; > + completedTransfer = true; > + notifyAll(); > + } > + synchronized(backgroundTransfers) { > + backgroundTransfers.notifyAll(); > + } > + if(!success) { > + synchronized(CHKInsertSender.this) { > + transferTimedOut = true; > + CHKInsertSender.this.notifyAll(); > + } > + } > + } > + > + void receivedNotice(boolean success) { > + synchronized(this) { > completionSucceeded = success; > receivedCompletionNotice = true; > notifyAll(); > @@ -86,8 +93,9 @@ > transferTimedOut = true; > CHKInsertSender.this.notifyAll(); > } > - } > + } > } > + > } > > CHKInsertSender(NodeCHK myKey, long uid, byte[] headers, short htl, > @@ -701,7 +709,7 @@ > PeerNode p = transfers[i].pn; > if(p == pn) { > boolean anyTimedOut = m.getBoolean(DMT.ANY_TIMED_OUT); > - transfers[i].completed(false, !anyTimedOut); > + transfers[i].receivedNotice(!anyTimedOut); > if(anyTimedOut) { > synchronized(CHKInsertSender.this) { > if(!transferTimedOut) { -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/b8d99ac3/attachment.pgp From toad at amphibian.dyndns.org Wed Dec 19 18:20:03 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 18:20:03 +0000 Subject: [freenet-dev] [freenet-cvs] r16705 - trunk/freenet/src/freenet/node In-Reply-To: <20071218221514.4594A400FA1@freenetproject.org> References: <20071218221514.4594A400FA1@freenetproject.org> Message-ID: <200712191820.03420.toad@amphibian.dyndns.org> Nice one! That code is messy... On Tuesday 18 December 2007 22:15, robert at freenetproject.org wrote: > Author: robert > Date: 2007-12-18 22:15:14 +0000 (Tue, 18 Dec 2007) > New Revision: 16705 > > Modified: > trunk/freenet/src/freenet/node/CHKInsertSender.java > Log: > Factor out CompletionWaiter (use 1/2 the threads for inserts; could still be simplied...) > > > Modified: trunk/freenet/src/freenet/node/CHKInsertSender.java > =================================================================== > --- trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 21:50:24 UTC (rev 16704) > +++ trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-18 22:15:14 UTC (rev 16705) > @@ -143,9 +143,6 @@ > /** Has a transfer timed out, either directly or downstream? */ > private boolean transferTimedOut; > > - /** Runnable which waits for completion of all transfers */ > - private CompletionWaiter cw; > - > private int status = -1; > /** Still running */ > static final int NOT_FINISHED = -1; > @@ -367,7 +364,6 @@ > backgroundTransfers.notifyAll(); > } > ac.start(); > - makeCompletionWaiter(); > > while (true) { > > @@ -532,26 +528,20 @@ > notifyAll(); > if(logMINOR) Logger.minor(this, "Set status code: "+getStatusString()+" on "+uid); > } > + > // Now wait for transfers, or for downstream transfer notifications. > - if(cw != null) { > + synchronized(backgroundTransfers) { > + if (!backgroundTransfers.isEmpty()) { > + waitForBackgroundTransferCompletions(); > + } else { > + if(logMINOR) Logger.minor(this, "No background transfers"); > + } > + } > + > synchronized(this) { > - while(!allTransfersCompleted && cw!=null) { > - try { > - wait(10*1000); > - } catch (InterruptedException e) { > - // Try again > - } > - } > - } > - } > - if (cw==null) { > - if(logMINOR) Logger.minor(this, "No completion waiter"); > - // There weren't any transfers > - synchronized(this) { > allTransfersCompleted = true; > notifyAll(); > } > - } > > if(status == SUCCESS && next != null) > next.onSuccess(true, false); > @@ -583,7 +573,8 @@ > synchronized(this) { > status = RECEIVE_FAILED; > allTransfersCompleted = true; > - cw = null; // Effectively ... we certainly don't want to wait for it. > + //FIXME: Won't this leak unclaimed FIFO elements? > + backgroundTransfers.clear(); // Effectively ... we certainly don't want to wait for it. > notifyAll(); > } > // Do not call finish(), that can only be called on the main thread and it will block. > @@ -613,28 +604,13 @@ > public synchronized boolean sentRequest() { > return sentRequest; > } > - > - private void makeCompletionWaiter() { > - if(logMINOR) > - Logger.minor(this, "Creating completion waiter for "+uid); > - synchronized (this) { > - if(cw == null) > - cw = new CompletionWaiter(); > - else > - return; > - } > - node.executor.execute(cw, "Completion waiter for "+uid); > - } > - > - private class CompletionWaiter implements Runnable { > > - public void run() { > + public void waitForBackgroundTransferCompletions() { > try { > freenet.support.Logger.OSThread.logPID(this); > if(logMINOR) Logger.minor(this, "Starting "+this); > > - // Wait for the request to reach a terminal stage. > - waitForStatus(); > + // We are presently at a terminal stage. > > BackgroundTransfer[] transfers; > synchronized(backgroundTransfers) { > @@ -781,10 +757,6 @@ > } > } > > - public String toString() { > - return super.toString()+" for "+uid; > - } > - } > > public synchronized boolean completed() { > return allTransfersCompleted; > @@ -855,6 +827,6 @@ > } > > public synchronized boolean startedSendingData() { > - return cw != null; > + return !backgroundTransfers.isEmpty(); > } > } > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/cdaab9ae/attachment.pgp From colin at sq7.org Wed Dec 19 18:23:28 2007 From: colin at sq7.org (Colin Davis) Date: Wed, 19 Dec 2007 13:23:28 -0500 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <47695E20.3040100@cs.ucl.ac.uk> References: <200712140024.29607.toad@amphibian.dyndns.org> <4762DF59.90109@sq7.org> <476914A9.10106@cs.ucl.ac.uk> <200712191447.16838.toad@amphibian.dyndns.org> <47695E20.3040100@cs.ucl.ac.uk> Message-ID: <476961A0.2000207@sq7.org> I think this conversation has denegrated into Bike Shed area. (http://wiki.netbsd.se/bikeshed) My vote is for the following, but I think at this point is almost doesn't matter. You're almost there! Your Freenet installation is almost complete. Freenet is currently running in insecure mode, which allows you to get up and running on the Freenet network, but isn't very secure. It is (relatively) easy for an attacker to find out that you are running a node, connect to it, or even trace your requests. You can dramatically improve your security by adding connections to people you already know and trust through the Friends page. Adding connections to people you do not trust will not increase your security. When you have added at least 10 people to the Friends page, turn off insecure mode and your Freenet node will become invisible to the outside world. // > Sounds good - can we get rid of "node" where possible, or is that > unnecessary in post-installation messages? > > "Freenet is currently running in insecure mode. It is trivial for > anyone, including attackers, to discover that you are running Freenet > and connect to you. Please get some connections to people you trust and > add them to the Friends page as soon as possible. (These must be people > you know in real life - adding connections to people you don't trust > will not increase your security.) When you have added at least 10 > friends, turn off insecure mode and your Freenet node will become > invisible to the outside world." > > Cheers, > Michael From toad at amphibian.dyndns.org Wed Dec 19 18:23:35 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 18:23:35 +0000 Subject: [freenet-dev] [freenet-cvs] r16710 - trunk/freenet/src/freenet/node In-Reply-To: <20071219145846.B43E33C0482@freenetproject.org> References: <20071219145846.B43E33C0482@freenetproject.org> Message-ID: <200712191823.35531.toad@amphibian.dyndns.org> I don't understand the fix. It's supposed to randomize it and then record the new port number? On Wednesday 19 December 2007 14:58, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2007-12-19 14:58:46 +0000 (Wed, 19 Dec 2007) > New Revision: 16710 > > Modified: > trunk/freenet/src/freenet/node/Node.java > Log: > Resolve #191: Bug with the configuration framework (node.listenPort=-1) > > Modified: trunk/freenet/src/freenet/node/Node.java > =================================================================== > --- trunk/freenet/src/freenet/node/Node.java 2007-12-19 14:50:50 UTC (rev 16709) > +++ trunk/freenet/src/freenet/node/Node.java 2007-12-19 14:58:46 UTC (rev 16710) > @@ -686,7 +686,9 @@ > sortOrder = ipDetector.registerConfigs(nodeConfig, sortOrder); > > // Determine the port number > - > + // @see #191 > + if("-1".equals(oldConfig.get("node.listenPort"))) > + throw new NodeInitException(NodeInitException.EXIT_COULD_NOT_BIND_USM, "Your freenet.ini file is corrupted! 'listenPort=-1'"); > NodeCryptoConfig darknetConfig = new NodeCryptoConfig(nodeConfig, sortOrder++, false); > sortOrder += NodeCryptoConfig.OPTION_COUNT; > darknetCrypto = new NodeCrypto(this, false, darknetConfig, startupTime); > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/37e325b3/attachment.pgp From toad at amphibian.dyndns.org Wed Dec 19 18:35:26 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 18:35:26 +0000 Subject: [freenet-dev] [freenet-cvs] r16725 - trunk/freenet/src/freenet/node In-Reply-To: <20071219172736.9B59E47B312@freenetproject.org> References: <20071219172736.9B59E47B312@freenetproject.org> Message-ID: <200712191835.31504.toad@amphibian.dyndns.org> It's a nice simplification... but it'll use a lot of threads. How about asynchronous callbacks? On Wednesday 19 December 2007 17:27, you wrote: > Author: robert > Date: 2007-12-19 17:27:36 +0000 (Wed, 19 Dec 2007) > New Revision: 16725 > > Modified: > trunk/freenet/src/freenet/node/CHKInsertSender.java > Log: > Wait for transfer ack's from background transfers independently (might catch more) > > > Modified: trunk/freenet/src/freenet/node/CHKInsertSender.java > =================================================================== > --- trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-19 16:44:52 UTC (rev 16724) > +++ trunk/freenet/src/freenet/node/CHKInsertSender.java 2007-12-19 17:27:36 UTC (rev 16725) > @@ -56,8 +56,10 @@ > try { > bt.send(node.executor); > this.completedTransfer(bt.failedDueToOverload()); > + this.receivedNotice(waitForReceivedNotification(this)); > } catch (Throwable t) { > this.completedTransfer(false); > + this.receivedNotice(false); > Logger.error(this, "Caught "+t, t); > } > } > @@ -617,12 +619,12 @@ > return sentRequest; > } > > - public void waitForBackgroundTransferCompletions() { > + private void waitForBackgroundTransferCompletions() { > try { > freenet.support.Logger.OSThread.logPID(this); > if(logMINOR) Logger.minor(this, "Starting "+this); > > - // We are presently at a terminal stage. > + // We must presently be at such a stage that no more background transfers will be added. > > BackgroundTransfer[] transfers; > synchronized(backgroundTransfers) { > @@ -631,10 +633,26 @@ > } > > // Wait for the outgoing transfers to complete. > - if(!waitForCompletedTransfers(transfers)) { > + if(!waitForBackgroundTransfers(transfers)) { > setTransferTimedOut(); > return; > } > + } finally { > + synchronized(CHKInsertSender.this) { > + allTransfersCompleted = true; > + CHKInsertSender.this.notifyAll(); > + } > + } > + } > + > + /** > + * Blocks and waits for a response from the given node asto the final transfer status in the chain. This will be longer/after > + * the local block transfer is complete, as it is neccesary to include the rount-trip-time in the allTransfersComplete() > + * function. > + * Returns true if received a successful notification of the downstream reception, false in every other case > + * (e.g. timeout, cancel, receiveFailed, etc). > + */ > + private boolean waitForReceivedNotification(BackgroundTransfer awc) { > > long transfersCompletedTime = System.currentTimeMillis(); > > @@ -643,7 +661,7 @@ > while(true) { > > synchronized(backgroundTransfers) { > - if(receiveFailed) return; > + if(receiveFailed) return false; > } > // First calculate the timeout > int timeout; > @@ -652,104 +670,92 @@ > if(timeout <= 0) { > Logger.error(this, "Timed out waiting for transfers to complete on "+uid); > setTransferTimedOut(); > - return; > + return false; > } > > - MessageFilter mf = null; > - > - //Build a message filter to capture acknowledgement messages from the nodes we are interested in. > - for(int i=0;i - BackgroundTransfer awc = transfers[i]; > // If disconnected, ignore. > if(!awc.pn.isRoutable()) { > Logger.normal(this, "Disconnected: "+awc.pn+" in "+CHKInsertSender.this); > - continue; > + return false; > } > // If transfer failed, probably won't be acknowledged. > if(!awc.transferSucceeded) { > - continue; > + if (logMINOR) Logger.minor(this, "waitForReceivedNotification: !transferSucceeded -> false"); > + return false; > } > - // Wait for completion. > - if(!awc.receivedCompletionNotice) { > - MessageFilter m = > + // See if redundant. > + if(awc.receivedCompletionNotice) { > + return awc.completionSucceeded; > + } > + > + MessageFilter mf = > MessageFilter.create().setField(DMT.UID, uid).setType(DMT.FNPInsertTransfersCompleted).setSource(awc.pn).setTimeout(timeout); > - if(mf == null) > - mf = m; > - else > - mf = m.or(mf); > + > if(logMINOR) Logger.minor(this, "Waiting for "+awc.pn.getPeer()); > - } > - } > > - if (mf==null) { > - if (logMINOR) Logger.minor(this, "Done waiting, no more completion listeners"); > - return; > - } else { > Message m; > try { > m = node.usm.waitFor(mf, CHKInsertSender.this); > } catch (DisconnectedException e) { > - // Which one? I have no idea. > - // Go around the loop again to find out. > - continue; > + Logger.normal(this, "Disconnected (on waitFor): "+awc.pn+" in "+this); > + return false; > } > if(m == null) { > - Logger.error(this, "Timed out waiting for a final ack from any nodes."); > - //Would looping again help? We could either: > - // (1) loop and notice that there is no more time left (handling the timeout), or > - // (2) notice that the nodes we are waiting on are down and possibly exit gracefully (not implemented). > - continue; > + Logger.error(this, "Timed out waiting for a final ack from: "+awc.pn); > + return false; > } else { > - // Process message > PeerNode pn = (PeerNode) m.getSource(); > // pn cannot be null, because the filters will prevent garbage collection of the nodes > - boolean processed = false; > - for(int i=0;i - PeerNode p = transfers[i].pn; > - if(p == pn) { > + > + if(awc.pn.equals(pn)) { > boolean anyTimedOut = m.getBoolean(DMT.ANY_TIMED_OUT); > - transfers[i].receivedNotice(!anyTimedOut); > if(anyTimedOut) { > setTransferTimedOut(); > } > - processed = true; > - break; > + return !anyTimedOut; > + } else { > + Logger.error(this, "received completion notice for wrong node: "+awc); > + continue; > } > - } > - if(!processed) { > - Logger.error(this, "Did not process message: "+m+" on "+this); > - } > } > } > - } > - } finally { > - synchronized(CHKInsertSender.this) { > - allTransfersCompleted = true; > - CHKInsertSender.this.notifyAll(); > - } > - } > } > > - /** Block until all transfers have finished. @return True if there is any point in waiting for acknowledgements. */ > - private boolean waitForCompletedTransfers(BackgroundTransfer[] transfers) { > + /** > + * Block until all transfers have reached a final-terminal state (success/failure). On success this means that a > + * successful 'received-notification' has been received. > + * @return True if all background transfers were successful. > + */ > + private boolean waitForBackgroundTransfers(BackgroundTransfer[] transfers) { > // MAYBE all done > while(true) { > + //If we want to be sure to exit as-soon-as the transfers are done, then we must hold the lock while we check. > + synchronized(backgroundTransfers) { > + if(receiveFailed) return false; > + > boolean noneRouteable = true; > boolean completedTransfers = true; > + boolean completedNotifications = true; > for(int i=0;i if(!transfers[i].pn.isRoutable()) continue; > noneRouteable = false; > if(!transfers[i].completedTransfer) { > + //must wait > completedTransfers = false; > break; > } > + if (!transfers[i].receivedCompletionNotice) { > + //must wait > + completedNotifications = false; > + break; > + } > + if (!transfers[i].completionSucceeded) > + return false; > } > - if(completedTransfers) return true; > if(noneRouteable) return false; > + if(completedTransfers && completedNotifications) return true; > > - synchronized(backgroundTransfers) { > - if(receiveFailed) return false; > - if(logMINOR) Logger.minor(this, "Waiting for completion"); > + if(logMINOR) Logger.minor(this, "Waiting for (completion="+!completedTransfers+", notification="+completedNotifications+")"); > try { > backgroundTransfers.wait(100*1000); > } catch (InterruptedException e) { > > _______________________________________________ > cvs mailing list > cvs at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20071219/50b32c93/attachment.pgp From toad at amphibian.dyndns.org Wed Dec 19 18:43:48 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Dec 2007 18:43:48 +0000 Subject: [freenet-dev] Shrinking the opennet warning, documentation toadlet In-Reply-To: <476961A0.2000207@sq7.org> References: <200712140024.29607.toad@amphibian.dyndns.org> <47695E20.3040100@cs.ucl.ac.uk> <476961A0.2000207@sq7.org> Message-ID: <200712191843.48395.toad@amphibian.dyndns.org> On Wednesday 19 December 2007 18:23, Colin Davis wrote: > I think this conversation has denegrated int