From toad at amphibian.dyndns.org Wed Aug 1 09:53:56 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 1 Aug 2007 10:53:56 +0100 Subject: [freenet-dev] Should swap identifiers be random? In-Reply-To: <46AE5ECA.3080908@cs.ucl.ac.uk> References: <46A263B9.9060608@cs.ucl.ac.uk> <200707251640.59607.toad@amphibian.dyndns.org> <46AE5ECA.3080908@cs.ucl.ac.uk> Message-ID: <200708011054.04000.toad@amphibian.dyndns.org> On Monday 30 July 2007 22:57, Michael Rogers wrote: > Matthew Toseland wrote: > > If we are to include UIDs for our > > peers then we must know them. > > Are peers we've never connected to included in the list of peers used > for swapping? > > Actually, come to think of it, why do swap requests need to include a > long-term node identifier at all? Why not just the node's location and > its peers' locations? (Sorry if I'm missing something obvious.) They don't, but it makes analysing the network *much* easier e.g. if we cross-reference with probe requests. Having said that, we haven't actually used this data much (unlike probe reqs), so maybe it can go..? > > > What can you do with that? Probably some fun things - but details > > would be nice. > > To be honest I can't think of any specific attacks, but a map of the > network still seems like a useful thing for an attacker to have. Of course. It's also a useful thing for us to have. I believe we discussed this. :) > Removing the routing location from the noderef and making swap > identifiers random would make it harder to map the network - it is worth > making those changes just in case, or is that just my usual paranoia > talking? ;-) Obviously this shouldn't be in the final freenet, but this is useful debugging info. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070801/229625d6/attachment.pgp From nextgens at freenetproject.org Wed Aug 1 20:23:10 2007 From: nextgens at freenetproject.org (NextGen$) Date: Wed, 1 Aug 2007 20:23:10 +0000 Subject: [freenet-dev] Propagating opennet refs through darknet nodes In-Reply-To: <200707262001.35110.toad@amphibian.dyndns.org> References: <200707251839.16584.toad@amphibian.dyndns.org> <20070726031405.GB23461@freenetproject.org> <200707262001.35110.toad@amphibian.dyndns.org> Message-ID: <20070801202310.GH23461@freenetproject.org> * Matthew Toseland [2007-07-26 20:01:34]: > On Thursday 26 July 2007 04:14, NextGen$ wrote: > > * Matthew Toseland [2007-07-25 18:39:11]: > > > I propose that darknet nodes be allowed to forward announcements and path > > > folding messages (ConnectDestination etc), without including their own > > > noderefs. > > > > > > Any objections? > > > > _o/ > > It's a bad idea. > > Be informed that my node wont behave that way... > > Why, precisely, is it a bad idea? As far as I can tell it doesn't compromise > the nodes which only relay and never send their own noderef? It makes flow analysis related attackes both trivial and more effective. I'm really dissapointed that you implemented it before leaving me time to respond. You did so even though I explained to you my concerns on IRC... I told you about an attack vector (flow analysis using request size - arguably already present but currently not easily exploitable) then we disccused about workarounds (padding noderefs, limitating the number of refs per request) and didn't manage to find any 'good' solution. Your implementation doesn't even feature basic workarounds we talked about and you have enabled that 'risky' option by default :( I don't mind about opennet beeing insecure but don't lower artificially the security level provided by darknet. NextGen$ From ian at locut.us Sun Aug 5 18:09:53 2007 From: ian at locut.us (Ian Clarke) Date: Sun, 5 Aug 2007 13:09:53 -0500 Subject: [freenet-dev] Creating initial node locations Message-ID: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> Here at DEFCON it has been pointed out by the Gnunet guys that we are creating a node location by combining locations supplied by two different nodes by taking one to the power of the other. Nice that *somebody* is reviewing our code :-) If true, this is a *serious* error, and could be the reason for some of the weirdness we are seeing in node location clustering. Math.pow(rand(1.0), rand(1.0)) may result in a number between 0.0 and 1.0, but it will not have a uniform distribution across this range. I suspect the problem here (after discussing it with Oskar) is that when he outlined the problem, he described it as A^B - however he intended ^ to mean bitwise XOR, *not* the exponent function. If this is the case, then we need to fix ASAP - although it shouldn't cause any network incompatibility. The Gnunet guys did a talk here about how it is possible to cause node locations to become highly clustered through attacking the location swapping mechanism. See: http://defcon.org/html/defcon-15/dc-15-speakers.html#Evans Despite the rather dramatic tone of their abstract, this was a problem that Oskar identified when he first came up with the location-swapping algorithm. Also, despite the fact that they claim the flaw cannot be easily addressed, it is entirely possible that it can actually be easily addressed by periodically resetting node locations (they outlined this possible solution on a slide in their talk but didn't talk about why they believed it wouldn't work). Location resetting was also Oskar's initial thought on how to counter this attack. Of course, if we reset node-locations too frequently then it will interfere with convergence to a good network configuration, so we should probably do some experiments to figure out a good balance. There is another possible solution which they suggested (not in the talk, but in a discussion afterwards) where we keep an eye out for locations are too close together based on a very rough estimate of the maximum network size. Clearly this is ugly, but could be effective in practice. Anyway, props to the Gnunet guys for raising these points, hopefully they will continue to help us identify or highlight potential problems. Ian. -- Founder and CEO, Thoof Inc Email: ian at thoof.com Office: +1 512 524 8934 x 100 Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From ian at locut.us Sun Aug 5 23:59:29 2007 From: ian at locut.us (Ian Clarke) Date: Sun, 5 Aug 2007 18:59:29 -0500 Subject: [freenet-dev] Creating initial node locations In-Reply-To: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> References: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> Message-ID: <823242bd0708051659h1638c539jf33f1e65530fc503@mail.gmail.com> On 8/5/07, Ian Clarke wrote: > Here at DEFCON it has been pointed out by the Gnunet guys that we are > creating a node location by combining locations supplied by two > different nodes by taking one to the power of the other. Actually, this wasn't quite right. This error is made when determining a random number between 0 and 1 to determine whether we should swap or not. This could explain all sorts of problems we've been having with swapping. Ian. -- Founder and CEO, Thoof Inc Email: ian at thoof.com Office: +1 512 524 8934 x 100 Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From toad at amphibian.dyndns.org Mon Aug 6 20:14:52 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 21:14:52 +0100 Subject: [freenet-dev] Creating initial node locations In-Reply-To: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> References: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> Message-ID: <200708062114.57747.toad@amphibian.dyndns.org> New node locations are created using RandomSource.nextDouble(). It is evenly distributed across [0.0,1.0]. So yes they were looking at A ^ B. I'm skeptical about Oskar's proposed solution; it appears to me that it could cause progressively more perverse clustering, but Oskar knows a good deal more about it than I do, and hopefully there will be attack simulations eventually. Enforcing location swaps might also be possible. And obviously any such attack would be somewhat easier from opennet. On Sunday 05 August 2007 19:09, Ian Clarke wrote: > Here at DEFCON it has been pointed out by the Gnunet guys that we are > creating a node location by combining locations supplied by two > different nodes by taking one to the power of the other. Nice that > *somebody* is reviewing our code :-) > > If true, this is a *serious* error, and could be the reason for some > of the weirdness we are seeing in node location clustering. > > Math.pow(rand(1.0), rand(1.0)) may result in a number between 0.0 and > 1.0, but it will not have a uniform distribution across this range. > > I suspect the problem here (after discussing it with Oskar) is that > when he outlined the problem, he described it as A^B - however he > intended ^ to mean bitwise XOR, *not* the exponent function. > > If this is the case, then we need to fix ASAP - although it shouldn't > cause any network incompatibility. > > The Gnunet guys did a talk here about how it is possible to cause node > locations to become highly clustered through attacking the location > swapping mechanism. See: > > http://defcon.org/html/defcon-15/dc-15-speakers.html#Evans > > Despite the rather dramatic tone of their abstract, this was a problem > that Oskar identified when he first came up with the location-swapping > algorithm. > > Also, despite the fact that they claim the flaw cannot be easily > addressed, it is entirely possible that it can actually be easily > addressed by periodically resetting node locations (they outlined this > possible solution on a slide in their talk but didn't talk about why > they believed it wouldn't work). Location resetting was also Oskar's > initial thought on how to counter this attack. > > Of course, if we reset node-locations too frequently then it will > interfere with convergence to a good network configuration, so we > should probably do some experiments to figure out a good balance. > > There is another possible solution which they suggested (not in the > talk, but in a discussion afterwards) where we keep an eye out for > locations are too close together based on a very rough estimate of the > maximum network size. Clearly this is ugly, but could be effective in > practice. > > Anyway, props to the Gnunet guys for raising these points, hopefully > they will continue to help us identify or highlight potential > problems. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/6a290771/attachment.pgp From toad at amphibian.dyndns.org Mon Aug 6 20:16:32 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 21:16:32 +0100 Subject: [freenet-dev] Creating initial node locations In-Reply-To: <823242bd0708051659h1638c539jf33f1e65530fc503@mail.gmail.com> References: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> <823242bd0708051659h1638c539jf33f1e65530fc503@mail.gmail.com> Message-ID: <200708062116.33285.toad@amphibian.dyndns.org> On Monday 06 August 2007 00:59, Ian Clarke wrote: > On 8/5/07, Ian Clarke wrote: > > Here at DEFCON it has been pointed out by the Gnunet guys that we are > > creating a node location by combining locations supplied by two > > different nodes by taking one to the power of the other. > > Actually, this wasn't quite right. This error is made when > determining a random number between 0 and 1 to determine whether we > should swap or not. No, the two random numbers are integers, and we XOR them to produce another random number. This is a trivial collaborative random number generation algorithm. > > This could explain all sorts of problems we've been having with swapping. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/8173f534/attachment.pgp From ian at locut.us Mon Aug 6 20:29:23 2007 From: ian at locut.us (Ian Clarke) Date: Mon, 6 Aug 2007 15:29:23 -0500 Subject: [freenet-dev] Creating initial node locations In-Reply-To: <200708062116.33285.toad@amphibian.dyndns.org> References: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> <823242bd0708051659h1638c539jf33f1e65530fc503@mail.gmail.com> <200708062116.33285.toad@amphibian.dyndns.org> Message-ID: <823242bd0708061329i4e9bedc9g14f111d58c04325e@mail.gmail.com> On 8/6/07, Matthew Toseland wrote: > > On Monday 06 August 2007 00:59, Ian Clarke wrote: > > On 8/5/07, Ian Clarke wrote: > > > Here at DEFCON it has been pointed out by the Gnunet guys that we are > > > creating a node location by combining locations supplied by two > > > different nodes by taking one to the power of the other. > > > > Actually, this wasn't quite right. This error is made when > > determining a random number between 0 and 1 to determine whether we > > should swap or not. > > No, the two random numbers are integers, and we XOR them to produce > another > random number. This is a trivial collaborative random number generation > algorithm. Right, and we don't use commit-reveal because it wouldn't actually add any security (Oskar explained the reasoning). Where exactly is this code? Ian. -- Founder and CEO, Thoof Inc Email: ian at thoof.com Office: +1 512 524 8934 x 100 Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20070806/3724beae/attachment.htm From toad at amphibian.dyndns.org Mon Aug 6 21:19:55 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 22:19:55 +0100 Subject: [freenet-dev] Creating initial node locations In-Reply-To: <823242bd0708061329i4e9bedc9g14f111d58c04325e@mail.gmail.com> References: <823242bd0708051109h54a7f983n437cdf2fa4979ad4@mail.gmail.com> <200708062116.33285.toad@amphibian.dyndns.org> <823242bd0708061329i4e9bedc9g14f111d58c04325e@mail.gmail.com> Message-ID: <200708062220.00588.toad@amphibian.dyndns.org> On Monday 06 August 2007 21:29, Ian Clarke wrote: > On 8/6/07, Matthew Toseland wrote: > > On Monday 06 August 2007 00:59, Ian Clarke wrote: > > > On 8/5/07, Ian Clarke wrote: > > > > Here at DEFCON it has been pointed out by the Gnunet guys that we are > > > > creating a node location by combining locations supplied by two > > > > different nodes by taking one to the power of the other. > > > > > > Actually, this wasn't quite right. This error is made when > > > determining a random number between 0 and 1 to determine whether we > > > should swap or not. > > > > No, the two random numbers are integers, and we XOR them to produce > > another > > random number. This is a trivial collaborative random number generation > > algorithm. > > Right, and we don't use commit-reveal because it wouldn't actually add any > security (Oskar explained the reasoning). We do use commit-reveal. Are you saying we should get rid of it? > > Where exactly is this code? LocationManager.java. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/2659f8bd/attachment.pgp From toad at amphibian.dyndns.org Mon Aug 6 21:26:54 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 22:26:54 +0100 Subject: [freenet-dev] Propagating opennet refs through darknet nodes In-Reply-To: <20070801202310.GH23461@freenetproject.org> References: <200707251839.16584.toad@amphibian.dyndns.org> <200707262001.35110.toad@amphibian.dyndns.org> <20070801202310.GH23461@freenetproject.org> Message-ID: <200708062226.54644.toad@amphibian.dyndns.org> On Wednesday 01 August 2007 21:23, NextGen$ wrote: > * Matthew Toseland [2007-07-26 20:01:34]: > > On Thursday 26 July 2007 04:14, NextGen$ wrote: > > > * Matthew Toseland [2007-07-25 18:39:11]: > > > > I propose that darknet nodes be allowed to forward announcements and > > > > path folding messages (ConnectDestination etc), without including > > > > their own noderefs. > > > > > > > > Any objections? > > > > > > _o/ > > > It's a bad idea. > > > Be informed that my node wont behave that way... > > > > Why, precisely, is it a bad idea? As far as I can tell it doesn't > > compromise the nodes which only relay and never send their own noderef? > > It makes flow analysis related attackes both trivial and more effective. Only if it's done in a braindead way. Of course it has been, but it will be fixed (the current way risks fragmentation, as well as being a security issue). The following bugs are related to this: https://bugs.freenetproject.org/view.php?id=1566 https://bugs.freenetproject.org/view.php?id=1571 https://bugs.freenetproject.org/view.php?id=1570 https://bugs.freenetproject.org/view.php?id=1569 https://bugs.freenetproject.org/view.php?id=1568 > > I'm really dissapointed that you implemented it before leaving me time to > respond. You did so even though I explained to you my concerns on IRC... > > I told you about an attack vector (flow analysis using request size - > arguably already present but currently not easily exploitable) then we > disccused about workarounds (padding noderefs, limitating the number of > refs per request) and didn't manage to find any 'good' solution. There is only one ref per request. And transferring the noderef as a bulk transfer with a fixed size of say 2kB (four 512 byte packets, assuming data transfer is also in 512 byte packets, or two 1024 byte packets) would entirely solve the problem. I do agree that variable sized fields are a big deal. > > Your implementation doesn't even feature basic workarounds we talked about > and you have enabled that 'risky' option by default :( > > I don't mind about opennet beeing insecure but don't lower artificially the > security level provided by darknet. Even the current implementation will significantly help opennet (once it's been debugged), and give another reason to have darknet connections. Saying that people will just get them from #freenet-refs is nonsense, because #freenet-refs is worthless: it's a great way to get bad (newbie, slow, soon to leave) peers, and it costs a lot of time. > > NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/e59a4f9d/attachment.pgp From toad at amphibian.dyndns.org Mon Aug 6 21:28:30 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 22:28:30 +0100 Subject: [freenet-dev] Propagating opennet refs through darknet nodes In-Reply-To: <20070801202310.GH23461@freenetproject.org> References: <200707251839.16584.toad@amphibian.dyndns.org> <200707262001.35110.toad@amphibian.dyndns.org> <20070801202310.GH23461@freenetproject.org> Message-ID: <200708062228.30582.toad@amphibian.dyndns.org> FROM FROST: On Wednesday 01 August 2007 21:23, NextGen$ wrote: > * Matthew Toseland [2007-07-26 20:01:34]: > > On Thursday 26 July 2007 04:14, NextGen$ wrote: > > > * Matthew Toseland [2007-07-25 18:39:11]: > > > > I propose that darknet nodes be allowed to forward announcements and > > > > path folding messages (ConnectDestination etc), without including > > > > their own noderefs. > > > > > > > > Any objections? > > > > > > _o/ > > > It's a bad idea. > > > Be informed that my node wont behave that way... > > > > Why, precisely, is it a bad idea? As far as I can tell it doesn't > > compromise the nodes which only relay and never send their own noderef? > > It makes flow analysis related attackes both trivial and more effective. > > I'm really dissapointed that you implemented it before leaving me time to > respond. You did so even though I explained to you my concerns on IRC... > > I told you about an attack vector (flow analysis using request size - > arguably already present but currently not easily exploitable) then we > disccused about workarounds (padding noderefs, limitating the number of > refs per request) and didn't manage to find any 'good' solution. > > Your implementation doesn't even feature basic workarounds we talked about > and you have enabled that 'risky' option by default :( > > I don't mind about opennet beeing insecure but don't lower artificially the > security level provided by darknet. > > NextGen$ First, what kind of flow attack do you mean and why is it currently not exploitable but would be exploitable after that new feature? You know, such stuff is somewhat interesting in networks like freenet, it doesn't hurt to mention it publically. It might even make it simpler to find a solution you know ... Second, "lower artificially the security level provided by darknet" is what was done in the past to debug it, or due to time constraints, talking about "this is alpha, bear with it until it gets removed/fixed in months/years". I don't see the point why another temporary weakening is that important to you. It may serve another purpose and that purpose may be less important or perhaps the security issues are worse (how should I know?), but contrary to the weakenings for debugging reasons it may actually get fixed in reasonable time. There is even the chance of fixing it before releasing another version. Third, as said above, it is only available in trunk, not in a live build. Forth, not making it the default would render it useless. How about not "fixing" the problem by removing stuff for which you don't care, crippling it or rendering it useless, but just fixing it completly, even if that means waiting with another build for a week. You know, given freenets history I would bet that else it will just be left as it is for months if not years, as everything that works somewhat. Talk about link crypto that allows MITM and is DoSable (the first was fixed after ~1 year iirc, the second is finally a SoC project after at least one and a half years), not resuming SSK inserts, not resuming requests, thread madness, plugin interface confusion, ... It all works somewhat, possibly not as well as it should, possibly not helping/protecting as it should but it works in one way or another. Other things seem more important, thus its left as it is. And left and left and ... Fix it right away or it will be postponed forever. Not to blame any dev for that, simple time constraints and the resulting considerations will cause that if it works, even if just somewhat, and time constraints are admittedly a huge problem with freenets development. But it still doesn't make the result any better. Sorry for this rant but I do get frustrated over such things, given enough time. :| -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/632a8c83/attachment.pgp From toad at amphibian.dyndns.org Mon Aug 6 21:35:57 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 6 Aug 2007 22:35:57 +0100 Subject: [freenet-dev] Propagating opennet refs through darknet nodes In-Reply-To: <200708062228.30582.toad@amphibian.dyndns.org> References: <200707251839.16584.toad@amphibian.dyndns.org> <20070801202310.GH23461@freenetproject.org> <200708062228.30582.toad@amphibian.dyndns.org> Message-ID: <200708062235.58782.toad@amphibian.dyndns.org> On Monday 06 August 2007 22:28, Matthew Toseland wrote: > FROM FROST: > > On Wednesday 01 August 2007 21:23, NextGen$ wrote: > > * Matthew Toseland [2007-07-26 20:01:34]: > > > On Thursday 26 July 2007 04:14, NextGen$ wrote: > > > > * Matthew Toseland [2007-07-25 18:39:11]: > > > > > I propose that darknet nodes be allowed to forward announcements > > > > > and path folding messages (ConnectDestination etc), without > > > > > including their own noderefs. > > > > > > > > > > Any objections? > > > > > > > > _o/ > > > > It's a bad idea. > > > > Be informed that my node wont behave that way... > > > > > > Why, precisely, is it a bad idea? As far as I can tell it doesn't > > > compromise the nodes which only relay and never send their own noderef? > > > > It makes flow analysis related attackes both trivial and more effective. > > > > I'm really dissapointed that you implemented it before leaving me time to > > respond. You did so even though I explained to you my concerns on IRC... > > > > I told you about an attack vector (flow analysis using request size - > > arguably already present but currently not easily exploitable) then we > > disccused about workarounds (padding noderefs, limitating the number of > > refs per request) and didn't manage to find any 'good' solution. > > > > Your implementation doesn't even feature basic workarounds we talked > > about and you have enabled that 'risky' option by default :( > > > > I don't mind about opennet beeing insecure but don't lower artificially > > the security level provided by darknet. > > > > NextGen$ > > First, what kind of flow attack do you mean and why is it currently not > exploitable but would be exploitable after that new feature? You know, such > stuff is somewhat interesting in networks like freenet, it doesn't hurt to > mention it publically. It might even make it simpler to find a solution you > know ... > > Second, "lower artificially the security level provided by darknet" is what > was done in the past to debug it, or due to time constraints, talking > about "this is alpha, bear with it until it gets removed/fixed in > months/years". I don't see the point why another temporary weakening is > that important to you. It may serve another purpose and that purpose may be > less important or perhaps the security issues are worse (how should I > know?), but contrary to the weakenings for debugging reasons it may > actually get fixed in reasonable time. There is even the chance of fixing > it before releasing another version. > > Third, as said above, it is only available in trunk, not in a live build. > > Forth, not making it the default would render it useless. How about > not "fixing" the problem by removing stuff for which you don't care, > crippling it or rendering it useless, but just fixing it completly, even if > that means waiting with another build for a week. > You know, given freenets history I would bet that else it will just be left > as it is for months if not years, as everything that works somewhat. > Talk about link crypto that allows MITM and is DoSable (the first was fixed > after ~1 year iirc, the second is finally a SoC project after at least one > and a half years), not resuming SSK inserts, not resuming requests, thread > madness, plugin interface confusion, ... > It all works somewhat, possibly not as well as it should, possibly not > helping/protecting as it should but it works in one way or another. Other > things seem more important, thus its left as it is. And left and left and > ... Fix it right away or it will be postponed forever. Not to blame any dev > for that, simple time constraints and the resulting considerations will > cause that if it works, even if just somewhat, and time constraints are > admittedly a huge problem with freenets development. But it still doesn't > make the result any better. > Sorry for this rant but I do get frustrated over such things, given enough > time. :| The flow attack is real. All an attacker has to do is send an exceptionally large noderef, either from the data source node to trace the requester, or from the requester node to trace the data source. This could be tracked back to source assuming you can observe the packets between each pair of nodes on the chain, because it would cause a very large (fragmented) packet to be sent at each node. However, we *will* solve the problem, most likely by making noderef transfer happen in a bulk transfer (the other option would be binary references). Whether that will be done before the next stable build, I dunno; how vital is it? DoS on link crypto is not a big deal. You can only do it if you have the noderef of the node you are DoSing, and if you do, you can do lots of other fun things to them too (seize their node, kidnap them and render them unto the interrogators...). I accept that there are lots of bugs and even more unfinished features; we have to work on what appears to be the highest priority at the time, right now that's opennet. Request resuming isn't absolutely necessary, but I've made some progress on architecture for it so I may implement it before 0.7. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070806/697334db/attachment.pgp From nextgens at freenetproject.org Mon Aug 6 22:38:22 2007 From: nextgens at freenetproject.org (NextGen$) Date: Mon, 6 Aug 2007 22:38:22 +0000 Subject: [freenet-dev] Propagating opennet refs through darknet nodes In-Reply-To: <200708062228.30582.toad@amphibian.dyndns.org> References: <200707251839.16584.toad@amphibian.dyndns.org> <200707262001.35110.toad@amphibian.dyndns.org> <20070801202310.GH23461@freenetproject.org> <200708062228.30582.toad@amphibian.dyndns.org> Message-ID: <20070806223822.GQ29506@freenetproject.org> * Matthew Toseland [2007-08-06 22:28:30]: > FROM FROST: > > On Wednesday 01 August 2007 21:23, NextGen$ wrote: > > * Matthew Toseland [2007-07-26 20:01:34]: > > > On Thursday 26 July 2007 04:14, NextGen$ wrote: > > > > * Matthew Toseland [2007-07-25 18:39:11]: > > > > > I propose that darknet nodes be allowed to forward announcements and > > > > > path folding messages (ConnectDestination etc), without including > > > > > their own noderefs. > > > > > > > > > > Any objections? > > > > > > > > _o/ > > > > It's a bad idea. > > > > Be informed that my node wont behave that way... > > > > > > Why, precisely, is it a bad idea? As far as I can tell it doesn't > > > compromise the nodes which only relay and never send their own noderef? > > > > It makes flow analysis related attackes both trivial and more effective. > > > > I'm really dissapointed that you implemented it before leaving me time to > > respond. You did so even though I explained to you my concerns on IRC... > > > > I told you about an attack vector (flow analysis using request size - > > arguably already present but currently not easily exploitable) then we > > disccused about workarounds (padding noderefs, limitating the number of > > refs per request) and didn't manage to find any 'good' solution. > > > > Your implementation doesn't even feature basic workarounds we talked about > > and you have enabled that 'risky' option by default :( > > > > I don't mind about opennet beeing insecure but don't lower artificially the > > security level provided by darknet. > > > > NextGen$ > Let's reply in order : > First, what kind of flow attack do you mean and why is it currently not > exploitable but would be exploitable after that new feature? You know, such > stuff is somewhat interesting in networks like freenet, it doesn't hurt to > mention it publically. It might even make it simpler to find a solution you > know ... > Reading the thread will tell you about it... If you can't figure out the threat by yourself I doubt you can be of any help, sorry. > Second, "lower artificially the security level provided by darknet" is what > was done in the past to debug it, or due to time constraints, talking > about "this is alpha, bear with it until it gets removed/fixed in > months/years". I don't see the point why another temporary weakening is that > important to you. That one benefits only to opennet. > It may serve another purpose and that purpose may be less > important or perhaps the security issues are worse (how should I know?), but > contrary to the weakenings for debugging reasons it may actually get fixed in > reasonable time. There is even the chance of fixing it before releasing > another version. > Here we are talking about a 'feature' that isn't planned to be removed until opennet dies. > Third, as said above, it is only available in trunk, not in a live build. > Yeah, I said it... better to yell before the harm is done. > Forth, not making it the default would render it useless. How about > not "fixing" the problem by removing stuff for which you don't care, > crippling it or rendering it useless, but just fixing it completly, even if > that means waiting with another build for a week. Again, here we are talking about a feature which has inherent problems; not an implementation bug. > You know, given freenets history I would bet that else it will just be left as > it is for months if not years, as everything that works somewhat. Hence reacting now is important. > Talk about link crypto that allows MITM and is DoSable (the first was fixed > after ~1 year iirc, the second is finally a SoC project after at least one > and a half years), Don't say that to me : I proposed to implement it last year! http://archives.freenetproject.org/message/20060503.130933.ccc20f47.en.html > not resuming SSK inserts, As far as I know that works and has been implemented a while ago. > not resuming requests, It gives a good incentive to users not to restart their node (helps limitating churn) hence I didn't try to implement it... others might have their reasons as well. > thread madness, I worked on that one... and have plans but no time to improve the situation even more. > plugin interface confusion, ... Toad was working on that one before opennet became 'the priority'... I won't do it : three different persons have been involved and now we have three different plugin systems. I wont do a fourth. > It all works somewhat, possibly not as well as it should, possibly not > helping/protecting as it should but it works in one way or another. Other > things seem more important, thus its left as it is. And left and left and ... I^we have a finite amount of time at our disposal, so we schedule according to priorities... Of course devs and users don't have the same views on what's important... Do it yourself or propose a bounty to someone if you really want something to be implemented in spite of its 'low priority'. > Fix it right away or it will be postponed forever. In that specific case it's 'revert toad's patch'. That feature is broken by design... there might be acceptable workarounds to mitigate its effects... but the main problem is that they haven't been discussed nor implemented. > Not to blame any dev for > that, simple time constraints and the resulting considerations will cause > that if it works, even if just somewhat, and time constraints are admittedly > a huge problem with freenets development. I wish it was a freenet-specific problem > But it still doesn't make the result any better. Ranting doesn't help either. Send patches ;) > Sorry for this rant but I do get frustrated over such things, given enough > time. :| Me too. NextGen$ From ian at locut.us Tue Aug 7 00:46:45 2007 From: ian at locut.us (Ian Clarke) Date: Mon, 6 Aug 2007 19:46:45 -0500 Subject: [freenet-dev] Link to Subversion repository Message-ID: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> I can't seem to find a link to our subversion repository anywhere on the website. Unless I'm missing something, that is a rather critical oversight. Ian. -- Founder and CEO, Thoof Inc Email: ian at thoof.com Office: +1 512 524 8934 x 100 Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From juiceman69 at gmail.com Tue Aug 7 00:57:14 2007 From: juiceman69 at gmail.com (Juiceman) Date: Mon, 6 Aug 2007 20:57:14 -0400 Subject: [freenet-dev] Link to Subversion repository In-Reply-To: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> References: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> Message-ID: <8b525dee0708061757u7b44af35x792d661c293ea796@mail.gmail.com> You need to go under the "Developer" section. ;-) http://emu.freenetproject.org/cgi-bin/viewcvs.cgi/trunk/ On 8/6/07, Ian Clarke wrote: > I can't seem to find a link to our subversion repository anywhere on > the website. Unless I'm missing something, that is a rather critical > oversight. > > Ian. > > -- > Founder and CEO, Thoof Inc > Email: ian at thoof.com > Office: +1 512 524 8934 x 100 > Cell: +1 512 422 3588 > AIM: ian.clarke at mac.com > Skype: sanity > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -- I may disagree with what you have to say, but I shall defend, to the death, your right to say it. - Voltaire Those who would give up Liberty, to purchase temporary Safety, deserve neither Liberty nor Safety. - Ben Franklin From ian at locut.us Tue Aug 7 04:02:32 2007 From: ian at locut.us (Ian Clarke) Date: Mon, 6 Aug 2007 23:02:32 -0500 Subject: [freenet-dev] Link to Subversion repository In-Reply-To: <8b525dee0708061757u7b44af35x792d661c293ea796@mail.gmail.com> References: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> <8b525dee0708061757u7b44af35x792d661c293ea796@mail.gmail.com> Message-ID: <823242bd0708062102p24898383yf6f5615a4ede30d7@mail.gmail.com> Nope, that isn't the repository URL, its just a link to a web interface for viewing the repository. I can't give that to Subversion and have it check out and commit to a repository. The correct repository URL is: https://emu.freenetproject.org/svn/ Ian. On 8/6/07, Juiceman wrote: > You need to go under the "Developer" section. ;-) > > http://emu.freenetproject.org/cgi-bin/viewcvs.cgi/trunk/ > > On 8/6/07, Ian Clarke wrote: > > I can't seem to find a link to our subversion repository anywhere on > > the website. Unless I'm missing something, that is a rather critical > > oversight. > > > > Ian. > > > > -- > > Founder and CEO, Thoof Inc > > Email: ian at thoof.com > > Office: +1 512 524 8934 x 100 > > Cell: +1 512 422 3588 > > AIM: ian.clarke at mac.com > > Skype: sanity > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > > -- > I may disagree with what you have to say, but I shall defend, to the > death, your right to say it. - Voltaire > Those who would give up Liberty, to purchase temporary Safety, deserve > neither Liberty nor Safety. - Ben Franklin > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -- Founder and CEO, Thoof Inc Email: ian at thoof.com Office: +1 512 524 8934 x 100 Cell: +1 512 422 3588 AIM: ian.clarke at mac.com Skype: sanity From sback at sback.it Tue Aug 7 06:16:34 2007 From: sback at sback.it (Alberto Bacchelli) Date: Tue, 07 Aug 2007 08:16:34 +0200 Subject: [freenet-dev] Link to Subversion repository In-Reply-To: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> References: <823242bd0708061746i5cfb48e9m5f90e710e645bdc7@mail.gmail.com> Message-ID: <46B80E42.3090301@sback.it> Ian Clarke wrote: > I can't seem to find a link to our subversion repository anywhere on > the website. Unless I'm missing something, that is a rather critical > oversight. > > Ian. > > Hi, at the end of Download page [ http://freenetproject.org/download.html ], there is the "Source Code" section with this link: https://emu.freenetproject.org/svn/trunk/freenet/ . I found it easly when I started working on Freenet before GSoC Cheers, Sback From vive at freenetproject.org Tue Aug 7 22:28:09 2007 From: vive at freenetproject.org (vive) Date: Wed, 8 Aug 2007 00:28:09 +0200 Subject: [freenet-dev] Load management proposal In-Reply-To: <200706181640.25133.toad@amphibian.dyndns.org> References: <200706181640.25133.toad@amphibian.dyndns.org> Message-ID: <20070807222809.GC6302@tim.hack.org> I could not find this discussion on the frost board any longer. Did it continue? Then I'd be happy if someone could post it to the list or send me a copy. :-) Also regarding the blocking-sockets method to let a node delay requests. This seems to have some sideeffects to me that may be unwanted later on. If Freenet would need a QoS mechanism or OOB data (some sort of control messages with higher priority) -- wouldnt blocking socket reads entirely for limiting requests also block these out? But I still find the idea interesting to delay traffic when the load is high rather than preemptively rejecting it. This would simplify adapting greedy routing to take delays into account (e.g. only having to consider delays as the extra factor when making routing decisions at a node will simplify things a lot by incorporating both the effect from link delay and node load). This is what I'm currently working on.. regards, Vilhelm On Mon, Jun 18, 2007 at 04:40:16PM +0100, Matthew Toseland wrote: > MRogers and Anonymous have been arguing that we should limit load only on the > immediate bandwidth level. Some of the thread is below, and my response > including a proposal for a bandwidth limiting mechanism: > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.14 - 21:32:43GMT ----- > > No, what I've been arguing is that we don't need to *explicitly* take the > lifetime overhead into account, because the feedback loop should be slow > enough that it will *implicitly* take it into account anyway. If the feedback > loop takes a few minutes to adapt (which it must if we don't want to > over-adapt to short-term conditions) then we're implicitly measuring the > lifetime overhead of a search, and we don't need to make an additional > explicit estimate of the lifetime overhead. > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.15 - 17:12:52GMT ----- > > What makes you think it would work at all? It seems to me that it would simply > oscillate - we accept too many requests, most of them time out, the we accept > too many requests again, and most of them timeout again. KISS is one thing, > but not slitting your own throat with occam's razor is another! > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.15 - 21:44:34GMT ----- > > :) If the feedback loop is slow enough it shouldn't oscillate - that applies > equally to bandwidth liability estimation, token passing or any other > mechanism - we musn't get into the situation of saying "I accepted 100 > requests in the last 10 seconds and nothing bad has happened yet, therefore I > can accept another 100". As far as I can see, the only way to avoid that > situation is by adapting very slowly, ie on the order of the longest timeout. > Luckily we expect the connections to be relatively long-lived (minutes or > hours) so we can afford to take a few minutes to get up to speed. > > Assuming that we have a suitably slow feedback loop in place, the next > question is how to tell our peers when we're ready to accept another search. > We could use tokens, preemptive rejection or blocking sockets. I don't have > any religious opinions on this issue, but I thought Anonymous made a fairly > good case for handling it in the same way as any other network app: don't > read from the socket until you're ready to process more data. I realise > there's a highly variable relationship between bytes in and bytes out, but > regardless of which mechanism we use we'll have to rely on the slow feedback > loop to smooth that out, because so far nobody's suggested a way of dealing > with it that doesn't involve favouring some kinds of traffic over others. (I > hope we agree that we don't want the whole network to end up favouring SSK > traffic over CHK traffic just because it happens to come in smaller quanta. > Maybe there are reasons for giving SSK traffic priority, but that isn't one > of them.) > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.18 - 14:25:49GMT ----- > > It's not possible using the mechanism you propose *on its own*, while it *is* > interesting. Here's why: I have 48kB/sec output. With your proposed > mechanism, the fact that I have 1MB/sec input won't matter. So a leaf node > requests a splitfile through me, which is all in the network, but is not in > my datastore. I route his requests. It takes 5 seconds for enough of the > requests to start transferring to make a difference. Over that 5 seconds, > he's sent me 240kB of requests. That's around 240k/50 = 4,915 requests. Which > will yield 160MB of data. Unfortunately it will take me 160M/48k = nearly an > hour to transfer all the data. > > This is not acceptable, because we want Freenet requests to complete in a > reasonable amount of time - if only for fproxy support, which IMHO is an > important application. And I don't like the idea of having different request > priorities either; it gives away information on the traffic and allows for > Bad Things. > > Imposing an arbitrary limit on the number of requests running is not the > solution either. A limit may be imposed because of threads/memory usage, but > this is likely to be heavily architecture dependant. The solution IMHO is to > take into account likely future bandwidth usage. If we want to guarantee that > there are no timeouts, this means bandwidth liability limiting. The current > code will accept an SSK request only if it could also accept a CHK request, > and vice versa, so I don't see any reason to think it is excessively biased > in favour of CHKs. However if you like I will add code to collect the > probability of rejection for SSKs and CHKs individually. > > For data blocks, clearly we cannot send one if there is no available space in > the congestion window to the peer in question. However we may want the data > for ourself, or for multiple peers, in which case the slowest peer should not > necessarily dictate the transfer rate; AFAICS we must accept the data as fast > as we can manage within memory/cpu/etc limits, and limit the incoming > bandwidth at a higher level. > > So given that we must limit traffic on the request level (as well as the > congestion control level), how can we do this? We can either: > > A) Wait for a request, and then either accept it or reject it, based on e.g. > how many requests are running already. > PRO: > - Simple > CON: > - Wasteful of time and bandwidth when we have to ask multiple peers before one > accepts > - Difficult to adapt to propagate load back to source > > Queueing doesn't help much here because we still have to complete the > request - including queueing it - in a reasonable time. > > or > > B) Tell our peers when they can send us a request. The obvious way to do this > is to compute our overall quota of requests (or request-success-bytes), and > allocate it to our peers (and tell them on every packet/response/etc), > initially equally, but progressively allowing more to busier nodes and less > to nodes that don't use their quota (but with diminishing returns: a node > with a low quota that suddenly starts using more will take quota away from an > established heavy requestor). Thus, initially, if most nodes aren't busy we > won't run many requests, but as we identify those nodes which need a bigger > quota, more requests run overall. Note that running fewer requests than we > could isn't necessarily a problem anyway unless they complete really slowly. > > How to avoid excessive misrouting? > Options: > 1: We don't need to queue requests, as they are queued on the next node for us > (running requests can be regarded as queued requests). When we get a request, > we identify the set of nodes that we are willing to route it to, and if none > of them have any free request slots, we reject it. > 2: Queueing requests helps, because we can match requests with nodes. When we > get a request, (if it's allowable by the node's current quota), we queue it. > When we have a slot in an outgoing node's quota, we send the request which is > closest to the node's location (which hasn't already been to that node), > regardless of the time it's been queued for (if multiple nodes have space, we > find the best request/node pair until they don't or we run out of requests). > If after a certain period a request hasn't been sent, we reject it. This > avoids excessive misrouting without requiring arbitrary parameters (as the > first option does), and sends more specialised requests to slower nodes. > 3: Simulated queueing: remember the locations of recent requests we could have > sent to a node, and calculate a threshold, which decays over time if we don't > accept a request (obviously this will result in fewer requests being sent, > but lower latency). > > Does this prevent flooding? Yes: we only accept, and offer, as many requests > as we can handle, and with the right fairness algorithms, a flood will be > limited to what the other peers don't need, although if our overall quota > calculation is too generous flooding may still be an effective attack. > Obviously on opennet, connecting to several hundred peers and flooding to > each of them will be fairly effective; opennet sucks, we all know this, it's > a transition phase. > > Does this propagate load back to the originator? Not explicitly, as it's not > an originator-throttles scheme (such schemes don't prevent flooding and may > make it easier to identify the originator). However, the rate at which a node > can send requests is quite clearly limited by the rate at which the rest of > the network can handle them: > > O - A - B - C > > If B is a bottleneck, then O will only send requests at the rate that can be > funneled through it (or satisfied on A). > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070808/bb7ae98c/attachment.pgp From batosai at batosai.net Wed Aug 8 10:49:51 2007 From: batosai at batosai.net (batosai) Date: Wed, 08 Aug 2007 12:49:51 +0200 Subject: [freenet-dev] The welcome.html page Message-ID: <46B99FCF.7060007@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I noticed two issues with that page : - - It doesn't mention the opennet (It even says that the user can get darknet connections on #freenet-refs, what I see as being BAD). I don't know when you plan to release the opennet officialy but I think the user should already know that he should enable opennet if he has no friends running freenet. - - It is the only part of the entire installation process that doesn't support internationalization. Isn't it possible to replace this static page with a bunch of translatable strings, like the wizard and the entire interface ? Regards -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGuZ/Po6N05NzaOvURAoLYAKCqlfoX3JrZOOYzdKJ6eLaV/CsNCQCdHB3i mmC/ZDiQZGVrVb3qMtFS6hg= =IS/5 -----END PGP SIGNATURE----- From toad at amphibian.dyndns.org Wed Aug 8 14:39:34 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 8 Aug 2007 15:39:34 +0100 Subject: [freenet-dev] Load management proposal In-Reply-To: <20070807222809.GC6302@tim.hack.org> References: <200706181640.25133.toad@amphibian.dyndns.org> <20070807222809.GC6302@tim.hack.org> Message-ID: <200708081539.34813.toad@amphibian.dyndns.org> There hasn't been any such debate for some time. IIRC it stopped just before one of the earlier spam attacks, and hasn't resumed since. You might want to talk to mrogers about it. I will go through the messages from Frost (which fortunately I still have on freenet.tech because of my long expiry setting) and post the most important ones on tech or devl. On Tuesday 07 August 2007 23:28, vive wrote: > I could not find this discussion on the frost board any longer. Did it > continue? Then I'd be happy if someone could post it to the list or send me > a copy. :-) > > Also regarding the blocking-sockets method to let a node delay requests. > This seems to have some sideeffects to me that may be unwanted later on. If > Freenet would need a QoS mechanism or OOB data (some sort of control > messages with higher priority) -- wouldnt blocking socket reads entirely > for limiting requests also block these out? But I still find the idea > interesting to delay traffic when the load is high rather than preemptively > rejecting it. This would simplify adapting greedy routing to take delays > into account (e.g. only having to consider delays as the extra factor when > making routing decisions at a node will simplify things a lot by > incorporating both the effect from link delay and node load). This is what > I'm currently working on.. > > regards, > Vilhelm > > On Mon, Jun 18, 2007 at 04:40:16PM +0100, Matthew Toseland wrote: > > MRogers and Anonymous have been arguing that we should limit load only on > > the immediate bandwidth level. Some of the thread is below, and my > > response including a proposal for a bandwidth limiting mechanism: > > > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.14 - 21:32:43GMT > > ----- > > > > No, what I've been arguing is that we don't need to *explicitly* take the > > lifetime overhead into account, because the feedback loop should be slow > > enough that it will *implicitly* take it into account anyway. If the > > feedback loop takes a few minutes to adapt (which it must if we don't > > want to over-adapt to short-term conditions) then we're implicitly > > measuring the lifetime overhead of a search, and we don't need to make an > > additional explicit estimate of the lifetime overhead. > > > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.15 - 17:12:52GMT > > ----- > > > > What makes you think it would work at all? It seems to me that it would > > simply oscillate - we accept too many requests, most of them time out, > > the we accept too many requests again, and most of them timeout again. > > KISS is one thing, but not slitting your own throat with occam's razor is > > another! > > > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.15 - 21:44:34GMT > > ----- > > > > :) If the feedback loop is slow enough it shouldn't oscillate - that > > : applies > > > > equally to bandwidth liability estimation, token passing or any other > > mechanism - we musn't get into the situation of saying "I accepted 100 > > requests in the last 10 seconds and nothing bad has happened yet, > > therefore I can accept another 100". As far as I can see, the only way to > > avoid that situation is by adapting very slowly, ie on the order of the > > longest timeout. Luckily we expect the connections to be relatively > > long-lived (minutes or hours) so we can afford to take a few minutes to > > get up to speed. > > > > Assuming that we have a suitably slow feedback loop in place, the next > > question is how to tell our peers when we're ready to accept another > > search. We could use tokens, preemptive rejection or blocking sockets. I > > don't have any religious opinions on this issue, but I thought Anonymous > > made a fairly good case for handling it in the same way as any other > > network app: don't read from the socket until you're ready to process > > more data. I realise there's a highly variable relationship between bytes > > in and bytes out, but regardless of which mechanism we use we'll have to > > rely on the slow feedback loop to smooth that out, because so far > > nobody's suggested a way of dealing with it that doesn't involve > > favouring some kinds of traffic over others. (I hope we agree that we > > don't want the whole network to end up favouring SSK traffic over CHK > > traffic just because it happens to come in smaller quanta. Maybe there > > are reasons for giving SSK traffic priority, but that isn't one of them.) > > > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.18 - 14:25:49GMT > > ----- > > > > It's not possible using the mechanism you propose *on its own*, while it > > *is* interesting. Here's why: I have 48kB/sec output. With your proposed > > mechanism, the fact that I have 1MB/sec input won't matter. So a leaf > > node requests a splitfile through me, which is all in the network, but is > > not in my datastore. I route his requests. It takes 5 seconds for enough > > of the requests to start transferring to make a difference. Over that 5 > > seconds, he's sent me 240kB of requests. That's around 240k/50 = 4,915 > > requests. Which will yield 160MB of data. Unfortunately it will take me > > 160M/48k = nearly an hour to transfer all the data. > > > > This is not acceptable, because we want Freenet requests to complete in a > > reasonable amount of time - if only for fproxy support, which IMHO is an > > important application. And I don't like the idea of having different > > request priorities either; it gives away information on the traffic and > > allows for Bad Things. > > > > Imposing an arbitrary limit on the number of requests running is not the > > solution either. A limit may be imposed because of threads/memory usage, > > but this is likely to be heavily architecture dependant. The solution > > IMHO is to take into account likely future bandwidth usage. If we want to > > guarantee that there are no timeouts, this means bandwidth liability > > limiting. The current code will accept an SSK request only if it could > > also accept a CHK request, and vice versa, so I don't see any reason to > > think it is excessively biased in favour of CHKs. However if you like I > > will add code to collect the probability of rejection for SSKs and CHKs > > individually. > > > > For data blocks, clearly we cannot send one if there is no available > > space in the congestion window to the peer in question. However we may > > want the data for ourself, or for multiple peers, in which case the > > slowest peer should not necessarily dictate the transfer rate; AFAICS we > > must accept the data as fast as we can manage within memory/cpu/etc > > limits, and limit the incoming bandwidth at a higher level. > > > > So given that we must limit traffic on the request level (as well as the > > congestion control level), how can we do this? We can either: > > > > A) Wait for a request, and then either accept it or reject it, based on > > e.g. how many requests are running already. > > PRO: > > - Simple > > CON: > > - Wasteful of time and bandwidth when we have to ask multiple peers > > before one accepts > > - Difficult to adapt to propagate load back to source > > > > Queueing doesn't help much here because we still have to complete the > > request - including queueing it - in a reasonable time. > > > > or > > > > B) Tell our peers when they can send us a request. The obvious way to do > > this is to compute our overall quota of requests (or > > request-success-bytes), and allocate it to our peers (and tell them on > > every packet/response/etc), initially equally, but progressively allowing > > more to busier nodes and less to nodes that don't use their quota (but > > with diminishing returns: a node with a low quota that suddenly starts > > using more will take quota away from an established heavy requestor). > > Thus, initially, if most nodes aren't busy we won't run many requests, > > but as we identify those nodes which need a bigger quota, more requests > > run overall. Note that running fewer requests than we could isn't > > necessarily a problem anyway unless they complete really slowly. > > > > How to avoid excessive misrouting? > > Options: > > 1: We don't need to queue requests, as they are queued on the next node > > for us (running requests can be regarded as queued requests). When we get > > a request, we identify the set of nodes that we are willing to route it > > to, and if none of them have any free request slots, we reject it. > > 2: Queueing requests helps, because we can match requests with nodes. > > When we get a request, (if it's allowable by the node's current quota), > > we queue it. When we have a slot in an outgoing node's quota, we send the > > request which is closest to the node's location (which hasn't already > > been to that node), regardless of the time it's been queued for (if > > multiple nodes have space, we find the best request/node pair until they > > don't or we run out of requests). If after a certain period a request > > hasn't been sent, we reject it. This avoids excessive misrouting without > > requiring arbitrary parameters (as the first option does), and sends more > > specialised requests to slower nodes. 3: Simulated queueing: remember the > > locations of recent requests we could have sent to a node, and calculate > > a threshold, which decays over time if we don't accept a request > > (obviously this will result in fewer requests being sent, but lower > > latency). > > > > Does this prevent flooding? Yes: we only accept, and offer, as many > > requests as we can handle, and with the right fairness algorithms, a > > flood will be limited to what the other peers don't need, although if our > > overall quota calculation is too generous flooding may still be an > > effective attack. Obviously on opennet, connecting to several hundred > > peers and flooding to each of them will be fairly effective; opennet > > sucks, we all know this, it's a transition phase. > > > > Does this propagate load back to the originator? Not explicitly, as it's > > not an originator-throttles scheme (such schemes don't prevent flooding > > and may make it easier to identify the originator). However, the rate at > > which a node can send requests is quite clearly limited by the rate at > > which the rest of the network can handle them: > > > > O - A - B - C > > > > If B is a bottleneck, then O will only send requests at the rate that can > > be funneled through it (or satisfied on A). > > > > > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070808/f729bbcd/attachment.pgp From toad at amphibian.dyndns.org Wed Aug 8 16:20:48 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 8 Aug 2007 17:20:48 +0100 Subject: [freenet-dev] Load management proposal In-Reply-To: <200708081539.34813.toad@amphibian.dyndns.org> References: <200706181640.25133.toad@amphibian.dyndns.org> <20070807222809.GC6302@tim.hack.org> <200708081539.34813.toad@amphibian.dyndns.org> Message-ID: <200708081720.50031.toad@amphibian.dyndns.org> On Wednesday 08 August 2007 15:39, Matthew Toseland wrote: > There hasn't been any such debate for some time. IIRC it stopped just > before one of the earlier spam attacks, and hasn't resumed since. You might > want to talk to mrogers about it. I will go through the messages from Frost > (which fortunately I still have on freenet.tech because of my long expiry > setting) and post the most important ones on tech or devl. I posted all of them to tech. > > On Tuesday 07 August 2007 23:28, vive wrote: > > I could not find this discussion on the frost board any longer. Did it > > continue? Then I'd be happy if someone could post it to the list or send > > me a copy. :-) > > > > Also regarding the blocking-sockets method to let a node delay requests. > > This seems to have some sideeffects to me that may be unwanted later on. > > If Freenet would need a QoS mechanism or OOB data (some sort of control > > messages with higher priority) -- wouldnt blocking socket reads entirely > > for limiting requests also block these out? But I still find the idea > > interesting to delay traffic when the load is high rather than > > preemptively rejecting it. This would simplify adapting greedy routing to > > take delays into account (e.g. only having to consider delays as the > > extra factor when making routing decisions at a node will simplify things > > a lot by incorporating both the effect from link delay and node load). > > This is what I'm currently working on.. > > > > regards, > > Vilhelm > > > > On Mon, Jun 18, 2007 at 04:40:16PM +0100, Matthew Toseland wrote: > > > MRogers and Anonymous have been arguing that we should limit load only > > > on the immediate bandwidth level. Some of the thread is below, and my > > > response including a proposal for a bandwidth limiting mechanism: > > > > > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.14 - > > > 21:32:43GMT ----- > > > > > > No, what I've been arguing is that we don't need to *explicitly* take > > > the lifetime overhead into account, because the feedback loop should be > > > slow enough that it will *implicitly* take it into account anyway. If > > > the feedback loop takes a few minutes to adapt (which it must if we > > > don't want to over-adapt to short-term conditions) then we're > > > implicitly measuring the lifetime overhead of a search, and we don't > > > need to make an additional explicit estimate of the lifetime overhead. > > > > > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.15 - 17:12:52GMT > > > ----- > > > > > > What makes you think it would work at all? It seems to me that it would > > > simply oscillate - we accept too many requests, most of them time out, > > > the we accept too many requests again, and most of them timeout again. > > > KISS is one thing, but not slitting your own throat with occam's razor > > > is another! > > > > > > ----- mrogers at UU62+3E1vKT1k+7fR0Gx7ZN2IB0 ----- 2007.06.15 - > > > 21:44:34GMT ----- > > > > > > :) If the feedback loop is slow enough it shouldn't oscillate - that > > > : applies > > > > > > equally to bandwidth liability estimation, token passing or any other > > > mechanism - we musn't get into the situation of saying "I accepted 100 > > > requests in the last 10 seconds and nothing bad has happened yet, > > > therefore I can accept another 100". As far as I can see, the only way > > > to avoid that situation is by adapting very slowly, ie on the order of > > > the longest timeout. Luckily we expect the connections to be relatively > > > long-lived (minutes or hours) so we can afford to take a few minutes to > > > get up to speed. > > > > > > Assuming that we have a suitably slow feedback loop in place, the next > > > question is how to tell our peers when we're ready to accept another > > > search. We could use tokens, preemptive rejection or blocking sockets. > > > I don't have any religious opinions on this issue, but I thought > > > Anonymous made a fairly good case for handling it in the same way as > > > any other network app: don't read from the socket until you're ready to > > > process more data. I realise there's a highly variable relationship > > > between bytes in and bytes out, but regardless of which mechanism we > > > use we'll have to rely on the slow feedback loop to smooth that out, > > > because so far nobody's suggested a way of dealing with it that doesn't > > > involve favouring some kinds of traffic over others. (I hope we agree > > > that we don't want the whole network to end up favouring SSK traffic > > > over CHK traffic just because it happens to come in smaller quanta. > > > Maybe there are reasons for giving SSK traffic priority, but that isn't > > > one of them.) > > > > > > ----- toad at zceUWxlSaHLmvEMnbr4RHnVfehA ----- 2007.06.18 - 14:25:49GMT > > > ----- > > > > > > It's not possible using the mechanism you propose *on its own*, while > > > it *is* interesting. Here's why: I have 48kB/sec output. With your > > > proposed mechanism, the fact that I have 1MB/sec input won't matter. So > > > a leaf node requests a splitfile through me, which is all in the > > > network, but is not in my datastore. I route his requests. It takes 5 > > > seconds for enough of the requests to start transferring to make a > > > difference. Over that 5 seconds, he's sent me 240kB of requests. That's > > > around 240k/50 = 4,915 requests. Which will yield 160MB of data. > > > Unfortunately it will take me 160M/48k = nearly an hour to transfer all > > > the data. > > > > > > This is not acceptable, because we want Freenet requests to complete in > > > a reasonable amount of time - if only for fproxy support, which IMHO is > > > an important application. And I don't like the idea of having different > > > request priorities either; it gives away information on the traffic and > > > allows for Bad Things. > > > > > > Imposing an arbitrary limit on the number of requests running is not > > > the solution either. A limit may be imposed because of threads/memory > > > usage, but this is likely to be heavily architecture dependant. The > > > solution IMHO is to take into account likely future bandwidth usage. If > > > we want to guarantee that there are no timeouts, this means bandwidth > > > liability limiting. The current code will accept an SSK request only if > > > it could also accept a CHK request, and vice versa, so I don't see any > > > reason to think it is excessively biased in favour of CHKs. However if > > > you like I will add code to collect the probability of rejection for > > > SSKs and CHKs individually. > > > > > > For data blocks, clearly we cannot send one if there is no available > > > space in the congestion window to the peer in question. However we may > > > want the data for ourself, or for multiple peers, in which case the > > > slowest peer should not necessarily dictate the transfer rate; AFAICS > > > we must accept the data as fast as we can manage within memory/cpu/etc > > > limits, and limit the incoming bandwidth at a higher level. > > > > > > So given that we must limit traffic on the request level (as well as > > > the congestion control level), how can we do this? We can either: > > > > > > A) Wait for a request, and then either accept it or reject it, based on > > > e.g. how many requests are running already. > > > PRO: > > > - Simple > > > CON: > > > - Wasteful of time and bandwidth when we have to ask multiple peers > > > before one accepts > > > - Difficult to adapt to propagate load back to source > > > > > > Queueing doesn't help much here because we still have to complete the > > > request - including queueing it - in a reasonable time. > > > > > > or > > > > > > B) Tell our peers when they can send us a request. The obvious way to > > > do this is to compute our overall quota of requests (or > > > request-success-bytes), and allocate it to our peers (and tell them on > > > every packet/response/etc), initially equally, but progressively > > > allowing more to busier nodes and less to nodes that don't use their > > > quota (but with diminishing returns: a node with a low quota that > > > suddenly starts using more will take quota away from an established > > > heavy requestor). Thus, initially, if most nodes aren't busy we won't > > > run many requests, but as we identify those nodes which need a bigger > > > quota, more requests run overall. Note that running fewer requests than > > > we could isn't necessarily a problem anyway unless they complete really > > > slowly. > > > > > > How to avoid excessive misrouting? > > > Options: > > > 1: We don't need to queue requests, as they are queued on the next node > > > for us (running requests can be regarded as queued requests). When we > > > get a request, we identify the set of nodes that we are willing to > > > route it to, and if none of them have any free request slots, we reject > > > it. 2: Queueing requests helps, because we can match requests with > > > nodes. When we get a request, (if it's allowable by the node's current > > > quota), we queue it. When we have a slot in an outgoing node's quota, > > > we send the request which is closest to the node's location (which > > > hasn't already been to that node), regardless of the time it's been > > > queued for (if multiple nodes have space, we find the best request/node > > > pair until they don't or we run out of requests). If after a certain > > > period a request hasn't been sent, we reject it. This avoids excessive > > > misrouting without requiring arbitrary parameters (as the first option > > > does), and sends more specialised requests to slower nodes. 3: > > > Simulated queueing: remember the locations of recent requests we could > > > have sent to a node, and calculate a threshold, which decays over time > > > if we don't accept a request (obviously this will result in fewer > > > requests being sent, but lower latency). > > > > > > Does this prevent flooding? Yes: we only accept, and offer, as many > > > requests as we can handle, and with the right fairness algorithms, a > > > flood will be limited to what the other peers don't need, although if > > > our overall quota calculation is too generous flooding may still be an > > > effective attack. Obviously on opennet, connecting to several hundred > > > peers and flooding to each of them will be fairly effective; opennet > > > sucks, we all know this, it's a transition phase. > > > > > > Does this propagate load back to the originator? Not explicitly, as > > > it's not an originator-throttles scheme (such schemes don't prevent > > > flooding and may make it easier to identify the originator). However, > > > the rate at which a node can send requests is quite clearly limited by > > > the rate at which the rest of the network can handle them: > > > > > > O - A - B - C > > > > > > If B is a bottleneck, then O will only send requests at the rate that > > > can be funneled through it (or satisfied on A). > > > > > > > > > > > > _______________________________________________ > > > Devl mailing list > > > Devl at freenetproject.org > > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070808/f1bdd77f/attachment.pgp From nextgens at freenetproject.org Thu Aug 9 11:00:03 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 9 Aug 2007 13:00:03 +0200 Subject: [freenet-dev] [freenet-cvs] r14551 - trunk/freenet/src/freenet/node In-Reply-To: <20070808235953.B48FC47A23C@freenetproject.org> References: <20070808235953.B48FC47A23C@freenetproject.org> Message-ID: <20070809110003.GF5467@freenetproject.org> * juiceman at freenetproject.org [2007-08-08 23:59:53]: > Author: juiceman > Date: 2007-08-08 23:59:53 +0000 (Wed, 08 Aug 2007) > New Revision: 14551 > > Modified: > trunk/freenet/src/freenet/node/NodeClientCore.java > Log: > Surely is time to set the default to false. Users have had months of notice. I will let someone else drop support for pre-1010 keys in the future. > We probably want to go futher than that ... with your patch only new setups will default to false. What about existing nodes ? NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/cd4baaca/attachment.pgp From nextgens at freenetproject.org Thu Aug 9 11:41:49 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 9 Aug 2007 13:41:49 +0200 Subject: [freenet-dev] The welcome.html page In-Reply-To: <46B99FCF.7060007@batosai.net> References: <46B99FCF.7060007@batosai.net> Message-ID: <20070809114149.GH5467@freenetproject.org> * batosai [2007-08-08 12:49:51]: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, I noticed two issues with that page : > > - - It doesn't mention the opennet (It even says that the user can get > darknet connections on #freenet-refs, what I see as being BAD). I don't > know when you plan to release the opennet officialy but I think the user > should already know that he should enable opennet if he has no friends > running freenet. There is already a bug ticket filled in for that iirc > > - - It is the only part of the entire installation process that doesn't > support internationalization. Isn't it possible to replace this static > page with a bunch of translatable strings, like the wizard and the > entire interface ? Fill in a ticket for that one :)... It's a bit tricky to do it well; I expect the user to be reading the page while the node is starting up : If we want the node to serve that page then we have to spawn fproxy earlier on NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/fb3819c5/attachment.pgp From batosai at batosai.net Thu Aug 9 12:15:46 2007 From: batosai at batosai.net (batosai) Date: Thu, 09 Aug 2007 14:15:46 +0200 Subject: [freenet-dev] The welcome.html page In-Reply-To: <20070809114149.GH5467@freenetproject.org> References: <46B99FCF.7060007@batosai.net> <20070809114149.GH5467@freenetproject.org> Message-ID: <46BB0572.7090003@batosai.net> Florent Daigni?re a ?crit : > * batosai [2007-08-08 12:49:51]: >> - - It is the only part of the entire installation process that doesn't >> support internationalization. Isn't it possible to replace this static >> page with a bunch of translatable strings, like the wizard and the >> entire interface ? > > Fill in a ticket for that one :)... It's a bit tricky to do it well; I > expect the user to be reading the page while the node is starting up : > If we want the node to serve that page then we have to spawn fproxy > earlier on Mmmm, I see. And what about starting the node in the background while the installer is downloading Frost, Thaw, etc. ? That would give us a little extra-time... From nextgens at freenetproject.org Thu Aug 9 13:28:44 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 9 Aug 2007 15:28:44 +0200 Subject: [freenet-dev] The welcome.html page In-Reply-To: <46BB0572.7090003@batosai.net> References: <46B99FCF.7060007@batosai.net> <20070809114149.GH5467@freenetproject.org> <46BB0572.7090003@batosai.net> Message-ID: <20070809132843.GJ5467@freenetproject.org> * batosai [2007-08-09 14:15:46]: > Florent Daigni?re a ?crit : > > * batosai [2007-08-08 12:49:51]: > >> - - It is the only part of the entire installation process that doesn't > >> support internationalization. Isn't it possible to replace this static > >> page with a bunch of translatable strings, like the wizard and the > >> entire interface ? > > > > Fill in a ticket for that one :)... It's a bit tricky to do it well; I > > expect the user to be reading the page while the node is starting up : > > If we want the node to serve that page then we have to spawn fproxy > > earlier on > > Mmmm, I see. And what about starting the node in the background while > the installer is downloading Frost, Thaw, etc. ? That would give us a > little extra-time... We are already doing that iirc NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/bfd6a936/attachment.pgp From toad at amphibian.dyndns.org Thu Aug 9 15:42:15 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 9 Aug 2007 16:42:15 +0100 Subject: [freenet-dev] The welcome.html page In-Reply-To: <20070809114149.GH5467@freenetproject.org> References: <46B99FCF.7060007@batosai.net> <20070809114149.GH5467@freenetproject.org> Message-ID: <200708091642.22857.toad@amphibian.dyndns.org> On Thursday 09 August 2007 12:41, Florent Daigni?re wrote: > * batosai [2007-08-08 12:49:51]: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Hi, I noticed two issues with that page : > > > > - - It doesn't mention the opennet (It even says that the user can get > > darknet connections on #freenet-refs, what I see as being BAD). I don't > > know when you plan to release the opennet officialy but I think the user > > should already know that he should enable opennet if he has no friends > > running freenet. > > There is already a bug ticket filled in for that iirc > > > - - It is the only part of the entire installation process that doesn't > > support internationalization. Isn't it possible to replace this static > > page with a bunch of translatable strings, like the wizard and the > > entire interface ? > > Fill in a ticket for that one :)... It's a bit tricky to do it well; I > expect the user to be reading the page while the node is starting up : > If we want the node to serve that page then we have to spawn fproxy > earlier on Why can't we just ship it in various languages and have the installer choose based on the setup language? > > NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/e954b091/attachment.pgp From toad at amphibian.dyndns.org Thu Aug 9 15:43:42 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 9 Aug 2007 16:43:42 +0100 Subject: [freenet-dev] [freenet-cvs] r14551 - trunk/freenet/src/freenet/node In-Reply-To: <20070809110003.GF5467@freenetproject.org> References: <20070808235953.B48FC47A23C@freenetproject.org> <20070809110003.GF5467@freenetproject.org> Message-ID: <200708091643.43228.toad@amphibian.dyndns.org> On Thursday 09 August 2007 12:00, Florent Daigni?re wrote: > * juiceman at freenetproject.org [2007-08-08 23:59:53]: > > Author: juiceman > > Date: 2007-08-08 23:59:53 +0000 (Wed, 08 Aug 2007) > > New Revision: 14551 > > > > Modified: > > trunk/freenet/src/freenet/node/NodeClientCore.java > > Log: > > Surely is time to set the default to false. Users have had months of > > notice. I will let someone else drop support for pre-1010 keys in the > > future. > > We probably want to go futher than that ... with your patch only new > setups will default to false. What about existing nodes ? No. It's not forceWrite enabled, so old nodes will set it to the new default (false). > > NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/3e86c1a3/attachment.pgp From batosai at batosai.net Thu Aug 9 15:48:44 2007 From: batosai at batosai.net (batosai) Date: Thu, 09 Aug 2007 17:48:44 +0200 Subject: [freenet-dev] The welcome.html page In-Reply-To: <200708091642.22857.toad@amphibian.dyndns.org> References: <46B99FCF.7060007@batosai.net> <20070809114149.GH5467@freenetproject.org> <200708091642.22857.toad@amphibian.dyndns.org> Message-ID: <46BB375C.5090606@batosai.net> Matthew Toseland a ?crit : > On Thursday 09 August 2007 12:41, Florent Daigni?re wrote: >> * batosai [2007-08-08 12:49:51]: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Hi, I noticed two issues with that page : >>> >>> - - It doesn't mention the opennet (It even says that the user can get >>> darknet connections on #freenet-refs, what I see as being BAD). I don't >>> know when you plan to release the opennet officialy but I think the user >>> should already know that he should enable opennet if he has no friends >>> running freenet. >> There is already a bug ticket filled in for that iirc >> >>> - - It is the only part of the entire installation process that doesn't >>> support internationalization. Isn't it possible to replace this static >>> page with a bunch of translatable strings, like the wizard and the >>> entire interface ? >> Fill in a ticket for that one :)... It's a bit tricky to do it well; I >> expect the user to be reading the page while the node is starting up : >> If we want the node to serve that page then we have to spawn fproxy >> earlier on > > Why can't we just ship it in various languages and have the installer choose > based on the setup language? That's what I suggested in the bugreport if it wasn't possible to do it in fProxy. From nextgens at freenetproject.org Thu Aug 9 17:57:23 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 9 Aug 2007 19:57:23 +0200 Subject: [freenet-dev] The welcome.html page In-Reply-To: <200708091642.22857.toad@amphibian.dyndns.org> References: <46B99FCF.7060007@batosai.net> <20070809114149.GH5467@freenetproject.org> <200708091642.22857.toad@amphibian.dyndns.org> Message-ID: <20070809175723.GL5467@freenetproject.org> * Matthew Toseland [2007-08-09 16:42:15]: > On Thursday 09 August 2007 12:41, Florent Daigni?re wrote: > > * batosai [2007-08-08 12:49:51]: > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > Hi, I noticed two issues with that page : > > > > > > - - It doesn't mention the opennet (It even says that the user can get > > > darknet connections on #freenet-refs, what I see as being BAD). I don't > > > know when you plan to release the opennet officialy but I think the user > > > should already know that he should enable opennet if he has no friends > > > running freenet. > > > > There is already a bug ticket filled in for that iirc > > > > > - - It is the only part of the entire installation process that doesn't > > > support internationalization. Isn't it possible to replace this static > > > page with a bunch of translatable strings, like the wizard and the > > > entire interface ? > > > > Fill in a ticket for that one :)... It's a bit tricky to do it well; I > > expect the user to be reading the page while the node is starting up : > > If we want the node to serve that page then we have to spawn fproxy > > earlier on > > Why can't we just ship it in various languages and have the installer choose > based on the setup language? It wouldn't be simple to maintain nor update. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/de2ccea9/attachment.pgp From toad at amphibian.dyndns.org Thu Aug 9 18:11:06 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 9 Aug 2007 19:11:06 +0100 Subject: [freenet-dev] Freenet 0.7 build 1051 Message-ID: <200708091911.18644.toad@amphibian.dyndns.org> Freenet 0.7 build 1051 is now available, please upgrade. It will be mandatory at midnight GMT next Tuesday. Major changes: - Your node will now get opennet peers from its darknet peers (config option node.passOpennetPeersThroughDarknet; it should be on by default but if you used an early trunk build, you might have it set to false). - node.allowInsecureSSKs / node.allowInsecureCHKs is now set to false by default. This means your node will not let you see freesites, files etc inserted before build 1010's major crypto bugfixes. You can re-enable this temporarily by setting the config options, but we will remove support for insecure keys completely soon, so please reinsert any important content. - lots of bugfixes to opennet, containers, losing the node file, load limiting stats (very low bandwidth usage etc), minor FCP fixes (include Global on more messages), i18n fixes (always send the correct charset in fproxy), and lots more. Please upgrade! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070809/f4e4edbb/attachment.pgp From luke771 at gmail.com Fri Aug 10 05:47:17 2007 From: luke771 at gmail.com (luke771) Date: Fri, 10 Aug 2007 07:47:17 +0200 Subject: [freenet-dev] translation ITA update Message-ID: <46BBFBE5.7060101@gmail.com> updated to 1052 r14570 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.it.override.properties_21-ph00 Url: http://emu.freenetproject.org/pipermail/devl/attachments/20070810/91736eab/attachment.txt From NEOatNHNG at users.sourceforge.net Fri Aug 10 11:07:16 2007 From: NEOatNHNG at users.sourceforge.net (=?ISO-8859-15?Q?Michael_T=E4nzer?=) Date: Fri, 10 Aug 2007 13:07:16 +0200 Subject: [freenet-dev] German Translation (freenet.l10n.de.v1052) Message-ID: <46BC46E4.2040602@users.sourceforge.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Here's the German translation update. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFGvEbkPUBAMhFf+J4RAjpBAJi19pc33p6luhTvqWyb75gCnhhQAKCp9Y+z i+ZQTAEr/9zZQG9U7sob/Q== =gHOT -----END PGP SIGNATURE----- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: freenet.l10n.de.override.properties Url: http://emu.freenetproject.org/pipermail/devl/attachments/20070810/d812998d/attachment.txt From toad at amphibian.dyndns.org Fri Aug 10 12:16:00 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 10 Aug 2007 13:16:00 +0100 Subject: [freenet-dev] [freenet-support] Freenet 0.7 build 1051 In-Reply-To: <200708091911.18644.toad@amphibian.dyndns.org> References: <200708091911.18644.toad@amphibian.dyndns.org> Message-ID: <200708101316.08076.toad@amphibian.dyndns.org> Freenet 0.7 build 1052 is now available. This build features another fix to the load stats code (which ultimately controls how many requests the node accepts and therefore how much bandwidth it uses etc), and a fix the the first time wizard (we were setting the output bandwidth limit to 15 bytes per second by default!). On Thursday 09 August 2007 19:11, Matthew Toseland wrote: > Freenet 0.7 build 1051 is now available, please upgrade. It will be > mandatory at midnight GMT next Tuesday. Major changes: > - Your node will now get opennet peers from its darknet peers (config > option node.passOpennetPeersThroughDarknet; it should be on by default but > if you used an early trunk build, you might have it set to false). > - node.allowInsecureSSKs / node.allowInsecureCHKs is now set to false by > default. This means your node will not let you see freesites, files etc > inserted before build 1010's major crypto bugfixes. You can re-enable this > temporarily by setting the config options, but we will remove support for > insecure keys completely soon, so please reinsert any important content. > - lots of bugfixes to opennet, containers, losing the node file, load > limiting stats (very low bandwidth usage etc), minor FCP fixes (include > Global on more messages), i18n fixes (always send the correct charset in > fproxy), and lots more. > > Please upgrade! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070810/5a96b406/attachment.pgp From toad at amphibian.dyndns.org Fri Aug 10 18:47:37 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 10 Aug 2007 19:47:37 +0100 Subject: [freenet-dev] How to fix Frost Message-ID: <200708101947.52397.toad@amphibian.dyndns.org> PROBLEM: We have a problem with Frost. The spammer automatically posts ALICE-generated replies to every message posted on a variety of boards on the list shipped with Frost. This is a minor annoyance for oldies - all they do is tell Frost to ignore CHECK messages (messages from posters who haven't been marked yet), and enable threaded view so they see replies from their friends to newbies. The problem is, newbies don't have a database of trustworthy people. If they post a question, they will get a nonsensical reply from a bot, probably before any real person responds. And half of the messages on Frost are such nonsense spam. Now, because this doesn't DoS Frost for oldies, very little is being done about it, but it does seriously harm Freenet by making it unlikely that newbies will use Frost, and therefore significantly increasing their chance of leaving... Apart from this, there are many other attacks possible on the KSK-queues that Frost uses, the simplest being the Message of Death: simply post continually to the KSK-queue, it doesn't matter what you post, it'll make it take a loooong time to either find new messages or post a message. If the spammer was to change tactic he could probably render Frost unusable even to oldies, at least on a few boards, with very little effort, and he could certainly make it impossible to identify newbie posts among all the spam. PROPOSED SOLUTION: 0. ULPRs. (Node) Ultra-Lightweight Persistent Requests are the basis of all that follows. Essentially this is a means to limit the load caused by polling clients such as Frost, to get messages to the clients faster, and to make messages which have been lost by being in the wrong place findable if they are popular. Issues: If this is deployed without the below, it will only make spam easier, because the messages will be propagated even faster. 1. True Web of Trust. (Frost) Frost must publish the list of users marked manually by users. So if you trust a particular user, you automatically have (a slightly reduced) trust in the users that he trusts. If you then mark somebody as not trustworthy, Frost will ask you if you want to reevaluate the people who trust him, and indicate how many/which people you have marked as bad are trusted by those posters. Benefits: For oldies, faster propagation of trust in newbies etc. For newbies, if we ship an initial list of likely to be trustworthy posters, much faster assimilation; they can have a fairly usable Frost, minus spam. But we still need some experienced people to watch the boards for posts from newbies. 2. Outbox Polling. (Frost) Frost, or some similar app, can poll outboxes of specific users rather than watching a global KSK queue. These might be global for that user (encrypted for specific boards), which might work well with ULPRs, or they might be per user per board. Prerequisites: ULPRs! This will generate a lot of traffic otherwise, but with ULPRs it should be feasible. Benefits: For oldies, Frost will work efficiently and is completely immune to Message of Death attacks and similar KSK queue DoS'es. Note that we still need a means for newbies to introduce themselves, initially this would be a KSK queue. Issues: Currently threaded view with CHECK disabled works relatively well. It doesn't require explicit trust settings. Maybe we should have automatic marginal trust when you reply to an untrusted post, unless you tell Frost not to? 3. Thinkcash/Hashcash introductions. (Frost) Each poster can publish hashcash/thinkcash puzzles. The puzzle when solved yields a key, which enables a newbie to send a message to the poster. This message will be seen regardless of trust settings, and the poster will be given an initial marginal trust (OBSERVE??). After that, if nobody marks the newbie poster as bad, he can post freely, and his messages will be seen by the people who trust the original poster. IMPLEMENTATION I may implement ULPRs after opennet is ready. The rest will have to be implemented by Frost devs etc. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070810/77879147/attachment.pgp From jflesch at gmail.com Fri Aug 10 21:58:29 2007 From: jflesch at gmail.com (Jerome Flesch) Date: Fri, 10 Aug 2007 23:58:29 +0200 Subject: [freenet-dev] How to fix Frost In-Reply-To: <200708101947.52397.toad@amphibian.dyndns.org> References: <200708101947.52397.toad@amphibian.dyndns.org> Message-ID: <200708102358.32029.jflesch@gmail.com> > PROPOSED SOLUTION: > 0. ULPRs. (Node) > Ultra-Lightweight Persistent Requests are the basis of all that follows. > Essentially this is a means to limit the load caused by polling clients > such as Frost, to get messages to the clients faster, and to make messages > which have been lost by being in the wrong place findable if they are > popular. > > Issues: If this is deployed without the below, it will only make spam > easier, because the messages will be propagated even faster. > > 1. True Web of Trust. (Frost) > Frost must publish the list of users marked manually by users. So if you > trust a particular user, you automatically have (a slightly reduced) trust > in the users that he trusts. If you then mark somebody as not trustworthy, > Frost will ask you if you want to reevaluate the people who trust him, and > indicate how many/which people you have marked as bad are trusted by those > posters. > > Benefits: For oldies, faster propagation of trust in newbies etc. For > newbies, if we ship an initial list of likely to be trustworthy posters, > much faster assimilation; they can have a fairly usable Frost, minus spam. > But we still need some experienced people to watch the boards for posts > from newbies. > With a WoT, if you accept to read the new-comers, you accept to read the spam. So in the end, you risk to end with no-one reading anymore the posts from the newbies. > 2. Outbox Polling. (Frost) > Frost, or some similar app, can poll outboxes of specific users rather than > watching a global KSK queue. These might be global for that user (encrypted > for specific boards), which might work well with ULPRs, or they might be > per user per board. > > Prerequisites: ULPRs! This will generate a lot of traffic otherwise, but > with ULPRs it should be feasible. > Hm, stupid question : What are ULPRs ? > Benefits: For oldies, Frost will work efficiently and is completely immune > to Message of Death attacks and similar KSK queue DoS'es. Note that we > still need a means for newbies to introduce themselves, initially this > would be a KSK queue. > Issues: Currently threaded view with CHECK disabled works relatively well. > It doesn't require explicit trust settings. Maybe we should have automatic > marginal trust when you reply to an untrusted post, unless you tell Frost > not to? > > 3. Thinkcash/Hashcash introductions. (Frost) > Each poster can publish hashcash/thinkcash puzzles. The puzzle when solved > yields a key, which enables a newbie to send a message to the poster. This > message will be seen regardless of trust settings, and the poster will be > given an initial marginal trust (OBSERVE??). After that, if nobody marks > the newbie poster as bad, he can post freely, and his messages will be seen > by the people who trust the original poster. > Hm, and the hashcash should be sized to take how many time ? Because if it takes less than 10 minutes, spamming is possible (some messages each 10 minutes can be really annonying), and if it takes more than 10 minutes, I'm note sure that most of the users will wait for the end of the process. > IMPLEMENTATION > > I may implement ULPRs after opennet is ready. The rest will have to be > implemented by Frost devs etc. Hm, can I suggest a fourth idea ? 4) Blacklists : - Let the user put the messages they don't like / the spams / troll's identities in a blacklist. - Let them upload on Freenet their blacklist. - Let the other people filter the messages with this blacklist (and possibly some others to make the filtering as good as possible) Of cource their software will have to refresh the list(s) as often as possible. I think that 'SubscribeUSK'[1] in FCP can help. Benefits : Less annoying for the users, easier to explain to the users, and compatible with the current Frost boards. Issues : Between the spam sending and the blacklist update, there may be a while (but as I said, SubscribeUSK[1] and many blacklists can probably help to reduce this delay). Another possibility would be a whitelist mechanism: Some people decide for the others what they can see (Like IRL in sum ;). But it would be as annoying as the solutions 1 to 3 for the users (probably even more). -- [1]: http://wiki.freenetproject.org/FCP2p0SubscribeUSK -- Jerome Flesch -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070810/8fb80709/attachment.pgp From jflesch at nerim.net Fri Aug 10 22:01:41 2007 From: jflesch at nerim.net (Jerome Flesch) Date: Sat, 11 Aug 2007 00:01:41 +0200 Subject: [freenet-dev] How to fix Frost In-Reply-To: <200708101947.52397.toad@amphibian.dyndns.org> References: <200708101947.52397.toad@amphibian.dyndns.org> Message-ID: <200708110001.44976.jflesch@nerim.net> > PROPOSED SOLUTION: > 0. ULPRs. (Node) > Ultra-Lightweight Persistent Requests are the basis of all that follows. > Essentially this is a means to limit the load caused by polling clients > such as Frost, to get messages to the clients faster, and to make messages > which have been lost by being in the wrong place findable if they are > popular. > > Issues: If this is deployed without the below, it will only make spam > easier, because the messages will be propagated even faster. > > 1. True Web of Trust. (Frost) > Frost must publish the list of users marked manually by users. So if you > trust a particular user, you automatically have (a slightly reduced) trust > in the users that he trusts. If you then mark somebody as not trustworthy, > Frost will ask you if you want to reevaluate the people who trust him, and > indicate how many/which people you have marked as bad are trusted by those > posters. > > Benefits: For oldies, faster propagation of trust in newbies etc. For > newbies, if we ship an initial list of likely to be trustworthy posters, > much faster assimilation; they can have a fairly usable Frost, minus spam. > But we still need some experienced people to watch the boards for posts > from newbies. > With a WoT, if you accept to read the new-comers, you accept to read the spam. So in the end, you risk to end with no-one reading anymore the posts from the newbies. > 2. Outbox Polling. (Frost) > Frost, or some similar app, can poll outboxes of specific users rather than > watching a global KSK queue. These might be global for that user (encrypted > for specific boards), which might work well with ULPRs, or they might be > per user per board. > > Prerequisites: ULPRs! This will generate a lot of traffic otherwise, but > with ULPRs it should be feasible. > Hm, stupid question : What are ULPRs ? > Benefits: For oldies, Frost will work efficiently and is completely immune > to Message of Death attacks and similar KSK queue DoS'es. Note that we > still need a means for newbies to introduce themselves, initially this > would be a KSK queue. > Issues: Currently threaded view with CHECK disabled works relatively well. > It doesn't require explicit trust settings. Maybe we should have automatic > marginal trust when you reply to an untrusted post, unless you tell Frost > not to? > > 3. Thinkcash/Hashcash introductions. (Frost) > Each poster can publish hashcash/thinkcash puzzles. The puzzle when solved > yields a key, which enables a newbie to send a message to the poster. This > message will be seen regardless of trust settings, and the poster will be > given an initial marginal trust (OBSERVE??). After that, if nobody marks > the newbie poster as bad, he can post freely, and his messages will be seen > by the people who trust the original poster. > Hm, and the hashcash should be sized to take how many time ? Because if it takes less than 10 minutes, spamming is possible (some messages each 10 minutes can be really annonying), and if it takes more than 10 minutes, I'm note sure that most of the users will wait for the end of the process. > IMPLEMENTATION > > I may implement ULPRs after opennet is ready. The rest will have to be > implemented by Frost devs etc. Hm, can I suggest a fourth idea ? 4) Blacklists : - Let the user put the messages they don't like / the spams / troll's identities in a blacklist. - Let them upload on Freenet their blacklist. - Let the other people filter the messages with this blacklist (and possibly some others to make the filtering as good as possible) Of cource their software will have to refresh the list(s) as often as possible. I think that 'SubscribeUSK'[1] in FCP can help. Benefits : Less annoying for the users, easier to explain to the users, and compatible with the current Frost boards. Issues : Between the spam sending and the blacklist update, there may be a while (but as I said, SubscribeUSK[1] and many blacklists can probably help to reduce this delay). Another possibility would be a whitelist mechanism: Some people decide for the others what they can see (Like IRL in sum ;). But it would be as annoying as the solutions 1 to 3 for the users (probably even more). -- [1]: http://wiki.freenetproject.org/FCP2p0SubscribeUSK -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/8819dd3a/attachment.pgp From thelema314 at gmail.com Fri Aug 10 20:23:01 2007 From: thelema314 at gmail.com (Edgar Friendly) Date: Fri, 10 Aug 2007 15:23:01 -0500 Subject: [freenet-dev] How to fix Frost In-Reply-To: <200708110001.44976.jflesch@nerim.net> References: <200708101947.52397.toad@amphibian.dyndns.org> <200708110001.44976.jflesch@nerim.net> Message-ID: <46BCC925.4070001@gmail.com> Jerome Flesch wrote: > With a WoT, if you accept to read the new-comers, you accept to read the spam. > So in the end, you risk to end with no-one reading anymore the posts from the > newbies. > Might it work to distribute the load of looking at unsigned posts to all readers? Instead of a few trusted people looking through mounds of spam to find a few newbie posts, maybe each user could get a dozen posts to filter for the community. I don't know if a random distribution or some sort of slicing method would work better, they each have their ups and downs. > Hm, can I suggest a fourth idea ? > > 4) Blacklists : > Won't ever work. Trying to blacklist spam will not succeed. Look through the mailing list archives for the huge debates related to negative trust for more info. > Another possibility would be a whitelist mechanism: Some people decide for the > others what they can see (Like IRL in sum ;). But it would be as annoying as > the solutions 1 to 3 for the users (probably even more). > Instead of whitelisting posts, one might as well whitelist people, and then we arrive back at the web of trust solution. E. From NEOatNHNG at users.sourceforge.net Fri Aug 10 21:32:14 2007 From: NEOatNHNG at users.sourceforge.net (=?ISO-8859-15?Q?Michael_T=E4nzer?=) Date: Fri, 10 Aug 2007 23:32:14 +0200 Subject: [freenet-dev] How to fix Frost Message-ID: <46BCD95E.50404@users.sourceforge.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 As I'm just a translator, I'm not really familiar with how exactly Freenet works, but I had some ideas on this topic so here they are: I think a kind of public rating system could help, which stores the username of the poster, and how many people classified it as SPAM/annoying(users that are not spammers but behave (sometimes) in a annoying way)/good As something like this rating system does not need anonymity itself, just the users of it, one could think of a system that is faster than normal freesites. The rating system could have a fixed address which is known by any node but the connection to it is routed through other nodes (a little bit like tor), this way there is just one place to look for the rating file, but even if the rating system is compromised, the attacker can't tell where the request originated from. On the rating server there could be a database which holds the ratings and creates a file which can be read by Frost. Pros: - -no complicated routing needed, because there's one server who hosts the system - -no site inserts needed therefore less load on the network - -the current version of the file is always available (given the server works well) - -just the first (about) 10 people need to read the (and mark it as) SPAM and newbies will be read (unless they're marked as SPAM too) Cons: - -single point of failure (could be diminished if every node holds a copy of the file in the last version it downloaded it (which it needs to store anyway) so if the server is down the node sends a request to it's peers and gets the last version available), if the server is down "on a long-time basis" we are where we are now, but we haven't lost anything - -could be abused by spammers (marking useful posts as SPAM so the system doesn't really work and is turned of by the users) maybe the web of trust idea could work in here, so if people who are trusted by many others (or maybe many trustworthy others (starting point: the developers)) can change the rating more significantly than others. Maybe even based on the own system: only people who are rated themselves with good a couple of times may mark something as anything (or are stored in a different field so the rating differentiates between trusted and untrusted counts) Maybe the special routing system could be used for similar services too (once implemented) although I can't think of examples right now. Looking forward to hear your thoughts about it Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGvNlePUBAMhFf+J4RApryAKCt+MtcAvz7P5xh1nFHZlJ5/z8bjQCaA8aV HYcK3Up97QzQKjKf23Lkvvc= =j0dy -----END PGP SIGNATURE----- -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x115FF89E.asc Type: application/pgp-keys Size: 2437 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070810/6409f294/attachment.key From toad at amphibian.dyndns.org Sat Aug 11 18:17:40 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 11 Aug 2007 19:17:40 +0100 Subject: [freenet-dev] Freenet 0.7 build 1053 Message-ID: <200708111917.45336.toad@amphibian.dyndns.org> Freenet 0.7 build 1053 is now available. Please upgrade! This build will be mandatory soon. Changes include fixes to temporary file leaks, thread pooling (which should solve the high system load bug), containers fixes, client layer fixes and security fixes. Please upgrade! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/cae90c4f/attachment.pgp From toad at amphibian.dyndns.org Sat Aug 11 18:27:43 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 11 Aug 2007 19:27:43 +0100 Subject: [freenet-dev] How to fix Frost In-Reply-To: <46BCD95E.50404@users.sourceforge.net> References: <46BCD95E.50404@users.sourceforge.net> Message-ID: <200708111927.43967.toad@amphibian.dyndns.org> Centralised is bad, spammable is bad. But in any case we don't have the technology to route requests to an anonymous central server over the network; that's one-to-one streams, maybe in 0.8. On Friday 10 August 2007 22:32, Michael T?nzer wrote: > As I'm just a translator, I'm not really familiar with how exactly > Freenet works, but I had some ideas on this topic so here they are: > > I think a kind of public rating system could help, which stores the > username of the poster, and how many people classified it as > SPAM/annoying(users that are not spammers but behave (sometimes) in a > annoying way)/good > > As something like this rating system does not need anonymity itself, > just the users of it, one could think of a system that is faster than > normal freesites. > > The rating system could have a fixed address which is known by any node > but the connection to it is routed through other nodes (a little bit > like tor), this way there is just one place to look for the rating file, > but even if the rating system is compromised, the attacker can't tell > where the request originated from. On the rating server there could be a > database which holds the ratings and creates a file which can be read by > Frost. > > Pros: > -no complicated routing needed, because there's one server who hosts the > system > -no site inserts needed therefore less load on the network > -the current version of the file is always available (given the server > works well) > -just the first (about) 10 people need to read the (and mark it as) SPAM > and newbies will be read (unless they're marked as SPAM too) > > Cons: > -single point of failure (could be diminished if every node holds a copy > of the file in the last version it downloaded it (which it needs to > store anyway) so if the server is down the node sends a request to it's > peers and gets the last version available), if the server is down "on a > long-time basis" we are where we are now, but we haven't lost anything > -could be abused by spammers (marking useful posts as SPAM so the system > doesn't really work and is turned of by the users) maybe the web of > trust idea could work in here, so if people who are trusted by many > others (or maybe many trustworthy others (starting point: the > developers)) can change the rating more significantly than others. Maybe > even based on the own system: only people who are rated themselves with > good a couple of times may mark something as anything (or are stored in > a different field so the rating differentiates between trusted and > untrusted counts) > > > Maybe the special routing system could be used for similar services too > (once implemented) although I can't think of examples right now. > > Looking forward to hear your thoughts about it > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/d9efe1ea/attachment.pgp From toad at amphibian.dyndns.org Sat Aug 11 18:32:25 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 11 Aug 2007 19:32:25 +0100 Subject: [freenet-dev] How to fix Frost In-Reply-To: <200708102358.32029.jflesch@gmail.com> References: <200708101947.52397.toad@amphibian.dyndns.org> <200708102358.32029.jflesch@gmail.com> Message-ID: <200708111932.26078.toad@amphibian.dyndns.org> On Friday 10 August 2007 22:58, Jerome Flesch wrote: > > PROPOSED SOLUTION: > > 0. ULPRs. (Node) > > Ultra-Lightweight Persistent Requests are the basis of all that follows. > > Essentially this is a means to limit the load caused by polling clients > > such as Frost, to get messages to the clients faster, and to make > > messages which have been lost by being in the wrong place findable if > > they are popular. > > > > Issues: If this is deployed without the below, it will only make spam > > easier, because the messages will be propagated even faster. > > > > 1. True Web of Trust. (Frost) > > Frost must publish the list of users marked manually by users. So if you > > trust a particular user, you automatically have (a slightly reduced) > > trust in the users that he trusts. If you then mark somebody as not > > trustworthy, Frost will ask you if you want to reevaluate the people who > > trust him, and indicate how many/which people you have marked as bad are > > trusted by those posters. > > > > Benefits: For oldies, faster propagation of trust in newbies etc. For > > newbies, if we ship an initial list of likely to be trustworthy posters, > > much faster assimilation; they can have a fairly usable Frost, minus > > spam. But we still need some experienced people to watch the boards for > > posts from newbies. > > With a WoT, if you accept to read the new-comers, you accept to read the > spam. So in the end, you risk to end with no-one reading anymore the posts > from the newbies. Which is exactly what we have now. But newbies would propagate faster, once they have been accepted by somebody. > > > 2. Outbox Polling. (Frost) > > Frost, or some similar app, can poll outboxes of specific users rather > > than watching a global KSK queue. These might be global for that user > > (encrypted for specific boards), which might work well with ULPRs, or > > they might be per user per board. > > > > Prerequisites: ULPRs! This will generate a lot of traffic otherwise, but > > with ULPRs it should be feasible. > > Hm, stupid question : What are ULPRs ? Ultra-Lightweight Passive/Persistent Requests. See the wiki, basically nodes remember that you wanted a key for a while and propagate it back to the original requestors (and throttle requests for it which won't be satisfied anyway). > > > Benefits: For oldies, Frost will work efficiently and is completely > > immune to Message of Death attacks and similar KSK queue DoS'es. Note > > that we still need a means for newbies to introduce themselves, initially > > this would be a KSK queue. > > Issues: Currently threaded view with CHECK disabled works relatively > > well. It doesn't require explicit trust settings. Maybe we should have > > automatic marginal trust when you reply to an untrusted post, unless you > > tell Frost not to? > > > > > > > > 3. Thinkcash/Hashcash introductions. (Frost) > > Each poster can publish hashcash/thinkcash puzzles. The puzzle when > > solved yields a key, which enables a newbie to send a message to the > > poster. This message will be seen regardless of trust settings, and the > > poster will be given an initial marginal trust (OBSERVE??). After that, > > if nobody marks the newbie poster as bad, he can post freely, and his > > messages will be seen by the people who trust the original poster. > > Hm, and the hashcash should be sized to take how many time ? Because if it > takes less than 10 minutes, spamming is possible (some messages each 10 > minutes can be really annonying), and if it takes more than 10 minutes, I'm > note sure that most of the users will wait for the end of the process. They only need to do it once. And if it's only one spam every 10 minutes, that's a massive improvement on what's possible now. > > > IMPLEMENTATION > > > > I may implement ULPRs after opennet is ready. The rest will have to be > > implemented by Frost devs etc. > > Hm, can I suggest a fourth idea ? > > 4) Blacklists : > > - Let the user put the messages they don't like / the spams / troll's > identities in a blacklist. > - Let them upload on Freenet their blacklist. > - Let the other people filter the messages with this blacklist (and > possibly some others to make the filtering as good as possible) The spammer is now faking random identities, reusing them for a small number of posts (not because they're expensive but because they don't want to be blocked by a heuristic "this identity only made one post"). Unless identities are costly (and I have shown above one way for them to be), blacklists are of no use: There is no such thing as negative trust in cyberspace. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/dfc76a51/attachment.pgp From toad at amphibian.dyndns.org Sat Aug 11 18:33:59 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 11 Aug 2007 19:33:59 +0100 Subject: [freenet-dev] How to fix Frost In-Reply-To: <46BCC925.4070001@gmail.com> References: <200708101947.52397.toad@amphibian.dyndns.org> <200708110001.44976.jflesch@nerim.net> <46BCC925.4070001@gmail.com> Message-ID: <200708111934.03244.toad@amphibian.dyndns.org> On Friday 10 August 2007 21:23, Edgar Friendly wrote: > Jerome Flesch wrote: > > With a WoT, if you accept to read the new-comers, you accept to read the > > spam. So in the end, you risk to end with no-one reading anymore the > > posts from the newbies. > > Might it work to distribute the load of looking at unsigned posts to all > readers? Instead of a few trusted people looking through mounds of spam > to find a few newbie posts, maybe each user could get a dozen posts to > filter for the community. I don't know if a random distribution or some > sort of slicing method would work better, they each have their ups and > downs. Hashcash/thinkcash puzzles set by each user approximate this. The difficulty could automatically escalate over multiple posts, and de-escalate over time. > > Instead of whitelisting posts, one might as well whitelist people, and > then we arrive back at the web of trust solution. > > E. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/4dff24f8/attachment.pgp From toad at amphibian.dyndns.org Sat Aug 11 19:21:35 2007 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 11 Aug 2007 20:21:35 +0100 Subject: [freenet-dev] Freenet 0.7 build 1054 In-Reply-To: <200708111917.45336.toad@amphibian.dyndns.org> References: <200708111917.45336.toad@amphibian.dyndns.org> Message-ID: <200708112021.40964.toad@amphibian.dyndns.org> Freenet 0.7 build 1054 is now available. This fixes some bugs in 1053 relating to thread pooling, in particular it won't reject requests due to shortage of threads unless the number of threads actually doing useful work is over the limit. On Saturday 11 August 2007 19:17, Matthew Toseland wrote: > Freenet 0.7 build 1053 is now available. Please upgrade! This build will be > mandatory soon. Changes include fixes to temporary file leaks, thread > pooling (which should solve the high system load bug), containers fixes, > client layer fixes and security fixes. Please upgrade! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20070811/05c1688a/attachment.pgp From thelema314 at gmail.com Sat Aug 11 22:05:26 2007 From: thelema314 at gmail.com (Edgar Friendly) Date: Sat, 11 Aug 2007 17:05:26 -0500 Subject: [freenet-dev] How to fix Frost In-Reply-To: <200708111934.03244.toad@amphibian.dyndns.org> References: <200708101947.52397.toad@amphibian.dyndns.org> <200708110001.44976.jflesch@nerim.net> <46BCC925.4070001@gmail.com> <200708111934.03244.toad@amphibian.dyndns.org> Message-ID: <46BE32A6.5070308@gmail.com> Matthew Toseland wrote: > On Friday 10 August 2007 21:23, Edgar Friendly wrote: >> Jerome Flesch wrote: >>> With a WoT, if you accept to read the new-comers, you accept to read the >>> spam. So in the end, you risk to end with no-one reading anymore the >>> posts from the newbies. >> Might it work to distribute the load of looking at unsigned posts to all >> readers? Instead of a few trusted people looking through mounds of spam >> to find a few newbie posts, maybe each user could get a dozen posts to >> filter for the community. I don't know if a random distribution or some >> sort of slicing method would work better, they each have their ups and >> downs. > > Hashcash/thinkcash puzzles set by each user approximate this. The difficulty > could automatically escalate over multiple posts, and de-escalate over time. Do you suggest that each established user create *cash puzzles, and new posters choose one or many of these puzzles to solve in order to get through untrusted/unknown-user-blocking so they can start to earn trust? I don't get your point about difficulty of puzzles, they'd go by the wayside once a person entered the web of trust, no? E. From batosai at batosai.net Sun Aug 12 22:44:55 2007 From: batosai at batosai.net (batosai) Date: Mon, 13 Aug 2007 00:44:55 +0200 Subject: [freenet-dev] Negative margins forbidden by Fproxy ? Message-ID: <46BF8D67.5000402@batosai.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I would like to know why fProxy blocks the CSS negative margins. Is it for security reasons ? I'm working on a new index and without them, my layout is... flat. Regards -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGv41no6N05NzaOvURAqYLAJ9BP7X0UfN52CuYGM7GG+6HXD3ZewCdFzt4 CRRDvfCtnuACClsVIgKL4hg= =sXsR -----END PGP SIGNATURE----- From thelema314 at gmail.com Mon Aug 13 02:55:58 2007 From: thelema314 at gmail.com (Edgar Friendly) Date: Sun, 12 Aug 2007 21:55:58 -0500 Subject: [freenet-dev] Negative margins forbidden by Fproxy ? In-Reply-To: <46BF8D67.5000402@batosai.net> References: <46BF8D67.5000402@batosai.net> Message-ID: <46BFC83E.6050300@gmail.com> batosai wrote: > Hi, > > I would like to know why fProxy blocks the CSS negative margins. Is it > for security reasons ? > > I'm working on a new index and without them, my layout is... flat. > > Regards It's probably a weakness in the parser - the node parses CSS and HTML completely, stripping out any potentially harmful content. I'm guessing that the potential for a "-" was missed, and can be added pretty easily, now that the problem is found. Do you have an example document that shouldn't be blocked? E. _______________________________________________ Devl mailing list Devl at freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl From batosai at batosai.net Mon Aug 13 08:35:49 2007 From: batosai at batosai.net (batosai) Date: Mon, 13 Aug 2007 10:35:49 +0200 Subject: [freenet-dev] Negative margins forbidden by Fproxy ? In-Reply-To: <46BFC83E.6050300@gmail.com> References: <46BF8D67.5000402@batosai.net> <46BFC83E.6050300@gmail.com> Message-ID: <46C017E5.1080306@batosai.net> Edgar Friendly a ?crit : > It's probably a weakness in the parser - the node parses CSS and HTML > completely, stripping out any potentially harmful content. I'm guessing > that the potential for a "-" was missed, and can be added pretty easily, > now that the problem is found. Do you have an example document that > shouldn't be blocked? Here is an example document. Negative margins are used in : - #content : for centering the page regardless of the browser. - .left .right : for overlapping outside of the page. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/devl/attachments/20070813/b932b9c8/attachment.html From batosai at batosai.net Mon Aug 13 08:43:56 2007 From: batosai at batosai.net (batosai) Date: Mon, 13 Aug 2007 10:43:56 +0200 Subject: [freenet-dev] Negative margins forbidden by Fproxy ? In-Reply-To: <46C017E5.1080306@batosai.net> References: <46BF8D67.5000402@batosai.net> <46BFC83E.6050300@gmail.com> <46C017E5.1080306@batosai.net> Message-ID: <46C019CC.8060404@batosai.net> batosai a ?crit : > Edgar Friendly a ?crit : >> It's probably a weakness in the parser - the node parses CSS and HTML >> completely, stripping out any potentially harmful content. I'm guessing >> that the potential for a "-" was missed, and can be added pretty easily, >> now that the problem is found. Do you have an example document that >> shouldn't be blocked? > > Here is an example document. Negative margins are used in : > > - #content : for centering the page regardless of the browser. > - .left .right : for overlapping outside of the page. Ooops, didn't think about displaying in the mail client. You should display the page in a real web browser. From nextgens at freenetproject.org Mon Aug 13 15:22:49 2007 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Mon, 13 Aug 2007 17:22:49 +0200 Subject: [freenet-dev] Negative margins forbidden by Fproxy ? In-Reply-To: <46C017E5.1080306@batosai.net> References: <46BF8D67.5000402@batosai.net> <46BFC83E.6050300@gmail.com> <46C017E5.1080306@batosai.net> Message-ID: <20070813152248.GB5465@freenetproject.org> * batosai [2007-08-13 10:35:49]: > Edgar Friendly a ?crit : > > It's probably a weakness in the parser - the node parses CSS and HTML > > completely, stripping out any potentially harmful content. I'm guessing > > that the potential for a "-"