From aeb99 at doc.ic.ac.uk Thu Jul 10 02:21:09 2008 From: aeb99 at doc.ic.ac.uk (Alexander Barnell) Date: Tue, 11 Apr 100 13:22:19 +0100 (BST) Subject: [Freenet-dev] Crypto, Metadata, Multicasting, and Multilinks... In-Reply-To: from "Jake Mannix" at Apr 11, 0 04:49:43 am Message-ID: <200004111222.NAA17618@baza.doc.ic.ac.uk> A non-text attachment was scrubbed... Name: not available Type: text Size: 1075 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000411/257a98b4/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Tue, 2 May 100 17:17:08 +1000 (EST) Subject: [Freenet-dev] I had a mad idea In-Reply-To: from "degs" at May 2, 0 09:07:44 am Message-ID: <200005020717.RAA30082@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 3169 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000503/96e7c9e5/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Tue, 2 May 100 19:46:09 +1000 (EST) Subject: [Freenet-dev] I had a mad idea In-Reply-To: from "lists" at May 2, 0 11:36:28 am Message-ID: <200005020946.TAA30482@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 2970 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000503/713a197d/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 17:15:49 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <006f01bfcad7$d04dc400$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 10:11:23 am Message-ID: <200005310715.RAA05413@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 944 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/4e59ea15/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 17:35:09 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <20000531093813.A755@linuxpower.org> from "Adam Langley" at May 31, 0 09:38:13 am Message-ID: <200005310735.RAA05660@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1820 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/1f185967/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 18:16:12 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <008501bfcadf$57b10540$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 11:05:17 am Message-ID: <200005310816.SAA05720@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1028 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/9e125fad/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Thu, 1 Jun 100 10:33:34 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <00a701bfcae9$e8d001c0$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 12:20:56 pm Message-ID: <200006010033.KAA06333@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1721 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/9ba59828/attachment.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Thu, 1 Jun 100 17:17:36 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: from "Lucky Green" at May 31, 0 11:02:35 pm Message-ID: <200006010717.RAA00728@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 2002 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/50c66ce8/attachment.txt From aeb99 at doc.ic.ac.uk Thu Jul 10 02:21:09 2008 From: aeb99 at doc.ic.ac.uk (Alexander Barnell) Date: Wed, 2 Aug 100 04:01:24 +0100 (BST) Subject: [Freenet-dev] Subspaces In-Reply-To: from "Brandon" at Aug 1, 0 09:36:59 pm Message-ID: <200008020301.EAA25621@baza.doc.ic.ac.uk> A non-text attachment was scrubbed... Name: not available Type: text Size: 810 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000803/94c9256e/attachment.txt From pete at petertodd.ca Thu Jul 10 02:21:09 2008 From: pete at petertodd.ca (Peter Todd) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] "Freegle" online In-Reply-To: <012401c0bc5d$c2bcfe00$0200000a@pete>; from doc@drjava.de on Tue, Apr 03, 2001 at 06:47:22PM +0200 References: <00ac01c0bbc7$68f274e0$0200000a@pete> <20010402194742.B2913@petes.localdomain> <012401c0bc5d$c2bcfe00$0200000a@pete> Message-ID: <20010403133133.B375@petes.localdomain> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/78473da9/attachment.bin From pete at petertodd.ca Thu Jul 10 02:21:09 2008 From: pete at petertodd.ca (Peter Todd) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] Push hard or Suck hard? In-Reply-To: <005401c0c673$e7d07170$1be836d2@private>; from david@rebirthing.co.nz on Tue, Apr 17, 2001 at 12:51:06AM +1200 References: <005401c0c673$e7d07170$1be836d2@private> Message-ID: <20010416120516.B358@petes.localdomain> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/8cb1a6fc/attachment.bin From agl at linuxpower.org Thu Jul 10 02:21:09 2008 From: agl at linuxpower.org (Adam Langley) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] versions and codenames In-Reply-To: <01a701c0cabe$e34104e0$0353aad0@classified>; from timm@madtimes.com on Sat, Apr 21, 2001 at 04:57:56PM -0700 References: <987888078.1465.1.camel@ywwg> <01a701c0cabe$e34104e0$0353aad0@classified> Message-ID: <20010422112419.A1233@linuxpower.org> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/a1da3eb1/attachment.bin From tavin at mailandnews.com Thu Jul 10 02:21:09 2008 From: tavin at mailandnews.com (Tavin Cole) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] 0.3.9 refinements In-Reply-To: <20010426114516.H3700@freenetproject.org>; from ian@hawk.freenetproject.org on Thu, Apr 26, 2001 at 11:45:16AM -0700 References: <20010426105723.C3700@freenetproject.org> <874rvbsfx6.fsf@priss.bad-people-of-the-future.san-francisco.ca.us> <20010426114516.H3700@freenetproject.org> Message-ID: <20010426165718.F4565@niss> From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 17:15:49 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <006f01bfcad7$d04dc400$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 10:11:23 am Message-ID: <200005310715.RAA05413@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 944 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/4e59ea15/attachment-0001.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 17:35:09 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <20000531093813.A755@linuxpower.org> from "Adam Langley" at May 31, 0 09:38:13 am Message-ID: <200005310735.RAA05660@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1820 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/1f185967/attachment-0001.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Wed, 31 May 100 18:16:12 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <008501bfcadf$57b10540$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 11:05:17 am Message-ID: <200005310816.SAA05720@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1028 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/9e125fad/attachment-0001.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Thu, 1 Jun 100 10:33:34 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: <00a701bfcae9$e8d001c0$0101a8c0@totor> from "Michel Bouissou" at May 31, 0 12:20:56 pm Message-ID: <200006010033.KAA06333@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 1721 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/9ba59828/attachment-0001.txt From jon at central.warehouse.net Thu Jul 10 02:21:09 2008 From: jon at central.warehouse.net (Jon) Date: Thu, 1 Jun 100 17:17:36 +1000 (EST) Subject: [Freenet-dev] Freenet forgets closing connections In-Reply-To: from "Lucky Green" at May 31, 0 11:02:35 pm Message-ID: <200006010717.RAA00728@central.warehouse.net> A non-text attachment was scrubbed... Name: not available Type: text Size: 2002 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000605/50c66ce8/attachment-0001.txt From aeb99 at doc.ic.ac.uk Thu Jul 10 02:21:09 2008 From: aeb99 at doc.ic.ac.uk (Alexander Barnell) Date: Wed, 2 Aug 100 04:01:24 +0100 (BST) Subject: [Freenet-dev] Subspaces In-Reply-To: from "Brandon" at Aug 1, 0 09:36:59 pm Message-ID: <200008020301.EAA25621@baza.doc.ic.ac.uk> A non-text attachment was scrubbed... Name: not available Type: text Size: 810 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20000803/94c9256e/attachment-0001.txt From pete at petertodd.ca Thu Jul 10 02:21:09 2008 From: pete at petertodd.ca (Peter Todd) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] "Freegle" online In-Reply-To: <012401c0bc5d$c2bcfe00$0200000a@pete>; from doc@drjava.de on Tue, Apr 03, 2001 at 06:47:22PM +0200 References: <00ac01c0bbc7$68f274e0$0200000a@pete> <20010402194742.B2913@petes.localdomain> <012401c0bc5d$c2bcfe00$0200000a@pete> Message-ID: <20010403133133.B375@petes.localdomain> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/78473da9/attachment-0001.bin From pete at petertodd.ca Thu Jul 10 02:21:09 2008 From: pete at petertodd.ca (Peter Todd) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] Push hard or Suck hard? In-Reply-To: <005401c0c673$e7d07170$1be836d2@private>; from david@rebirthing.co.nz on Tue, Apr 17, 2001 at 12:51:06AM +1200 References: <005401c0c673$e7d07170$1be836d2@private> Message-ID: <20010416120516.B358@petes.localdomain> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/8cb1a6fc/attachment-0001.bin From agl at linuxpower.org Thu Jul 10 02:21:09 2008 From: agl at linuxpower.org (Adam Langley) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] versions and codenames In-Reply-To: <01a701c0cabe$e34104e0$0353aad0@classified>; from timm@madtimes.com on Sat, Apr 21, 2001 at 04:57:56PM -0700 References: <987888078.1465.1.camel@ywwg> <01a701c0cabe$e34104e0$0353aad0@classified> Message-ID: <20010422112419.A1233@linuxpower.org> A non-text attachment was scrubbed... Name: not available Type: multipart/signed Size: 0 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/a1da3eb1/attachment-0001.bin From tavin at mailandnews.com Thu Jul 10 02:21:09 2008 From: tavin at mailandnews.com (Tavin Cole) Date: Thu, 10 Jul 2008 02:21:09 -0000 Subject: [freenet-devl] 0.3.9 refinements In-Reply-To: <20010426114516.H3700@freenetproject.org>; from ian@hawk.freenetproject.org on Thu, Apr 26, 2001 at 11:45:16AM -0700 References: <20010426105723.C3700@freenetproject.org> <874rvbsfx6.fsf@priss.bad-people-of-the-future.san-francisco.ca.us> <20010426114516.H3700@freenetproject.org> Message-ID: <20010426165718.F4565@niss> From nextgens at freenetproject.org Tue Jul 1 06:41:25 2008 From: nextgens at freenetproject.org (NextGen$) Date: Tue, 1 Jul 2008 06:41:25 +0000 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <20080629170308.E00424791C6@freenetproject.org> References: <20080629170308.E00424791C6@freenetproject.org> Message-ID: <20080701064125.GC26659@freenetproject.org> * bombe at freenetproject.org [2008-06-29 17:03:08]: > Author: bombe > Date: 2008-06-29 17:03:08 +0000 (Sun, 29 Jun 2008) > New Revision: 20868 > > Modified: > trunk/freenet/src/freenet/client/async/SingleFileInserter.java > trunk/freenet/src/freenet/support/io/BucketChainBucket.java > trunk/freenet/src/freenet/support/io/BucketChainBucketFactory.java > trunk/freenet/src/freenet/support/io/BucketTools.java > Log: > fix backport of BucketChainBucket > As far as I can see you have restored the old behaviour bypassing the non-working code... but that's hardly a fix :) > Modified: trunk/freenet/src/freenet/client/async/SingleFileInserter.java > =================================================================== > --- trunk/freenet/src/freenet/client/async/SingleFileInserter.java 2008-06-29 16:23:27 UTC (rev 20867) > +++ trunk/freenet/src/freenet/client/async/SingleFileInserter.java 2008-06-29 17:03:08 UTC (rev 20868) > @@ -14,7 +14,6 @@ > import freenet.keys.BaseClientKey; > import freenet.keys.CHKBlock; > import freenet.keys.FreenetURI; > - import freenet.keys.NodeCHK; > import freenet.keys.SSKBlock; > import freenet.support.Logger; > import freenet.support.OOMHandler; > @@ -22,7 +21,6 @@ > import freenet.support.api.Bucket; > import freenet.support.compress.CompressionOutputSizeException; > import freenet.support.compress.Compressor; > -import freenet.support.io.BucketChainBucketFactory; > import freenet.support.io.BucketTools; > > /** > @@ -172,7 +170,7 @@ > ctx.eventProducer.produceEvent(new StartedCompressionEvent(i)); > Compressor comp = Compressor.getCompressionAlgorithmByDifficulty(i); > Bucket result; > - result = comp.compress(origData, new BucketChainBucketFactory(ctx.persistentBucketFactory, NodeCHK.BLOCK_SIZE), origData.size()); > + result = comp.compress(origData, ctx.persistentBucketFactory, origData.size()); > if(result.size() < oneBlockCompressedSize) { > bestCodec = comp; > if(bestCompressedData != null) > That's the gist of the patch, right ? If so, where is the BucketChainBucketFactory class used in the code ? I have no problem with you disabling non-working code or even reverting my backports... but please make it in separate commits with explicit commit messages! This diff is huge for a 1 line workaround which isn't a "plain" revert. Florent. From nextgens at freenetproject.org Tue Jul 1 12:23:42 2008 From: nextgens at freenetproject.org (NextGen$) Date: Tue, 1 Jul 2008 12:23:42 +0000 Subject: [freenet-dev] Bug submission In-Reply-To: <823242bd0806220833g6ad525f1u508a9d067fe1799b@mail.gmail.com> References: <823242bd0806220833g6ad525f1u508a9d067fe1799b@mail.gmail.com> Message-ID: <20080701122342.GE26659@freenetproject.org> * Ian Clarke [2008-06-22 10:33:09]: > I was surprised to read this on https://bugs.freenetproject.org/ :- > > "The first step in filing a bug report is a post to the support > mailing list. You must subscribe to post. If you get told it's a bug, > then come back here and file a bug report. When filing, please include > a URL into the mailing list archives so we can easily locate the > original conversations. This process greatly helps us to help you. > Many "bugs" are actually configuration or comprehension issues that > can be easily and more quickly resolved through the users mailing > list." > > I don't agree with this, and its creating a significant hurdle for > people filing bugs. Its basically using a mailing list as a > first-line bug tracker, something mailing lists aren't optomised for. > We should be using the bugtracker as our bugtracker. > > If people are repeatedly running into the same "configuration or > comprehension" issue, then that means there is a usability bug, and it > should absolutely be filed as a bug in Mantis. > > If people submit an invalid bug then its as easy as anything to mark > it as invalid in the bugtracker. Requiring that they post to support > first isn't making life easier for anyone, and it is probably > discouraging quite a few entirely legitimate bug reports. > > Ian. I wish we could rely on users to use mantis properly... but experience has shown we can't. Most of them don't know how to categorize bugs, not to mention they don't look for duplicates. Anyway, what do you want me to do? Rephrase the page or just remove it and display mantis directly? Florent From toad at amphibian.dyndns.org Tue Jul 1 12:25:33 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:25:33 +0100 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <20080701064125.GC26659@freenetproject.org> References: <20080629170308.E00424791C6@freenetproject.org> <20080701064125.GC26659@freenetproject.org> Message-ID: <200807011325.39428.toad@amphibian.dyndns.org> On Tuesday 01 July 2008 07:41, NextGen$ wrote: > * bombe at freenetproject.org [2008-06-29 17:03:08]: > > > Author: bombe > > Date: 2008-06-29 17:03:08 +0000 (Sun, 29 Jun 2008) > > New Revision: 20868 > > > > Modified: > > trunk/freenet/src/freenet/client/async/SingleFileInserter.java > > trunk/freenet/src/freenet/support/io/BucketChainBucket.java > > trunk/freenet/src/freenet/support/io/BucketChainBucketFactory.java > > trunk/freenet/src/freenet/support/io/BucketTools.java > > Log: > > fix backport of BucketChainBucket > > > > As far as I can see you have restored the old behaviour bypassing the > non-working code... but that's hardly a fix :) > > > Modified: trunk/freenet/src/freenet/client/async/SingleFileInserter.java > > =================================================================== > > --- trunk/freenet/src/freenet/client/async/SingleFileInserter.java 2008-06-29 16:23:27 UTC (rev 20867) > > +++ trunk/freenet/src/freenet/client/async/SingleFileInserter.java 2008-06-29 17:03:08 UTC (rev 20868) > > @@ -14,7 +14,6 @@ > > import freenet.keys.BaseClientKey; > > import freenet.keys.CHKBlock; > > import freenet.keys.FreenetURI; > > - import freenet.keys.NodeCHK; > > import freenet.keys.SSKBlock; > > import freenet.support.Logger; > > import freenet.support.OOMHandler; > > @@ -22,7 +21,6 @@ > > import freenet.support.api.Bucket; > > import freenet.support.compress.CompressionOutputSizeException; > > import freenet.support.compress.Compressor; > > -import freenet.support.io.BucketChainBucketFactory; > > import freenet.support.io.BucketTools; > > > > /** > > @@ -172,7 +170,7 @@ > > ctx.eventProducer.produceEvent(new StartedCompressionEvent(i)); > > Compressor comp = Compressor.getCompressionAlgorithmByDifficulty(i); > > Bucket result; > > - result = comp.compress(origData, new BucketChainBucketFactory(ctx.persistentBucketFactory, NodeCHK.BLOCK_SIZE), origData.size()); > > + result = comp.compress(origData, ctx.persistentBucketFactory, origData.size()); > > if(result.size() < oneBlockCompressedSize) { > > bestCodec = comp; > > if(bestCompressedData != null) > > > > That's the gist of the patch, right ? If so, where is the > BucketChainBucketFactory class used in the code ? > > I have no problem with you disabling non-working code or even reverting my > backports... but please make it in separate commits with explicit commit > messages! This diff is huge for a 1 line workaround which isn't a "plain" revert. I did fix this in the db4o branch, sorry for not labelling the final fix as BACKPORT... it's right after the original backport-labelled commit though, it should be easy to find. > > Florent. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/0506cb51/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 12:26:16 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:26:16 +0100 Subject: [freenet-dev] [freenet-cvs] r20750 - branches/db4o/freenet/src/freenet/client/async In-Reply-To: References: <20080626154314.D3AF6478ECD@freenetproject.org> Message-ID: <200807011326.17063.toad@amphibian.dyndns.org> On Sunday 29 June 2008 15:10, Daniel Cheng wrote: > On Thu, Jun 26, 2008 at 11:43 PM, wrote: > > Author: toad > > Date: 2008-06-26 15:43:14 +0000 (Thu, 26 Jun 2008) > > New Revision: 20750 > > > > Modified: > > branches/db4o/freenet/src/freenet/client/async/SplitFileInserterSegment.java > > Log: > > Don't activate and deactivate parent here > > > > activate() is very cheap on already activated object.. Yes but deactivating the parent and then returning to them would cause problems. > > > > Modified: branches/db4o/freenet/src/freenet/client/async/SplitFileInserterSegment.java > > =================================================================== > > --- branches/db4o/freenet/src/freenet/client/async/SplitFileInserterSegment.java 2008-06-26 15:33:10 UTC (rev 20749) > > +++ branches/db4o/freenet/src/freenet/client/async/SplitFileInserterSegment.java 2008-06-26 15:43:14 UTC (rev 20750) > > @@ -401,8 +401,8 @@ > > } > > > > public void start(ObjectContainer container, ClientContext context) throws InsertException { > > + // Always called by parent, so don't activate or deactivate parent. > > if(persistent) { > > - container.activate(parent, 1); > > container.activate(parent.parent, 1); > > } > > if (logMINOR) > > @@ -473,8 +473,6 @@ > > if (finished) { > > parent.segmentFinished(this, container, context); > > } > > - if(persistent) > > - container.deactivate(parent, 1); > > } > > > > public void onDecodedSegment(ObjectContainer container, ClientContext context, FECJob job, Bucket[] dataBuckets, Bucket[] checkBuckets, SplitfileBlock[] dataBlockStatus, SplitfileBlock[] checkBlockStatus) {} // irrevelant > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/c1f40094/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 12:28:01 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:28:01 +0100 Subject: [freenet-dev] [freenet-cvs] r20796 - branches/db4o/freenet/src/freenet/keys In-Reply-To: References: <20080626225401.7C7AF478EAE@freenetproject.org> Message-ID: <200807011328.02883.toad@amphibian.dyndns.org> On Sunday 29 June 2008 15:02, Daniel Cheng wrote: > On Fri, Jun 27, 2008 at 6:54 AM, wrote: > > Author: toad > > Date: 2008-06-26 22:54:01 +0000 (Thu, 26 Jun 2008) > > New Revision: 20796 > > > > Modified: > > branches/db4o/freenet/src/freenet/keys/FreenetURI.java > > Log: > > Hopefully prevent NPE in logging > > > > > Modified: branches/db4o/freenet/src/freenet/keys/FreenetURI.java > > =================================================================== > > --- branches/db4o/freenet/src/freenet/keys/FreenetURI.java 2008-06-26 22:48:25 UTC (rev 20795) > > +++ branches/db4o/freenet/src/freenet/keys/FreenetURI.java 2008-06-26 22:54:01 UTC (rev 20796) > > @@ -540,6 +540,10 @@ > > } > > > > public String toString(boolean prefix, boolean pureAscii) { > > + if(keyType == null) { > > + // Not activated or something... > > Log a minor/debug message here? Probably a good idea. Done. > > > + return null; > > + } > > StringBuffer b; > > if(prefix) > > b = new StringBuffer("freenet:"); -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/5397d93d/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 12:33:20 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:33:20 +0100 Subject: [freenet-dev] [freenet-cvs] r20845 - branches/db4o/freenet/src/freenet/support/io In-Reply-To: References: <20080628144923.D3988479050@freenetproject.org> Message-ID: <200807011333.22090.toad@amphibian.dyndns.org> On Sunday 29 June 2008 14:52, Daniel Cheng wrote: > On Sat, Jun 28, 2008 at 10:49 PM, wrote: > > Author: toad > > Date: 2008-06-28 14:49:23 +0000 (Sat, 28 Jun 2008) > > New Revision: 20845 > > > > Modified: > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > > Log: > > Don't store the RNG here. Just store a seed created on startup. > > > > If we restarted the node twice... The PRNG will be seeded with the > same seed in the two times... -- Is this behavior expected? The PRNG is only instantiated when we need it for padding... what's the problem? > > IMO, make Node a static field would make the code much simpler in most > code -- we can refer to the node object when we need a prng, database, > etc.... For "many node in a jvm", we can use some ClassLoader > hacks.... Yay God Object. :) When we have many node in a jvm, we want to be able to deal with them easily... e.g. in the simulation code. Not to have to go through 10 layers of indirection because of the hacks we're using. W.r.t. the database, it was a deliberate decision to pass it around, this makes writing relatively non-buggy code easier at this point. It could be changed later ... but the single-database-thread single-simultaneous-transaction design really doesn't go well with having static access to the database: it would be far too easy to just grab the database handle and write something, when there might be 10 other threads doing the same at the same time. So we'd have to use an embedded client-server design, which gives some benefits, but a number of major headaches too. Performance-wise I'm not sure it would be a benefit: what we gain in parallel reads/writes (which isn't always a gain especially on commodity drives), we lose in caching and refreshing. Having said that, the weak and strong random sources could reasonably be static. These are self-contained and can be common between many nodes. That's a sensible optimisation both to code quality and to performance. > > > Modified: branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > > =================================================================== > > --- branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java 2008-06-28 14:48:55 UTC (rev 20844) > > +++ branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java 2008-06-28 14:49:23 UTC (rev 20845) > > @@ -18,6 +18,8 @@ > > import freenet.support.api.Bucket; > > import java.util.Random; > > > > +import org.spaceroots.mantissa.random.MersenneTwister; > > + > > import com.db4o.ObjectContainer; > > > > /** > > @@ -29,10 +31,10 @@ > > > > private final Bucket bucket; > > private final int minPaddedSize; > > - private final Random randomSource; > > private transient SoftReference /* */ aesRef; > > /** The decryption key. */ > > private final byte[] key; > > + private final byte[] randomSeed; > > private long dataLength; > > private boolean readOnly; > > private int lastOutputStream; > > @@ -49,10 +51,11 @@ > > * @throws UnsupportedCipherException > > */ > > public PaddedEphemerallyEncryptedBucket(Bucket bucket, int minSize, RandomSource strongPRNG, Random weakPRNG) { > > - this.randomSource = weakPRNG; > > this.bucket = bucket; > > if(bucket.size() != 0) throw new IllegalArgumentException("Bucket must be empty"); > > byte[] tempKey = new byte[32]; > > + randomSeed = new byte[32]; > > + weakPRNG.nextBytes(randomSeed); > > strongPRNG.nextBytes(tempKey); > > this.key = tempKey; > > this.minPaddedSize = minSize; > > @@ -76,9 +79,10 @@ > > if(bucket.size() < knownSize) > > throw new IOException("Bucket "+bucket+" is too small on disk - knownSize="+knownSize+" but bucket.size="+bucket.size()+" for "+bucket); > > this.dataLength = knownSize; > > - this.randomSource = origRandom; > > this.bucket = bucket; > > if(key.length != 32) throw new IllegalArgumentException("Key wrong length: "+key.length); > > + randomSeed = new byte[32]; > > + origRandom.nextBytes(randomSeed); > > this.key = key; > > this.minPaddedSize = minSize; > > readOnly = false; > > @@ -86,7 +90,6 @@ > > } > > > > public PaddedEphemerallyEncryptedBucket(SimpleFieldSet fs, RandomSource origRandom, PersistentFileTracker f) throws CannotCreateFromFieldSetException { > > - this.randomSource = origRandom; > > String tmp = fs.get("DataLength"); > > if(tmp == null) > > throw new CannotCreateFromFieldSetException("No DataLength"); > > @@ -116,6 +119,8 @@ > > } > > if(dataLength > bucket.size()) > > throw new CannotCreateFromFieldSetException("Underlying bucket "+bucket+" is too small: should be "+dataLength+" actually "+bucket.size()); > > + randomSeed = new byte[32]; > > + origRandom.nextBytes(randomSeed); > > } > > > > public OutputStream getOutputStream() throws IOException { > > @@ -184,6 +189,7 @@ > > Logger.normal(this, "Not padding out to length because have been superceded: "+getName()); > > return; > > } > > + Random random = new MersenneTwister(randomSeed); > > synchronized(PaddedEphemerallyEncryptedBucket.this) { > > long finalLength = paddedLength(); > > long padding = finalLength - dataLength; > > @@ -191,7 +197,7 @@ > > long writtenPadding = 0; > > while(writtenPadding < padding) { > > int left = (int) Math.min((padding - writtenPadding), (long)buf.length); > > - randomSource.nextBytes(buf); > > + random.nextBytes(buf); > > out.write(buf, 0, left); > > writtenPadding += left; > > } > > > > _______________________________________________ > > cvs mailing list > > cvs at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/47b925cc/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 12:34:37 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:34:37 +0100 Subject: [freenet-dev] [freenet-cvs] r20865 - branches/db4o/freenet/src/freenet/node/fcp In-Reply-To: References: <20080628213339.D4070479075@freenetproject.org> Message-ID: <200807011334.39127.toad@amphibian.dyndns.org> On Sunday 29 June 2008 14:37, Daniel Cheng wrote: > On Sun, Jun 29, 2008 at 5:33 AM, wrote: > > Author: toad > > Date: 2008-06-28 21:33:39 +0000 (Sat, 28 Jun 2008) > > New Revision: 20865 > > > > Modified: > > branches/db4o/freenet/src/freenet/node/fcp/FCPServer.java > > Log: > > Fix renaming > > > > Should we use FileUtil#renameTo() ? No, that's mostly useful for renaming-over-a-file or renaming between directories. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/eac861e8/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 12:40:39 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 13:40:39 +0100 Subject: [freenet-dev] l10n update jun13-08 In-Reply-To: <1213363490.897.3.camel@u64> References: <1213363490.897.3.camel@u64> Message-ID: <200807011340.40477.toad@amphibian.dyndns.org> On Friday 13 June 2008 14:24, Luke771 wrote: > Updated to 1152 r20268 > - Translated new strings about Java 1.4 => 1.5 > - a couple of typos and minor rewordings. > Has this been applied to trunk? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/01959811/attachment.pgp From nextgens at freenetproject.org Tue Jul 1 12:54:16 2008 From: nextgens at freenetproject.org (NextGen$) Date: Tue, 1 Jul 2008 12:54:16 +0000 Subject: [freenet-dev] l10n update jun13-08 In-Reply-To: <200807011340.40477.toad@amphibian.dyndns.org> References: <1213363490.897.3.camel@u64> <200807011340.40477.toad@amphibian.dyndns.org> Message-ID: <20080701125416.GF26659@freenetproject.org> * Matthew Toseland [2008-07-01 13:40:39]: > On Friday 13 June 2008 14:24, Luke771 wrote: > > Updated to 1152 r20268 > > - Translated new strings about Java 1.4 => 1.5 > > - a couple of typos and minor rewordings. > > > Has this been applied to trunk? No. From j16sdiz+freenet at gmail.com Tue Jul 1 13:22:57 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Tue, 1 Jul 2008 21:22:57 +0800 Subject: [freenet-dev] [freenet-cvs] r20845 - branches/db4o/freenet/src/freenet/support/io In-Reply-To: <200807011333.22090.toad@amphibian.dyndns.org> References: <20080628144923.D3988479050@freenetproject.org> <200807011333.22090.toad@amphibian.dyndns.org> Message-ID: On Tue, Jul 1, 2008 at 8:33 PM, Matthew Toseland wrote: > On Sunday 29 June 2008 14:52, Daniel Cheng wrote: >> On Sat, Jun 28, 2008 at 10:49 PM, wrote: >> > Author: toad >> > Date: 2008-06-28 14:49:23 +0000 (Sat, 28 Jun 2008) >> > New Revision: 20845 >> > >> > Modified: >> > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java >> > Log: >> > Don't store the RNG here. Just store a seed created on startup. >> > >> >> If we restarted the node twice... The PRNG will be seeded with the >> same seed in the two times... -- Is this behavior expected? > > The PRNG is only instantiated when we need it for padding... what's the > problem? >> >> IMO, make Node a static field would make the code much simpler in most >> code -- we can refer to the node object when we need a prng, database, >> etc.... For "many node in a jvm", we can use some ClassLoader >> hacks.... > > Yay God Object. :) > > When we have many node in a jvm, we want to be able to deal with them > easily... e.g. in the simulation code. Not to have to go through 10 layers of > indirection because of the hacks we're using. The simulator is just ~1% of all source (in terms of line of code, exclude unit test). And we don't really need 10 layers of indirection... I saw you passing the ObjectContainer and ClientContext around recently, hence the suggestion. > W.r.t. the database, it was a deliberate decision to pass it around, this > makes writing relatively non-buggy code easier at this point. It could be > changed later ... but the single-database-thread > single-simultaneous-transaction design really doesn't go well with having > static access to the database: it would be far too easy to just grab the > database handle and write something, when there might be 10 other threads > doing the same at the same time. So we'd have to use an embedded > client-server design, which gives some benefits, but a number of major > headaches too. Performance-wise I'm not sure it would be a benefit: what we > gain in parallel reads/writes (which isn't always a gain especially on > commodity drives), we lose in caching and refreshing. > > Having said that, the weak and strong random sources could reasonably be > static. These are self-contained and can be common between many nodes. That's > a sensible optimisation both to code quality and to performance. >> >> > Modified: > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java >> > =================================================================== >> > --- > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > 2008-06-28 14:48:55 UTC (rev 20844) >> > +++ > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > 2008-06-28 14:49:23 UTC (rev 20845) >> > @@ -18,6 +18,8 @@ >> > import freenet.support.api.Bucket; >> > import java.util.Random; >> > >> > +import org.spaceroots.mantissa.random.MersenneTwister; >> > + >> > import com.db4o.ObjectContainer; >> > >> > /** >> > @@ -29,10 +31,10 @@ >> > >> > private final Bucket bucket; >> > private final int minPaddedSize; >> > - private final Random randomSource; >> > private transient SoftReference /* */ aesRef; >> > /** The decryption key. */ >> > private final byte[] key; >> > + private final byte[] randomSeed; >> > private long dataLength; >> > private boolean readOnly; >> > private int lastOutputStream; >> > @@ -49,10 +51,11 @@ >> > * @throws UnsupportedCipherException >> > */ >> > public PaddedEphemerallyEncryptedBucket(Bucket bucket, int minSize, > RandomSource strongPRNG, Random weakPRNG) { >> > - this.randomSource = weakPRNG; >> > this.bucket = bucket; >> > if(bucket.size() != 0) throw new > IllegalArgumentException("Bucket must be empty"); >> > byte[] tempKey = new byte[32]; >> > + randomSeed = new byte[32]; >> > + weakPRNG.nextBytes(randomSeed); >> > strongPRNG.nextBytes(tempKey); >> > this.key = tempKey; >> > this.minPaddedSize = minSize; >> > @@ -76,9 +79,10 @@ >> > if(bucket.size() < knownSize) >> > throw new IOException("Bucket "+bucket+" is too > small on disk - knownSize="+knownSize+" but bucket.size="+bucket.size()+" > for "+bucket); >> > this.dataLength = knownSize; >> > - this.randomSource = origRandom; >> > this.bucket = bucket; >> > if(key.length != 32) throw new > IllegalArgumentException("Key wrong length: "+key.length); >> > + randomSeed = new byte[32]; >> > + origRandom.nextBytes(randomSeed); >> > this.key = key; >> > this.minPaddedSize = minSize; >> > readOnly = false; >> > @@ -86,7 +90,6 @@ >> > } >> > >> > public PaddedEphemerallyEncryptedBucket(SimpleFieldSet fs, > RandomSource origRandom, PersistentFileTracker f) throws > CannotCreateFromFieldSetException { >> > - this.randomSource = origRandom; >> > String tmp = fs.get("DataLength"); >> > if(tmp == null) >> > throw new CannotCreateFromFieldSetException("No > DataLength"); >> > @@ -116,6 +119,8 @@ >> > } >> > if(dataLength > bucket.size()) >> > throw new > CannotCreateFromFieldSetException("Underlying bucket "+bucket+" is too small: > should be "+dataLength+" actually "+bucket.size()); >> > + randomSeed = new byte[32]; >> > + origRandom.nextBytes(randomSeed); >> > } >> > >> > public OutputStream getOutputStream() throws IOException { >> > @@ -184,6 +189,7 @@ >> > Logger.normal(this, "Not padding > out to length because have been superceded: "+getName()); >> > return; >> > } >> > + Random random = new > MersenneTwister(randomSeed); >> > > synchronized(PaddedEphemerallyEncryptedBucket.this) { >> > long finalLength = paddedLength(); >> > long padding = finalLength - > dataLength; >> > @@ -191,7 +197,7 @@ >> > long writtenPadding = 0; >> > while(writtenPadding < padding) { >> > int left = (int) > Math.min((padding - writtenPadding), (long)buf.length); >> > - > randomSource.nextBytes(buf); >> > + random.nextBytes(buf); >> > out.write(buf, 0, left); >> > writtenPadding += left; >> > } >> > >> > _______________________________________________ >> > cvs mailing list >> > cvs at freenetproject.org >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs >> > >> _______________________________________________ >> Devl mailing list >> Devl at freenetproject.org >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> >> > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > From j16sdiz+freenet at gmail.com Tue Jul 1 13:26:02 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Tue, 1 Jul 2008 21:26:02 +0800 Subject: [freenet-dev] [freenet-cvs] r20865 - branches/db4o/freenet/src/freenet/node/fcp In-Reply-To: <200807011334.39127.toad@amphibian.dyndns.org> References: <20080628213339.D4070479075@freenetproject.org> <200807011334.39127.toad@amphibian.dyndns.org> Message-ID: On Tue, Jul 1, 2008 at 8:34 PM, Matthew Toseland wrote: > On Sunday 29 June 2008 14:37, Daniel Cheng wrote: >> On Sun, Jun 29, 2008 at 5:33 AM, wrote: >> > Author: toad >> > Date: 2008-06-28 21:33:39 +0000 (Sat, 28 Jun 2008) >> > New Revision: 20865 >> > >> > Modified: >> > branches/db4o/freenet/src/freenet/node/fcp/FCPServer.java >> > Log: >> > Fix renaming >> > >> >> Should we use FileUtil#renameTo() ? > > No, that's mostly useful for renaming-over-a-file or renaming between > directories. > May you elaborate a bit (and fix the javadoc for FileUtil)? FileUtil#renameTo() does _NOT_ work for moving file across mount point for me. From toad at amphibian.dyndns.org Tue Jul 1 21:12:48 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 22:12:48 +0100 Subject: [freenet-dev] [freenet-cvs] r20845 - branches/db4o/freenet/src/freenet/support/io In-Reply-To: References: <20080628144923.D3988479050@freenetproject.org> <200807011333.22090.toad@amphibian.dyndns.org> Message-ID: <200807012212.48819.toad@amphibian.dyndns.org> On Tuesday 01 July 2008 14:22, Daniel Cheng wrote: > On Tue, Jul 1, 2008 at 8:33 PM, Matthew Toseland > wrote: > > On Sunday 29 June 2008 14:52, Daniel Cheng wrote: > >> On Sat, Jun 28, 2008 at 10:49 PM, wrote: > >> > Author: toad > >> > Date: 2008-06-28 14:49:23 +0000 (Sat, 28 Jun 2008) > >> > New Revision: 20845 > >> > > >> > Modified: > >> > > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > >> > Log: > >> > Don't store the RNG here. Just store a seed created on startup. > >> > > >> > >> If we restarted the node twice... The PRNG will be seeded with the > >> same seed in the two times... -- Is this behavior expected? > > > > The PRNG is only instantiated when we need it for padding... what's the > > problem? > >> > >> IMO, make Node a static field would make the code much simpler in most > >> code -- we can refer to the node object when we need a prng, database, > >> etc.... For "many node in a jvm", we can use some ClassLoader > >> hacks.... > > > > Yay God Object. :) > > > > When we have many node in a jvm, we want to be able to deal with them > > easily... e.g. in the simulation code. Not to have to go through 10 layers of > > indirection because of the hacks we're using. > > The simulator is just ~1% of all source (in terms of line of code, > exclude unit test). > And we don't really need 10 layers of indirection... > > I saw you passing the ObjectContainer and ClientContext around recently, hence > the suggestion. Actually, ClientContext could reasonably be static too... But not ObjectContainer. > > > W.r.t. the database, it was a deliberate decision to pass it around, this > > makes writing relatively non-buggy code easier at this point. It could be > > changed later ... but the single-database-thread > > single-simultaneous-transaction design really doesn't go well with having > > static access to the database: it would be far too easy to just grab the > > database handle and write something, when there might be 10 other threads > > doing the same at the same time. So we'd have to use an embedded > > client-server design, which gives some benefits, but a number of major > > headaches too. Performance-wise I'm not sure it would be a benefit: what we > > gain in parallel reads/writes (which isn't always a gain especially on > > commodity drives), we lose in caching and refreshing. > > > > Having said that, the weak and strong random sources could reasonably be > > static. These are self-contained and can be common between many nodes. That's > > a sensible optimisation both to code quality and to performance. > >> > >> > Modified: > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > >> > =================================================================== > >> > --- > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > > 2008-06-28 14:48:55 UTC (rev 20844) > >> > +++ > > branches/db4o/freenet/src/freenet/support/io/PaddedEphemerallyEncryptedBucket.java > > 2008-06-28 14:49:23 UTC (rev 20845) > >> > @@ -18,6 +18,8 @@ > >> > import freenet.support.api.Bucket; > >> > import java.util.Random; > >> > > >> > +import org.spaceroots.mantissa.random.MersenneTwister; > >> > + > >> > import com.db4o.ObjectContainer; > >> > > >> > /** > >> > @@ -29,10 +31,10 @@ > >> > > >> > private final Bucket bucket; > >> > private final int minPaddedSize; > >> > - private final Random randomSource; > >> > private transient SoftReference /* */ aesRef; > >> > /** The decryption key. */ > >> > private final byte[] key; > >> > + private final byte[] randomSeed; > >> > private long dataLength; > >> > private boolean readOnly; > >> > private int lastOutputStream; > >> > @@ -49,10 +51,11 @@ > >> > * @throws UnsupportedCipherException > >> > */ > >> > public PaddedEphemerallyEncryptedBucket(Bucket bucket, int minSize, > > RandomSource strongPRNG, Random weakPRNG) { > >> > - this.randomSource = weakPRNG; > >> > this.bucket = bucket; > >> > if(bucket.size() != 0) throw new > > IllegalArgumentException("Bucket must be empty"); > >> > byte[] tempKey = new byte[32]; > >> > + randomSeed = new byte[32]; > >> > + weakPRNG.nextBytes(randomSeed); > >> > strongPRNG.nextBytes(tempKey); > >> > this.key = tempKey; > >> > this.minPaddedSize = minSize; > >> > @@ -76,9 +79,10 @@ > >> > if(bucket.size() < knownSize) > >> > throw new IOException("Bucket "+bucket+" is too > > small on disk - knownSize="+knownSize+" but bucket.size="+bucket.size()+" > > for "+bucket); > >> > this.dataLength = knownSize; > >> > - this.randomSource = origRandom; > >> > this.bucket = bucket; > >> > if(key.length != 32) throw new > > IllegalArgumentException("Key wrong length: "+key.length); > >> > + randomSeed = new byte[32]; > >> > + origRandom.nextBytes(randomSeed); > >> > this.key = key; > >> > this.minPaddedSize = minSize; > >> > readOnly = false; > >> > @@ -86,7 +90,6 @@ > >> > } > >> > > >> > public PaddedEphemerallyEncryptedBucket(SimpleFieldSet fs, > > RandomSource origRandom, PersistentFileTracker f) throws > > CannotCreateFromFieldSetException { > >> > - this.randomSource = origRandom; > >> > String tmp = fs.get("DataLength"); > >> > if(tmp == null) > >> > throw new CannotCreateFromFieldSetException("No > > DataLength"); > >> > @@ -116,6 +119,8 @@ > >> > } > >> > if(dataLength > bucket.size()) > >> > throw new > > CannotCreateFromFieldSetException("Underlying bucket "+bucket+" is too small: > > should be "+dataLength+" actually "+bucket.size()); > >> > + randomSeed = new byte[32]; > >> > + origRandom.nextBytes(randomSeed); > >> > } > >> > > >> > public OutputStream getOutputStream() throws IOException { > >> > @@ -184,6 +189,7 @@ > >> > Logger.normal(this, "Not padding > > out to length because have been superceded: "+getName()); > >> > return; > >> > } > >> > + Random random = new > > MersenneTwister(randomSeed); > >> > > > synchronized(PaddedEphemerallyEncryptedBucket.this) { > >> > long finalLength = paddedLength(); > >> > long padding = finalLength - > > dataLength; > >> > @@ -191,7 +197,7 @@ > >> > long writtenPadding = 0; > >> > while(writtenPadding < padding) { > >> > int left = (int) > > Math.min((padding - writtenPadding), (long)buf.length); > >> > - > > randomSource.nextBytes(buf); > >> > + random.nextBytes(buf); > >> > out.write(buf, 0, left); > >> > writtenPadding += left; > >> > } > >> > > >> > _______________________________________________ > >> > cvs mailing list > >> > cvs at freenetproject.org > >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > >> > > >> _______________________________________________ > >> Devl mailing list > >> Devl at freenetproject.org > >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >> > >> > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/83e131b6/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 21:13:29 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 22:13:29 +0100 Subject: [freenet-dev] [freenet-cvs] r20865 - branches/db4o/freenet/src/freenet/node/fcp In-Reply-To: References: <20080628213339.D4070479075@freenetproject.org> <200807011334.39127.toad@amphibian.dyndns.org> Message-ID: <200807012213.29910.toad@amphibian.dyndns.org> On Tuesday 01 July 2008 14:26, Daniel Cheng wrote: > On Tue, Jul 1, 2008 at 8:34 PM, Matthew Toseland > wrote: > > On Sunday 29 June 2008 14:37, Daniel Cheng wrote: > >> On Sun, Jun 29, 2008 at 5:33 AM, wrote: > >> > Author: toad > >> > Date: 2008-06-28 21:33:39 +0000 (Sat, 28 Jun 2008) > >> > New Revision: 20865 > >> > > >> > Modified: > >> > branches/db4o/freenet/src/freenet/node/fcp/FCPServer.java > >> > Log: > >> > Fix renaming > >> > > >> > >> Should we use FileUtil#renameTo() ? > > > > No, that's mostly useful for renaming-over-a-file or renaming between > > directories. > > > > May you elaborate a bit (and fix the javadoc for FileUtil)? > > FileUtil#renameTo() does _NOT_ work for moving file across mount point for me. Yeah you need copyTo for that. :| -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/d958da44/attachment.pgp From bombe at pterodactylus.net Tue Jul 1 22:25:51 2008 From: bombe at pterodactylus.net (David =?utf-8?q?=E2=80=98Bombe=E2=80=99_Roden?=) Date: Wed, 2 Jul 2008 00:25:51 +0200 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <200807011325.39428.toad@amphibian.dyndns.org> References: <20080629170308.E00424791C6@freenetproject.org> <20080701064125.GC26659@freenetproject.org> <200807011325.39428.toad@amphibian.dyndns.org> Message-ID: <200807020025.54909.bombe@pterodactylus.net> On Tuesday 01 July 2008 14:25:33 Matthew Toseland wrote: > I did fix this in the db4o branch, sorry for not labelling the final fix as > BACKPORT... it's right after the original backport-labelled commit though, > it should be easy to find. Fix for what? SingleFileInserter in your branch looks exactly like it looks in trunk now. There is no single reference to BucketChainBucket(Factory) in it. I have merged 20571-20575 into trunk. Did that include your fix or not? Bombe -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080702/3f70169d/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 22:53:18 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 1 Jul 2008 23:53:18 +0100 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <200807020025.54909.bombe@pterodactylus.net> References: <20080629170308.E00424791C6@freenetproject.org> <200807011325.39428.toad@amphibian.dyndns.org> <200807020025.54909.bombe@pterodactylus.net> Message-ID: <200807012353.30196.toad@amphibian.dyndns.org> On Tuesday 01 July 2008 23:25, David ?Bombe? Roden wrote: > On Tuesday 01 July 2008 14:25:33 Matthew Toseland wrote: > > > I did fix this in the db4o branch, sorry for not labelling the final fix as > > BACKPORT... it's right after the original backport-labelled commit though, > > it should be easy to find. > > Fix for what? SingleFileInserter in your branch looks exactly like it looks in > trunk now. There is no single reference to BucketChainBucket(Factory) in it. > > I have merged 20571-20575 into trunk. Did that include your fix or not? BucketChainFactory is used in InsertCompressor. > > > Bombe > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080701/a3c93564/attachment.pgp From bombe at pterodactylus.net Tue Jul 1 23:01:48 2008 From: bombe at pterodactylus.net (David =?utf-8?q?=E2=80=98Bombe=E2=80=99_Roden?=) Date: Wed, 2 Jul 2008 01:01:48 +0200 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <200807012353.30196.toad@amphibian.dyndns.org> References: <20080629170308.E00424791C6@freenetproject.org> <200807020025.54909.bombe@pterodactylus.net> <200807012353.30196.toad@amphibian.dyndns.org> Message-ID: <200807020101.49109.bombe@pterodactylus.net> On Wednesday 02 July 2008 00:53:18 Matthew Toseland wrote: > BucketChainFactory is used in InsertCompressor. Right, and InsertCompressor does not exist in trunk. It looks like nextgens applied the patch for InsertCompressor to SingleFileInserter. Bombe -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080702/17c8e332/attachment.pgp From toad at amphibian.dyndns.org Tue Jul 1 23:15:43 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 2 Jul 2008 00:15:43 +0100 Subject: [freenet-dev] Request for wider testing of the db4o branch Message-ID: <200807020015.49166.toad@amphibian.dyndns.org> The db4o branch is now approaching usability. This is a branch (temporary fork) based on build 1151, which uses the object database db4o to store persistent requests and inserts. The objectives and intentions here: - True request resuming: The db4o branch resumes requests (not only inserts) from exactly where they were when the node shut down. The current trunk/stable Freenet code remembers what it was requesting, but has to resume requests from the datastore, which means heavy disk access for a long time after startup. - Faster startup: The db4o branch does not have to read in a list of persistent requests on startup, so startup is quicker. - Memory usage decoupled from queue size: The db4o branch "pages in" requests on demand, so in theory can run with a relatively low memory limit, even with a very large queue. In practice this would involve a lot of disk I/O, so it may not work so well in practice, but at least it will degrade gradually, rather than getting OutOfMemoryException's! Wider testing would be appreciated at this point, to identify any bugs and work towards merging. As with any testing of prerelease Freenet code, you should make a backup of all files with a port number in them (node-12345 etc) before trying it. It will attempt to automatically migrate your downloads and uploads, and will rename downloads.dat.gz to downloads.dat.gz.old.pre-db4o. That is tested with requests but not yet with inserts. It may use more disk space than the trunk version as it does not yet clean up as thoroughly as the trunk version. It may erase your hard drive, eat your cat and turn you in to the FBI for downloading pictures of naked chickens. The data is stored in node.db4o, the format should be relatively stable but *may* change if serious issues arise, in which case you *may* lose all your data (persistent downloads/uploads) and have to wipe it and start again. How to install: - Download db4o 6.4 for Java from www.db4o.com (they also do a .net version). - You will need the following libraries from the zip file (in lib/): db4o-6.4.44.10817-java1.2.jar db4o-6.4.44.10817-nqopt.jar bloat-1.0.jar - Add them to your classpath in wrapper.conf: wrapper.java.classpath.1=freenet.jar wrapper.java.classpath.2=freenet-ext.jar wrapper.java.classpath.3=db4o-6.4.38.10595-java1.2.jar wrapper.java.classpath.4=db4o-6.4.38.10595-nqopt.jar wrapper.java.classpath.5=bloat-1.0.jar The first two lines (or something similar) should be already in your wrapper.conf. If you have more classpath lines already, then increment the number after "classpath". - Install SVN (subversion.tigris.org), and check out the db4o branch sources: svn co --ignore-externals http://freenet.googlecode.com/svn/branches/db4o/freenet/ - Once checked out, you can update the source: svn update --ignore-externals - Build the code: ant - Copy the resulting lib/freenet-cvs-latest.jar to freenet.jar in your node. - Start the node up, and tell me how it goes! Any feedback can go to the devl mailing list, the FMS board, me personally, or the #freenet-testers channel on irc.freenode.org. Have fun! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080702/5b4eecff/attachment.pgp From nextgens at freenetproject.org Wed Jul 2 07:04:01 2008 From: nextgens at freenetproject.org (NextGen$) Date: Wed, 2 Jul 2008 07:04:01 +0000 Subject: [freenet-dev] [freenet-cvs] r20868 - in trunk/freenet/src/freenet: client/async support/io In-Reply-To: <200807020101.49109.bombe@pterodactylus.net> References: <20080629170308.E00424791C6@freenetproject.org> <200807020025.54909.bombe@pterodactylus.net> <200807012353.30196.toad@amphibian.dyndns.org> <200807020101.49109.bombe@pterodactylus.net> Message-ID: <20080702070401.GG26659@freenetproject.org> * David ???Bombe??? Roden [2008-07-02 01:01:48]: > On Wednesday 02 July 2008 00:53:18 Matthew Toseland wrote: > > > BucketChainFactory is used in InsertCompressor. > > Right, and InsertCompressor does not exist in trunk. It looks like nextgens > applied the patch for InsertCompressor to SingleFileInserter. > I don't pretend to understand the client-layer stuffs but it seems to be the right thing to do to me ;) How come your wonderful merging tool didn't do it by itself? :p Anyway, will you fix it or shall I have a look? From m.rogers at cs.ucl.ac.uk Wed Jul 2 09:50:20 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Wed, 02 Jul 2008 10:50:20 +0100 Subject: [freenet-dev] Request for wider testing of the db4o branch In-Reply-To: <200807020015.49166.toad@amphibian.dyndns.org> References: <200807020015.49166.toad@amphibian.dyndns.org> Message-ID: <486B4F5C.30003@cs.ucl.ac.uk> Matthew Toseland wrote: > It may erase your hard drive, eat your cat and turn you in to > the FBI for downloading pictures of naked chickens. This is a great warning - any chance of porting it to trunk? :-) Cheers, Michael From nextgens at freenetproject.org Wed Jul 2 14:08:49 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Wed, 2 Jul 2008 16:08:49 +0200 Subject: [freenet-dev] l10n update jun13-08 In-Reply-To: <20080701125416.GF26659@freenetproject.org> References: <1213363490.897.3.camel@u64> <200807011340.40477.toad@amphibian.dyndns.org> <20080701125416.GF26659@freenetproject.org> Message-ID: <20080702140848.GC3513@freenetproject.org> * NextGen$ [2008-07-01 12:54:16]: > * Matthew Toseland [2008-07-01 13:40:39]: > > > On Friday 13 June 2008 14:24, Luke771 wrote: > > > Updated to 1152 r20268 > > > - Translated new strings about Java 1.4 => 1.5 > > > - a couple of typos and minor rewordings. > > > > > Has this been applied to trunk? > > No. Applied in r20927 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080702/da4a7182/attachment.pgp From nextgens at freenetproject.org Wed Jul 2 16:35:33 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Wed, 2 Jul 2008 18:35:33 +0200 Subject: [freenet-dev] Freenet build #1153 is released Message-ID: <20080702163531.GF3513@freenetproject.org> A new build of freenet has been released. It will be mandatory on the 9th. of July. We don't provide 1.4-compatible bytecode anymore; if you are still using a 1.4 jvm time, it's time to upgrade! The changelog is available at http://cia.vc/stats/project/Freenet/.message/a8d723 Please report any bug you encounter so that we can keep on improving the code :) NextGen$ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080702/e9139c5f/attachment.pgp From nextgens at freenetproject.org Thu Jul 3 00:07:50 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Thu, 3 Jul 2008 02:07:50 +0200 Subject: [freenet-dev] [freenet-cvs] r20157 - in trunk/apps/new_installer: . res src In-Reply-To: <200806260050.59314.toad@amphibian.dyndns.org> References: <20080531133254.81A384791B3@freenetproject.org> <200806260050.59314.toad@amphibian.dyndns.org> Message-ID: <20080703000749.GA3519@freenetproject.org> * Matthew Toseland [2008-06-26 00:50:53]: > On Saturday 31 May 2008 14:32, nextgens at freenetproject.org wrote: > > Author: nextgens > > Date: 2008-05-31 13:32:54 +0000 (Sat, 31 May 2008) > > New Revision: 20157 > > > > Added: > > trunk/apps/new_installer/res/startssl.pem > > Modified: > > trunk/apps/new_installer/install.xml > > trunk/apps/new_installer/src/Sha1Test.java > > Log: > > new_installer: commit a new, improved Sha1Test class: it will get the > checksums from emu using a secured connection > > > ... > > > > // now we compare > > - byte[] digest=new byte[160]; > .. > > + byte[] digest = new byte[160]; > > Eh? Why does digest need to be initialized here? And it's 160 BITS not 160 > BYTES. Doesn't need to be; fixed in r20941 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080703/87d81015/attachment.pgp From m.rogers at cs.ucl.ac.uk Thu Jul 3 07:21:25 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Thu, 03 Jul 2008 08:21:25 +0100 Subject: [freenet-dev] Request for wider testing of the db4o branch In-Reply-To: <200807020015.49166.toad@amphibian.dyndns.org> References: <200807020015.49166.toad@amphibian.dyndns.org> Message-ID: <486C7DF5.3030303@cs.ucl.ac.uk> Matthew Toseland wrote: > - Memory usage decoupled from queue size: The db4o branch "pages in" requests > on demand, so in theory can run with a relatively low memory limit, even with > a very large queue. In practice this would involve a lot of disk I/O, so it > may not work so well in practice, but at least it will degrade gradually, > rather than getting OutOfMemoryException's! With three large downloads running, the db4o branch consistently uses about 50-70% CPU (I guess this is the paging you mentioned). The main branch uses 90-100% during startup but only 5% after that. Cheers, Michael From ian.clarke at gmail.com Fri Jul 4 11:58:36 2008 From: ian.clarke at gmail.com (Ian Clarke) Date: Fri, 4 Jul 2008 12:58:36 +0100 Subject: [freenet-dev] Bug submission In-Reply-To: <20080701122342.GE26659@freenetproject.org> References: <823242bd0806220833g6ad525f1u508a9d067fe1799b@mail.gmail.com> <20080701122342.GE26659@freenetproject.org> Message-ID: <823242bd0807040458x2d0cabb9ta3b67eaf73a40a19@mail.gmail.com> On Tue, Jul 1, 2008 at 1:23 PM, NextGen$ wrote: > Most of them don't know how to categorize bugs, not to > mention they don't look for duplicates. Fine, but having them email the support mailing list before filing the bug is a far less efficient way to deal with these problems than just making the necessary modifications to the bug (such as marking it as a dupe) after the fact. > Anyway, what do you want me to do? Rephrase the page or just remove it and > display mantis directly? Display Mantis directly. Ian. -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From nextgens at freenetproject.org Fri Jul 4 13:33:42 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Fri, 4 Jul 2008 15:33:42 +0200 Subject: [freenet-dev] Bug submission In-Reply-To: <823242bd0807040458x2d0cabb9ta3b67eaf73a40a19@mail.gmail.com> References: <823242bd0806220833g6ad525f1u508a9d067fe1799b@mail.gmail.com> <20080701122342.GE26659@freenetproject.org> <823242bd0807040458x2d0cabb9ta3b67eaf73a40a19@mail.gmail.com> Message-ID: <20080704133340.GA3522@freenetproject.org> * Ian Clarke [2008-07-04 12:58:36]: > On Tue, Jul 1, 2008 at 1:23 PM, NextGen$ wrote: > > Most of them don't know how to categorize bugs, not to > > mention they don't look for duplicates. > > Fine, but having them email the support mailing list before filing the > bug is a far less efficient way to deal with these problems than just > making the necessary modifications to the bug (such as marking it as a > dupe) after the fact. > > > Anyway, what do you want me to do? Rephrase the page or just remove it and > > display mantis directly? > > Display Mantis directly. > > Ian. Done -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080704/c93349cf/attachment.pgp From toad at amphibian.dyndns.org Fri Jul 4 21:57:34 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 4 Jul 2008 22:57:34 +0100 Subject: [freenet-dev] Request for wider testing of the db4o branch In-Reply-To: <486C7DF5.3030303@cs.ucl.ac.uk> References: <200807020015.49166.toad@amphibian.dyndns.org> <486C7DF5.3030303@cs.ucl.ac.uk> Message-ID: <200807042257.45443.toad@amphibian.dyndns.org> On Thursday 03 July 2008 08:21, Michael Rogers wrote: > Matthew Toseland wrote: > > - Memory usage decoupled from queue size: The db4o branch "pages in" requests > > on demand, so in theory can run with a relatively low memory limit, even with > > a very large queue. In practice this would involve a lot of disk I/O, so it > > may not work so well in practice, but at least it will degrade gradually, > > rather than getting OutOfMemoryException's! > > With three large downloads running, the db4o branch consistently uses > about 50-70% CPU (I guess this is the paging you mentioned). The main > branch uses 90-100% during startup but only 5% after that. :( Optimisation is in progress. A lot of work has been done already (especially yesterday), the next step is to separate the onGotKey() API from the main SendableRequest (queued requests) interface, so that we almost never need to touch pendingKeys; at the moment, access to pendingKeys is a significant proportion of total time spent on the database thread. On my test node, with a single big download running, the node's normal priority queue grows indefinitely ("StatisticsToadlet.databaseJobsByPriority", priority 5). Granted that's with heavy logging... > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080704/c26edafd/attachment.pgp From bigswim at gmail.com Sat Jul 5 22:19:48 2008 From: bigswim at gmail.com (Cole Brown) Date: Sat, 5 Jul 2008 18:19:48 -0400 Subject: [freenet-dev] Google Summer of Code, etc Message-ID: <291ff3790807051519j68aaa595l83feebeaca21f20@mail.gmail.com> Hey developers of freenet! I'm sorry it took me so long to get this message out -- i've had quite the busy summer. As some (few :x) of you know, I am the Google Summer of Code applicant that is working for freenet. My project (detailed at http://code.google.com/soc/2008/freenet/appinfo.html?csaid=93A6CF793E6ABEF5) is the creation of a web interface and diagnostic tools for the freemail plugin. While I have basically been silent, I've actually just about created the entire GUI and started to fill in the dynamic java backend for the site. I look forward to (hopefully) meeting all of you over the rest of the summer. I'll be making a better effort to idle in the channel. My nick there is v1rtuous. It has been a pleasure to work on this project so far and look forward to the rest of the summer. Sincerely, Cole Brown From j16sdiz+freenet at gmail.com Mon Jul 7 04:52:59 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Mon, 7 Jul 2008 12:52:59 +0800 Subject: [freenet-dev] [freenet-cvs] r20993 - in branches/db4o/freenet/src/freenet: client/async node In-Reply-To: <20080704142859.4F1654799DD@freenetproject.org> References: <20080704142859.4F1654799DD@freenetproject.org> Message-ID: On Fri, Jul 4, 2008 at 10:28 PM, wrote: > Author: toad > Date: 2008-07-04 14:28:59 +0000 (Fri, 04 Jul 2008) > New Revision: 20993 > > Added: > branches/db4o/freenet/src/freenet/client/async/PendingKeyItem.java > Modified: > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerBase.java > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerCore.java > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerNonPersistent.java > branches/db4o/freenet/src/freenet/node/Node.java > Log: > Implement pendingKeys differently for transient vs persistent. > For transient, keep the existing Map-based code. > For persistent, use PendingKeyItem's and queries. > Hopefully this is a speedup. > [...] > + > + ObjectSet pending = container.query(new Predicate() { > + public boolean match(PendingKeyItem item) { > + if(!key.equals(item.key)) return false; ob4o does not seems to be using the index for key.equals here. I have misinformed you something about indexing.... -- From toad at amphibian.dyndns.org Mon Jul 7 08:49:49 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 7 Jul 2008 09:49:49 +0100 Subject: [freenet-dev] [freenet-cvs] r20993 - in branches/db4o/freenet/src/freenet: client/async node In-Reply-To: References: <20080704142859.4F1654799DD@freenetproject.org> Message-ID: <20080707084949.GA32014@amphibian.dyndns.org> On Mon, Jul 07, 2008 at 12:52:59PM +0800, Daniel Cheng wrote: > On Fri, Jul 4, 2008 at 10:28 PM, wrote: > > Author: toad > > Date: 2008-07-04 14:28:59 +0000 (Fri, 04 Jul 2008) > > New Revision: 20993 > > > > Added: > > branches/db4o/freenet/src/freenet/client/async/PendingKeyItem.java > > Modified: > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerBase.java > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerCore.java > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerNonPersistent.java > > branches/db4o/freenet/src/freenet/node/Node.java > > Log: > > Implement pendingKeys differently for transient vs persistent. > > For transient, keep the existing Map-based code. > > For persistent, use PendingKeyItem's and queries. > > Hopefully this is a speedup. > > > [...] > > + > > + ObjectSet pending = container.query(new Predicate() { > > + public boolean match(PendingKeyItem item) { > > + if(!key.equals(item.key)) return false; > > ob4o does not seems to be using the index for key.equals here. > I have misinformed you something about indexing.... Yikes.... :( What to do about it? Can db4o index byte[]'s? From toad at amphibian.dyndns.org Mon Jul 7 11:17:33 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 7 Jul 2008 12:17:33 +0100 Subject: [freenet-dev] Summary of conclusions from Freenet mini-conference Message-ID: <20080707111733.GA2175@amphibian.dyndns.org> On Sunday, Oskar, myself, Ian, Theo, Michael, Vilhelm (vive), and a couple of newbies met up in a pub in London and had a long productive conversation about the current status of Freenet and the near future priorities. Vive has some notes; proper minutes will not be produced, but I include the important points and action items below, most of which inevitably fall to me! :) One important side benefit was to explain various current issues to people who haven't been involved very much for a while... hopefully they will contribute more now. FINANCIAL STATUS: Freenet Project Inc has $16,000 in the bank and paypal combined. This is enough to keep me paid up until the end of the year. Google contributed $18K some months ago, also iirc we had a $1500 donation more recently, but we had expected much of that to be used by now; the difference may be private (paypal) donations. VERSIONING: Given the significant gains available from some of the below items in security and performance, we should IMHO skip 0.7.1 and release 0.8-alpha1 in approximately December. Hopefully we will be able to raise enough money on the back of this to continue into next year. Many features will need to be postponed to 0.9; hopefully 1.0 will be largely stabilisation work. ACTION ITEMS/TODO LIST ADDITIONS: Somewhat theoretical stuff: should work, but needs simulations to be absolutely certain. Mostly relatively easy to implement once simulations confirm that it works, but actual implementation should probably be postponed to 0.9 as mostly darknet specific and potentially destabilising. Responsible: Vive. (After he comes back from holiday, assisted by Oskar). 1. Selective randomization. It is necessary to randomize node locations from time to time in order to prevent the network from degenerating, with some parts of the keyspace becoming increasingly crowded and other parts of the keyspace becoming huge voids with no nodes in that area at all. This can be either caused by join-leave churn (causing an unhelpful form of "evolution", with locations in less dense areas being shifted to new nodes and then dropped), or by deliberate attack on the swapping algorithm, again with the goal of destroying locations and concentrating all nodes' locations over an increasingly small area of the keyspace. For more detail read the Pitch Black paper (is it linked from the website??). Currently we randomize on a regular basis. This is bad because it is not sensitive to the network size or whether an active attack is happening, hence it either does too much randomization or too little. Oskar has proposed a new way to decide when to randomize: We send out 5 probe requests (not the same as the current probe requests), to 5 random locations, and each reports how close the reached node is to its peers' locations. We then take the overall median, and if we are much closer (tuning factor) to our peers than the probed nodes on average are, we reset our location to that of one of the random locations we probed for. This requires simulation to: a) Determine a reasonable tuning factor. b) Prove that it works. c) Prove that it is less vulnerable to attack than the current mechanism. An attacker would have to compromize 3 of the 5 paths, but for each path he only needs to be on the path to fake it... the paths might be limited to 5 hops, but even so, that's a lot of opportunities. On the other hand, it is unlikely that regular randomization will beat location spamming attacks, unless there is so much randomization as to have a significant impact on performance when there is no attack. 2. Targeted swapping. Freenet initially assigns random locations to nodes on the darknet, and constantly attempts to swap locations between pairs of nodes to improve the routability of the network, using the Metropolis-Hastings algorithm. At the moment, these swaps are proposed completely randomly, from 10 hop random walks. Vive proposed in a paper last year that some proportion of swaps should be "directed" i.e. the node figures out where would be optimal for its peers, and then routes to that location and proposes a swap with the "ideal" node. Not all of the swaps can be directed, in order to avoid long-lived false minima. This would speed up location swapping, the network would stabilise faster, achieve more efficient routing, and hopefully less location churn. But mostly this is about making routing work well on darknet: see the next item. 3. Opennet nodes and swapping. Opennet nodes use path folding, which Oskar's work shows is a very fast, reliable algorithm for making a navigable network by changing the links rather than the locations, and therefore the do not need swapping. Swapping in fact may cause significant location churn for opennet nodes, leading to poor data storage. But simply turning it off for opennet nodes may have unexpected side-effects on a hybrid network. One proposal is that opennet nodes not initiate swaps, but relay them for darknet nodes; or even accept swaps, but not initiate them. The question is whether the differences in swap pressure will cause problems... We need some detailed simulations, there is the potential to improve performance (location stability, hence data reachability) significantly for opennet (i.e. the majority of the network at present), without jeopardising our long term strategy of moving everyone progressively to darknet in order to reduce our attackability (especially as ISPs may begin blocking p2p networks in many countries in the next few years). More straightforward todo items (mostly me), for 0.8-beta1 1. Finish the db4o branch. I have been working on the db4o branch for some time, around a month. This is an attempt to move the client layer (for persistent requests) into a lightweight object database. This should allow fast resuming of requests, and for users to queue larger numbers of requests than will fit in RAM, and have performance degrade gracefully. Right now it is essentially working, but has major performance problems, resulting in the in-RAM queue of database jobs continually expanding when a big request is running (even on my fairly heavy hardware, admittedly with heavy logging enabled). There is room for significant optimisations, the immediate thing being to not use pendingKeys (the big persistent map of which key is wanted by which request) when we merely retry a block. Hopefully it will be possible to achieve reasonable performance, but some work will also need to be done to ensure that on a slow system it still works even if slowly (i.e. that it slows down rather than the job queue expanding indefinitely). There is a nonzero chance that we will have to abandon the whole project. Simple snapshot based resuming would be possible in that case (and IMHO essential for 0.8), but would of course take time on startup and memory proportional to the queue size. 2. Friend-of-a-friend routing. (Toad) Published work shows that taking into account the locations of our neighbour's neighbours when routing can dramatically improve performance. Oskar estimates that we can reduce the path length by an average factor of two, this may be even better in practice on a darknet. Apparently beyond two hops we rapidly run into diminishing returns. The security impact is minimal since this data is already exposed, albeit somewhat less reliably, by swapping. In theory the performance gain should be around a factor of log log N, assuming N nodes and log N peers per node, but no more than 5. 3. Eliminate all checkboxes in the installer, move to the wizard, and create an advanced options button in the installer. (Toad) This would streamline installation significantly: Paranoid users could click a box to turn UPnP etc on/off, everyone else doesn't need to worry about it. 4. Non-convergent splitfile encryption. Encrypting splitfiles using a random key independant of the content of the splitfile would dramatically reduce the rate at which an attacker can approach the inserter in the case where he can guess the content. Implementing this will be reasonably easy, it will however cost some redundancy: if two people insert the same file, they won't share blocks. In the future there are a couple of (relatively complex) schemes to reduce this cost, which we may implement if they appear to be necessary. This should probably be on by default but easy to turn off if the user is sure the file content can't be predicted by the attacker, or just doesn't care. 5. THROTTLE EVERYTHING !! (Toad) Not all traffic is throttled at the moment. We should subject all traffic, not only data transfer packets, to both congestion control and bandwidth limiting. This is related to the low-level streams proposal, but can be done without it, and should be easier. This is especially problematic for online gaming. See bug 2312. Note that a lot of Freenet runs on altruism, thus it is *essential* that it not severely disrupt a user's internet connection! 6. Reinstate uninstall survey. (Toad) Nextgens had an uninstall survey on emu for a while. Some useful data came from it, and since 90%+ of our users don't stick around, it would be a good idea to reinstate it, and try to optimise the questions. 7. Automatic bandwidth limit calibration. (Toad) Several other p2p apps implement this, we should too. Bandwidth is *the* scarce resource most of the time, we want to use as much of it as we can without significantly slowing down the user's internet connection (see above). Vive's students' possible projects: 1. Facebook application. A facebook app for (semi-?) automatic swapping of noderefs with people already marked as your friends on Facebook. Obviously this has security implications, it would be turned off by default but would be a useful option. We could explain this in the post-install wizard. TO IMPLEMENT IN 0.9: 1. Streams (low level optimisation) At the moment, Freenet is quite inefficient in bandwidth usage: the payload percentage, especially on nodes with low bandwidth limits but also on nodes with high bandwidth limits, can be quite low, 70%, maybe 50%... this is unacceptable. A lot of this is due to message padding. If we split block transfers (and large messages) into streams which can be sliced up byte-wise however big we want them, rather than into 1K blocks, then we will need much less padding and so acheive a higher payload percentage. This also has significant side benefits for steganography and for connections with low MTU, since we can (on a link with live streams) construct useful packets of almost any size. 2. Tunnels. There have been several proposals for creating encrypted tunnels to send requests through. The oldest proposal is cellular premix routing, which we have now abandoned, because it only offers an anonymity set of the cell, and costs twice the cell's diameter in hops. The recent proposals create tunnels by doing rendezvous of two or more "anchors" sent out by a node: either we rendezvous at a location, with the first anchor sent direct and the second random routed for a few hops, or we send two or more random routed anchors. Either way when the anchors meet, we create a tunnel. There can be just two anchors, or it can be an M of N secret sharing scheme. Once a tunnel is created, it will be fairly long lived, and related requests will be sent down the same tunnel(s). Related meaning firstly part of the same splitfile, and secondly marked as belonging to the same pseudonymous identity by FCP. (This is a GOOD thing to reduce the number of samples, as long as separate pseudonymous identities can be kept separate). Tunnels will help with each of the three main attacks on requests which most concern me at present: - Datastore seizure / remote probing. - Local correlation attacks. - Key-based search. (The splitfile changes above also help for this, but only for inserts; put both together and we have a radical improvement of security for insertors). Because tunnels mean that requests are started many hops away, we won't have to cache local requests in the local datastore. We will have a separate client cache, which may be ephemeral (key in RAM only, not saved). This means that we can publish Bloom filters for our datastore to our direct neighbours. This will likely save us several hops on average, so the overall cost of tunneling will not be huge. The only real problem is what to do if the user turns off tunneling... turn off Bloom filters too? 3. Passive requests. This feature has been planned for some time, I explained the current vision to Oskar et al, and the main justification for it (FMS!). There seemed to be general support in the group, in the medium term. Passive requests are a way of subscribing to a key by doing a request for it, and the nodes visited remembering the request and forwarding the data if it is found. As long as routing works, this should work adequately. Currently we have a simple form of this, ULPRs (Ultra-Lightweight Passive/Persistent Requests), but this is inherently unreliable, and only lasts an hour. Full passive requests would be reliable as long as routing works. This means when a connection is broken, or there are significant changes to the network topology/locations, the node involved would need to reroute the request. This is somewhat similar to the publish/subscribe system I tried to get working early in the 0.7 cycle, but IMHO we have a much better idea what we are doing now. Note that we may have a large number of passive request subscriptions on any one node, big enough that they will need to be stored in a database. Also note that if a lot of nodes are subscribed to a key, they form a tree/web structure so that it will be efficiently and near instantly propagated to all of them when it is found (in a place where it shouldn't have been, or because it was inserted). The objective here is primarily to reduce the cost of polling, particularly for FMS, but also for requests in general, and to increase the speed at which data can be found if people subscribe to it in advance. High-bandwidth / medium-latency publish/subscribe (audio streams e.g.) can be built on top of this; whether we want to have a separate system for low-bandwidth low latency streams (IRC etc) is an open question. DATA COLLECTION: 1. Push/pull tests. We should do push/pull tests regularly on the real network. On darknet this won't work well because of the difficulty of getting somebody you're not near to to fetch your data, although this is IMHO possible with FMS etc volunteers. On opennet we can simply select a different pair of peers every time to avoid the two nodes converging on each other as they are reused. OPEN QUESTIONS: 1. CPU profiling. Is there any more we can do to improve the node's CPU usage? It can still be high on some systems. What I found last time I tried was that mostly this is due to low memory conditions and FEC encoding/decoding ... but it may be worth re-investigating. 2. Oskar's NGR-like-routing paper. Is there an opportunity for significant performance gain here? Oskar proved in a paper that it is possible to take link costs into account in routing and still achieve good routing... it's not clear how close this is to something we could use. 3. Darknet link trust levels. Should we have a trust level dropdown box for each darknet peer? Oskar had originally assumed that we would solve the datastore problems by simply not caching local requests, on the grounds that we trust our (darknet) peers to not attempt to determine whether our requests are local from whether we cached them. This logic doesn't work for most links, even most darknet links; but for some connections, it may be useful, since it would significantly reduce our vulnerability if our store is seized since only a fraction of the file would be cached in our store, and likewise with remote store probing. Also, we may not want to send Bloom filters that are potentially incriminating to all our peers, especially if we turn off tunneling for some requests??? 4. Migration of data. After we swap locations, should we try to migrate data to match the new location? From ian.clarke at gmail.com Mon Jul 7 11:17:47 2008 From: ian.clarke at gmail.com (ian.clarke at gmail.com) Date: Mon, 7 Jul 2008 12:17:47 +0100 Subject: [freenet-dev] Google Summer of Code, etc In-Reply-To: <291ff3790807051519j68aaa595l83feebeaca21f20@mail.gmail.com> References: <291ff3790807051519j68aaa595l83feebeaca21f20@mail.gmail.com> Message-ID: <823242bd0807070417l34e90546vfffe41971e9f84cf@mail.gmail.com> Sounds great Cole, I'm on the road just now but am looking forward to checking it out as soon as I can, Ian. On 7/5/08, Cole Brown wrote: > Hey developers of freenet! > > I'm sorry it took me so long to get this message out -- i've had quite > the busy summer. As some (few :x) of you know, I am the Google Summer > of Code applicant that is working for freenet. My project (detailed > at > http://code.google.com/soc/2008/freenet/appinfo.html?csaid=93A6CF793E6ABEF5) > is the creation of a web interface and diagnostic tools for the > freemail plugin. While I have basically been silent, I've actually > just about created the entire GUI and started to fill in the dynamic > java backend for the site. I look forward to (hopefully) meeting all > of you over the rest of the summer. I'll be making a better effort to > idle in the channel. My nick there is v1rtuous. It has been a > pleasure to work on this project so far and look forward to the rest > of the summer. > > Sincerely, > Cole Brown > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From j16sdiz+freenet at gmail.com Mon Jul 7 15:32:20 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Mon, 7 Jul 2008 23:32:20 +0800 Subject: [freenet-dev] Summary of conclusions from Freenet mini-conference In-Reply-To: <20080707111733.GA2175@amphibian.dyndns.org> References: <20080707111733.GA2175@amphibian.dyndns.org> Message-ID: On Mon, Jul 7, 2008 at 7:17 PM, Matthew Toseland wrote: [...] > 1. Selective randomization. > 2. Targeted swapping. > 3. Opennet nodes and swapping. I think (3) deserve a higher priority then (1) and (2). Location chunk have been a big problem for a long time. The change is simple: disable swapping on opennet should relief the problem, result is deterministic, no dark magic are involved... [...] > 4. Non-convergent splitfile encryption. Should we introduce a new url type (CHK@/USK@/SSK@) for this? [...] > 1. CPU profiling. I have been doing this for some time. Most CPU time are spend on FEC and client layers. I will do more test after db4o branch is merged. From toad at amphibian.dyndns.org Wed Jul 9 12:20:28 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 9 Jul 2008 13:20:28 +0100 Subject: [freenet-dev] Summary of conclusions from Freenet mini-conference In-Reply-To: References: <20080707111733.GA2175@amphibian.dyndns.org> Message-ID: <200807091320.28647.toad@amphibian.dyndns.org> On Monday 07 July 2008 16:32, Daniel Cheng wrote: > On Mon, Jul 7, 2008 at 7:17 PM, Matthew Toseland > wrote: > [...] > > 1. Selective randomization. > > 2. Targeted swapping. > > 3. Opennet nodes and swapping. > > I think (3) deserve a higher priority then (1) and (2). > Location chunk have been a big problem for a long time. The change is > simple: disable swapping on opennet should relief the problem, result > is deterministic, no dark magic are involved... Dark magic are most definitely involved in the interface between opennet and darknet on a hybrid network. We need simulations here IMHO. > > [...] > > 4. Non-convergent splitfile encryption. > > Should we introduce a new url type (CHK@/USK@/SSK@) for this? CHKs can be happily encrypted with any key, this is by design. > > [...] > > 1. CPU profiling. > > I have been doing this for some time. Most CPU time are spend on FEC > and client layers. I will do more test after db4o branch is merged. That was more or less my impression, but Ian or somebody was complaining about CPU usage for an idle node... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080709/964bcfd2/attachment.pgp From toad at amphibian.dyndns.org Thu Jul 10 01:35:46 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 10 Jul 2008 02:35:46 +0100 Subject: [freenet-dev] =?iso-8859-1?q?=5Bfreenet-cvs=5D_r20993_-_in=09bran?= =?iso-8859-1?q?ches/db4o/freenet/src/freenet=3A_client/async_node?= In-Reply-To: <20080707084949.GA32014@amphibian.dyndns.org> References: <20080704142859.4F1654799DD@freenetproject.org> <20080707084949.GA32014@amphibian.dyndns.org> Message-ID: <200807100235.52694.toad@amphibian.dyndns.org> On Monday 07 July 2008 09:49, Matthew Toseland wrote: > On Mon, Jul 07, 2008 at 12:52:59PM +0800, Daniel Cheng wrote: > > On Fri, Jul 4, 2008 at 10:28 PM, wrote: > > > Author: toad > > > Date: 2008-07-04 14:28:59 +0000 (Fri, 04 Jul 2008) > > > New Revision: 20993 > > > > > > Added: > > > branches/db4o/freenet/src/freenet/client/async/PendingKeyItem.java > > > Modified: > > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerBase.java > > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerCore.java > > > branches/db4o/freenet/src/freenet/client/async/ClientRequestSchedulerNonPersistent.java > > > branches/db4o/freenet/src/freenet/node/Node.java > > > Log: > > > Implement pendingKeys differently for transient vs persistent. > > > For transient, keep the existing Map-based code. > > > For persistent, use PendingKeyItem's and queries. > > > Hopefully this is a speedup. > > > > > [...] > > > + > > > + ObjectSet pending = container.query(new Predicate() { > > > + public boolean match(PendingKeyItem item) { > > > + if(!key.equals(item.key)) return false; > > > > ob4o does not seems to be using the index for key.equals here. > > I have misinformed you something about indexing.... > > Yikes.... :( > > What to do about it? Can db4o index byte[]'s? You're right. :( It doesn't index byte[]'s either. What does work is converting to String's and indexing them. So that's what we're going to do. It sucks, but it works fast. http://amphibian.dyndns.org/index-test/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080710/b75e1d27/attachment.pgp