From jUrner at arcor.de Sat Mar 1 00:24:32 2008 From: jUrner at arcor.de (juergen urner) Date: Sat, 01 Mar 2008 01:24:32 +0100 Subject: [Tech] Opera JPEG vulnerability In-Reply-To: <1204310947.17396.34.camel@mustafar.winstonsmith.info> References: <200802271851.38357.toad@amphibian.dyndns.org> <1204310947.17396.34.camel@mustafar.winstonsmith.info> Message-ID: <47C8A240.5060609@arcor.de> Marco A. Calamari schrieb: > On Wed, 2008-02-27 at 18:51 +0000, Matthew Toseland wrote: > >> This is interesting because it came at the end of a thread on Frost where the >> OP was arguing that Freenet shouldn't filter JPEGs. (Freenet strips out EXIF >> data and other unknown chunks from JPEGs on download to maximize security; in >> the future we will do something similar on inserts). >> > > IMHO changing in any way information inserted in Freenet *must* > be documented, evident in user interface, up by default > but easily user selectable. > > Is it really up to Freenet fixing issues in software people may use along with it? Sounds like opening a can of worms with scarce devel time at hand. If it is a problem related to fproxy, let fproxy deal with it. Or tell users not to use Opera. As I see it, Freenet has a tendency of mixing node and client stuff a bit too much. My 2 cents, Juergen From toad at amphibian.dyndns.org Sat Mar 1 12:42:04 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Mar 2008 12:42:04 +0000 Subject: [Tech] Opera JPEG vulnerability In-Reply-To: <47C8A240.5060609@arcor.de> References: <200802271851.38357.toad@amphibian.dyndns.org> <1204310947.17396.34.camel@mustafar.winstonsmith.info> <47C8A240.5060609@arcor.de> Message-ID: <200803011242.12654.toad@amphibian.dyndns.org> On Saturday 01 March 2008 00:24, juergen urner wrote: > Marco A. Calamari schrieb: > > On Wed, 2008-02-27 at 18:51 +0000, Matthew Toseland wrote: > > > >> This is interesting because it came at the end of a thread on Frost where the > >> OP was arguing that Freenet shouldn't filter JPEGs. (Freenet strips out EXIF > >> data and other unknown chunks from JPEGs on download to maximize security; in > >> the future we will do something similar on inserts). > > > > IMHO changing in any way information inserted in Freenet *must* > > be documented, evident in user interface, up by default > > but easily user selectable. > > Is it really up to Freenet fixing issues in software people may use > along with it? Sounds like opening a can of worms with scarce > devel time at hand. If it is a problem related to fproxy, let fproxy > deal with it. Or tell users not to use Opera. > > As I see it, Freenet has a tendency of mixing node and client > stuff a bit too much. We have to implement filtering because our threat model is completely different to that of the typical web browser. This explains why we implement filtering of HTML, and it explains why we warn the user on any type we can't filter - for example, mp3s can contain id3 tags, which might be interpreted by some players so you can click on the url to go to the song's author's homepage - or even as HTML. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080301/475f3ac5/attachment.pgp From jUrner at arcor.de Sat Mar 1 14:48:20 2008 From: jUrner at arcor.de (juergen urner) Date: Sat, 01 Mar 2008 15:48:20 +0100 Subject: [Tech] Opera JPEG vulnerability In-Reply-To: <200803011242.12654.toad@amphibian.dyndns.org> References: <200802271851.38357.toad@amphibian.dyndns.org> <1204310947.17396.34.camel@mustafar.winstonsmith.info> <47C8A240.5060609@arcor.de> <200803011242.12654.toad@amphibian.dyndns.org> Message-ID: <47C96CB4.1030700@arcor.de> Matthew Toseland schrieb: > On Saturday 01 March 2008 00:24, juergen urner wrote: > >> Marco A. Calamari schrieb: >> >>> On Wed, 2008-02-27 at 18:51 +0000, Matthew Toseland wrote: >>> >>> >>>> This is interesting because it came at the end of a thread on Frost where >>>> > the > >>>> OP was arguing that Freenet shouldn't filter JPEGs. (Freenet strips out >>>> > EXIF > >>>> data and other unknown chunks from JPEGs on download to maximize >>>> > security; in > >>>> the future we will do something similar on inserts). >>>> >>> IMHO changing in any way information inserted in Freenet *must* >>> be documented, evident in user interface, up by default >>> but easily user selectable. >>> >> Is it really up to Freenet fixing issues in software people may use >> along with it? Sounds like opening a can of worms with scarce >> devel time at hand. If it is a problem related to fproxy, let fproxy >> deal with it. Or tell users not to use Opera. >> >> As I see it, Freenet has a tendency of mixing node and client >> stuff a bit too much. >> > > We have to implement filtering because our threat model is completely > different to that of the typical web browser. This explains why we implement > filtering of HTML, and it explains why we warn the user on any type we can't > filter - for example, mp3s can contain id3 tags, which might be interpreted > by some players so you can click on the url to go to the song's author's > homepage - or even as HTML. > I may watermark my mp3s. From toad at amphibian.dyndns.org Sat Mar 1 14:54:07 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Sat, 1 Mar 2008 14:54:07 +0000 Subject: [Tech] Opera JPEG vulnerability In-Reply-To: <47C96CB4.1030700@arcor.de> References: <200802271851.38357.toad@amphibian.dyndns.org> <200803011242.12654.toad@amphibian.dyndns.org> <47C96CB4.1030700@arcor.de> Message-ID: <200803011454.15507.toad@amphibian.dyndns.org> On Saturday 01 March 2008 14:48, juergen urner wrote: > Matthew Toseland schrieb: > > > > We have to implement filtering because our threat model is completely > > different to that of the typical web browser. This explains why we implement > > filtering of HTML, and it explains why we warn the user on any type we can't > > filter - for example, mp3s can contain id3 tags, which might be interpreted > > by some players so you can click on the url to go to the song's author's > > homepage - or even as HTML. > > > I may watermark my mp3s. Which is bad because...? The only threat here is that a download site may watermark each download of the same song differently in order to identify where the leak came from. There's nothing we can realistically do about that at the node level, but it's not a direct threat, certainly not to the downloader-of-the-mp3-from-freenet. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080301/60505c2b/attachment.pgp From m.rogers at cs.ucl.ac.uk Sat Mar 1 17:03:07 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat, 01 Mar 2008 17:03:07 +0000 Subject: [Tech] Hash cash keys Message-ID: <47C98C4B.4090900@cs.ucl.ac.uk> This week's half-baked proposal: hash cash keys, an extensions of SSKs where the key includes an extra field, b, and the data must be accompanied by a string whose hash shares a b-bit prefix with the hash of the public part of the key. Nodes check the partial hash collision before forwarding or storing the data; the string is forwarded and stored with the data. The key is written in the form SSK.b at foo,bar/baz, for example SSK.32 at foo,bar/baz for a 32-bit collision. This is for backwards compatibility with KSKs: you can't put b after the @ sign because everything after the @ sign is part of the name. This should make it pretty trivial to add hash cash to Frost: just use KSK.32 (for example) instead of KSK for new messages, and stop checking the old KSK queues after a transition period of a week or two. The value of b can be increased over time as computers get faster. I'm probably missing something obvious here - anyone care to point out what it is? :-) Cheers, Michael From m.rogers at cs.ucl.ac.uk Sat Mar 1 17:33:00 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat, 01 Mar 2008 17:33:00 +0000 Subject: [Tech] Hash cash keys In-Reply-To: <47C98C4B.4090900@cs.ucl.ac.uk> References: <47C98C4B.4090900@cs.ucl.ac.uk> Message-ID: <47C9934C.5000604@cs.ucl.ac.uk> Michael Rogers wrote: > the data must be > accompanied by a string whose hash shares a b-bit prefix with the hash > of the public part of the key. Sorry for replying to myself but I just spotted a problem: you could build up a dictionary of strings for all b-bit prefixes and reuse them. So instead of hash(string) matching hash(public_part) in the first b bits, hash(xor(string,public_part)) should match hash(public_part) in the first b bits. Cheers, Michael From bbackde at googlemail.com Sat Mar 1 19:45:39 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sat, 1 Mar 2008 20:45:39 +0100 Subject: [Tech] Hash cash keys In-Reply-To: <47C9934C.5000604@cs.ucl.ac.uk> References: <47C98C4B.4090900@cs.ucl.ac.uk> <47C9934C.5000604@cs.ucl.ac.uk> Message-ID: If I understand this correct, the node of the sender will have to compute a valid hash cash before actually sending the key? And receivers can easily check the hash cash and reject keys without a valid hash cash? On Sat, Mar 1, 2008 at 6:33 PM, Michael Rogers wrote: > Michael Rogers wrote: > > the data must be > > accompanied by a string whose hash shares a b-bit prefix with the hash > > of the public part of the key. > > Sorry for replying to myself but I just spotted a problem: you could > build up a dictionary of strings for all b-bit prefixes and reuse them. > So instead of hash(string) matching hash(public_part) in the first b > bits, hash(xor(string,public_part)) should match hash(public_part) in > the first b bits. > > > > Cheers, > Michael > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From m.rogers at cs.ucl.ac.uk Sat Mar 1 19:56:58 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat, 01 Mar 2008 19:56:58 +0000 Subject: [Tech] Hash cash keys In-Reply-To: References: <47C98C4B.4090900@cs.ucl.ac.uk> <47C9934C.5000604@cs.ucl.ac.uk> Message-ID: <47C9B50A.1000806@cs.ucl.ac.uk> bbackde at googlemail.com wrote: > If I understand this correct, the node of the sender will have to > compute a valid > hash cash before actually sending the key? And receivers can easily check the > hash cash and reject keys without a valid hash cash? Right - inserts without valid hash cash won't even be forwarded. Cheers, Michael From bbackde at googlemail.com Sat Mar 1 20:05:16 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sat, 1 Mar 2008 21:05:16 +0100 Subject: [Tech] Hash cash keys In-Reply-To: <47C9B50A.1000806@cs.ucl.ac.uk> References: <47C98C4B.4090900@cs.ucl.ac.uk> <47C9934C.5000604@cs.ucl.ac.uk> <47C9B50A.1000806@cs.ucl.ac.uk> Message-ID: This sounds like a good idea! No matter who uses it (fms, frost, thaw) could at least be sure that the sending costed the sender some cpu time. It could also restrict the amount of sent messages at all, considering the same cpu time. So it becomes more expensive and annoying for the spammer. Now its quite easy for him, he could even play crysis in parallell ;) On Sat, Mar 1, 2008 at 8:56 PM, Michael Rogers wrote: > bbackde at googlemail.com wrote: > > If I understand this correct, the node of the sender will have to > > compute a valid > > hash cash before actually sending the key? And receivers can easily check the > > hash cash and reject keys without a valid hash cash? > > Right - inserts without valid hash cash won't even be forwarded. > > > > Cheers, > Michael > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From bbackde at googlemail.com Sun Mar 2 12:49:57 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sun, 2 Mar 2008 13:49:57 +0100 Subject: [Tech] Hash cash keys In-Reply-To: References: <47C98C4B.4090900@cs.ucl.ac.uk> <47C9934C.5000604@cs.ucl.ac.uk> <47C9B50A.1000806@cs.ucl.ac.uk> Message-ID: No complains from anyone? So we can expect this feature soon? On Sat, Mar 1, 2008 at 9:05 PM, wrote: > This sounds like a good idea! No matter who uses it (fms, frost, thaw) > could at least be sure > that the sending costed the sender some cpu time. It could also > restrict the amount of sent messages > at all, considering the same cpu time. So it becomes more expensive > and annoying for the spammer. > Now its quite easy for him, he could even play crysis in parallell ;) > > > > On Sat, Mar 1, 2008 at 8:56 PM, Michael Rogers wrote: > > bbackde at googlemail.com wrote: > > > If I understand this correct, the node of the sender will have to > > > compute a valid > > > hash cash before actually sending the key? And receivers can easily check the > > > hash cash and reject keys without a valid hash cash? > > > > Right - inserts without valid hash cash won't even be forwarded. > > > > > > > > Cheers, > > Michael > > _______________________________________________ > > Tech mailing list > > Tech at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > > > > > > > > -- > __________________________________________________ > GnuPG key: (0x48DBFA8A) > Keyserver: pgpkeys.pca.dfn.de > Fingerprint: > 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A > __________________________________________________ > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From bbackde at googlemail.com Sun Mar 2 13:35:53 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sun, 2 Mar 2008 14:35:53 +0100 Subject: [Tech] FMS (Java) design Message-ID: >From a discussion about the fms java port. Please discuss. We need a final agreement how to implement it (as plugin, standalone, ...) bbackde at googlemail.com wrote: > While I think about it, maybe the best idea is to have your fms code > running inside the > node (as a plugin). The plugin provides only an FCP2 interface. All > other interfaces should > be on top, e.g. NNTP provided by another plugin, and Frost would use > FCP2 commands > to access your fms library code. > > What about this? I could help with the FCP2 interface, and we can > remove all other interfaces > from the code... > > I think it's a good idea with a few "but". 1) The question is - do you think it should handle everything, including messaging ? That would be: - Management of local identities (create/delete/change properties) - View/editing of trusts (in my version TrustList is a part of local identity) - Introductions of new local identities and accepting introductions from others - Messaging (should the messaging store be on the node ? ) 2) [BIG issue] If messages store is on a Freenet node and not in Frost folder, a node cannot be shared by two or more people. I know that people _do_ share freenet nodes. For example, Tin0 from Germany offers access to his Freenet node on http://i2p.to/ and people run Frost agains it without running own node. While not a good idea from privacy point of view and risky for operator because of Vorratsdatenspeicherung & co, sharing a node is a way to get more newbies onboard. 3) [Small issue] Can we make protocol flexible enough that it can handle possible changes in FMS ? FMS is very much under development - we just recently changed the MessageId format to avoid hijacking, people want audio-captchas, I will propose a change to announce solved captchas (people re-solve the same capthcas again and again which is kinda stupid) So there are changes and protocol should be flexible enough to handle them. From batosai at batosai.net Sun Mar 2 14:22:27 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Sun, 02 Mar 2008 15:22:27 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: References: Message-ID: <47CAB823.8080001@batosai.net> bbackde at googlemail.com a ?crit : > From a discussion about the fms java port. Please discuss. We need > a final agreement how to implement it (as plugin, standalone, ...) > > > bbackde at googlemail.com wrote: >> While I think about it, maybe the best idea is to have your fms code >> running inside the >> node (as a plugin). The plugin provides only an FCP2 interface. All >> other interfaces should >> be on top, e.g. NNTP provided by another plugin, and Frost would use >> FCP2 commands >> to access your fms library code. >> >> What about this? I could help with the FCP2 interface, and we can >> remove all other interfaces >> from the code... >> >> > I think it's a good idea with a few "but". I agree. I always thought it was the node's job and client applications should concentrate on "shiny" interfaces. > 1) The question is - do you think it should handle everything, > including messaging ? > That would be: > - Management of local identities (create/delete/change properties) > - View/editing of trusts (in my version TrustList is a part of local > identity) > - Introductions of new local identities and accepting introductions from > others > - Messaging (should the messaging store be on the node ? ) In my opinion, trusts should be handled by an independant plugin, so it could be used by other apps. Most apps need a WoT and it would be a waste of time (and maybe a security risk) if each developper had to write his own. I might work on it but not right now : I've got another project to finish first. > 2) [BIG issue] If messages store is on a Freenet node and not in Frost > folder, a node cannot be shared by two or more people. > I know that people _do_ share freenet nodes. > For example, Tin0 from Germany offers access to his Freenet node on > http://i2p.to/ and people run Frost agains it > without running own node. While not a good idea from privacy point of > view and risky for operator because of Vorratsdatenspeicherung & co, > sharing a node is a way to get more newbies onboard. Maybe it's possible to tag messages for a client ID, as we do with private download queues. > 3) [Small issue] Can we make protocol flexible enough that it can handle > possible changes in FMS ? > FMS is very much under development - we just recently changed the > MessageId format to avoid hijacking, > people want audio-captchas, I will propose a change to announce solved > captchas (people re-solve the same capthcas again and again > which is kinda stupid) > So there are changes and protocol should be flexible enough to handle them. > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080302/1d4c0ba2/attachment.pgp From bbackde at googlemail.com Sun Mar 2 15:01:50 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sun, 2 Mar 2008 16:01:50 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: <47CAB823.8080001@batosai.net> References: <47CAB823.8080001@batosai.net> Message-ID: 2008/3/2 Julien Cornuwel : > bbackde at googlemail.com a ?crit : > > > From a discussion about the fms java port. Please discuss. We need > > a final agreement how to implement it (as plugin, standalone, ...) > > > > > > bbackde at googlemail.com wrote: > >> While I think about it, maybe the best idea is to have your fms code > >> running inside the > >> node (as a plugin). The plugin provides only an FCP2 interface. All > >> other interfaces should > >> be on top, e.g. NNTP provided by another plugin, and Frost would use > >> FCP2 commands > >> to access your fms library code. > >> > >> What about this? I could help with the FCP2 interface, and we can > >> remove all other interfaces > >> from the code... > >> > >> > > I think it's a good idea with a few "but". > > I agree. I always thought it was the node's job and client applications > should concentrate on "shiny" interfaces. > > > > 1) The question is - do you think it should handle everything, > > including messaging ? > > That would be: > > - Management of local identities (create/delete/change properties) > > - View/editing of trusts (in my version TrustList is a part of local > > identity) > > - Introductions of new local identities and accepting introductions from > > others > > - Messaging (should the messaging store be on the node ? ) > > In my opinion, trusts should be handled by an independant plugin, so it > could be used by other apps. Most apps need a WoT and it would be a > waste of time (and maybe a security risk) if each developper had to > write his own. > > I might work on it but not right now : I've got another project to > finish first. > > > > 2) [BIG issue] If messages store is on a Freenet node and not in Frost > > folder, a node cannot be shared by two or more people. > > I know that people _do_ share freenet nodes. > > For example, Tin0 from Germany offers access to his Freenet node on > > http://i2p.to/ and people run Frost agains it > > without running own node. While not a good idea from privacy point of > > view and risky for operator because of Vorratsdatenspeicherung & co, > > sharing a node is a way to get more newbies onboard. > > Maybe it's possible to tag messages for a client ID, as we do with > private download queues. But this would require the node to fetch and store messages for ALL existing boards and identities. Just for the case that someone maybe wants to see it. This is a big change from the current situation (frost and fms), where you can choose to ignore boards and identities and we do not have to request the boards/identities at all. > > > > 3) [Small issue] Can we make protocol flexible enough that it can handle > > possible changes in FMS ? > > FMS is very much under development - we just recently changed the > > MessageId format to avoid hijacking, > > people want audio-captchas, I will propose a change to announce solved > > captchas (people re-solve the same capthcas again and again > > which is kinda stupid) > > So there are changes and protocol should be flexible enough to handle them. > > _______________________________________________ > > Tech mailing list > > Tech at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From batosai at batosai.net Sun Mar 2 15:14:19 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Sun, 02 Mar 2008 16:14:19 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: References: <47CAB823.8080001@batosai.net> Message-ID: <47CAC44B.5030402@batosai.net> bbackde at googlemail.com a ?crit : >> > 2) [BIG issue] If messages store is on a Freenet node and not in Frost >> > folder, a node cannot be shared by two or more people. >> > I know that people _do_ share freenet nodes. >> > For example, Tin0 from Germany offers access to his Freenet node on >> > http://i2p.to/ and people run Frost agains it >> > without running own node. While not a good idea from privacy point of >> > view and risky for operator because of Vorratsdatenspeicherung & co, >> > sharing a node is a way to get more newbies onboard. >> >> Maybe it's possible to tag messages for a client ID, as we do with >> private download queues. > > But this would require the node to fetch and store messages for ALL existing > boards and identities. Just for the case that someone maybe wants to see it. > This is a big change from the current situation (frost and fms), where > you can choose > to ignore boards and identities and we do not have to request the > boards/identities at all. No, the node would just fetch messages requested by one of its clients. And it would show each client only the messages it requested. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080302/c86d090c/attachment.pgp From bbackde at googlemail.com Sun Mar 2 15:25:51 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Sun, 2 Mar 2008 16:25:51 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: <47CAC44B.5030402@batosai.net> References: <47CAB823.8080001@batosai.net> <47CAC44B.5030402@batosai.net> Message-ID: 2008/3/2 Julien Cornuwel : > bbackde at googlemail.com a ?crit : > > > >> > 2) [BIG issue] If messages store is on a Freenet node and not in Frost > >> > folder, a node cannot be shared by two or more people. > >> > I know that people _do_ share freenet nodes. > >> > For example, Tin0 from Germany offers access to his Freenet node on > >> > http://i2p.to/ and people run Frost agains it > >> > without running own node. While not a good idea from privacy point of > >> > view and risky for operator because of Vorratsdatenspeicherung & co, > >> > sharing a node is a way to get more newbies onboard. > >> > >> Maybe it's possible to tag messages for a client ID, as we do with > >> private download queues. > > > > But this would require the node to fetch and store messages for ALL existing > > boards and identities. Just for the case that someone maybe wants to see it. > > This is a big change from the current situation (frost and fms), where > > you can choose > > to ignore boards and identities and we do not have to request the > > boards/identities at all. > > No, the node would just fetch messages requested by one of its clients. > And it would show each client only the messages it requested. > This becomes complicated: what messages to fetch depends on the WoT trust level of each client. If WoT is independent, how would the underlying system know what to fetch, but not fetch messages from spam identities? Another point brought up by a Frost user: his node runs on a slow old PC (I also know people who run the node on silent PCs), and his system is utilized. He cannot run another CPU/memory eating plugininside the node, he wants to run fms from another box, using FCP2 to request keys only. > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From martin.nyhus at sensewave.com Sun Mar 2 17:50:05 2008 From: martin.nyhus at sensewave.com (Martin Nyhus) Date: Sun, 02 Mar 2008 18:50:05 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: References: Message-ID: <1204480205.21500.28.camel@decoy.lan> On Sun, 2008-03-02 at 14:35 +0100, bbackde at googlemail.com wrote: > 2) [BIG issue] If messages store is on a Freenet node and not in Frost > folder, a node cannot be shared by two or more people. I'm sure I've forgot something, but what if client A tells the plugin to poll key X. The plugin will then keep track of which keys have been requested on a per-client basis and poll those keys. If client A then tells the plugin to stop polling key X, it will remove the key from the list of client A, and stop polling that key if there are no other 'subscribers'. If all clients only get to know their own keys, the only attack I can think of is based on the time it takes from you subscribe to the first message arrives. If this takes very little time, you can assume that someone else is already subscribed to that key. The above could of course be combined with some sort of login so the plugin could remember subscriptions on a per-user basis rather than a per-client basis. Nogaso -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080302/548d41bc/attachment.pgp From batosai at batosai.net Sun Mar 2 21:40:05 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Sun, 02 Mar 2008 22:40:05 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: References: <47CAB823.8080001@batosai.net> <47CAC44B.5030402@batosai.net> Message-ID: <47CB1EB5.9030702@batosai.net> bbackde at googlemail.com a ?crit : > 2008/3/2 Julien Cornuwel : >> bbackde at googlemail.com a ?crit : >> >> >>>> > 2) [BIG issue] If messages store is on a Freenet node and not in Frost >> >> > folder, a node cannot be shared by two or more people. >> >> > I know that people _do_ share freenet nodes. >> >> > For example, Tin0 from Germany offers access to his Freenet node on >> >> > http://i2p.to/ and people run Frost agains it >> >> > without running own node. While not a good idea from privacy point of >> >> > view and risky for operator because of Vorratsdatenspeicherung & co, >> >> > sharing a node is a way to get more newbies onboard. >> >> >> >> Maybe it's possible to tag messages for a client ID, as we do with >> >> private download queues. >> > >> > But this would require the node to fetch and store messages for ALL existing >> > boards and identities. Just for the case that someone maybe wants to see it. >> > This is a big change from the current situation (frost and fms), where >> > you can choose >> > to ignore boards and identities and we do not have to request the >> > boards/identities at all. >> >> No, the node would just fetch messages requested by one of its clients. >> And it would show each client only the messages it requested. >> > > This becomes complicated: what messages to fetch depends on the WoT > trust level of each client. > If WoT is independent, how would the underlying system know what to > fetch, but not fetch > messages from spam identities? The client decides what to fetch with the informations given by the WoT plugin. It then asks it to the FMS plugin and it works as Martin described in another post. The only issue I see here is that we would have to create 2 SSKs per user : one for the WoT, one for FMS stuff (messages...). > Another point brought up by a Frost user: his node runs on a slow old > PC (I also know people > who run the node on silent PCs), and his system is utilized. He cannot > run another CPU/memory > eating plugininside the node, he wants to run fms from another box, > using FCP2 to request > keys only. In that particular case, he would need an external program. But AFAIK, most users are using freenet and all its components on a single system. From toad at amphibian.dyndns.org Mon Mar 3 14:23:20 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 14:23:20 +0000 Subject: [Tech] FMS (Java) design In-Reply-To: References: <47CAC44B.5030402@batosai.net> Message-ID: <200803031423.21298.toad@amphibian.dyndns.org> On Sunday 02 March 2008 15:25, bbackde at googlemail.com wrote: > 2008/3/2 Julien Cornuwel : > > bbackde at googlemail.com a ?crit : > > > > >> > 2) [BIG issue] If messages store is on a Freenet node and not in Frost > > >> > folder, a node cannot be shared by two or more people. > > >> > I know that people _do_ share freenet nodes. > > >> > For example, Tin0 from Germany offers access to his Freenet node on > > >> > http://i2p.to/ and people run Frost agains it > > >> > without running own node. While not a good idea from privacy point of > > >> > view and risky for operator because of Vorratsdatenspeicherung & co, > > >> > sharing a node is a way to get more newbies onboard. > > >> > > >> Maybe it's possible to tag messages for a client ID, as we do with > > >> private download queues. > > > > > > But this would require the node to fetch and store messages for ALL existing > > > boards and identities. Just for the case that someone maybe wants to see it. > > > This is a big change from the current situation (frost and fms), where > > > you can choose > > > to ignore boards and identities and we do not have to request the > > > boards/identities at all. > > > > No, the node would just fetch messages requested by one of its clients. > > And it would show each client only the messages it requested. > > This becomes complicated: what messages to fetch depends on the WoT > trust level of each client. > If WoT is independent, how would the underlying system know what to > fetch, but not fetch > messages from spam identities? Well, at worst, we'd have a completely separate database and data structures for each client. Of course this means that running a public node with FCP support and the WoT plugin wouldn't work - but really, why would anyone install Frost but then connect it to a public node? It's madness. I can understand people trying out FProxy via a public node... but Frost?! If this is vital then a public node would have to require round-trip email verification for creating an account or something, but that's really not our problem. > > Another point brought up by a Frost user: his node runs on a slow old > PC (I also know people > who run the node on silent PCs), and his system is utilized. He cannot > run another CPU/memory > eating plugininside the node, he wants to run fms from another box, > using FCP2 to request > keys only. Long term it should be possible to run most plugins outside the node if absolutely necessary. For example, this particular plugin would generally only need to do requests and inserts, which could certainly be done from a plugin container running on another computer and talking to the node over FCP. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/69b6de25/attachment.pgp From toad at amphibian.dyndns.org Mon Mar 3 14:24:31 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 14:24:31 +0000 Subject: [Tech] FMS (Java) design In-Reply-To: <47CB1EB5.9030702@batosai.net> References: <47CB1EB5.9030702@batosai.net> Message-ID: <200803031424.32261.toad@amphibian.dyndns.org> On Sunday 02 March 2008 21:40, Julien Cornuwel wrote: > > The client decides what to fetch with the informations given by the WoT > plugin. It then asks it to the FMS plugin and it works as Martin > described in another post. > > The only issue I see here is that we would have to create 2 SSKs per > user : one for the WoT, one for FMS stuff (messages...). No, each user must have entirely separate publications. Otherwise an attacker can identify that two users share the same plugin / the same node. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/36983e3e/attachment.pgp From toad at amphibian.dyndns.org Mon Mar 3 14:29:56 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 14:29:56 +0000 Subject: [Tech] Hash cash keys In-Reply-To: References: <47C98C4B.4090900@cs.ucl.ac.uk> Message-ID: <200803031429.57074.toad@amphibian.dyndns.org> On Sunday 02 March 2008 12:49, bbackde at googlemail.com wrote: > No complains from anyone? So we can expect this feature soon? We're in a feature freeze. That means you have to convince me that it's essential to have this for 0.7.0. Hash cash is not a panacea: IMHO it will have to be combined with CAPTCHAs and web of trust, therefore we will still need FMS. Because an attacker will probably have a powerful system, whereas a lot of users don't; an attacker might have native-optimised hashcash libraries, and a typical user won't, at least in the short term. A user might not be willing to wait more than 1 minute for a message's hashcash to be computed, on a slow system; this means an attacker can solve them every 5 seconds on a faster system (twice the clock speed and four cores). Hash cash can be implemented in FMS without any change to the core. And it should only be necessary for FMS introductions, not later on. Just bolting it on to Frost would mean a very weak protection, at the cost of a new keytype. On the other hand it's a nice idea... > > On Sat, Mar 1, 2008 at 9:05 PM, wrote: > > This sounds like a good idea! No matter who uses it (fms, frost, thaw) > > could at least be sure > > that the sending costed the sender some cpu time. It could also > > restrict the amount of sent messages > > at all, considering the same cpu time. So it becomes more expensive > > and annoying for the spammer. > > Now its quite easy for him, he could even play crysis in parallell ;) > > > > On Sat, Mar 1, 2008 at 8:56 PM, Michael Rogers wrote: > > > bbackde at googlemail.com wrote: > > > > If I understand this correct, the node of the sender will have to > > > > compute a valid > > > > hash cash before actually sending the key? And receivers can easily check the > > > > hash cash and reject keys without a valid hash cash? > > > > > > Right - inserts without valid hash cash won't even be forwarded. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/2667d438/attachment.pgp From bbackde at googlemail.com Mon Mar 3 15:43:45 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Mon, 3 Mar 2008 16:43:45 +0100 Subject: [Tech] Hash cash keys In-Reply-To: <200803031429.57074.toad@amphibian.dyndns.org> References: <47C98C4B.4090900@cs.ucl.ac.uk> <200803031429.57074.toad@amphibian.dyndns.org> Message-ID: > On the other hand it's a nice idea... That is what I thought. I don't expect to see it in 070, but I think its great for a later release. 2008/3/3 Matthew Toseland : > On Sunday 02 March 2008 12:49, bbackde at googlemail.com wrote: > > No complains from anyone? So we can expect this feature soon? > > We're in a feature freeze. That means you have to convince me that it's > essential to have this for 0.7.0. Hash cash is not a panacea: IMHO it will > have to be combined with CAPTCHAs and web of trust, therefore we will still > need FMS. Because an attacker will probably have a powerful system, whereas a > lot of users don't; an attacker might have native-optimised hashcash > libraries, and a typical user won't, at least in the short term. A user might > not be willing to wait more than 1 minute for a message's hashcash to be > computed, on a slow system; this means an attacker can solve them every 5 > seconds on a faster system (twice the clock speed and four cores). > > Hash cash can be implemented in FMS without any change to the core. And it > should only be necessary for FMS introductions, not later on. Just bolting it > on to Frost would mean a very weak protection, at the cost of a new keytype. > On the other hand it's a nice idea... > > > > > > On Sat, Mar 1, 2008 at 9:05 PM, wrote: > > > This sounds like a good idea! No matter who uses it (fms, frost, thaw) > > > could at least be sure > > > that the sending costed the sender some cpu time. It could also > > > restrict the amount of sent messages > > > at all, considering the same cpu time. So it becomes more expensive > > > and annoying for the spammer. > > > Now its quite easy for him, he could even play crysis in parallell ;) > > > > > > On Sat, Mar 1, 2008 at 8:56 PM, Michael Rogers > wrote: > > > > bbackde at googlemail.com wrote: > > > > > If I understand this correct, the node of the sender will have to > > > > > compute a valid > > > > > hash cash before actually sending the key? And receivers can easily > check the > > > > > hash cash and reject keys without a valid hash cash? > > > > > > > > Right - inserts without valid hash cash won't even be forwarded. > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From m.rogers at cs.ucl.ac.uk Mon Mar 3 16:48:28 2008 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: 03 Mar 2008 16:48:28 +0000 Subject: [Tech] Hash cash keys In-Reply-To: References: <47C98C4B.4090900@cs.ucl.ac.uk> <200803031429.57074.toad@amphibian.dyndns.org> Message-ID: On Mar 3 2008, bbackde at googlemail.com wrote: >That is what I thought. I don't expect to see it in 070, but I think >its great for a later release. Funny, I had the opposite in mind: use it as a short-term solution so we can bundle Frost with 0.7, giving us time to work on long-term solutions like WoT without delaying the release. (Of course hash cash could be one component of a long-term solution too.) Matthew's figures sound about right to me: one minute of hash cash for a typical user translates to five seconds for an attacker, or five minutes for a user with an old machine. That means the attacker can generate 17,280 messages per day, across all boards. When combined with the new features for avoiding jammed boards in Frost, and with the right default settings to hide spam from newbies, that's a tolerable level of spam in my opinion. Better than releasing an unfinished messaging system, anyway. Cheers, Michael From batosai at batosai.net Mon Mar 3 17:20:13 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Mon, 03 Mar 2008 18:20:13 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: <200803031424.32261.toad@amphibian.dyndns.org> References: <47CB1EB5.9030702@batosai.net> <200803031424.32261.toad@amphibian.dyndns.org> Message-ID: <47CC334D.7030307@batosai.net> Matthew Toseland a ?crit : > On Sunday 02 March 2008 21:40, Julien Cornuwel wrote: >> The client decides what to fetch with the informations given by the WoT >> plugin. It then asks it to the FMS plugin and it works as Martin >> described in another post. >> >> The only issue I see here is that we would have to create 2 SSKs per >> user : one for the WoT, one for FMS stuff (messages...). > > No, each user must have entirely separate publications. Otherwise an attacker > can identify that two users share the same plugin / the same node. I don't understand your point. Of course each identity should have its own SSK for publications. What's the problem if the same plugin fetches messages for all its clients ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/c0b02548/attachment.pgp From batosai at batosai.net Mon Mar 3 17:39:55 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Mon, 03 Mar 2008 18:39:55 +0100 Subject: [Tech] WoT plugin Message-ID: <47CC37EB.8010305@batosai.net> Hi, I've been thinking about the discussion initiated by bback (FMS Java design) and wondered if I shouldn't put my actual project on standby and start working on a WoT plugin. It seems to me that a WoT plugin would be more usefull than a filesharing tool... And of course, my first project would benefit of the WoT. So, unless someone with more experience volunteers for it, I think I'll start working on it. About the implementation, I think a relational database would be faster for trust calculation. I see a solution based only on the filesystem but it will use *a lot* of disk space to ensure decent performances. If I use a relational database, my preference goes to derby, which has to be loaded by the node itself. On my own machine, I load it in wrapper.conf. We'll have to find a more elegant solution... Your thoughts ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/dc5cc32c/attachment.pgp From bbackde at googlemail.com Mon Mar 3 17:43:40 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Mon, 3 Mar 2008 18:43:40 +0100 Subject: [Tech] WoT plugin In-Reply-To: <47CC37EB.8010305@batosai.net> References: <47CC37EB.8010305@batosai.net> Message-ID: Try perst. Seagull already made good experiences with the perst performance, as I did in Frost. He works on the Java port of fms. As far as I know there is already some kind of trust system inside the fms code? So lets start using the Java port code and maybe separate it. 2008/3/3 Julien Cornuwel : > Hi, > > > I've been thinking about the discussion initiated by bback (FMS Java > design) and wondered if I shouldn't put my actual project on standby and > start working on a WoT plugin. > > It seems to me that a WoT plugin would be more usefull than a > filesharing tool... And of course, my first project would benefit of the > WoT. > > So, unless someone with more experience volunteers for it, I think I'll > start working on it. > > About the implementation, I think a relational database would be faster > for trust calculation. I see a solution based only on the filesystem but > it will use *a lot* of disk space to ensure decent performances. > > If I use a relational database, my preference goes to derby, which has > to be loaded by the node itself. On my own machine, I load it in > wrapper.conf. We'll have to find a more elegant solution... > > > Your thoughts ? > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From batosai at batosai.net Mon Mar 3 21:18:18 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Mon, 03 Mar 2008 22:18:18 +0100 Subject: [Tech] WoT plugin In-Reply-To: References: <47CC37EB.8010305@batosai.net> Message-ID: <47CC6B1A.9080008@batosai.net> bbackde at googlemail.com a ?crit : > Try perst. Seagull already made good experiences with the perst > performance, as I did in Frost. > He works on the Java port of fms. As far as I know there is already > some kind of trust system inside > the fms code? So lets start using the Java port code and maybe separate it. Mmm, I don't think perst will fit that plugin's needs. Its operation will consist in : - The user sets a trust value for an identity. Get all identities that are trusted by this one and re-calculate*. - We fetched a new trustlist (from a trusted identity), re-calculate* all listed identities. * Get all the identities that trust a particular identity and calculate the resulting trust. If we assume it has a lot of identities to handle, the re-calculation process will happen very often and I can't load every identities in memory and perform selections on them, each time. I need a way to only get those I'm interrested in. I see two solutions : - SQL (easy, but has to be embedded in freenet) - Files (named like the identity they represent), listing the identity they trusts AND the identity that trust them. That would use a lot of disk space but garantees a fast access to the data. I think the SQL is cleaner but am open to suggestions that are going to work with, let's say, 100.000 identities. From bbackde at googlemail.com Mon Mar 3 22:21:15 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Mon, 3 Mar 2008 23:21:15 +0100 Subject: [Tech] WoT plugin In-Reply-To: <47CC6B1A.9080008@batosai.net> References: <47CC37EB.8010305@batosai.net> <47CC6B1A.9080008@batosai.net> Message-ID: I'm not sure if you know perst, it fits the needs you listed. I could give you some first step advise if you want. On Mon, Mar 3, 2008 at 10:18 PM, Julien Cornuwel wrote: > bbackde at googlemail.com a ?crit : > > > Try perst. Seagull already made good experiences with the perst > > performance, as I did in Frost. > > He works on the Java port of fms. As far as I know there is already > > some kind of trust system inside > > the fms code? So lets start using the Java port code and maybe separate it. > > Mmm, I don't think perst will fit that plugin's needs. Its operation > will consist in : > - The user sets a trust value for an identity. Get all identities that > are trusted by this one and re-calculate*. > - We fetched a new trustlist (from a trusted identity), re-calculate* > all listed identities. > > * Get all the identities that trust a particular identity and calculate > the resulting trust. > > If we assume it has a lot of identities to handle, the re-calculation > process will happen very often and I can't load every identities in > memory and perform selections on them, each time. I need a way to only > get those I'm interrested in. > > I see two solutions : > - SQL (easy, but has to be embedded in freenet) > - Files (named like the identity they represent), listing the identity > they trusts AND the identity that trust them. That would use a lot of > disk space but garantees a fast access to the data. > > I think the SQL is cleaner but am open to suggestions that are going to > work with, let's say, 100.000 identities. > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From toad at amphibian.dyndns.org Mon Mar 3 22:31:53 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 22:31:53 +0000 Subject: [Tech] Hash cash keys In-Reply-To: References: <47C98C4B.4090900@cs.ucl.ac.uk> Message-ID: <200803032231.59028.toad@amphibian.dyndns.org> On Monday 03 March 2008 16:48, Michael Rogers wrote: > On Mar 3 2008, bbackde at googlemail.com wrote: > >That is what I thought. I don't expect to see it in 070, but I think > >its great for a later release. > > Funny, I had the opposite in mind: use it as a short-term solution so we > can bundle Frost with 0.7, giving us time to work on long-term solutions > like WoT without delaying the release. (Of course hash cash could be one > component of a long-term solution too.) > > Matthew's figures sound about right to me: one minute of hash cash for a > typical user translates to five seconds for an attacker, or five minutes > for a user with an old machine. However, if we use this with FMS announcements, we can reasonably happily take an hour to generate the hashcash part, because it won't be fully announced for 24 hours anyway. That means much less volume gets through. > That means the attacker can generate 17,280 > messages per day, across all boards. When combined with the new features > for avoiding jammed boards in Frost, and with the right default settings to > hide spam from newbies, that's a tolerable level of spam in my opinion. > Better than releasing an unfinished messaging system, anyway. The current spam is on the order of 30,000 messages per week. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/daf48063/attachment.pgp From toad at amphibian.dyndns.org Mon Mar 3 22:35:52 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 22:35:52 +0000 Subject: [Tech] FMS (Java) design In-Reply-To: <47CC334D.7030307@batosai.net> References: <200803031424.32261.toad@amphibian.dyndns.org> <47CC334D.7030307@batosai.net> Message-ID: <200803032235.55369.toad@amphibian.dyndns.org> On Monday 03 March 2008 17:20, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > On Sunday 02 March 2008 21:40, Julien Cornuwel wrote: > >> The client decides what to fetch with the informations given by the WoT > >> plugin. It then asks it to the FMS plugin and it works as Martin > >> described in another post. > >> > >> The only issue I see here is that we would have to create 2 SSKs per > >> user : one for the WoT, one for FMS stuff (messages...). > > > > No, each user must have entirely separate publications. Otherwise an attacker > > can identify that two users share the same plugin / the same node. > > I don't understand your point. Of course each identity should have its > own SSK for publications. What's the problem if the same plugin fetches > messages for all its clients ? "we would have to create 2 SSKs per user : one for the WoT, one for FMS stuff" I misunderstood this. But it's unnecessary anyway, you can use the same SSK and change the document name. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/6cb94cef/attachment.pgp From toad at amphibian.dyndns.org Mon Mar 3 22:37:10 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 3 Mar 2008 22:37:10 +0000 Subject: [Tech] WoT plugin In-Reply-To: References: <47CC37EB.8010305@batosai.net> Message-ID: <200803032237.12740.toad@amphibian.dyndns.org> On Monday 03 March 2008 17:43, bbackde at googlemail.com wrote: > Try perst. Seagull already made good experiences with the perst > performance, as I did in Frost. > He works on the Java port of fms. As far as I know there is already > some kind of trust system inside > the fms code? So lets start using the Java port code and maybe separate it. Perst sounds good but has whole-database-level locking. That's bad, we probably won't use it for the datastore for that reason. > > 2008/3/3 Julien Cornuwel : > > Hi, > > > > > > I've been thinking about the discussion initiated by bback (FMS Java > > design) and wondered if I shouldn't put my actual project on standby and > > start working on a WoT plugin. > > > > It seems to me that a WoT plugin would be more usefull than a > > filesharing tool... And of course, my first project would benefit of the > > WoT. > > > > So, unless someone with more experience volunteers for it, I think I'll > > start working on it. > > > > About the implementation, I think a relational database would be faster > > for trust calculation. I see a solution based only on the filesystem but > > it will use *a lot* of disk space to ensure decent performances. > > > > If I use a relational database, my preference goes to derby, which has > > to be loaded by the node itself. On my own machine, I load it in > > wrapper.conf. We'll have to find a more elegant solution... > > > > > > Your thoughts ? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080303/2b5dd18f/attachment.pgp From evanbd at gmail.com Tue Mar 4 01:59:42 2008 From: evanbd at gmail.com (Evan Daniel) Date: Mon, 3 Mar 2008 20:59:42 -0500 Subject: [Tech] WoT plugin In-Reply-To: <47CC6B1A.9080008@batosai.net> References: <47CC37EB.8010305@batosai.net> <47CC6B1A.9080008@batosai.net> Message-ID: <4f9383510803031759lc86795fve4d331923fb650bd@mail.gmail.com> On Mon, Mar 3, 2008 at 4:18 PM, Julien Cornuwel wrote: > - The user sets a trust value for an identity. Get all identities that > are trusted by this one and re-calculate*. > - We fetched a new trustlist (from a trusted identity), re-calculate* > all listed identities. > > * Get all the identities that trust a particular identity and calculate > the resulting trust. How do you plan to do trust calculation? The FMS system, as proposed on the freesite when I last looked at it, looks overly simplistic to me. If you haven't already, I highly recommend reading about the Advogato trust metric; it's specifically intended to be resistant to attacks, including spamming. http://www.advogato.org/trust-metric.html (simple version) http://www.levien.com/thesis/compact.pdf (the thesis it's based on) Evan Daniel From batosai at batosai.net Tue Mar 4 17:31:44 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 18:31:44 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: <200803032235.55369.toad@amphibian.dyndns.org> References: <200803031424.32261.toad@amphibian.dyndns.org> <47CC334D.7030307@batosai.net> <200803032235.55369.toad@amphibian.dyndns.org> Message-ID: <47CD8780.6080407@batosai.net> Matthew Toseland a ?crit : > On Monday 03 March 2008 17:20, Julien Cornuwel wrote: >> Matthew Toseland a ?crit : >>> On Sunday 02 March 2008 21:40, Julien Cornuwel wrote: >>>> The client decides what to fetch with the informations given by the WoT >>>> plugin. It then asks it to the FMS plugin and it works as Martin >>>> described in another post. >>>> >>>> The only issue I see here is that we would have to create 2 SSKs per >>>> user : one for the WoT, one for FMS stuff (messages...). >>> No, each user must have entirely separate publications. Otherwise an > attacker >>> can identify that two users share the same plugin / the same node. >> I don't understand your point. Of course each identity should have its >> own SSK for publications. What's the problem if the same plugin fetches >> messages for all its clients ? > > "we would have to create 2 SSKs per user : one for the WoT, one for FMS stuff" > > I misunderstood this. But it's unnecessary anyway, you can use the same SSK > and change the document name. I must have misunderstood something with how SSK works. Do you mean it's possible for the plugin to insert a new edition without knowledge of other files in it (FMS files) ? If so, can you explain how ? I'm interrested. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/0cf490ed/attachment.pgp From batosai at batosai.net Tue Mar 4 17:36:22 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 18:36:22 +0100 Subject: [Tech] WoT plugin In-Reply-To: <4f9383510803031759lc86795fve4d331923fb650bd@mail.gmail.com> References: <47CC37EB.8010305@batosai.net> <47CC6B1A.9080008@batosai.net> <4f9383510803031759lc86795fve4d331923fb650bd@mail.gmail.com> Message-ID: <47CD8896.5010505@batosai.net> Evan Daniel a ?crit : > On Mon, Mar 3, 2008 at 4:18 PM, Julien Cornuwel wrote: > >> - The user sets a trust value for an identity. Get all identities that >> are trusted by this one and re-calculate*. >> - We fetched a new trustlist (from a trusted identity), re-calculate* >> all listed identities. >> >> * Get all the identities that trust a particular identity and calculate >> the resulting trust. > > How do you plan to do trust calculation? As I didn't wrote a single line on it, I'm open to suggestions. The idea is to have a reference implementation of the WoT in an official plugin that can be reviewed by core devs and corrected in case a flaw is discovered in it. > The FMS system, as proposed on the freesite when I last looked at it, > looks overly simplistic to me. > > If you haven't already, I highly recommend reading about the Advogato > trust metric; it's specifically intended to be resistant to attacks, > including spamming. > > http://www.advogato.org/trust-metric.html (simple version) > http://www.levien.com/thesis/compact.pdf (the thesis it's based on) Thanks, I'll read that (the simple one ;) ) > Evan Daniel > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/1b315adb/attachment.pgp From toad at amphibian.dyndns.org Tue Mar 4 17:38:32 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 17:38:32 +0000 Subject: [Tech] FMS (Java) design In-Reply-To: <47CD8780.6080407@batosai.net> References: <200803032235.55369.toad@amphibian.dyndns.org> <47CD8780.6080407@batosai.net> Message-ID: <200803041738.37073.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 17:31, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > On Monday 03 March 2008 17:20, Julien Cornuwel wrote: > >> Matthew Toseland a ?crit : > >>> On Sunday 02 March 2008 21:40, Julien Cornuwel wrote: > >>>> The client decides what to fetch with the informations given by the WoT > >>>> plugin. It then asks it to the FMS plugin and it works as Martin > >>>> described in another post. > >>>> > >>>> The only issue I see here is that we would have to create 2 SSKs per > >>>> user : one for the WoT, one for FMS stuff (messages...). > >>> No, each user must have entirely separate publications. Otherwise an > > attacker > >>> can identify that two users share the same plugin / the same node. > >> I don't understand your point. Of course each identity should have its > >> own SSK for publications. What's the problem if the same plugin fetches > >> messages for all its clients ? > > > > "we would have to create 2 SSKs per user : one for the WoT, one for FMS stuff" > > > > I misunderstood this. But it's unnecessary anyway, you can use the same SSK > > and change the document name. > > I must have misunderstood something with how SSK works. Do you mean it's > possible for the plugin to insert a new edition without knowledge of > other files in it (FMS files) ? If so, can you explain how ? I'm > interrested. I mean you could use the same SSK pubkey. But maybe you could share more than that, we really need to minimise polling here.. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/2ef28988/attachment.pgp From toad at amphibian.dyndns.org Tue Mar 4 17:39:18 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 17:39:18 +0000 Subject: [Tech] WoT plugin In-Reply-To: <47CD8896.5010505@batosai.net> References: <47CC37EB.8010305@batosai.net> <4f9383510803031759lc86795fve4d331923fb650bd@mail.gmail.com> <47CD8896.5010505@batosai.net> Message-ID: <200803041739.18854.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 17:36, Julien Cornuwel wrote: > Evan Daniel a ?crit : > > On Mon, Mar 3, 2008 at 4:18 PM, Julien Cornuwel wrote: > > > >> - The user sets a trust value for an identity. Get all identities that > >> are trusted by this one and re-calculate*. > >> - We fetched a new trustlist (from a trusted identity), re-calculate* > >> all listed identities. > >> > >> * Get all the identities that trust a particular identity and calculate > >> the resulting trust. > > > > How do you plan to do trust calculation? > > As I didn't wrote a single line on it, I'm open to suggestions. The idea > is to have a reference implementation of the WoT in an official plugin > that can be reviewed by core devs and corrected in case a flaw is > discovered in it. And such review isn't going to happen until we have some code to look at! > > > The FMS system, as proposed on the freesite when I last looked at it, > > looks overly simplistic to me. > > > > If you haven't already, I highly recommend reading about the Advogato > > trust metric; it's specifically intended to be resistant to attacks, > > including spamming. > > > > http://www.advogato.org/trust-metric.html (simple version) > > http://www.levien.com/thesis/compact.pdf (the thesis it's based on) > > Thanks, I'll read that (the simple one ;) ) > > > Evan Daniel -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/c1c77dc6/attachment.pgp From batosai at batosai.net Tue Mar 4 17:50:39 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 18:50:39 +0100 Subject: [Tech] WoT plugin In-Reply-To: References: <47CC37EB.8010305@batosai.net> <47CC6B1A.9080008@batosai.net> Message-ID: <47CD8BEF.7040403@batosai.net> bbackde at googlemail.com a ?crit : > I'm not sure if you know perst, it fits the needs you listed. I could > give you some > first step advise if you want. No, I don't really know perst. I've been thinking to it a few hours today (stolen time from my employer ;) ) and I now have a way to simply use the filesystem. My method would be very simple, fast and low ressource consuming. I'll try to write the all idea down by the end of the week so you can all discuss it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/5aaa08ee/attachment.pgp From batosai at batosai.net Tue Mar 4 17:54:19 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 18:54:19 +0100 Subject: [Tech] FMS (Java) design In-Reply-To: <200803041738.37073.toad@amphibian.dyndns.org> References: <200803032235.55369.toad@amphibian.dyndns.org> <47CD8780.6080407@batosai.net> <200803041738.37073.toad@amphibian.dyndns.org> Message-ID: <47CD8CCB.2020309@batosai.net> Matthew Toseland a ?crit : >> I must have misunderstood something with how SSK works. Do you mean it's >> possible for the plugin to insert a new edition without knowledge of >> other files in it (FMS files) ? If so, can you explain how ? I'm >> interrested. > > I mean you could use the same SSK pubkey. But maybe you could share more than > that, we really need to minimise polling here.. Agreed. If it is possible to update a key without loosing the files listed in the previous edition you don't know about, that is the way to go. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/58ef6a25/attachment.pgp From bbackde at googlemail.com Tue Mar 4 18:04:50 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Tue, 4 Mar 2008 19:04:50 +0100 Subject: [Tech] FMS (Java), WoT, ... - Work together! Message-ID: In the current discussions I miss one thing. Everyone wants to start his own thingy. It would be much more effective if all interested parties get together, decide about a design and then implement it _together_! SomeDude wrote fms and got some experience with the web of trust, now we start a WoT from scratch. Seagull already has a working (not completed) version of FMS in Java, but we start something different? Is the FMS port unneeded then? Or should WoT and FMS be competitors? No offense, I just wanted to read your thinkings about this :) I also experienced this with Frost: I am the only true Frost developer for a long time now, and again and again someone came up with its own new tools. I love competition, but as you all know there is alot to do in Frost, and we would have made a much better tool for all users if we would have worked together on Frost. -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From toad at amphibian.dyndns.org Tue Mar 4 18:08:00 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 18:08:00 +0000 Subject: [Tech] FMS (Java) design In-Reply-To: <47CD8CCB.2020309@batosai.net> References: <200803041738.37073.toad@amphibian.dyndns.org> <47CD8CCB.2020309@batosai.net> Message-ID: <200803041808.06501.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 17:54, Julien Cornuwel wrote: > Matthew Toseland a ?crit : > > >> I must have misunderstood something with how SSK works. Do you mean it's > >> possible for the plugin to insert a new edition without knowledge of > >> other files in it (FMS files) ? If so, can you explain how ? I'm > >> interrested. > > > > I mean you could use the same SSK pubkey. But maybe you could share more than > > that, we really need to minimise polling here.. > > Agreed. If it is possible to update a key without loosing the files > listed in the previous edition you don't know about, that is the way to go. An SSK key consists of a public key and a filename. Some SSKs are also manifests, this is denoted by a slash. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/7ed702ab/attachment.pgp From toad at amphibian.dyndns.org Tue Mar 4 18:10:54 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 18:10:54 +0000 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: References: Message-ID: <200803041810.55598.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 18:04, bbackde at googlemail.com wrote: > In the current discussions I miss one thing. Everyone wants to start > his own thingy. > It would be much more effective if all interested parties get > together, decide about a design > and then implement it _together_! SomeDude wrote fms and got some > experience with the > web of trust, now we start a WoT from scratch. Seagull already has a > working (not completed) > version of FMS in Java, but we start something different? It would be great to see some code! > Is the FMS port unneeded then? Or should WoT and FMS be competitors? > > No offense, I just wanted to read your thinkings about this :) > > I also experienced this with Frost: I am the only true Frost developer > for a long time now, > and again and again someone came up with its own new tools. I love > competition, but > as you all know there is alot to do in Frost, and we would have made a > much better tool > for all users if we would have worked together on Frost. Well, with FMS, hopefully there will only be one implementation for the foreseeable future. I get the impression batosai is getting impatient though. I would be if I wasn't barred from working on it. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/7f37fea1/attachment.pgp From bbackde at googlemail.com Tue Mar 4 18:16:56 2008 From: bbackde at googlemail.com (bbackde at googlemail.com) Date: Tue, 4 Mar 2008 19:16:56 +0100 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: <200803041810.55598.toad@amphibian.dyndns.org> References: <200803041810.55598.toad@amphibian.dyndns.org> Message-ID: 2008/3/4 Matthew Toseland : > On Tuesday 04 March 2008 18:04, bbackde at googlemail.com wrote: > > In the current discussions I miss one thing. Everyone wants to start > > his own thingy. > > It would be much more effective if all interested parties get > > together, decide about a design > > and then implement it _together_! SomeDude wrote fms and got some > > experience with the > > web of trust, now we start a WoT from scratch. Seagull already has a > > working (not completed) > > version of FMS in Java, but we start something different? > > It would be great to see some code! > Ack! I am in contact with seagull, I hope we see the code in some SVN soon. > > > Is the FMS port unneeded then? Or should WoT and FMS be competitors? > > > > No offense, I just wanted to read your thinkings about this :) > > > > I also experienced this with Frost: I am the only true Frost developer > > for a long time now, > > and again and again someone came up with its own new tools. I love > > competition, but > > as you all know there is alot to do in Frost, and we would have made a > > much better tool > > for all users if we would have worked together on Frost. > > Well, with FMS, hopefully there will only be one implementation for the > foreseeable future. I get the impression batosai is getting impatient though. > I would be if I wasn't barred from working on it. :) I understand, really. But Julien, do you really want to start with a new WoT design instead of using something which currently works (fms)? We have the C code for this implementation, and its Java port also (soon). Do you think current fms isn't the right thing? > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- __________________________________________________ GnuPG key: (0x48DBFA8A) Keyserver: pgpkeys.pca.dfn.de Fingerprint: 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A __________________________________________________ From batosai at batosai.net Tue Mar 4 18:50:45 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 19:50:45 +0100 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: References: <200803041810.55598.toad@amphibian.dyndns.org> Message-ID: <47CD9A05.3010603@batosai.net> bbackde at googlemail.com a ?crit : > 2008/3/4 Matthew Toseland : >> On Tuesday 04 March 2008 18:04, bbackde at googlemail.com wrote: >> > In the current discussions I miss one thing. Everyone wants to start >> > his own thingy. >> > It would be much more effective if all interested parties get >> > together, decide about a design >> > and then implement it _together_! SomeDude wrote fms and got some >> > experience with the >> > web of trust, now we start a WoT from scratch. Seagull already has a >> > working (not completed) >> > version of FMS in Java, but we start something different? >> >> It would be great to see some code! >> > > Ack! I am in contact with seagull, I hope we see the code in some SVN soon. > >> > Is the FMS port unneeded then? Or should WoT and FMS be competitors? >> > >> > No offense, I just wanted to read your thinkings about this :) >> > >> > I also experienced this with Frost: I am the only true Frost developer >> > for a long time now, >> > and again and again someone came up with its own new tools. I love >> > competition, but >> > as you all know there is alot to do in Frost, and we would have made a >> > much better tool >> > for all users if we would have worked together on Frost. >> >> Well, with FMS, hopefully there will only be one implementation for the >> foreseeable future. I get the impression batosai is getting impatient though. >> I would be if I wasn't barred from working on it. :) > > I understand, really. But Julien, do you really want to start with a > new WoT design instead > of using something which currently works (fms)? We have the C code for > this implementation, > and its Java port also (soon). Do you think current fms isn't the right thing? I've got no problem with FMS and I personnally think that it is the way to go. But I don't think the WoT feature should be a part of it. As I explained before, most of Freenet features will, sooner or later, need a WoT. Making it a plugin accessible through FCP will save some time for other developpments. And all client apps could benefit of a common WoT : if an identity is a valuable contributor to discussion boards, there is a good chance that the files he shares are valuable too. Of course I will re-use a lot of the existing code (why re-invent the wheel). My idea is just to get the WoT feature out of client apps. @Toad : You're right. I'm impatient ;) I'm finally learning Java, and seeking for an idea that could help the project. Maybe I'm wasting my time with that idea of independant plugin and should help FMS' java porter to make his WoT accessible to other apps ? What is your opinion about what I should start with ? If your answer is my original idea of a filesharing tool, note that I'll need a WoT ;) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/467c3751/attachment.pgp From toad at amphibian.dyndns.org Tue Mar 4 18:52:53 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 18:52:53 +0000 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: References: <200803041810.55598.toad@amphibian.dyndns.org> Message-ID: <200803041852.58466.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 18:16, bbackde at googlemail.com wrote: > 2008/3/4 Matthew Toseland : > > On Tuesday 04 March 2008 18:04, bbackde at googlemail.com wrote: > > > In the current discussions I miss one thing. Everyone wants to start > > > his own thingy. > > > It would be much more effective if all interested parties get > > > together, decide about a design > > > and then implement it _together_! SomeDude wrote fms and got some > > > experience with the > > > web of trust, now we start a WoT from scratch. Seagull already has a > > > working (not completed) > > > version of FMS in Java, but we start something different? > > > > It would be great to see some code! > > Ack! I am in contact with seagull, I hope we see the code in some SVN soon. It doesn't even have to be in SVN, if he'd just post tarballs... or longer term publish via pyFreenetHg (I haven't heard much feedback on it, I wonder if there are problems? It's likely the node would be way too big, but I'd have thought it'd be ideal for something relatively small). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/33a6cd8f/attachment.pgp From toad at amphibian.dyndns.org Tue Mar 4 18:55:12 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 18:55:12 +0000 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: <47CD9A05.3010603@batosai.net> References: <47CD9A05.3010603@batosai.net> Message-ID: <200803041855.12536.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 18:50, Julien Cornuwel wrote: > bbackde at googlemail.com a ?crit : > > 2008/3/4 Matthew Toseland : > >> On Tuesday 04 March 2008 18:04, bbackde at googlemail.com wrote: > >> > In the current discussions I miss one thing. Everyone wants to start > >> > his own thingy. > >> > It would be much more effective if all interested parties get > >> > together, decide about a design > >> > and then implement it _together_! SomeDude wrote fms and got some > >> > experience with the > >> > web of trust, now we start a WoT from scratch. Seagull already has a > >> > working (not completed) > >> > version of FMS in Java, but we start something different? > >> > >> It would be great to see some code! > >> > > > > Ack! I am in contact with seagull, I hope we see the code in some SVN soon. > > > >> > Is the FMS port unneeded then? Or should WoT and FMS be competitors? > >> > > >> > No offense, I just wanted to read your thinkings about this :) > >> > > >> > I also experienced this with Frost: I am the only true Frost developer > >> > for a long time now, > >> > and again and again someone came up with its own new tools. I love > >> > competition, but > >> > as you all know there is alot to do in Frost, and we would have made a > >> > much better tool > >> > for all users if we would have worked together on Frost. > >> > >> Well, with FMS, hopefully there will only be one implementation for the > >> foreseeable future. I get the impression batosai is getting impatient though. > >> I would be if I wasn't barred from working on it. :) > > > > I understand, really. But Julien, do you really want to start with a > > new WoT design instead > > of using something which currently works (fms)? We have the C code for > > this implementation, > > and its Java port also (soon). Do you think current fms isn't the right thing? > > I've got no problem with FMS and I personnally think that it is the way > to go. But I don't think the WoT feature should be a part of it. As I > explained before, most of Freenet features will, sooner or later, need a > WoT. Making it a plugin accessible through FCP will save some time for > other developpments. I thought FMS was going to be a plugin accessible through FCP? > > And all client apps could benefit of a common WoT : if an identity is a > valuable contributor to discussion boards, there is a good chance that > the files he shares are valuable too. > > Of course I will re-use a lot of the existing code (why re-invent the > wheel). My idea is just to get the WoT feature out of client apps. > > > @Toad : You're right. I'm impatient ;) I'm finally learning Java, and > seeking for an idea that could help the project. Maybe I'm wasting my > time with that idea of independant plugin and should help FMS' java > porter to make his WoT accessible to other apps ? What is your opinion > about what I should start with ? If your answer is my original idea of a > filesharing tool, note that I'll need a WoT ;) :) Well there's not much you can do without the code. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/7e420385/attachment.pgp From batosai at batosai.net Tue Mar 4 18:55:28 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 19:55:28 +0100 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: <47CD9A05.3010603@batosai.net> References: <200803041810.55598.toad@amphibian.dyndns.org> <47CD9A05.3010603@batosai.net> Message-ID: <47CD9B20.9040108@batosai.net> Julien Cornuwel a ?crit : > I've got no problem with FMS and I personnally think that it is the way > to go. Correction : I've got a problem with FMS' choice of NNTP interface. But I like the design. I even stole a lot of it in my filesharing project ;) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/ce846048/attachment.pgp From batosai at batosai.net Tue Mar 4 19:04:15 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Tue, 04 Mar 2008 20:04:15 +0100 Subject: [Tech] FMS (Java), WoT, ... - Work together! In-Reply-To: <200803041855.12536.toad@amphibian.dyndns.org> References: <47CD9A05.3010603@batosai.net> <200803041855.12536.toad@amphibian.dyndns.org> Message-ID: <47CD9D2F.7070504@batosai.net> Matthew Toseland a ?crit : >> I've got no problem with FMS and I personnally think that it is the way >> to go. But I don't think the WoT feature should be a part of it. As I >> explained before, most of Freenet features will, sooner or later, need a >> WoT. Making it a plugin accessible through FCP will save some time for >> other developpments. > > I thought FMS was going to be a plugin accessible through FCP? I really have to see that code ! I'll try to contact seagull to see if it is possible to work together. >> @Toad : You're right. I'm impatient ;) I'm finally learning Java, and >> seeking for an idea that could help the project. Maybe I'm wasting my >> time with that idea of independant plugin and should help FMS' java >> porter to make his WoT accessible to other apps ? What is your opinion >> about what I should start with ? If your answer is my original idea of a >> filesharing tool, note that I'll need a WoT ;) > > :) > > Well there's not much you can do without the code. AGREED ! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/cbee55be/attachment.pgp From stwa4647000 at yahoo.co.uk Tue Mar 4 21:27:51 2008 From: stwa4647000 at yahoo.co.uk (Stephen Walford) Date: Tue, 4 Mar 2008 21:27:51 +0000 (GMT) Subject: [Tech] Outside web access? In-Reply-To: <47C67387.6020405@home.se> Message-ID: <706559.79185.qm@web25410.mail.ukl.yahoo.com> An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/tech/attachments/20080304/967c20e2/attachment.htm From toad at amphibian.dyndns.org Tue Mar 4 21:46:08 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 4 Mar 2008 21:46:08 +0000 Subject: [Tech] Outside web access? In-Reply-To: <706559.79185.qm@web25410.mail.ukl.yahoo.com> References: <706559.79185.qm@web25410.mail.ukl.yahoo.com> Message-ID: <200803042146.18456.toad@amphibian.dyndns.org> On Tuesday 04 March 2008 21:27, Stephen Walford wrote: > --- On Thu, 28/2/08, John B?ckstrand wrote: > > see some sensitive information. Hey, I should open up a http proxy for > freenet, and then manually add filters when government agencies start > complaining. The filtered pages would simply say: "please install > freenet to see this page, since the legal system has asked me not to > show this". The hope is that by moderating things manually, I would not > be prosecuted for anything (hopefully) and people will at least get a > chance to see information before it is banned. It doesn't work that way. They'll sue you for contributory copyright infringement and you'll settle out of court, not being able to afford a good lawyer. Oh and they'll serve *your ISP* with a takedown notice for the entire site. Not you for the one infringing page. Oh and then they'll contact the local paper about your child porn distribution... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080304/50387d37/attachment.pgp From alejandro at mosteo.com Wed Mar 5 14:15:53 2008 From: alejandro at mosteo.com (Jano) Date: Wed, 05 Mar 2008 15:15:53 +0100 Subject: [Tech] WoT plugin References: <47CC37EB.8010305@batosai.net> Message-ID: bbackde at googlemail.com wrote: > Try perst. Seagull already made good experiences with the perst Is perst relational? My impression is that's a key/object based database. >From their introduction page: "What Perst Is Not Perst is not an enterprise-class database management system, or an object-relational mapping tool. Perst is not a client/server architecture database system and cannot support multiple concurrent processes sharing a common database. (Multiple threads within a single process are supported.) Perst does not have a high level query language, such as SQL." From jUrner at arcor.de Thu Mar 6 09:36:47 2008 From: jUrner at arcor.de (juergen urner) Date: Thu, 06 Mar 2008 10:36:47 +0100 Subject: [Tech] Fcp woes Message-ID: <47CFBB2F.7040504@arcor.de> Help! My Fcp wrapper drowns in if ..else cases. It is a bit unclear from the wiki when the message terminator of a message is "EndData" and when it is "Data". My current headscratcher FCPPluginMessage. I hope everyone out there uses "Data" as terminator in case data is attatched. Would be way easier if "DataLength" was reserved to indicate attatched data and "Data" as terminator would go away, but DataFound gets in the way here. If.. else.. elif... Juergen From jUrner at arcor.de Thu Mar 6 10:51:37 2008 From: jUrner at arcor.de (juergen urner) Date: Thu, 06 Mar 2008 11:51:37 +0100 Subject: [Tech] Fcp woes In-Reply-To: <47CFBB2F.7040504@arcor.de> References: <47CFBB2F.7040504@arcor.de> Message-ID: <47CFCCB9.4050102@arcor.de> juergen urner schrieb: > Help! My Fcp wrapper drowns in if ..else cases. > > > It is a bit unclear from the wiki when the message terminator of > a message is "EndData" and when it is "Data". My current headscratcher > FCPPluginMessage. I hope everyone out there uses "Data" as terminator > in case data is attatched. > > Would be way easier if "DataLength" was reserved to indicate attatched > data and "Data" as terminator would go away, but DataFound gets in the > way here. > If.. else.. elif... > > Juergen > Correction, I meant FCPPluginReply not FCPPluginMessage J. From toad at amphibian.dyndns.org Thu Mar 6 11:51:08 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 6 Mar 2008 11:51:08 +0000 Subject: [Tech] Fcp woes In-Reply-To: <47CFBB2F.7040504@arcor.de> References: <47CFBB2F.7040504@arcor.de> Message-ID: <200803061151.09934.toad@amphibian.dyndns.org> On Thursday 06 March 2008 09:36, juergen urner wrote: > Help! My Fcp wrapper drowns in if ..else cases. > > > It is a bit unclear from the wiki when the message terminator of > a message is "EndData" and when it is "Data". My current headscratcher > FCPPluginMessage. I hope everyone out there uses "Data" as terminator > in case data is attatched. > > Would be way easier if "DataLength" was reserved to indicate attatched > data and "Data" as terminator would go away, but DataFound gets in the > way here. > If.. else.. elif... I don't see that it matters. Any line that doesn't start or end a message will have an '=' in it. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080306/2f804b84/attachment.pgp From jUrner at arcor.de Thu Mar 6 12:20:16 2008 From: jUrner at arcor.de (juergen urner) Date: Thu, 06 Mar 2008 13:20:16 +0100 Subject: [Tech] Fcp woes In-Reply-To: <200803061151.09934.toad@amphibian.dyndns.org> References: <47CFBB2F.7040504@arcor.de> <200803061151.09934.toad@amphibian.dyndns.org> Message-ID: <47CFE180.7060500@arcor.de> Matthew Toseland schrieb: > On Thursday 06 March 2008 09:36, juergen urner wrote: > >> Help! My Fcp wrapper drowns in if ..else cases. >> >> >> It is a bit unclear from the wiki when the message terminator of >> a message is "EndData" and when it is "Data". My current headscratcher >> FCPPluginMessage. I hope everyone out there uses "Data" as terminator >> in case data is attatched. >> >> Would be way easier if "DataLength" was reserved to indicate attatched >> data and "Data" as terminator would go away, but DataFound gets in the >> way here. >> If.. else.. elif... >> > > I don't see that it matters. Any line that doesn't start or end a message will > have an '=' in it. > Java coding style? From jUrner at arcor.de Sun Mar 9 15:25:07 2008 From: jUrner at arcor.de (juergen urner) Date: Sun, 09 Mar 2008 16:25:07 +0100 Subject: [Tech] Fcp woes II Message-ID: <47D40153.7050800@arcor.de> There is no distinction between ClientPutDiskDir and ClientPutComplexDir in PersistentPut*. This makes it hard to tell both apart. I need this to do automatic type conversions. Just thinking aloud ..I wish ClientPut* (PersistentPut*) would go. Afaics a new message would make live much easier on both sides. Any thoughts? Put Identifer=any NItems=N Persistence=whatever (...) EndMessage ...emidiately followed by N items to put DataItem DataLength=N Name=any (...) EndMessage FileItem Filename=filename Name=any (...) EndMessage (...) Btw, on input Fcp does not seem to care if Files.N.* in ClientPutComplexDir start at 0. This broke persistents. Juergen From toad at amphibian.dyndns.org Mon Mar 10 20:21:08 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Mon, 10 Mar 2008 20:21:08 +0000 Subject: [Tech] Fcp woes II In-Reply-To: <47D40153.7050800@arcor.de> References: <47D40153.7050800@arcor.de> Message-ID: <200803102021.19623.toad@amphibian.dyndns.org> On Sunday 09 March 2008 15:25, juergen urner wrote: > There is no distinction between ClientPutDiskDir and ClientPutComplexDir > in PersistentPut*. This makes it hard to tell both apart. I need this to > do automatic > type conversions. We can make a distinction if we need to, however there really isn't much difference, ClientPutDiskDir is just a simple way to access the same functionality, any ClientPutDiskDir can be expressed as a ClientPutComplexDir if need be. > > Just thinking aloud ..I wish ClientPut* (PersistentPut*) would go. > Afaics a new message > would make live much easier on both sides. Any thoughts? > > Put > Identifer=any > NItems=N > Persistence=whatever > (...) > EndMessage > > ...emidiately followed by N items to put > > DataItem > DataLength=N > Name=any > (...) > EndMessage > FileItem > Filename=filename > Name=any > (...) > EndMessage > (...) Why is this better? > > Btw, on input Fcp does not seem to care if Files.N.* in ClientPutComplexDir > start at 0. This broke persistents. You mean it expects it to start at 0? Or what? > > Juergen -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080310/a3b5e350/attachment.pgp From jUrner at arcor.de Mon Mar 10 23:48:11 2008 From: jUrner at arcor.de (juergen urner) Date: Tue, 11 Mar 2008 00:48:11 +0100 Subject: [Tech] Fcp woes II In-Reply-To: <200803102021.19623.toad@amphibian.dyndns.org> References: <47D40153.7050800@arcor.de> <200803102021.19623.toad@amphibian.dyndns.org> Message-ID: <47D5C8BB.3030803@arcor.de> Matthew Toseland schrieb: > On Sunday 09 March 2008 15:25, juergen urner wrote: > >> There is no distinction between ClientPutDiskDir and ClientPutComplexDir >> in PersistentPut*. This makes it hard to tell both apart. I need this to >> do automatic >> type conversions. >> > > We can make a distinction if we need to, however there really isn't much > difference, ClientPutDiskDir is just a simple way to access the same > functionality, any ClientPutDiskDir can be expressed as a ClientPutComplexDir > if need be. > Yes, I noticed that. And I noticed that 'Filename' is not present in PeristentPutDir when I pass ClientPutComplexDir. just a bit of ugly parsing to find out 'Filename' I'd like to avoid. So, at least passing 'Filename' as indicator would be helpful. >> Just thinking aloud ..I wish ClientPut* (PersistentPut*) would go. >> Afaics a new message >> would make live much easier on both sides. Any thoughts? >> >> Put >> Identifer=any >> NItems=N >> Persistence=whatever >> (...) >> EndMessage >> >> ...emidiately followed by N items to put >> >> DataItem >> DataLength=N >> Name=any >> (...) >> EndMessage >> FileItem >> Filename=filename >> Name=any >> (...) >> EndMessage >> (...) >> > > Why is this better? > Put* is somewhat overcomplicated. I am already scared of future extensions to it. As I see it, all that is needed is is to break the container into its peaces to make a nice and clean api for both sides on only one message and one page in the wiki. >> Btw, on input Fcp does not seem to care if Files.N.* in ClientPutComplexDir >> start at 0. This broke persistents. >> > > You mean it expects it to start at 0? Or what? > Yes something breaks. Too lazy to run more tests, but NodeHello does not arrive anymore. Btw, I ran a test throwing 'Plum' as message terminator at the node. >>> If '=' not in chunk: >>> endOfMessageEncountered() Fcp doesn' t care at all? From toad at amphibian.dyndns.org Tue Mar 11 11:40:07 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Mar 2008 11:40:07 +0000 Subject: [Tech] Fcp woes II In-Reply-To: <47D5C8BB.3030803@arcor.de> References: <47D40153.7050800@arcor.de> <200803102021.19623.toad@amphibian.dyndns.org> <47D5C8BB.3030803@arcor.de> Message-ID: <200803111140.07644.toad@amphibian.dyndns.org> On Monday 10 March 2008 23:48, juergen urner wrote: > Matthew Toseland schrieb: > > On Sunday 09 March 2008 15:25, juergen urner wrote: > > > >> There is no distinction between ClientPutDiskDir and ClientPutComplexDir > >> in PersistentPut*. This makes it hard to tell both apart. I need this to > >> do automatic > >> type conversions. > >> > > > > We can make a distinction if we need to, however there really isn't much > > difference, ClientPutDiskDir is just a simple way to access the same > > functionality, any ClientPutDiskDir can be expressed as a ClientPutComplexDir > > if need be. > > Yes, I noticed that. And I noticed that 'Filename' is not present in > PeristentPutDir > when I pass ClientPutComplexDir. just a bit of ugly parsing to find out > 'Filename' > I'd like to avoid. So, at least passing 'Filename' as indicator would be > helpful. It's present for each file, but there is no overall Filename because a ClientPutComplexDir can put files from anywhere. > > >> Just thinking aloud ..I wish ClientPut* (PersistentPut*) would go. > >> Afaics a new message > >> would make live much easier on both sides. Any thoughts? > >> > >> Put > >> Identifer=any > >> NItems=N > >> Persistence=whatever > >> (...) > >> EndMessage > >> > >> ...emidiately followed by N items to put > >> > >> DataItem > >> DataLength=N > >> Name=any > >> (...) > >> EndMessage > >> FileItem > >> Filename=filename > >> Name=any > >> (...) > >> EndMessage > >> (...) > >> > > > > Why is this better? > > Put* is somewhat overcomplicated. I am already scared of future > extensions to it. The example given on the wiki has only the minimum info needed for each file: ----- ClientPutComplexDir Identifier=My Identifier Verbosity=1023 MaxRetries=999 PriorityClass=2 URI=SSK at Fk6sQ6...../myinsert-4/ GetCHKOnly=false DontCompress=true ClientToken=My Client Token Persistence=reboot Global=true DefaultName=hello.txt Files.0.Name=hello.txt Files.0.UploadFrom=direct Files.0.Metadata.ContentType=text/plain Files.0.DataLength=59 Files.1.Name=something.pdf Files.1.UploadFrom=disk Files.1.Filename=something.pdf Files.2.Name=gpl.txt Files.2.UploadFrom=redirect Files.2.TargetURI=KSK at sample.txt EndMessage hello, this is the contents of the file called "hello.txt" ----- > > As I see it, all that is needed is is to break the container into its > peaces to > make a nice and clean api for both sides on only one message and one page > in the wiki. AFAICS the only difficult thing about the above is that you have to prefix each file with , rather than sending them as separate messages... what's the big deal? > > >> Btw, on input Fcp does not seem to care if Files.N.* in ClientPutComplexDir > >> start at 0. This broke persistents. > > > > You mean it expects it to start at 0? Or what? > > Yes something breaks. Too lazy to run more tests, but NodeHello does > not arrive anymore. Look for a NullPointerException in the logs. > > Btw, I ran a test throwing 'Plum' as message terminator at the node. > > >>> If '=' not in chunk: > >>> endOfMessageEncountered() > > Fcp doesn' t care at all? Indeed. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080311/e0da1e2d/attachment.pgp From jUrner at arcor.de Tue Mar 11 12:38:48 2008 From: jUrner at arcor.de (juergen urner) Date: Tue, 11 Mar 2008 13:38:48 +0100 Subject: [Tech] Fcp woes II In-Reply-To: <200803111140.07644.toad@amphibian.dyndns.org> References: <47D40153.7050800@arcor.de> <200803102021.19623.toad@amphibian.dyndns.org> <47D5C8BB.3030803@arcor.de> <200803111140.07644.toad@amphibian.dyndns.org> Message-ID: <47D67D58.6030605@arcor.de> Matthew Toseland schrieb: > On Monday 10 March 2008 23:48, juergen urner wrote: > >> Matthew Toseland schrieb: >> >> Yes, I noticed that. And I noticed that 'Filename' is not present in >> PeristentPutDir >> when I pass ClientPutComplexDir. just a bit of ugly parsing to find out >> 'Filename' >> I'd like to avoid. So, at least passing 'Filename' as indicator would be >> helpful. >> > > It's present for each file, but there is no overall Filename because a > ClientPutComplexDir can put files from anywhere. > ClientPutDiskDir can not. Thinking it over it is impossible to tell if a PeristentPutDir results from a complex or a diskdir. I have to tell my user, sorry, I don't know what your initial request was. I throw the put directory mask at you, right? Yes | No | Cancel >> Put* is somewhat overcomplicated. I am already scared of future >> extensions to it. >> > > The example given on the wiki has only the minimum info needed for each file: > ----- > ClientPutComplexDir > Identifier=My Identifier > Verbosity=1023 > MaxRetries=999 > PriorityClass=2 > URI=SSK at Fk6sQ6...../myinsert-4/ > GetCHKOnly=false > DontCompress=true > ClientToken=My Client Token > Persistence=reboot > Global=true > DefaultName=hello.txt > Files.0.Name=hello.txt > Files.0.UploadFrom=direct > Files.0.Metadata.ContentType=text/plain > Files.0.DataLength=59 > Files.1.Name=something.pdf > Files.1.UploadFrom=disk > Files.1.Filename=something.pdf > Files.2.Name=gpl.txt > Files.2.UploadFrom=redirect > Files.2.TargetURI=KSK at sample.txt > EndMessage > hello, this is the contents > of the file called "hello.txt" > ----- > > AFAICS the only difficult thing about the above is that you have to prefix > each file with , rather than sending them as separate messages... > what's the big deal? > 1. what is nice and clean about it? 2. parsing - instead of running a loop for items in container one has to gather items by hand. if ..else ..elif, checkIndexIntergrity() 3. there may be many* items. In future extensions it may be possible to pass items one by one and query them on demand. 4. readable docs? Put: ------------------ Requests the node to upload one or more items Put params here Plum Items have to follow the Put message emidiately. Items can be one or more of the following: DataItem --------------------- Uploads raw data DataItem params here Plum Attatched data has to follow emidiately the newline after the item terminator (...) >> Btw, I ran a test throwing 'Plum' as message terminator at the node. >> >> >>> If '=' not in chunk: >> >>> endOfMessageEncountered() >> >> Fcp doesn' t care at all? >> > > Indeed. > What thought am I missing? :-) From toad at amphibian.dyndns.org Tue Mar 11 14:08:16 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Mar 2008 14:08:16 +0000 Subject: [Tech] Fcp woes II In-Reply-To: <47D67D58.6030605@arcor.de> References: <47D40153.7050800@arcor.de> <200803111140.07644.toad@amphibian.dyndns.org> <47D67D58.6030605@arcor.de> Message-ID: <200803111408.22041.toad@amphibian.dyndns.org> On Tuesday 11 March 2008 12:38, juergen urner wrote: > Matthew Toseland schrieb: > > On Monday 10 March 2008 23:48, juergen urner wrote: > > > >> Matthew Toseland schrieb: > >> > >> Yes, I noticed that. And I noticed that 'Filename' is not present in > >> PeristentPutDir > >> when I pass ClientPutComplexDir. just a bit of ugly parsing to find out > >> 'Filename' > >> I'd like to avoid. So, at least passing 'Filename' as indicator would be > >> helpful. > > > > It's present for each file, but there is no overall Filename because a > > ClientPutComplexDir can put files from anywhere. > > ClientPutDiskDir can not. Thinking it over it is impossible > to tell if a PeristentPutDir results from a complex or a diskdir. There is no easy and reliable way to tell yes. > > I have to tell my user, sorry, I don't know what your initial > request was. I throw the put directory mask at you, right? Why do you need to tell your user? I assume you need to be compatible with PutDiskDir's sent by other apps than your own? > > Yes | No | Cancel > > >> Put* is somewhat overcomplicated. I am already scared of future > >> extensions to it. > > > > The example given on the wiki has only the minimum info needed for each file: > > ----- > > ClientPutComplexDir > > Identifier=My Identifier > > Verbosity=1023 > > MaxRetries=999 > > PriorityClass=2 > > URI=SSK at Fk6sQ6...../myinsert-4/ > > GetCHKOnly=false > > DontCompress=true > > ClientToken=My Client Token > > Persistence=reboot > > Global=true > > DefaultName=hello.txt > > Files.0.Name=hello.txt > > Files.0.UploadFrom=direct > > Files.0.Metadata.ContentType=text/plain > > Files.0.DataLength=59 > > Files.1.Name=something.pdf > > Files.1.UploadFrom=disk > > Files.1.Filename=something.pdf > > Files.2.Name=gpl.txt > > Files.2.UploadFrom=redirect > > Files.2.TargetURI=KSK at sample.txt > > EndMessage > > hello, this is the contents > > of the file called "hello.txt" > > ----- > > > > AFAICS the only difficult thing about the above is that you have to prefix > > each file with , rather than sending them as separate messages... > > what's the big deal? > > 1. what is nice and clean about it? > 2. parsing - instead of running a loop for items in container one has to > gather > items by hand. if ..else ..elif, checkIndexIntergrity() Okay, I see. Because you don't turn the message into a tree first (using SimpleFieldSet), as the node does, but parse it directly, the current format is harder to deal with than the proposed format. This may be an argument to change PersistentPutDir. Would it help much to add a Count before the Files list? I'm curious what you are trying to do that needs to parse a PersistentPutDir ? > 3. there may be many* items. In future extensions it may be possible to pass > items one by one and query them on demand. I don't see how that would help, surely it would only complicate matters? > 4. readable docs? What's wrong with the current documentation? > > > Put: > ------------------ > Requests the node to upload one or more items > > Put > params here > Plum > > Items have to follow the Put message emidiately. No they don't. FCPv2 is designed to be multiplexable. Therefore the items will have to have an identifier referring back to the PutDir. > Items can be one or more of the following: > > DataItem > --------------------- > Uploads raw data > > DataItem > params here > Plum > > Attatched data has to follow emidiately the newline after the item > terminator Of course. And a data-item-with-data, as opposed to a data-item-referring-to-a-file-on-disk, will need to have a DataLength. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080311/2e18617b/attachment.pgp From jUrner at arcor.de Tue Mar 11 15:52:10 2008 From: jUrner at arcor.de (juergen urner) Date: Tue, 11 Mar 2008 16:52:10 +0100 Subject: [Tech] Fcp woes II In-Reply-To: <200803111408.22041.toad@amphibian.dyndns.org> References: <47D40153.7050800@arcor.de> <200803111140.07644.toad@amphibian.dyndns.org> <47D67D58.6030605@arcor.de> <200803111408.22041.toad@amphibian.dyndns.org> Message-ID: <47D6AAAA.5000103@arcor.de> Matthew Toseland schrieb: >> ClientPutDiskDir can not. Thinking it over it is impossible >> to tell if a PeristentPutDir results from a complex or a diskdir. >> > > There is no easy and reliable way to tell yes. > >> I have to tell my user, sorry, I don't know what your initial >> request was. I throw the put directory mask at you, right? >> > > Why do you need to tell your user? I assume you need to be compatible with > PutDiskDir's sent by other apps than your own? > No, no other apps. User requests upload of a directory, later he wants to send the request again. That is the directory, not the files listed. No problem attatching the directory as part of a struct in ClientToken, but I rather raise for 'Filename' being passed. >>>> Put* is somewhat overcomplicated. I am already scared of future >>>> extensions to it. >>>> >>> The example given on the wiki has only the minimum info needed for each >>> > file: > >>> ----- >>> ClientPutComplexDir >>> Identifier=My Identifier >>> Verbosity=1023 >>> MaxRetries=999 >>> PriorityClass=2 >>> URI=SSK at Fk6sQ6...../myinsert-4/ >>> GetCHKOnly=false >>> DontCompress=true >>> ClientToken=My Client Token >>> Persistence=reboot >>> Global=true >>> DefaultName=hello.txt >>> Files.0.Name=hello.txt >>> Files.0.UploadFrom=direct >>> Files.0.Metadata.ContentType=text/plain >>> Files.0.DataLength=59 >>> Files.1.Name=something.pdf >>> Files.1.UploadFrom=disk >>> Files.1.Filename=something.pdf >>> Files.2.Name=gpl.txt >>> Files.2.UploadFrom=redirect >>> Files.2.TargetURI=KSK at sample.txt >>> EndMessage >>> hello, this is the contents >>> of the file called "hello.txt" >>> ----- >>> >>> AFAICS the only difficult thing about the above is that you have to prefix >>> each file with , rather than sending them as separate >>> > messages... > >>> what's the big deal? >>> >> 1. what is nice and clean about it? >> 2. parsing - instead of running a loop for items in container one has to >> gather >> items by hand. if ..else ..elif, checkIndexIntergrity() >> > > Okay, I see. Because you don't turn the message into a tree first (using > SimpleFieldSet), as the node does, but parse it directly, the current format > is harder to deal with than the proposed format. This may be an argument to > change PersistentPutDir. > > Would it help much to add a Count before the Files list? > No, not much. > I'm curious what you are trying to do that needs to parse a PersistentPutDir ? > The need originated from building test cases for my Fcp client. Later I stumbled over the case where a user uploads a ComplexDir. On next connect I have to parse PeristentPutDir in order to present a graphical mask containg details to the user. >> 3. there may be many* items. In future extensions it may be possible to pass >> items one by one and query them on demand. >> > > I don't see how that would help, surely it would only complicate matters? > Maybe you are right. But what I would like to see is an api for querying ConfigData and persistents on demand. So, for uniformity reasons some (uh, how do you say) standard for container processing would be nice. >> 4. readable docs? >> > > What's wrong with the current documentation? > Nothing. But (hope this does not sound impolite) I always assume that if the documentation is hard to follow, my api is in need of a redesign. >> Put: >> ------------------ >> Requests the node to upload one or more items >> >> Put >> params here >> Plum >> >> Items have to follow the Put message emidiately. >> > > No they don't. FCPv2 is designed to be multiplexable. Therefore the items will > have to have an identifier referring back to the PutDir. > For sanity reasons, I hope Identifier=ParentRequestIdentifier is enough. >> Items can be one or more of the following: >> >> DataItem >> --------------------- >> Uploads raw data >> >> DataItem >> params here >> Plum >> >> Attatched data has to follow emidiately the newline after the item >> terminator >> > > Of course. And a data-item-with-data, as opposed to a > data-item-referring-to-a-file-on-disk, will need to have a DataLength. > Yes. And DataLength is reserved as message metadata field. Or something like 'NBytes' if you like it more formal and backwards compatible. From toad at amphibian.dyndns.org Tue Mar 11 18:09:59 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 11 Mar 2008 18:09:59 +0000 Subject: [Tech] Fcp woes II In-Reply-To: <47D6AAAA.5000103@arcor.de> References: <47D40153.7050800@arcor.de> <200803111408.22041.toad@amphibian.dyndns.org> <47D6AAAA.5000103@arcor.de> Message-ID: <200803111809.59531.toad@amphibian.dyndns.org> I have added a bug. It is unlikely that it will happen before 0.7.0. https://bugs.freenetproject.org/view.php?id=2159 On Tuesday 11 March 2008 15:52, juergen urner wrote: > Matthew Toseland schrieb: > > >> ClientPutDiskDir can not. Thinking it over it is impossible > >> to tell if a PeristentPutDir results from a complex or a diskdir. > >> > > > > There is no easy and reliable way to tell yes. > > > >> I have to tell my user, sorry, I don't know what your initial > >> request was. I throw the put directory mask at you, right? > >> > > > > Why do you need to tell your user? I assume you need to be compatible with > > PutDiskDir's sent by other apps than your own? > > > > No, no other apps. User requests upload of a directory, later he wants to > send the request again. That is the directory, not the files listed. > > No problem attatching the directory as part of a struct in ClientToken, > but I rather > raise for 'Filename' being passed. > > >>>> Put* is somewhat overcomplicated. I am already scared of future > >>>> extensions to it. > >>>> > >>> The example given on the wiki has only the minimum info needed for each > >>> > > file: > > > >>> ----- > >>> ClientPutComplexDir > >>> Identifier=My Identifier > >>> Verbosity=1023 > >>> MaxRetries=999 > >>> PriorityClass=2 > >>> URI=SSK at Fk6sQ6...../myinsert-4/ > >>> GetCHKOnly=false > >>> DontCompress=true > >>> ClientToken=My Client Token > >>> Persistence=reboot > >>> Global=true > >>> DefaultName=hello.txt > >>> Files.0.Name=hello.txt > >>> Files.0.UploadFrom=direct > >>> Files.0.Metadata.ContentType=text/plain > >>> Files.0.DataLength=59 > >>> Files.1.Name=something.pdf > >>> Files.1.UploadFrom=disk > >>> Files.1.Filename=something.pdf > >>> Files.2.Name=gpl.txt > >>> Files.2.UploadFrom=redirect > >>> Files.2.TargetURI=KSK at sample.txt > >>> EndMessage > >>> hello, this is the contents > >>> of the file called "hello.txt" > >>> ----- > >>> > >>> AFAICS the only difficult thing about the above is that you have to prefix > >>> each file with , rather than sending them as separate > >>> > > messages... > > > >>> what's the big deal? > >>> > >> 1. what is nice and clean about it? > >> 2. parsing - instead of running a loop for items in container one has to > >> gather > >> items by hand. if ..else ..elif, checkIndexIntergrity() > >> > > > > Okay, I see. Because you don't turn the message into a tree first (using > > SimpleFieldSet), as the node does, but parse it directly, the current format > > is harder to deal with than the proposed format. This may be an argument to > > change PersistentPutDir. > > > > Would it help much to add a Count before the Files list? > > > > No, not much. > > > I'm curious what you are trying to do that needs to parse a PersistentPutDir ? > > > > The need originated from building test cases for my Fcp client. Later I > stumbled > over the case where a user uploads a ComplexDir. On next connect I have > to parse > PeristentPutDir in order to present a graphical mask containg details to > the user. > > >> 3. there may be many* items. In future extensions it may be possible to pass > >> items one by one and query them on demand. > >> > > > > I don't see how that would help, surely it would only complicate matters? > > > > Maybe you are right. But what I would like to see is an api for querying > ConfigData and persistents on demand. So, for uniformity reasons > some (uh, how do you say) standard for container processing would be nice. > > >> 4. readable docs? > >> > > > > What's wrong with the current documentation? > > > > Nothing. But (hope this does not sound impolite) I always assume that if > the > documentation is hard to follow, my api is in need of a redesign. > > >> Put: > >> ------------------ > >> Requests the node to upload one or more items > >> > >> Put > >> params here > >> Plum > >> > >> Items have to follow the Put message emidiately. > >> > > > > No they don't. FCPv2 is designed to be multiplexable. Therefore the items will > > have to have an identifier referring back to the PutDir. > > > > For sanity reasons, I hope Identifier=ParentRequestIdentifier is enough. > > >> Items can be one or more of the following: > >> > >> DataItem > >> --------------------- > >> Uploads raw data > >> > >> DataItem > >> params here > >> Plum > >> > >> Attatched data has to follow emidiately the newline after the item > >> terminator > >> > > > > Of course. And a data-item-with-data, as opposed to a > > data-item-referring-to-a-file-on-disk, will need to have a DataLength. > > > > Yes. And DataLength is reserved as message metadata field. Or something > like 'NBytes' if you like it more formal and backwards compatible. > > > > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080311/420fc569/attachment.pgp From jUrner at arcor.de Thu Mar 13 12:00:59 2008 From: jUrner at arcor.de (juergen urner) Date: Thu, 13 Mar 2008 13:00:59 +0100 Subject: [Tech] wiki question Message-ID: <47D9177B.80701@arcor.de> Maybe off topic here ...does the wiki support automatic backlinks? Something like a page name "Fcp20/Foo/Bar"" gives a nice chain of backlinks in the header? Could not find anything in the docs. Juergen From jUrner at arcor.de Wed Mar 19 14:27:37 2008 From: jUrner at arcor.de (juergen urner) Date: Wed, 19 Mar 2008 15:27:37 +0100 Subject: [Tech] Fcp woes ..final Message-ID: <47E122D9.9010507@arcor.de> Just dropping by to tell, that I finally resign on writing a highlevel Python wrapper for the protocol. There is still too many show stoppers in there to write it nice and clean. I'd like to give some feedback on what made my live hard. Maybe s.o. can make use of it. In no particular order.... ------------------------------------------------------------------------------------------------------------- [Global queue and persistence] Already reported. Maybe one last note regarding this. If apps feel the need to share data it should be up to them to organize this. [Persistent requests vs. non persistents] x. one may want to modify or remove non persistent requests aswell this should eliminate all the case switches if Persistence==connection else... x. there should be an explicite way to stop, resume all kinds of requests [DataFound vs. AllData] Another case switch that looks rather unnecessary. AllData is not send for persistent requests. Looks loke a very special case to me, maybe some Frost. Imo, if there is a need to split data fetching and whatever-client-wants-to-do-with-data apart, doing it explicitely could be much clearer. Something like... client --> node GetData End node-->client DataFound End client-->node GetDataAs Filename= Direct= End [SimpleFiledSet and Fcp] This is a nasty one: dotted names. It may be easy or not for some languages to deal with this data type. Actually it is easy in python, but doing additional processing based on message signature (like type conversions) is pretty painful and costy. Flat messages shopuld be way easier to handle. Already made a suggestion regarding this... ContainerMessage NItems=N End Item End [Message signatures and consts] (already mentioned before) a good to have would be a file containg all consts and message signatures used in Fcp. It should go into very detail regarding types aswell. TypeIPAddress. TypeIPAddressList (...) This would save much typing work and could help in autogenarating huge parts of protocol wrappers. [Organize requests hirarchically] This may sound exotic, but it could help in organizing requests. Especially when it comes to restoring requests, where it may be desireable to process requests on demand. Something like GetRootRequest ..ListChildRequests [Reserve DataLenght as indicator of data attatched to a message] (Already mentioned) makes message parsing much easier if one can rely on this field being reserved as message metadata. [ConfigData, NodeData, Peer should be traversable] See [Organize requests hirarchically] [Drop Started field in Persistent* messages] Maybe use simple progress instead, and / or some dedicated field to indicate request status. This could be helpful anyways if requests may be stopped, resumed (...) at some point [Dedicated Result field in messages] In case something goes wrong, it would be nice to have the message casing the error at hand right away. ++ ProtocolError, FetchError, InsertError are not very handy. They should be numbered consecutively. I found myself writing a wrapper to emulate this (uniformity). [ClientToken field for all messages] It may be desireable to keep track of all messages and responses. Like for example GetConfig and Config to tell when all tasks a user has posted are completed. ClientToken would come in handy. ...maybe other things I can't remember now. -------------------------------------------------------------------------------- Finally, in case someone is interested the wrapper as complete or incomplete as it is: [http://fclient.svn.sourceforge.net/viewvc/fclient/trunk/sandbox/fcp2/] Thanks, Juergen From toad at amphibian.dyndns.org Wed Mar 19 15:01:55 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 19 Mar 2008 15:01:55 +0000 Subject: [Tech] Fcp woes ..final In-Reply-To: <47E122D9.9010507@arcor.de> References: <47E122D9.9010507@arcor.de> Message-ID: <200803191502.04015.toad@amphibian.dyndns.org> On Wednesday 19 March 2008 14:27, juergen urner wrote: > Just dropping by to tell, that I finally resign on writing a highlevel > Python wrapper > for the protocol. There is still too many show stopper