From dan.bruce at gmail.com Fri May 9 15:11:09 2008 From: dan.bruce at gmail.com (Dan Bruce) Date: Fri, 9 May 2008 09:11:09 -0600 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> I have two idea's to add the first is increased usage, bundling this with a secondary tool that would have more universal use would expand the network function. Part of the system depends on the number of users to help with obscurity; the more people in the network the less likely to be able to track a single source. Adding in an "add to the cause tool" that the everyday user can run being able to donate both cpu - drive and bandwidth would expand the function. The concept is basically Gramma and Grampa may not be using the system as a free net assured of the privacy but would be fairly willing to run a background process that gives them something upfront in terms of data - weather report, a PRIVATE IM that is secured to talk with the grand kids etc and also provide both the space and bandwidth to the cause. The distributed processing projects have generated some of the largest Super Computer ideas on earth, people will give to a cause even if they are not using it. The "Add To The Cause" factor as long as it remained in the back ground was clean a single icon in the tray with a focus on trying to remain as minimal impact on the persons machine and internet as possible. Something that is able to limit its use to off times as to not impact the users, and able to keep bandwidth limits in mind (off peak and not exceeding the max up/down) for the people willing to add to the cause. A secure chat tool (mod the Jabber Client and Server to run on the freenet) would give the user something back for adding to the cause if they are not fully participating. The secure IM would also be likely something that would see increased people using it for the secure chat functions alone. The second is with regard to the long term storage issue. The Hash functions being used in the other P2P tool breaking a file into 128k chunks doing a hash and the file ID being a Hash of the Hash has great advantage, both in ID of the file and being able to identify and distribute clear ID'd chunks. Including a means to allow adding in the older materials requested from CD/DVD copies users have burned would give the project hard storage in the form of sectors that are CD or DVD sized. The addition of a tool that helps with off line storage and location for users would increase the use of the feature. For the user to be able to scan in a CD/DVD for use in a location database (generating the file hashes and such) as well as the users cd/dvd numbering to locate gives the user an incentive. What this would give is the ability for the system to request OLD information from the freenet drive, having 100 or 1000 people protected by the anonymous factors would see a willingness to insert the media for upload, also in terms of protection the system having 100 or 1000 copies available the speed on file transfer increases along with the obscurity. It does essentially become a double blind function the request for an older File can see 1000 copies where it actually comes from at the source level is unable to be defined, if 300 blocks are needed and 1000 copies are put into Rom Drives around the world not only is the speed present the 1000 people putting in the older CD/DVD would not even know if their copy was used. It would be asking the user to put up a CD size or DVD size or both temp drive cache, and the other further blind function is with the system being able to ask for DVD #8 from the users archives there is no way to know even what file was requested unless it is a 4 gig file on the DVD The advantage this offers is not having to re-propagate an old file into the system if 100 copies can be brought online by request not only has the speed amplified 100X the bandwidth savings also have in being able to distribute it so the source is obscured. I have some idea's on implantation and indexing though I have not programmed in years to be able to add to this in coding at this time. For now its just a concept that the group will likely be able to add to before trying to nail down the specifics. Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/tech/attachments/20080509/bed25775/attachment.htm From toad at amphibian.dyndns.org Tue May 13 17:44:00 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 13 May 2008 18:44:00 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> Message-ID: <200805131844.05385.toad@amphibian.dyndns.org> On Friday 09 May 2008 16:11, Dan Bruce wrote: > I have two idea's to add the first is increased usage, bundling this with a > secondary tool that would have more universal use would expand the network > function. Possibly, however bundling Freenet with anything is problematic, because of the overhead (in every sense): - Exchanging noderefs. - CPU/disk/memory usage. (CPU usage is low, memory usage will improve significantly in 0.7.1). - The political/ideological overhead is IMHO rather significant: Your computer could have encrypted child porn / encrypted terorrist training manuals / encrypted Tibettan photographs etc on it. However, I'm by no means opposed to finding Killer Apps for Freenet. > Part of the system depends on the number of users to help with > obscurity; the more people in the network the less likely to be able to > track a single source. True, more users is always better. > > Adding in an "add to the cause tool" that the everyday user can run being > able to donate both cpu - drive and bandwidth would expand the function. The > concept is basically Gramma and Grampa may not be using the system as a free > net assured of the privacy but would be fairly willing to run a background > process that gives them something upfront in terms of data - weather report, > a PRIVATE IM that is secured to talk with the grand kids etc and also > provide both the space and bandwidth to the cause. The distributed > processing projects have generated some of the largest Super Computer ideas > on earth, people will give to a cause even if they are not using it. Perhaps. Most grandparents' politics would likely exclude Freenet though. Certainly not all, judging by even some folk who've been active on this list. > > The "Add To The Cause" factor as long as it remained in the back ground was > clean a single icon in the tray with a focus on trying to remain as minimal > impact on the persons machine and internet as possible. Something that is > able to limit its use to off times as to not impact the users, and able to > keep bandwidth limits in mind (off peak and not exceeding the max up/down) > for the people willing to add to the cause. We should have a systray icon. One focus of 0.7.1 will be to reduce the memory usage significantly, and this should also help with CPU usage etc. More accurate bandwidth limiting, and auto-detection of bandwidth usage, is also planned. Bandwidth scheduling is being actively worked on. > > A secure chat tool (mod the Jabber Client and Server to run on the freenet) > would give the user something back for adding to the cause if they are not > fully participating. The secure IM would also be likely something that would > see increased people using it for the secure chat functions alone. IM between darknet peers is feasible, it's just nobody's done it yet. Having said that, the only real benefit to doing it over Freenet is the fact that your conversations are not visible on passive traffic analysis (because Freenet is using lots of bandwidth for other things). > > The second is with regard to the long term storage issue. > > The Hash functions being used in the other P2P tool breaking a file into > 128k chunks doing a hash and the file ID being a Hash of the Hash has great > advantage, both in ID of the file and being able to identify and distribute > clear ID'd chunks. > > Including a means to allow adding in the older materials requested from > CD/DVD copies users have burned would give the project hard storage in > the form of sectors that are CD or DVD sized. The addition of a tool that > helps with off line storage and location for users would increase the use of > the feature. For the user to be able to scan in a CD/DVD for use in a > location database (generating the file hashes and such) as well as the users > cd/dvd numbering to locate gives the user an incentive. > > What this would give is the ability for the system to request OLD > information from the freenet drive, having 100 or 1000 people protected by > the anonymous factors would see a willingness to insert the media for > upload, also in terms of protection the system having 100 or 1000 copies > available the speed on file transfer increases along with the obscurity. Okay so we are talking about either: - High latency insert on demand, whether manual or client layer. - Some network level mechanism that equates to the same thing ("inverse passive requests" for example, although they assumed lower latency). Basically this becomes a manually operated offline storage system (non-manual systems have robots and libraries of tape drives). It *might* be possible to implement such a thing within the context of Freenet, however it's probably not something that we'll see within the 0.7 or 0.8 timeframe. For the time being, manual insert on demand is probably it: somebody asks on FRost for a given file, and somebody else inserts it. > > It does essentially become a double blind function the request for an older > File can see 1000 copies where it actually comes from at the source level is > unable to be defined, if 300 blocks are needed and 1000 copies are put into > Rom Drives around the world not only is the speed present the 1000 people > putting in the older CD/DVD would not even know if their copy was used. Of course they would know, they only put it in the drive because the system asked them to. > > It would be asking the user to put up a CD size or DVD size or both temp > drive cache, and the other further blind function is with the system being > able to ask for DVD #8 from the users archives there is no way to know even > what file was requested unless it is a 4 gig file on the DVD I'm not sure this is cost effective right now. DVD-R is cheap, but bulky, and relatively small (4G for the cheap ones, 8G for the big ones). Blu-ray disks are too expensive for this sort of usage. Hard disks are online, cheap, and large. > > The advantage this offers is not having to re-propagate an old file into the > system if 100 copies can be brought online by request not only has the speed > amplified 100X the bandwidth savings also have in being able to distribute > it so the source is obscured. > > I have some idea's on implantation and indexing though I have not > programmed in years to be able to add to this in coding at this time. > > For now its just a concept that the group will likely be able to add to > before trying to nail down the specifics. > > Dan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080513/16f80093/attachment.pgp From ian.clarke at gmail.com Tue May 13 17:51:55 2008 From: ian.clarke at gmail.com (Ian Clarke) Date: Tue, 13 May 2008 12:51:55 -0500 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805131844.05385.toad@amphibian.dyndns.org> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> <200805131844.05385.toad@amphibian.dyndns.org> Message-ID: <823242bd0805131051l7b7ec944vf545a568892858b3@mail.gmail.com> 2008/5/13 Matthew Toseland : > Possibly, however bundling Freenet with anything is problematic Being bundled is one of our best hopes for getting Freenet widely deployed, we need to be encouraging it, not looking for reasons not to do it. Freenet is a platform on which we want people to build apps. The best way to encourage that, given that Freenet isn't already widely deployed, is to make it easy for those apps to bundle Freenet. > , because of the overhead (in every sense): > - Exchanging noderefs. Not necessary if they use opennet - the app that bundles Freenet would be responsible for encouraging the user to behave correctly. > - CPU/disk/memory usage. (CPU usage is low, memory usage will improve > significantly in 0.7.1). If the user wants to use Frost/FMS/whatever, then they need Freenet. Whatever overheads Freenet has are inevitable, although we need to work to reduce them. Its certainly no reason to discourage people from bundling Freenet. > - The political/ideological overhead is IMHO rather significant: Your computer > could have encrypted child porn / encrypted terorrist training manuals / > encrypted Tibettan photographs etc on it. It would be the responsibility of the app bundling Freenet to explain these things to the user. Ian. -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From toad at amphibian.dyndns.org Tue May 13 18:54:28 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 13 May 2008 19:54:28 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <823242bd0805131051l7b7ec944vf545a568892858b3@mail.gmail.com> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> <200805131844.05385.toad@amphibian.dyndns.org> <823242bd0805131051l7b7ec944vf545a568892858b3@mail.gmail.com> Message-ID: <200805131954.28231.toad@amphibian.dyndns.org> On Tuesday 13 May 2008 18:51, Ian Clarke wrote: > 2008/5/13 Matthew Toseland : > > Possibly, however bundling Freenet with anything is problematic > > Being bundled is one of our best hopes for getting Freenet widely > deployed, we need to be encouraging it, not looking for reasons not to > do it. Freenet is a platform on which we want people to build apps. > The best way to encourage that, given that Freenet isn't already > widely deployed, is to make it easy for those apps to bundle Freenet. Okay, how do we make it easy to bundle Freenet? > > > , because of the overhead (in every sense): > > - Exchanging noderefs. > > Not necessary if they use opennet - the app that bundles Freenet would > be responsible for encouraging the user to behave correctly. You think they would? :) > > > - CPU/disk/memory usage. (CPU usage is low, memory usage will improve > > significantly in 0.7.1). > > If the user wants to use Frost/FMS/whatever, then they need Freenet. > Whatever overheads Freenet has are inevitable, although we need to > work to reduce them. Its certainly no reason to discourage people > from bundling Freenet. > > > - The political/ideological overhead is IMHO rather significant: Your computer > > could have encrypted child porn / encrypted terorrist training manuals / > > encrypted Tibettan photographs etc on it. > > It would be the responsibility of the app bundling Freenet to explain > these things to the user. Sure, but for a lot of apps, this is a blocker. For example, the mainstream version of Thingamablog can't realistically bundle Freenet. We are here talking about apps which are specifically written for Freenet, or plugins to mainstream apps, the plugin being specifically written for Freenet. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080513/41d8228f/attachment.pgp From ian.clarke at gmail.com Tue May 13 19:33:23 2008 From: ian.clarke at gmail.com (Ian Clarke) Date: Tue, 13 May 2008 14:33:23 -0500 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805131954.28231.toad@amphibian.dyndns.org> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> <200805131844.05385.toad@amphibian.dyndns.org> <823242bd0805131051l7b7ec944vf545a568892858b3@mail.gmail.com> <200805131954.28231.toad@amphibian.dyndns.org> Message-ID: <823242bd0805131233r4581a678l79a7b17033fbbba5@mail.gmail.com> 2008/5/13 Matthew Toseland : > > Being bundled is one of our best hopes for getting Freenet widely > > deployed, we need to be encouraging it, not looking for reasons not to > > do it. Freenet is a platform on which we want people to build apps. > > The best way to encourage that, given that Freenet isn't already > > widely deployed, is to make it easy for those apps to bundle Freenet. > > Okay, how do we make it easy to bundle Freenet? Streamline the installation process, minimize the amount of user interaction required - make it easy for an app's installer to provide whatever startup information Freenet might need. > > > , because of the overhead (in every sense): > > > - Exchanging noderefs. > > > > Not necessary if they use opennet - the app that bundles Freenet would > > be responsible for encouraging the user to behave correctly. > > You think they would? :) Hopefully, either way, its their responsibility as the "primary" app, not ours. Certainly, its no reason for us not to try to make Freenet easy to bundle. > > It would be the responsibility of the app bundling Freenet to explain > > these things to the user. > > Sure, but for a lot of apps, this is a blocker. For example, the mainstream > version of Thingamablog can't realistically bundle Freenet. We are here > talking about apps which are specifically written for Freenet, or plugins to > mainstream apps, the plugin being specifically written for Freenet. Of course, not every app will bundle Freenet, but we need to make it easy for those that can. Ian. -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From toad at amphibian.dyndns.org Tue May 13 21:03:44 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 13 May 2008 22:03:44 +0100 Subject: [Tech] Location churn: empirical data Message-ID: <200805132203.45605.toad@amphibian.dyndns.org> Below are links to my node's location logs. A lot of the location churn is within roughly a slice of 10% of the keyspace, and the central signal is reasonably clean... but this degree of variation is going to cause problems with routing and especially data retention IMHO (on a network of several thousand nodes, with *at least* 1000 nodes online at a time). There were a whole bunch of resets during the week 19 April to 26 April, and on the 24th, RC2 was released; on May 8th, 0.7.0 was released. These may explain some but not all of the long jumps... http://amphibian.dyndns.org/location.log.txt http://amphibian.dyndns.org/locations.png Random resets: It is possible there were random resets before the first mentioned below, because we didn't log them until some time after we started logging locations. 20-Apr-2008 18:40:16 : 0.08100045552596624 (random reset) 21-Apr-2008 00:47:04 : 0.6871948987851273 (random reset) 21-Apr-2008 09:49:15 : 0.9095478261922363 (random reset from duplicated location) 23-Apr-2008 13:04:35 : 0.3044855058133896 (random reset from duplicated location) 23-Apr-2008 19:08:05 : 0.141007692250818 (random reset) 04-May-2008 14:33:46 : 0.18902390631238708 (random reset) I am asking Oskar about this... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080513/cb630ee7/attachment.pgp From toad at amphibian.dyndns.org Tue May 13 21:22:30 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 13 May 2008 22:22:30 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <823242bd0805131233r4581a678l79a7b17033fbbba5@mail.gmail.com> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> <200805131954.28231.toad@amphibian.dyndns.org> <823242bd0805131233r4581a678l79a7b17033fbbba5@mail.gmail.com> Message-ID: <200805132222.30845.toad@amphibian.dyndns.org> On Tuesday 13 May 2008 20:33, Ian Clarke wrote: > 2008/5/13 Matthew Toseland : > > > Being bundled is one of our best hopes for getting Freenet widely > > > deployed, we need to be encouraging it, not looking for reasons not to > > > do it. Freenet is a platform on which we want people to build apps. > > > The best way to encourage that, given that Freenet isn't already > > > widely deployed, is to make it easy for those apps to bundle Freenet. > > > > Okay, how do we make it easy to bundle Freenet? > > Streamline the installation process, minimize the amount of user > interaction required - make it easy for an app's installer to provide > whatever startup information Freenet might need. Heh, that's your answer to everything! :) If the app wants to specify *everything* then they can just bundle the files in their own installer, force their own settings, and not send the user to the wizard. This may in fact be the easiest option in most cases, certainly it's what they will do if they want freenet to be "invisible". We could maybe provide hooks to skip stages where appropriate otherwise. But launching a browser would distract from whatever it is they are installing with Freenet, no? > > > > > , because of the overhead (in every sense): > > > > - Exchanging noderefs. > > > > > > Not necessary if they use opennet - the app that bundles Freenet would > > > be responsible for encouraging the user to behave correctly. > > > > You think they would? :) > > Hopefully, either way, its their responsibility as the "primary" app, > not ours. Certainly, its no reason for us not to try to make Freenet > easy to bundle. I hope this isn't simply a cunning plan to reduce our support load. Right now that's not a big problem imho. > > > > It would be the responsibility of the app bundling Freenet to explain > > > these things to the user. > > > > Sure, but for a lot of apps, this is a blocker. For example, the mainstream > > version of Thingamablog can't realistically bundle Freenet. We are here > > talking about apps which are specifically written for Freenet, or plugins to > > mainstream apps, the plugin being specifically written for Freenet. > > Of course, not every app will bundle Freenet, but we need to make it > easy for those that can. > > Ian. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080513/a3020180/attachment.pgp From dan.bruce at gmail.com Tue May 13 13:36:36 2008 From: dan.bruce at gmail.com (Dan.Bruce@gmail.com) Date: Tue, 13 May 2008 15:36:36 +0200 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <48299964.3050601@gmail.com> Some times it needs a bit of discussion to be able to make idea's come clear, lol, I am bad for missing a few steps in the thought process and jumping over them. The first is FreeNet allows for a couple ways to be set up. As an OPEN NET concept and it also allows for establishing a Private Network that literally does not connect to anyone but a secured group, 10 People can use it to establish a private Internet of there own and quite literally ignore the fact there is an internet. The "Add to the Cause" concept in terms of an application that allows a person to say OK I will give 20k up 20k down towards the cause in this. It gives the chance to incorporate essentially a TOR style router to the entire freenet. It would allow use of this to relay traffic through them, or if the user enables to function as a temporary distributed MINI Server for the freenet. It would depend on the application. I suggested CHAT as that is an application that most often is left on 24/7 on a machine, and as long as the bandwidth use stays with in reason, does like the SETI application did if the machine was idle it went to work if it was in use it went to a crawl people will add to the cause, a pause button for it if its interfering with the operation. MINI Server: you take 8 of these mini nodes that have the ability to work as a server for a short time and bundle them as a single server. The IRC Channel Server or what ever then works off one IP for 15 mins and then changes to the next this gives the redundancy and obscuring it with others working as relays to them. The ability to hide in the open is the concept I am working with 20 darknet networks that function alone are better hidden if there are 2 or 3 OPEN NET functions working and 20 or 30 Private/Open Net functions working. If you have 80 million people that have 20k in 20k out all with the same sized blocks of traffic and its all private/encrypted you cant say which traffic is in which of the numerous networks. What gets Bundled with an application for everyone to use does not have to be a FULL FreeNet it could be a service function that the freenet can use to increase its performance and its security. The user gets encrypted SECURE chat Freenet gets a mini service... With the Storage Idea I am not thinking of the new stuff to burn on DVD though it might be feasible to ask users to burn a FREENET SYSTEM DVD/CD every now and then. What I am thinking of is the thousands and millions of CD/DVD that people have burned already. The offline storage is already burnt and the potential archive goes back to the early 90's in terms of information archive. For the FreeNet to have an application that allows scanning in the files on the Users CD/DVD into a Database that the user can use to locate files in the users own library. What is needed is to generate the Hash ID's for what they have burned and include that in the database ideally a flag saying that some stuff is not to be included. A copy of a 1991 abandoned game would see some requests and if the person scans in a disc that also has wedding pictures obviously they would not be included in the database except for personal use. Not that anyone would have the ID Hash to request it. A brand new node instead of transfering data is able to scan in what they have already on cd/dvd rather than have to wait on the data coming from the freenet to fill the filestore. They downloaded something off of the newsgroups that maybe the most popular file on the freenet, rather than having to allow their cache to eventually get the blocks the scan can allow putting the entire copy in and not just a few blocks of it. In terms of the Bit Torrent layout. A user able to scan the files in would become essentially a new SEED having a full copy of it rather than one of the distributed copies that has 100 blocks of a 1000 block file in the cache. 10 people having 100 blocks to send makes it fast.... 1 person having all 1000 makes it reliable. With a request coming in from Freenet to put Disc 10 in... It would be your disc 10 and not related to anyone elses personal index. The double blind in this is freenet asks for a disc and not a file and your disc 10 is not anyone elses disc 10. Freenet gets a request for a FILE ID HASH, looks up in its database if its on the system or on the users off line storage. If it is then the freenet reads it into the users cache re-encrypting it and breaking it into the required block sizes and such. Unless you are running a program to monitor what is actually copied off the disc all you know is some one wanted you to put in a CD you burned in 1992. What actual file was called for is unknown by the person placing the disc in. As for political / ideological overheads.... You either have privacy or you don't at the moment much of our world offers its citizens only the illusion of privacy and the powers that be want it that way. True Privacy is something that is feared by all Governments world wide. The influx of porn and anarchy materials are common to any system at the start, it took them about 2 weeks to rename porn with an mp3 extenstion and jump onto napster when it first started. The percentage of the system that is porn will change instantly with the influx of a fileswapping system that can offer some elements of speed as well as security. 20 Million Users waters the percentages down real fast. I would never say it needs to be as fast as the current Bit Torrents allow, esentially this is a trade you would see about 1/2 the speed in return for the privacy. As the saying goes when it comes to buying things you can have any two the price costs you the third..... Cheap, Fast, or Quality encrypted anonymous private internet, any system that offers true privacy is not going to be pretty. Outside of the virtual world you remove law from our land and give the general person the ability to live a totally free private life in real life and you will have a society that is pretty much anarchy. Yes it offers a great deal to a person but it also means you need to take the good with the bad, when it comes to privacy you either have it or you don't. Secure Chat that is obscured completely by the freenet would give people a reason to run the app. With respect to the fileswap at the moment potential idea maybe to have a polling idea to allow users to catagorize files better. Each File having basically an NFO that can allow for some organization. If the FileSwap was able to allow files to be voted on as to them being movie, book, game, application, picture, X it would allow for some simple filtering which would improve the image of the system. Removing files will just see them reposted but being able to screen them into categories would allow for the users to get rid of materials they find offensive, 200 people voting on where it belongs would make it possible with out allowing for abuse in that fashion as well Dan From toad at amphibian.dyndns.org Tue May 13 21:57:58 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Tue, 13 May 2008 22:57:58 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <48299964.3050601@gmail.com> References: <48299964.3050601@gmail.com> Message-ID: <200805132258.03188.toad@amphibian.dyndns.org> On Tuesday 13 May 2008 14:36, Dan.Bruce at gmail.com wrote: > Some times it needs a bit of discussion to be able to make idea's come > clear, lol, I am bad for missing a few steps in the thought process and > jumping over them. > > The first is FreeNet allows for a couple ways to be set up. As an OPEN > NET concept and it also allows for establishing a Private Network that > literally does not connect to anyone but a secured group, 10 People can > use it to establish a private Internet of there own and quite literally > ignore the fact there is an internet. Not true. Lots and lots of people seem to not get the darknet concept. Just because you only connect to your trusted friends, does not mean that your friends only connect to YOUR trusted friends: they will connect to THEIR trusted friends, which may not be the same as yours, although there will be a good deal of overlap. Some of them may connect to the opennet, but even without that, it is possible to have a global darknet. And to have it routable, hopefully, in a reasonably small number of hops (= good performance), with Freenet 0.7. > > The "Add to the Cause" concept in terms of an application that allows a > person to say OK I will give 20k up 20k down towards the cause in this. > It gives the chance to incorporate essentially a TOR style router to the > entire freenet. It would allow use of this to relay traffic through > them, or if the user enables to function as a temporary distributed > MINI Server for the freenet. It would depend on the application. You can just install Freenet if you want to contribute to Freenet. I don't see the problem, except that it's a bit heavier than it ought to be. Is there any other barrier here? > > I suggested CHAT as that is an application that most often is left on > 24/7 on a machine, and as long as the bandwidth use stays with in > reason, does like the SETI application did if the machine was idle it > went to work if it was in use it went to a crawl people will add to the > cause, a pause button for it if its interfering with the operation. Sure, but it's rather heavyweight for a chat client! If people want a chat client they'll just get a chat client, won't they? > > MINI Server: you take 8 of these mini nodes that have the ability to > work as a server for a short time and bundle them as a single server. > The IRC Channel Server or what ever then works off one IP for 15 mins > and then changes to the next this gives the redundancy and obscuring it > with others working as relays to them. Eh? > > The ability to hide in the open is the concept I am working with 20 > darknet networks that function alone are better hidden if there are 2 or > 3 OPEN NET functions working and 20 or 30 Private/Open Net functions > working. > > If you have 80 million people that have 20k in 20k out all with the same > sized blocks of traffic and its all private/encrypted you cant say which > traffic is in which of the numerous networks. > > What gets Bundled with an application for everyone to use does not have > to be a FULL FreeNet it could be a service function that the freenet can > use to increase its performance and its security. The user gets > encrypted SECURE chat Freenet gets a mini service... I don't see any way to divide Freenet up beyond the node level. That means you will need to deploy a full Freenet node. However, most of its local resource consumption (memory, CPU) is related to queued downloads and so on. > > With the Storage Idea I am not thinking of the new stuff to burn on DVD > though it might be feasible to ask users to burn a FREENET SYSTEM DVD/CD > every now and then. What I am thinking of is the thousands and millions > of CD/DVD that people have burned already. That wasn't very clear from your post; in the first part I assumed it was existing disks but in the second it looked like you were talking about disks people had burned especially. > > The offline storage is already burnt and the potential archive goes back > to the early 90's in terms of information archive. > > For the FreeNet to have an application that allows scanning in the files > on the Users CD/DVD into a Database that the user can use to locate > files in the users own library. What is needed is to generate the Hash > ID's for what they have burned and include that in the database ideally > a flag saying that some stuff is not to be included. A copy of a 1991 > abandoned game would see some requests and if the person scans in a disc > that also has wedding pictures obviously they would not be included in > the database except for personal use. Not that anyone would have the ID > Hash to request it. The problem with this is that if it relies on the user coming up with the disk, not only will latency be terrible, but in many cases the user won't be bothered - it's just way too much work to find a specific disk, at least in my house! This makes it difficult to regulate such a system against attacks such as inserting lots of pointers to bogus content. > > A brand new node instead of transfering data is able to scan in what > they have already on cd/dvd rather than have to wait on the data coming > from the freenet to fill the filestore. They downloaded something off of > the newsgroups that maybe the most popular file on the freenet, rather > than having to allow their cache to eventually get the blocks the scan > can allow putting the entire copy in and not just a few blocks of it. > > In terms of the Bit Torrent layout. A user able to scan the files in > would become essentially a new SEED having a full copy of it rather than > one of the distributed copies that has 100 blocks of a 1000 block file > in the cache. 10 people having 100 blocks to send makes it fast.... 1 > person having all 1000 makes it reliable. > > With a request coming in from Freenet to put Disc 10 in... It would be > your disc 10 and not related to anyone elses personal index. The double > blind in this is freenet asks for a disc and not a file and your disc 10 > is not anyone elses disc 10. > > Freenet gets a request for a FILE ID HASH, looks up in its database if > its on the system or on the users off line storage. If it is then the > freenet reads it into the users cache re-encrypting it and breaking it > into the required block sizes and such. > > Unless you are running a program to monitor what is actually copied off > the disc all you know is some one wanted you to put in a CD you burned > in 1992. What actual file was called for is unknown by the person > placing the disc in. > > As for political / ideological overheads.... > > You either have privacy or you don't at the moment much of our world > offers its citizens only the illusion of privacy and the powers that be > want it that way. True Privacy is something that is feared by all > Governments world wide. > > The influx of porn and anarchy materials are common to any system at the > start, it took them about 2 weeks to rename porn with an mp3 extenstion > and jump onto napster when it first started. The percentage of the > system that is porn will change instantly with the influx of a > fileswapping system that can offer some elements of speed as well as > security. 20 Million Users waters the percentages down real fast. > > I would never say it needs to be as fast as the current Bit Torrents > allow, esentially this is a trade you would see about 1/2 the speed in > return for the privacy. As the saying goes when it comes to buying > things you can have any two the price costs you the third..... Cheap, > Fast, or Quality IMHO Freenet is not likely to be even half as fast as bittorrent for really popular files, however, for not-so-popular files it is much faster already, because *there aren't any seeders* on bittorrent for a lot of unpopular files! > > encrypted anonymous private internet, any system that offers true > privacy is not going to be pretty. > > Outside of the virtual world you remove law from our land and give the > general person the ability to live a totally free private life in real > life and you will have a society that is pretty much anarchy. > > Yes it offers a great deal to a person but it also means you need to > take the good with the bad, when it comes to privacy you either have it > or you don't. > > Secure Chat that is obscured completely by the freenet would give people > a reason to run the app. > > With respect to the fileswap at the moment potential idea maybe to have > a polling idea to allow users to catagorize files better. Each File > having basically an NFO that can allow for some organization. If the > FileSwap was able to allow files to be voted on as to them being movie, > book, game, application, picture, X it would allow for some simple > filtering which would improve the image of the system. This is client layer stuff. Freenet itself doesn't know or care what content is. > > Removing files will just see them reposted but being able to screen them > into categories would allow for the users to get rid of materials they > find offensive, 200 people voting on where it belongs would make it > possible with out allowing for abuse in that fashion as well > > Dan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080513/c7f688fa/attachment.pgp From ian.clarke at gmail.com Tue May 13 23:06:30 2008 From: ian.clarke at gmail.com (Ian Clarke) Date: Tue, 13 May 2008 18:06:30 -0500 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805132222.30845.toad@amphibian.dyndns.org> References: <70d0b61b0805090811k30ba116i7829fa1a62db006@mail.gmail.com> <200805131954.28231.toad@amphibian.dyndns.org> <823242bd0805131233r4581a678l79a7b17033fbbba5@mail.gmail.com> <200805132222.30845.toad@amphibian.dyndns.org> Message-ID: <823242bd0805131606y577fb8c0s88371797fecae59b@mail.gmail.com> 2008/5/13 Matthew Toseland : > I hope this isn't simply a cunning plan to reduce our support load. Right now > that's not a big problem imho. ?! No, hopefully its a cunning plan to increase the size of Freenet's user base. Obviously. Ian. -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From dan.bruce at gmail.com Tue May 13 15:57:19 2008 From: dan.bruce at gmail.com (Dan) Date: Tue, 13 May 2008 17:57:19 +0200 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <4829BA5F.7080009@gmail.com> DarkNet VS OpenNet Elements With FreeNet yes a DarkNet can leak public but a group that is using it to cover communications and information can work to restrict it even if its like the olden days; a memo form of security about not deleting a file on the mainframe. At present there is no way to force the restrictions on it but in time the groups that begin to implement this as a private internet will work on limiting it from breaking out of the set group. I can see this project resulting in not a single world wide private anonymous internet but resulting in many and the various nets all may or may not come together in the world. Its not too hard to see there being a Christian Crusader Network forming and an Eastern Jihad Network, a Republican Network, a Democratic Network, and so on. People have a natural tendency to associate with like minded groups. This project offers 2 elements in privacy the first is privacy from the current network and the internet as its known today and the second is privacy from each other. The latter in that is one that is under personal control while a person may not want the world to know they are part of the network they may be quite happy to have the group with in the network know even their home address's and names. The Chat Application Function and a stripped down NODE, there is likely a fair demand for this form of an application for a chat program running on the freenet network it offers more than just a simple CHAT. Running on freenet as a mini node would mean there is continued traffic making it impossible to pick out a conversation by watching a connection from a setting of an ISP. The system is encrypted and secured that can not be monitored. The Anonymous Factor on the chat is also one that would benefit people as well. Yes talking to your Brother the anon function is useless but the communications are secured by the system, on the other hand your also able to chat with a total stranger and there is no concern they may be able to locate you physically. A Chat Application would bring in a huge number of people to this, the mini node concept would also see there being a couple million TOR style routers available to the freenet expanding both the speed and security. The Seti at Home application running the distributed processing for the Seti Project was a fairly burdensome program when your computer when idle on screen saver mode it was quite happy to take over what ever system resources it could. Often times I would see my system hitting 100% cpu when I wasnt using it. People are willing to "Donate To the Cause" IF your machine is on 24/7 the 8 hours your sleeping even the Gamers out there wont care so long as when they are playing the game they dont loose latency. Mini Node.... enough to access it for chatting, and to function as a relay for freenet is one idea. The second idea is to use the Mini Nodes as MINI SERVERS.... You would not be able to keep a server on a single IP long 10 - 15 mins and it would need to be redundant by putting 4 or 8 or what ever together so that any one server set looses one member the other 4 take over. They hang out on some ones chat node for 10 mins and move on... A FreeNet Server Applet..... The Web Pages are fairly static in nature due to the security using the Chat Mini Nodes as temp servers would bring a bit of dynamics to the system would be one idea as a use for a MINI NODE. With the Offline Storage Concept. >The problem with this is that if it relies on the user coming up with the >disk, not only will latency be terrible, but in many cases the user won't be >bothered - it's just way too much work to find a specific disk, at least in >my house! In a system where only 3 people have a copy of the file wanted yes latency would be wait till it gets put in.... but in a system where 5 million people could be asked if it was a common burned file ( IE: a win98 DLL that went bad and some one needed a copy of) out of 5 million that could be asked and the million that have it some one will have it handy and put it in. And the idea with it is to make it of a benefit to the user as well not just to the freenet. The application I have in mind would provide the user with an index of the off line files they have stored, as they scan in the cd/dvd the system builds them a nice little storage index database so that they are able to find things.... all they need to do is mark them #1 #2 etc. The system would also need to generate the file id hash's ideally for a number of the P2P apps as well as part of the Database. With the Hash's all ready if a person goes to click on a file the database can look it up and tell the user you have that file on disc #12 already.... (benefit is saves the bandwidth for the freenet and for the user as well as time) The DataBase does not get published your not all of a sudden sharing everything burnt publicly even on an anon system.... the Freenet uses it to when it has a file that is either no longer on the system or is only 78% complete as a few of the nodes no longer have the blocks that would have completed it. In a small user base system this would be SLOW but in a large user base system it could see 500 copies brought online by various users instead of being a brand new seed in terms of Bit Torrent and it being propagated back into the free net, with 500 copies it becomes and instant swarm and you have 90k/sec down. with non of them upping more than a few K each. From j16sdiz+freenet at gmail.com Wed May 14 00:26:54 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Wed, 14 May 2008 08:26:54 +0800 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805132258.03188.toad@amphibian.dyndns.org> References: <48299964.3050601@gmail.com> <200805132258.03188.toad@amphibian.dyndns.org> Message-ID: On 5/14/08, Matthew Toseland wrote: > On Tuesday 13 May 2008 14:36, Dan.Bruce at gmail.com wrote: > > Some times it needs a bit of discussion to be able to make idea's come > > clear, lol, I am bad for missing a few steps in the thought process and > > jumping over them. > > > > The first is FreeNet allows for a couple ways to be set up. As an OPEN > > NET concept and it also allows for establishing a Private Network that > > literally does not connect to anyone but a secured group, 10 People can > > use it to establish a private Internet of there own and quite literally > > ignore the fact there is an internet. > > Not true. Lots and lots of people seem to not get the darknet concept. Just > because you only connect to your trusted friends, does not mean that your > friends only connect to YOUR trusted friends: they will connect to THEIR > trusted friends, which may not be the same as yours, although there will be a > good deal of overlap. Some of them may connect to the opennet, but even > without that, it is possible to have a global darknet. And to have it > routable, hopefully, in a reasonably small number of hops (= good > performance), with Freenet 0.7. Darknet user have to be motivated to add new trusted friends, and their friends have to be motivated too. We don't have enough motivated users.. In my experience, I have *tried* to convince two friends to run freenet. One of them refused because of the disk space usage (this is a lame reason, but still). The another friend installed that (and found the installation quite managable), considered to add some content, but don't want to start it because of the high memory usage. I am too depressed to try to convince another friends. If we want a real working darknet, I think we have to (sort from lowest to highest priority): 1) lower the memory usage (0.7.1 ?) 2) easier friend finding. There was a facebook app for this, but it's not working. 3) small (<128MiB) store mode Currently freenet is very slow without a good store/cache. > [...] From nextgens at freenetproject.org Wed May 14 02:18:42 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Wed, 14 May 2008 04:18:42 +0200 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: References: <48299964.3050601@gmail.com> <200805132258.03188.toad@amphibian.dyndns.org> Message-ID: <20080514021841.GD3554@freenetproject.org> * Daniel Cheng [2008-05-14 08:26:54]: > On 5/14/08, Matthew Toseland wrote: > > On Tuesday 13 May 2008 14:36, Dan.Bruce at gmail.com wrote: > > > Some times it needs a bit of discussion to be able to make idea's come > > > clear, lol, I am bad for missing a few steps in the thought process and > > > jumping over them. > > > > > > The first is FreeNet allows for a couple ways to be set up. As an OPEN > > > NET concept and it also allows for establishing a Private Network that > > > literally does not connect to anyone but a secured group, 10 People can > > > use it to establish a private Internet of there own and quite literally > > > ignore the fact there is an internet. > > > > Not true. Lots and lots of people seem to not get the darknet concept. Just > > because you only connect to your trusted friends, does not mean that your > > friends only connect to YOUR trusted friends: they will connect to THEIR > > trusted friends, which may not be the same as yours, although there will be a > > good deal of overlap. Some of them may connect to the opennet, but even > > without that, it is possible to have a global darknet. And to have it > > routable, hopefully, in a reasonably small number of hops (= good > > performance), with Freenet 0.7. > > Darknet user have to be motivated to add new trusted friends, and > their friends have to be motivated too. We don't have enough motivated > users.. > > In my experience, I have *tried* to convince two friends to run > freenet. One of them refused because of the disk space usage (this is > a lame reason, but still). The another friend installed that (and > found the installation quite managable), considered to add some > content, but don't want to start it because of the high memory usage. > I am too depressed to try to convince another friends. > > If we want a real working darknet, I think we have to (sort from > lowest to highest priority): > 1) lower the memory usage (0.7.1 ?) > Here freenet takes less RAM than firefox. > 2) easier friend finding. > There was a facebook app for this, but it's not working. > > 3) small (<128MiB) store mode > Currently freenet is very slow without a good store/cache. It can't possibly work without request resuming: such a small store will be poisoned by your requests. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/8dc84f0e/attachment.pgp From alejandro at mosteo.com Wed May 14 09:32:21 2008 From: alejandro at mosteo.com (Jano) Date: Wed, 14 May 2008 11:32:21 +0200 Subject: [Tech] Location churn: empirical data References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: Matthew Toseland wrote: > Below are links to my node's location logs. A lot of the location churn is > within roughly a slice of 10% of the keyspace, and the central signal is > reasonably clean... but this degree of variation is going to cause problems > with routing and especially data retention IMHO (on a network of several > thousand nodes, with *at least* 1000 nodes online at a time). There were a > whole bunch of resets during the week 19 April to 26 April, and on the 24th, > RC2 was released; on May 8th, 0.7.0 was released. These may explain some but > not all of the long jumps... > > http://amphibian.dyndns.org/location.log.txt > http://amphibian.dyndns.org/locations.png See mine attached. This node has not been steadly 24/7 up as of late, which can be more typical. It bounces a bit... -------------- next part -------------- A non-text attachment was scrubbed... Name: location.png Type: image/png Size: 11687 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/f498cb9b/attachment.png -------------- next part -------------- A non-text attachment was scrubbed... Name: location.txt.gz Type: application/x-gzip Size: 11420 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/f498cb9b/attachment.bin From toad at amphibian.dyndns.org Wed May 14 12:26:01 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 14 May 2008 13:26:01 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <20080514021841.GD3554@freenetproject.org> References: <48299964.3050601@gmail.com> <20080514021841.GD3554@freenetproject.org> Message-ID: <200805141326.09252.toad@amphibian.dyndns.org> On Wednesday 14 May 2008 03:18, Florent Daigni?re wrote: > * Daniel Cheng [2008-05-14 08:26:54]: > > > On 5/14/08, Matthew Toseland wrote: > > > On Tuesday 13 May 2008 14:36, Dan.Bruce at gmail.com wrote: > > > > Some times it needs a bit of discussion to be able to make idea's come > > > > clear, lol, I am bad for missing a few steps in the thought process and > > > > jumping over them. > > > > > > > > The first is FreeNet allows for a couple ways to be set up. As an OPEN > > > > NET concept and it also allows for establishing a Private Network that > > > > literally does not connect to anyone but a secured group, 10 People can > > > > use it to establish a private Internet of there own and quite literally > > > > ignore the fact there is an internet. > > > > > > Not true. Lots and lots of people seem to not get the darknet concept. Just > > > because you only connect to your trusted friends, does not mean that your > > > friends only connect to YOUR trusted friends: they will connect to THEIR > > > trusted friends, which may not be the same as yours, although there will be a > > > good deal of overlap. Some of them may connect to the opennet, but even > > > without that, it is possible to have a global darknet. And to have it > > > routable, hopefully, in a reasonably small number of hops (= good > > > performance), with Freenet 0.7. > > > > Darknet user have to be motivated to add new trusted friends, and > > their friends have to be motivated too. We don't have enough motivated > > users.. > > > > In my experience, I have *tried* to convince two friends to run > > freenet. One of them refused because of the disk space usage (this is > > a lame reason, but still). The another friend installed that (and > > found the installation quite managable), considered to add some > > content, but don't want to start it because of the high memory usage. > > I am too depressed to try to convince another friends. > > > > If we want a real working darknet, I think we have to (sort from > > lowest to highest priority): > > 1) lower the memory usage (0.7.1 ?) > > > > Here freenet takes less RAM than firefox. > > > 2) easier friend finding. > > There was a facebook app for this, but it's not working. > > > > 3) small (<128MiB) store mode > > Currently freenet is very slow without a good store/cache. > > It can't possibly work without request resuming: such a small store will > be poisoned by your requests. > Request resuming is well up the priority list too, in fact it's a prerequisite for queue-on-disk imho. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/35015ead/attachment.pgp From nextgens at freenetproject.org Wed May 14 12:49:23 2008 From: nextgens at freenetproject.org (Florent =?iso-8859-1?Q?Daigni=E8re?=) Date: Wed, 14 May 2008 14:49:23 +0200 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805141326.09252.toad@amphibian.dyndns.org> References: <48299964.3050601@gmail.com> <20080514021841.GD3554@freenetproject.org> <200805141326.09252.toad@amphibian.dyndns.org> Message-ID: <20080514124921.GC3486@freenetproject.org> * Matthew Toseland [2008-05-14 13:26:01]: > On Wednesday 14 May 2008 03:18, Florent Daigni?re wrote: > > * Daniel Cheng [2008-05-14 08:26:54]: > > > > > On 5/14/08, Matthew Toseland wrote: > > > > On Tuesday 13 May 2008 14:36, Dan.Bruce at gmail.com wrote: > > > > > Some times it needs a bit of discussion to be able to make idea's come > > > > > clear, lol, I am bad for missing a few steps in the thought process > and > > > > > jumping over them. > > > > > > > > > > The first is FreeNet allows for a couple ways to be set up. As an OPEN > > > > > NET concept and it also allows for establishing a Private Network that > > > > > literally does not connect to anyone but a secured group, 10 People > can > > > > > use it to establish a private Internet of there own and quite > literally > > > > > ignore the fact there is an internet. > > > > > > > > Not true. Lots and lots of people seem to not get the darknet concept. > Just > > > > because you only connect to your trusted friends, does not mean that > your > > > > friends only connect to YOUR trusted friends: they will connect to THEIR > > > > trusted friends, which may not be the same as yours, although there will > be a > > > > good deal of overlap. Some of them may connect to the opennet, but even > > > > without that, it is possible to have a global darknet. And to have it > > > > routable, hopefully, in a reasonably small number of hops (= good > > > > performance), with Freenet 0.7. > > > > > > Darknet user have to be motivated to add new trusted friends, and > > > their friends have to be motivated too. We don't have enough motivated > > > users.. > > > > > > In my experience, I have *tried* to convince two friends to run > > > freenet. One of them refused because of the disk space usage (this is > > > a lame reason, but still). The another friend installed that (and > > > found the installation quite managable), considered to add some > > > content, but don't want to start it because of the high memory usage. > > > I am too depressed to try to convince another friends. > > > > > > If we want a real working darknet, I think we have to (sort from > > > lowest to highest priority): > > > 1) lower the memory usage (0.7.1 ?) > > > > > > > Here freenet takes less RAM than firefox. > > > > > 2) easier friend finding. > > > There was a facebook app for this, but it's not working. > > > > > > 3) small (<128MiB) store mode > > > Currently freenet is very slow without a good store/cache. > > > > It can't possibly work without request resuming: such a small store will > > be poisoned by your requests. > > > Request resuming is well up the priority list too, in fact it's a prerequisite > for queue-on-disk imho. That's weird how point of views can differ. I could understand why queue-on-disk would be a prerequisite to request resuming... definitely not the other way around. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/fd35d352/attachment.pgp From dan.bruce at gmail.com Wed May 14 09:22:24 2008 From: dan.bruce at gmail.com (Dan) Date: Wed, 14 May 2008 11:22:24 +0200 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <482AAF50.9040201@gmail.com> File System Addition. The insertion of files needs to have a category flag system in place with it with at least the initial first few flags being a forced issue related to being able to insert the files. The file system in many of the P2P systems lacks a structure beyond that of a massive root directory which is dumped into and a total mess. This can be handled with a couple bytes to allow for flagging the files for basic organization with a hard coded structure, then maybe some additional space for user breakdowns. The benefit I can see of this would be to bring essentially a few directories into the file system and allow for the break down of the file listings into a more specific chunks suited to the category being sought. With even a 100,000 files the approach of a root directory catch all is easy enough to work with. When it approaches 20 million or 100 million files given the exchange of file data I have seen taking place and the time this takes it will becomes much more of a problematic issue. Even with the large news servers streaming full it can take an hour or two for a header list to come in and Free-Net does not make that process something that can be streamed direct at high speed. a Two ? Byte flag system would allow breaking those 16 bits into an initial category system to allow for some organization. With the use of a hard coded breakdown of that the forcing of the initial organization could be made. The debate would be establishing the hard code structure as to be able to make use of this and once implemented it would serve as permanent with in Free-Net. For it to be universally used the first part does have to be hard coded into the system at a core level To start off the structuring of it the initial debate would be in the breakdown of the bit pattern. To work with a possible idea for the bit mask breakdown is this: 1bits/3bit /4bits/4bits/ 1bits/3bit/ for a total of 16 bits/2 bytes Flags Description Flags First 4 Bits ? used as 1/1/2 First bit being a flag for System File Second Bit Default Too Lazy to say what the file is Third Bit being a flag for essentially Hidden File or Not like on a HD file system Fourth Bit being Personal or not Second 4 Bits being 16 category choices..... FreeNet-Internal, Video, Audio, Book, Game, Application, Picture, OS, News, Sit-Rep, Other, Misc .... Personal Third Set of 4 Bits a Secondary Description Classification Educational, Entertainment, Informative, Reference Material, Porn, Urgent, Misc, ...... Personal The option with the third set would be hard coding a sub category for each of the first choice, or coming up with a generalized list of 16 items that would fill the general need for a secondary descriptor for the file describing the first level. Fourth Set of Bits potentially being broken into 3/1 3 bits being a Content Rating 1 being N/A and the remaining 7 choices a content rating scale as to age 2 for all ages to 8 being adult 1 bit being Descriptor is Locked if its a public file this is by a polling vote. If its a private file this is by owner .................................................. This adds a step to the insertion procedure and adds an additional 2 bytes to the file key length. With the use of a Hard Coded System at the very core level this become something that can be used universally used and can have some level of being enforced. Failure to describe the file results it in being dumped in the ?I Was Too Lazy DIR? The Comments, Keywords, and so forth in the file insertion process that require a user to actually type in various data is by its very nature something that people will by pass in the process as it does take time and more importantly thought. Hard Coding a set of flags that function on a few radio button choices would result in better usage IMO as it is faster and it does not take much thought. The benefit of this system is one that benefits all !!!! First Benefit: Bandwidth Savings and easing of congestion is one factor this saves, in terms of there being a file storage system present within FreeNet this gives at least some structure to it. A Person looking for reference materials with in the system are then getting only those file lists. Essentially Some one looking for a Video File likely has no interest in the latest Operating System files that are available. This structure allows for the overhead of all those elements in a file list to be removed from the transmissions till called upon. Second Benefit: Better Presentation !!! Every System that has ever been used that provides for a singular list function for all services or provided what ever leads to the use of computer native sort organization attempts to bring a form of order and grouping to it. Structuring the file system with a hard coded category function in this fashion allows for better presentation along with better system resource use. The Fight to be the first on the list seems to be present in all public list functions and in a file system this happens as well, be this intentional or by nature of the chosen naming conventions. The nature of the Freenet is not traceable and non removable Data, a means to filter that then at least must be present. The Internet its self is about being able to put on a set of rose colored glass's and not see reality. This allows for the opening of the file sharing programs to be one that allows for a more PG view on the entire contents. From toad at amphibian.dyndns.org Wed May 14 19:28:30 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 14 May 2008 20:28:30 +0100 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: <200805142028.30866.toad@amphibian.dyndns.org> On Wednesday 14 May 2008 10:32, Jano wrote: > Matthew Toseland wrote: > > > Below are links to my node's location logs. A lot of the location churn is > > within roughly a slice of 10% of the keyspace, and the central signal is > > reasonably clean... but this degree of variation is going to cause problems > > with routing and especially data retention IMHO (on a network of several > > thousand nodes, with *at least* 1000 nodes online at a time). There were a > > whole bunch of resets during the week 19 April to 26 April, and on the 24th, > > RC2 was released; on May 8th, 0.7.0 was released. These may explain some but > > not all of the long jumps... > > > > http://amphibian.dyndns.org/location.log.txt > > http://amphibian.dyndns.org/locations.png > > See mine attached. This node has not been steadly 24/7 up as of late, which can > be more typical. It bounces a bit... What's the average uptime then? Has it been consistently up at certain times, or more downtime some weeks etc? Your graph is really ugly. :( -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/6f440bb3/attachment.pgp From toad at amphibian.dyndns.org Wed May 14 19:37:29 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 14 May 2008 20:37:29 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <4829BA5F.7080009@gmail.com> References: <4829BA5F.7080009@gmail.com> Message-ID: <200805142037.29598.toad@amphibian.dyndns.org> On Tuesday 13 May 2008 16:57, Dan wrote: > DarkNet VS OpenNet Elements > > With FreeNet yes a DarkNet can leak public but a group that is using it > to cover communications and information can work to restrict it even if > its like the olden days; a memo form of security about not deleting a > file on the mainframe. At present there is no way to force the > restrictions on it but in time the groups that begin to implement this > as a private internet will work on limiting it from breaking out of the > set group. Sure it can, but I'm not convinced as to how useful this is: there's a lot you can do on a small darknet that's much harder on a big darknet, and Freenet is generally designed for big networks. > > I can see this project resulting in not a single world wide private > anonymous internet but resulting in many and the various nets all may or > may not come together in the world. > > Its not too hard to see there being a Christian Crusader Network forming > and an Eastern Jihad Network, a Republican Network, a Democratic > Network, and so on. People have a natural tendency to associate with > like minded groups. :) And *all of them* will have content their users consider objectionable. > > This project offers 2 elements in privacy the first is privacy from the > current network and the internet as its known today and the second is > privacy from each other. The latter in that is one that is under > personal control while a person may not want the world to know they are > part of the network they may be quite happy to have the group with in > the network know even their home address's and names. Privacy from each other is less strong than privacy from an external attacker. > > The Chat Application Function and a stripped down NODE, there is likely > a fair demand for this form of an application for a chat program running > on the freenet network it offers more than just a simple CHAT. > > Running on freenet as a mini node would mean there is continued traffic > making it impossible to pick out a conversation by watching a connection > from a setting of an ISP. The system is encrypted and secured that can > not be monitored. The Anonymous Factor on the chat is also one that > would benefit people as well. Yes talking to your Brother the anon > function is useless but the communications are secured by the system, on > the other hand your also able to chat with a total stranger and there is > no concern they may be able to locate you physically. That's completely different. Real time chat over Freenet, with both peers anonymised, is not currently feasible (well there are prototypes with ~ 15 second latency, but IMHO it will be slow and rely on some unreliable mechanisms...) > > A Chat Application would bring in a huge number of people to this, the > mini node concept would also see there being a couple million TOR style > routers available to the freenet expanding both the speed and security. > > The Seti at Home application running the distributed processing for the > Seti Project was a fairly burdensome program when your computer when > idle on screen saver mode it was quite happy to take over what ever > system resources it could. Often times I would see my system hitting > 100% cpu when I wasnt using it. > > People are willing to "Donate To the Cause" IF your machine is on 24/7 > the 8 hours your sleeping even the Gamers out there wont care so long as > when they are playing the game they dont loose latency. Only if they leave their PC on overnight. Which they won't because it uses power and emits noise (and carbon). > > Mini Node.... enough to access it for chatting, and to function as a > relay for freenet is one idea. > > The second idea is to use the Mini Nodes as MINI SERVERS.... You would > not be able to keep a server on a single IP long 10 - 15 mins and it > would need to be redundant by putting 4 or 8 or what ever together so > that any one server set looses one member the other 4 take over. They > hang out on some ones chat node for 10 mins and move on... I still don't see how you are going to distribute one node over 8 peers. And I don't understand what you are talking about with 10 - 15 minutes either. > > A FreeNet Server Applet..... The Web Pages are fairly static in nature > due to the security using the Chat Mini Nodes as temp servers would > bring a bit of dynamics to the system would be one idea as a use for a > MINI NODE. > > With the Offline Storage Concept. > > >The problem with this is that if it relies on the user coming up with the > >disk, not only will latency be terrible, but in many cases the user > won't be > >bothered - it's just way too much work to find a specific disk, at > least in > >my house! > > In a system where only 3 people have a copy of the file wanted yes > latency would be wait till it gets put in.... but in a system where 5 > million people could be asked if it was a common burned file ( IE: a > win98 DLL that went bad and some one needed a copy of) out of 5 million > that could be asked and the million that have it some one will have it > handy and put it in. If it's a common file it should be already on their system, they shouldn't need to put a disk in. > > And the idea with it is to make it of a benefit to the user as well not > just to the freenet. > > The application I have in mind would provide the user with an index of > the off line files they have stored, as they scan in the cd/dvd the > system builds them a nice little storage index database so that they are > able to find things.... all they need to do is mark them #1 #2 etc. > > The system would also need to generate the file id hash's ideally for a > number of the P2P apps as well as part of the Database. > > With the Hash's all ready if a person goes to click on a file the > database can look it up and tell the user you have that file on disc #12 > already.... (benefit is saves the bandwidth for the freenet and for the > user as well as time) If they can be bothered to find it. In my home there are hundreds of disks... > > The DataBase does not get published your not all of a sudden sharing > everything burnt publicly even on an anon system.... the Freenet uses it > to when it has a file that is either no longer on the system or is only > 78% complete as a few of the nodes no longer have the blocks that would > have completed it. It would have to be published, but it could be published in an encrypted form. > > In a small user base system this would be SLOW but in a large user base > system it could see 500 copies brought online by various users instead > of being a brand new seed in terms of Bit Torrent and it being > propagated back into the free net, with 500 copies it becomes and > instant swarm and you have 90k/sec down. with non of them upping more > than a few K each. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/49e7b127/attachment.pgp From toad at amphibian.dyndns.org Wed May 14 19:40:23 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 14 May 2008 20:40:23 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <482AAF50.9040201@gmail.com> References: <482AAF50.9040201@gmail.com> Message-ID: <200805142040.24020.toad@amphibian.dyndns.org> On Wednesday 14 May 2008 10:22, Dan wrote: > File System Addition. > > > The insertion of files needs to have a category flag system in place > with it with at least the initial first few flags being a forced issue > related to being able to insert the files. The file system in many of > the P2P systems lacks a structure beyond that of a massive root > directory which is dumped into and a total mess. > > > This can be handled with a couple bytes to allow for flagging the files > for basic organization with a hard coded structure, then maybe some > additional space for user breakdowns. > > > The benefit I can see of this would be to bring essentially a few > directories into the file system and allow for the break down of the > file listings into a more specific chunks suited to the category being > sought. > > > With even a 100,000 files the approach of a root directory catch all is > easy enough to work with. When it approaches 20 million or 100 million > files given the exchange of file data I have seen taking place and the > time this takes it will becomes much more of a problematic issue. True, however there is a bigger reason not to have a single global directory: SPAM! Any such system would be flooded, just as Frost has been flooded, by people who want to break the system (or force its devs to build something more secure). > > > Even with the large news servers streaming full it can take an hour or > two for a header list to come in and Free-Net does not make that process > something that can be streamed direct at high speed. > > a Two ? Byte flag system would allow breaking those 16 bits into an > initial category system to allow for some organization. With the use of > a hard coded breakdown of that the forcing of the initial organization > could be made. Any category index would also be attacked. Have you used Freenet recently? > > > The debate would be establishing the hard code structure as to be able > to make use of this and once implemented it would serve as permanent > with in Free-Net. For it to be universally used the first part does have > to be hard coded into the system at a core level > > > To start off the structuring of it the initial debate would be in the > breakdown of the bit pattern. > > > To work with a possible idea for the bit mask breakdown is this: > > > 1bits/3bit /4bits/4bits/ 1bits/3bit/ for a total of 16 bits/2 bytes > > Flags Description Flags > > > First 4 Bits ? used as 1/1/2 > > > First bit being a flag for System File > > Second Bit Default Too Lazy to say what the file is > > Third Bit being a flag for essentially Hidden File or Not like on a HD > file system > > Fourth Bit being Personal or not > > Second 4 Bits being 16 category choices..... > > > FreeNet-Internal, Video, Audio, Book, Game, Application, Picture, OS, > News, Sit-Rep, Other, Misc .... Personal > > > Third Set of 4 Bits a Secondary Description Classification > > > Educational, Entertainment, Informative, Reference Material, Porn, > Urgent, Misc, ...... Personal > > > The option with the third set would be hard coding a sub category for > each of the first choice, or coming up with a generalized list of 16 > items that would fill the general need for a secondary descriptor for > the file describing the first level. > > > Fourth Set of Bits potentially being broken into 3/1 > > > 3 bits being a Content Rating > > 1 being N/A > > and the remaining 7 choices a content rating scale as to age > > 2 for all ages to 8 being adult > > 1 bit being Descriptor is Locked > > if its a public file this is by a polling vote. > > If its a private file this is by owner > > > .................................................. > > > This adds a step to the insertion procedure and adds an additional 2 > bytes to the file key length. With the use of a Hard Coded System at the > very core level this become something that can be used universally used > and can have some level of being enforced. Failure to describe the file > results it in being dumped in the ?I Was Too Lazy DIR? > > > The Comments, Keywords, and so forth in the file insertion process that > require a user to actually type in various data is by its very nature > something that people will by pass in the process as it does take time > and more importantly thought. > > > Hard Coding a set of flags that function on a few radio button choices > would result in better usage IMO as it is faster and it does not take > much thought. > > > The benefit of this system is one that benefits all !!!! > > > First Benefit: > > > Bandwidth Savings and easing of congestion is one factor this saves, in > terms of there being a file storage system present within FreeNet this > gives at least some structure to it. A Person looking for reference > materials with in the system are then getting only those file lists. > > > Essentially Some one looking for a Video File likely has no interest in > the latest Operating System files that are available. This structure > allows for the overhead of all those elements in a file list to be > removed from the transmissions till called upon. > > > Second Benefit: > > > Better Presentation !!! > > > Every System that has ever been used that provides for a singular list > function for all services or provided what ever leads to the use of > computer native sort organization attempts to bring a form of order and > grouping to it. > > > Structuring the file system with a hard coded category function in this > fashion allows for better presentation along with better system resource > use. The Fight to be the first on the list seems to be present in all > public list functions and in a file system this happens as well, be this > intentional or by nature of the chosen naming conventions. > > > The nature of the Freenet is not traceable and non removable Data, a > means to filter that then at least must be present. The Internet its > self is about being able to put on a set of rose colored glass's and not > see reality. This allows for the opening of the file sharing programs to > be one that allows for a more PG view on the entire contents. > > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080514/56a77a02/attachment.pgp From ian.clarke at gmail.com Wed May 14 19:42:44 2008 From: ian.clarke at gmail.com (Ian Clarke) Date: Wed, 14 May 2008 14:42:44 -0500 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: <823242bd0805141242h34494cd6we468212b1321b36f@mail.gmail.com> 2008/5/14 Jano : > See mine attached. This node has not been steadly 24/7 up as of late, which can > be more typical. It bounces a bit... This is just a somewhat far-fetched theory, but I wonder if somehow large changes in location cause a "cascade" effect in the network. So, one node's location is reset, but instead of it returning to its previous position after some time, it causes one or more of its neighbors to change position - and the effect spreads through the network. If this was happening, one remedy would be to only count node's locations in the swap algorithm if they haven't had a significant change recently. Of course, this is only a theory... We should simulate node resetting to see if we can observe this kind of effect. Ian. -- Email: ian at uprizer.com Cell: +1 512 422 3588 Skype: sanity From dan.bruce at gmail.com Wed May 14 15:22:37 2008 From: dan.bruce at gmail.com (Dan) Date: Wed, 14 May 2008 17:22:37 +0200 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <482B03BD.904@gmail.com> >I still don't see how you are going to distribute one node over 8 >peers. And I don't understand what you are talking about with 10 - 15 >minutes either. Using the people running a Chat Application as a Miniture Server to run an applet, Say a 6 person Chat Room. The idea would be that the Mini Server maintains basic communications I would guess in a "T" form to a back up server which in turn has 2 it maintains. The primary server being located at the bottom of the "T" The Channel as it is has the locations where to meet up should the primary server fail for what ever reason Net Burp, BSoD, Power Outage. The Primary Server functions in that role for 10-15-20 minutes then pass's the Role of Primary to the Center of the "T" being released from the role of server for that group and seeks out a new job for another 20 mins. The New Primary Mini Server once it has its 3 new replacements releases its 2 back ups and those two then seek out new job assignments. The Chat Clients pick the 4 for the job not the servers. Moving the Server Element in this form every what ever minutes limits the value of any recording being done at that location, IP Tracking on the mini - server would result in 20 mins or what ever of activity that is further buffered by being communications between the server applet and 6 Chat Clients functioning as a TOR style relay This prevents any tracking of the data exchange by ISP or even by an official serving as a CHATTER. 1. the Channel that forms the members have the keys not the server so the converstaion is not able to be monitored. 2. The Client Members are the ones that pick up the untrusted components to fill the "T" not the mini server. This being the case even if the Mini Server is being ran on an officials site for monitoring they are only getting relay address points and not user IP anything. 3. Moving in a fairly short time also prevents any pattern checking to be used against it. Ignoring the concept of requiring a Court Order and such, the Red Tape on just an ISP to ISP call asking to quickly start to trace on the connections happening around IP Blaa would see the server applet move and the group before any form of cooperation in that form could be attained. Granted if the IP's are all with in a single ISP they would not have that issue but the internet spans not only companies but borders. The bandwidth use could also allow the movement of 1/2 a data package as well for the FreeNet allowing for a server that is chatting only to still play TOR Relay for the Free Net. From j16sdiz+freenet at gmail.com Thu May 15 00:03:13 2008 From: j16sdiz+freenet at gmail.com (Daniel Cheng) Date: Thu, 15 May 2008 08:03:13 +0800 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: 2008/5/14 Jano : > Matthew Toseland wrote: > >> Below are links to my node's location logs. A lot of the location churn is >> within roughly a slice of 10% of the keyspace, and the central signal is >> reasonably clean... but this degree of variation is going to cause problems >> with routing and especially data retention IMHO (on a network of several >> thousand nodes, with *at least* 1000 nodes online at a time). There were a >> whole bunch of resets during the week 19 April to 26 April, and on the 24th, >> RC2 was released; on May 8th, 0.7.0 was released. These may explain some but >> not all of the long jumps... >> >> http://amphibian.dyndns.org/location.log.txt >> http://amphibian.dyndns.org/locations.png > > See mine attached. This node has not been steadly 24/7 up as of late, which can > be more typical. It bounces a bit... > How many opennet connection do you have? From alejandro at mosteo.com Thu May 15 10:46:00 2008 From: alejandro at mosteo.com (Jano) Date: Thu, 15 May 2008 12:46:00 +0200 Subject: [Tech] Location churn: empirical data References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: Daniel Cheng wrote: > 2008/5/14 Jano : >> Matthew Toseland wrote: >> >>> Below are links to my node's location logs. A lot of the location churn is >>> within roughly a slice of 10% of the keyspace, and the central signal is >>> reasonably clean... but this degree of variation is going to cause problems >>> with routing and especially data retention IMHO (on a network of several >>> thousand nodes, with *at least* 1000 nodes online at a time). There were a >>> whole bunch of resets during the week 19 April to 26 April, and on the >>> 24th, RC2 was released; on May 8th, 0.7.0 was released. These may explain >>> some but not all of the long jumps... >>> >>> http://amphibian.dyndns.org/location.log.txt >>> http://amphibian.dyndns.org/locations.png >> >> See mine attached. This node has not been steadly 24/7 up as of late, which >> can be more typical. It bounces a bit... >> > > How many opennet connection do you have? This node is pure opennet. So 20, I think. From alejandro at mosteo.com Thu May 15 10:58:30 2008 From: alejandro at mosteo.com (Jano) Date: Thu, 15 May 2008 12:58:30 +0200 Subject: [Tech] Location churn: empirical data References: <200805132203.45605.toad@amphibian.dyndns.org> <200805142028.30866.toad@amphibian.dyndns.org> Message-ID: Matthew Toseland wrote: > On Wednesday 14 May 2008 10:32, Jano wrote: >> Matthew Toseland wrote: >> >> > Below are links to my node's location logs. A lot of the location churn is >> > within roughly a slice of 10% of the keyspace, and the central signal is >> > reasonably clean... but this degree of variation is going to cause > problems >> > with routing and especially data retention IMHO (on a network of several >> > thousand nodes, with *at least* 1000 nodes online at a time). There were a >> > whole bunch of resets during the week 19 April to 26 April, and on the > 24th, >> > RC2 was released; on May 8th, 0.7.0 was released. These may explain some > but >> > not all of the long jumps... >> > >> > http://amphibian.dyndns.org/location.log.txt >> > http://amphibian.dyndns.org/locations.png >> >> See mine attached. This node has not been steadly 24/7 up as of late, which > can >> be more typical. It bounces a bit... > > What's the average uptime then? Has it been consistently up at certain times, > or more downtime some weeks etc? This node used to be 24/7 up until recently when I started having HW problems. The location is sampled every hour, not on changes (we could be missing resets here). So, in the graph, you can see the downtimes as the periods without samples. As you can see, the steady drifting and most jumps aren't caused by downtime. I see big downtimes in 29-31/mar 11-15/apr 3-12/may There are no "scheduled" downtimes, this box is always on, unless some crash takes it down until I'm back from work or week-end. I think I've solved the HW issues now, and I have a particular interest in having this node up, so we can see more evolution in the future. > Your graph is really ugly. :( The spikes in times of good uptime are particularly noticeable. Would they be randomization resets? From toad at amphibian.dyndns.org Thu May 15 14:09:07 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 15 May 2008 15:09:07 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <482B03BD.904@gmail.com> References: <482B03BD.904@gmail.com> Message-ID: <200805151509.08296.toad@amphibian.dyndns.org> Sounds centralised, and not very freenet-ish. Note that if an ISP had a court order, they'd just wait for it to rotate, and then track it. On Wednesday 14 May 2008 16:22, Dan wrote: > >I still don't see how you are going to distribute one node over 8 > >peers. And I don't understand what you are talking about with 10 - 15 > >minutes either. > > Using the people running a Chat Application as a Miniture Server to run > an applet, Say a 6 person Chat Room. The idea would be that the Mini > Server maintains basic communications I would guess in a "T" form to a > back up server which in turn has 2 it maintains. > > The primary server being located at the bottom of the "T" > > The Channel as it is has the locations where to meet up should the > primary server fail for what ever reason Net Burp, BSoD, Power Outage. > > The Primary Server functions in that role for 10-15-20 minutes then > pass's the Role of Primary to the Center of the "T" being released from > the role of server for that group and seeks out a new job for another 20 > mins. The New Primary Mini Server once it has its 3 new replacements > releases its 2 back ups and those two then seek out new job assignments. > > The Chat Clients pick the 4 for the job not the servers. > > Moving the Server Element in this form every what ever minutes limits > the value of any recording being done at that location, IP Tracking on > the mini - server would result in 20 mins or what ever of activity that > is further buffered by being communications between the server applet > and 6 Chat Clients functioning as a TOR style relay > > This prevents any tracking of the data exchange by ISP or even by an > official serving as a CHATTER. > > 1. the Channel that forms the members have the keys not the server so > the converstaion is not able to be monitored. > > 2. The Client Members are the ones that pick up the untrusted components > to fill the "T" not the mini server. This being the case even if the > Mini Server is being ran on an officials site for monitoring they are > only getting relay address points and not user IP anything. > > 3. Moving in a fairly short time also prevents any pattern checking to > be used against it. > > Ignoring the concept of requiring a Court Order and such, the Red Tape > on just an ISP to ISP call asking to quickly start to trace on the > connections happening around IP Blaa would see the server applet move > and the group before any form of cooperation in that form could be attained. > > Granted if the IP's are all with in a single ISP they would not have > that issue but the internet spans not only companies but borders. > > The bandwidth use could also allow the movement of 1/2 a data package as > well for the FreeNet allowing for a server that is chatting only to > still play TOR Relay for the Free Net. > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080515/9d7caa9c/attachment.pgp From toad at amphibian.dyndns.org Thu May 15 14:11:12 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 15 May 2008 15:11:12 +0100 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> <200805142028.30866.toad@amphibian.dyndns.org> Message-ID: <200805151511.12244.toad@amphibian.dyndns.org> On Thursday 15 May 2008 11:58, Jano wrote: > Matthew Toseland wrote: > > > On Wednesday 14 May 2008 10:32, Jano wrote: > >> Matthew Toseland wrote: > >> > >> > Below are links to my node's location logs. A lot of the location churn is > >> > within roughly a slice of 10% of the keyspace, and the central signal is > >> > reasonably clean... but this degree of variation is going to cause > > problems > >> > with routing and especially data retention IMHO (on a network of several > >> > thousand nodes, with *at least* 1000 nodes online at a time). There were a > >> > whole bunch of resets during the week 19 April to 26 April, and on the > > 24th, > >> > RC2 was released; on May 8th, 0.7.0 was released. These may explain some > > but > >> > not all of the long jumps... > >> > > >> > http://amphibian.dyndns.org/location.log.txt > >> > http://amphibian.dyndns.org/locations.png > >> > >> See mine attached. This node has not been steadly 24/7 up as of late, which > > can > >> be more typical. It bounces a bit... > > > > What's the average uptime then? Has it been consistently up at certain times, > > or more downtime some weeks etc? > > This node used to be 24/7 up until recently when I started having HW problems. > The location is sampled every hour, not on changes (we could be missing resets > here). So, in the graph, you can see the downtimes as the periods without > samples. As you can see, the steady drifting and most jumps aren't caused by > downtime. > > I see big downtimes in > > 29-31/mar > 11-15/apr > 3-12/may > > There are no "scheduled" downtimes, this box is always on, unless some crash > takes it down until I'm back from work or week-end. > > I think I've solved the HW issues now, and I have a particular interest in > having this node up, so we can see more evolution in the future. > > > Your graph is really ugly. :( > > The spikes in times of good uptime are particularly noticeable. Would they be > randomization resets? It looks like the spikes (long lines) result from long downtime, and when the node reconnects, it resets for some reason, presumably because its neighbours have moved on?? Is it pure opennet? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080515/e4765d45/attachment.pgp From toad at amphibian.dyndns.org Thu May 15 14:11:56 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Thu, 15 May 2008 15:11:56 +0100 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> Message-ID: <200805151511.56521.toad@amphibian.dyndns.org> On Thursday 15 May 2008 11:46, Jano wrote: > Daniel Cheng wrote: > > > 2008/5/14 Jano : > >> Matthew Toseland wrote: > >> > >>> Below are links to my node's location logs. A lot of the location churn is > >>> within roughly a slice of 10% of the keyspace, and the central signal is > >>> reasonably clean... but this degree of variation is going to cause problems > >>> with routing and especially data retention IMHO (on a network of several > >>> thousand nodes, with *at least* 1000 nodes online at a time). There were a > >>> whole bunch of resets during the week 19 April to 26 April, and on the > >>> 24th, RC2 was released; on May 8th, 0.7.0 was released. These may explain > >>> some but not all of the long jumps... > >>> > >>> http://amphibian.dyndns.org/location.log.txt > >>> http://amphibian.dyndns.org/locations.png > >> > >> See mine attached. This node has not been steadly 24/7 up as of late, which > >> can be more typical. It bounces a bit... > >> > > > > How many opennet connection do you have? > > This node is pure opennet. So 20, I think. Ok, in that case each time there is long downtime it will have to reseed... even so, seeding is directional... but your new nodes won't be the same as your old nodes, so it may partly explain it. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080515/e75921a0/attachment.pgp From dan.bruce at gmail.com Thu May 15 07:45:21 2008 From: dan.bruce at gmail.com (Dan) Date: Thu, 15 May 2008 09:45:21 +0200 Subject: [Tech] Distributed Drive and Increased Usage Message-ID: <482BEA11.4060706@gmail.com> What I can see happening with this as FreeNet advances is the migration of the crowds that are on the other p2p services running. Emule and the related at a quick glance this morning has 3.2 Million Users showing, 316 Million files. For a rough on Bit Torrent TPB is showing 10 million peers and 1.2 Million Torrents going. Both Systems have a form of server contact that makes them open to attack not only in disruptions of service but legal issues related to "File Sharing" regardless of what the content is. Free-Net offers considerable more in a service than just file swapping, and it is very much like the old Gopher Service. That was not a very neatly organized system relying heavily on bookmarks and old fashion digging to find the gems on the system. There where a few index's but it was a lot of looking around and exploring. Should freenet see the migration of the current p2p file swapping this can see the system go from a couple thousand nodes into the range of double digit millions. A large portion of that crowd will remain on an open net set up its human nature, click install it opens type your search click download if it works never look back. With the file system alone given the insert proceedure if you take 300 million files calling it say 250meg as a an average that is 75 million GB. Using a 3 copy redundancy 225 million gig, at 10 million users even that is a 22.5 Gig File Store Each. If FreeNet achieves the security making it impossible to then trace the two alternatives left to the powers that be are out right banning the program making it illegal to run in any form. And the second is attack the network directly from the inside with the security in place the only option would be to then Flood and Abuse it to a point of no longer usable. Easiest abuse in this would be the continuous insertion of faked files, 200 locations uploading a dvd or blu ray image and a couple hundred others making requests to keep the system burdened. OpenNet/DarkNet offers some defense to this form of attack From dan.bruce at gmail.com Thu May 15 09:06:08 2008 From: dan.bruce at gmail.com (Dan) Date: Thu, 15 May 2008 11:06:08 +0200 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805151509.08296.toad@amphibian.dyndns.org> References: <482B03BD.904@gmail.com> <200805151509.08296.toad@amphibian.dyndns.org> Message-ID: <482BFD00.8070800@gmail.com> Matthew Toseland wrote: > Sounds centralised, and not very freenet-ish. Note that if an ISP had a court > order, they'd just wait for it to rotate, and then track it. The Server/Client model is what has made the internet and the basis of 99% of the workings of the internet. A single IRC Server running 20,000 channels and serving 2 million users is impractical under the freenet model. A centralized server becomes then the target of any action. This is de-centralizing that model, using the bundled application concept to give an ICQ type tool to the general public that gives the user a true anonymous private and secured chat function. The return being able to utilize them as an IP Bounce Relay and run an applet. What applets they could possible run would depend on bandwidth and resource availability and permissions. To eliminate anyones ability to trace what is needed is one of two things. Either a variable burst which is what the PGP Remailers do store incoming messages a variable time to prevent knowing when or if a message was relayed and sending multiples out at the same time. Or a continuous flow of identical information making it impossible to determine which of the packets is meaning full. In terms of The Powers That Be(TPTB) with either of these options they are limited in what information can be gained. An ISP or TPTB watching a mini node working in this roll would not see a change in anything at the location in terms of traffic flow or even a packet change other than the connections made changing. A true anonymous secure private chat utility if it is taken on by the general public to become the norm would then bring in 60+ million users. With respect to the previous post if there is 10 Million Users in the file swap crowd migrating to this it results in a 6 to 1 ratio in additional bandwith and HOP Points. Mini Servers that the Chat Crowd would be able to bring services into the freenet functioning in an encrypted fashion would be as some ideas: Small Chat Room: 6 or 8 depending on bandwith want more you chain two together but you become the relay and not the mini server as to not swamp someone donating to this; Confernce call style. You would never be able to get the 5,000 user rooms that irc had but 25 by chaining 4 maybe possible. MINI DNS Server: they could serve as a miniture DNS Server for the freenet Virtual Router: Interactive Content Feed: A varriation on an RSS type server. File List Distribution. .................................................. With regard to the post on the file system having a hard coded category system in the actual protocol. The best explanation of the idea is the hard coded flags a hard drive has hidden,system,archive, read only. There will be abuse of this I am sure, but the idea does bring structure and it has to be implemented at the very core level to be used and remain being used. Further thinking on the concept of two category flags for the bit it may be better to expand it to a 3 byte flag setting and increase each section by a bit from 16 to 32 choices. The other is each section in the 2 choices needs to be independent of the other and not rely on the first choice. Rather than the system seeking out the entire files list, based on either of the 2 catagory settings a user would be requesting the return of one of 16X2-32 lists or 32X2-64 list. A file being added would then appear twice, once in each category list. The retrieval of files would be based on what you are seeking if you are looking for a video file them MP3 List and Books is essentially wasted bandwidth. The Enqued File List Download in Frost 1500 some days and can take the day to finish would be a nightmare with 10 Million users coming into this, 380days left to complete type of a setting. .................................................. Hash Key form of retrieval would allow resuming and such of files partially down from any number of sources and things like http://bitzi.com/search/ have the hash data on some 2.7 Million Files SHA1, TigerTree, ED2K, KZHASH hash's are all present and I would suggest where possible freenet should emply all 4 along with the MHD5 hash and also a freenet hash system From dan.bruce at gmail.com Thu May 15 16:09:31 2008 From: dan.bruce at gmail.com (Dan) Date: Thu, 15 May 2008 18:09:31 +0200 Subject: [Tech] Some Questions In-Reply-To: <200805151509.08296.toad@amphibian.dyndns.org> References: <482B03BD.904@gmail.com> <200805151509.08296.toad@amphibian.dyndns.org> Message-ID: <482C603B.4040801@gmail.com> Some Questions: Is there any stats with regard to how many users are on FreeNet both 0.5 and 0.7 ? and any way for there to be a users online function / files online with in it. I am having a hard time getting a clear picture of the Protocol Break Down that FreeNet is using for net transportation, and does anyone have some idea on the fragmentation and re - transmission rates for packets and the sizes of the packets being resent. Specifics I am looking for is a byte digram showing how many bytes for each step of the packet and then the data size. I am trying to understand is the byte overhead level the packets have with in the freenet, in the transfer process. I have a potential idea for a one way direction transfer needing only a handshake to initiate there is no ack needed and requires only a request for partial re-transmission if there is loss. It is able to withstand some packet loss but to be of value I need to have some idea on what the overheads are on the current system. From alejandro at mosteo.com Fri May 16 09:36:19 2008 From: alejandro at mosteo.com (Jano) Date: Fri, 16 May 2008 11:36:19 +0200 Subject: [Tech] Location churn: empirical data References: <200805132203.45605.toad@amphibian.dyndns.org> <200805151511.56521.toad@amphibian.dyndns.org> Message-ID: Matthew Toseland wrote: > On Thursday 15 May 2008 11:46, Jano wrote: >> Daniel Cheng wrote: >> >> > 2008/5/14 Jano : >> >> Matthew Toseland wrote: >> >> >> >>> Below are links to my node's location logs. A lot of the location churn > is >> >>> within roughly a slice of 10% of the keyspace, and the central signal is >> >>> reasonably clean... but this degree of variation is going to cause > problems >> >>> with routing and especially data retention IMHO (on a network of several >> >>> thousand nodes, with *at least* 1000 nodes online at a time). There were > a >> >>> whole bunch of resets during the week 19 April to 26 April, and on the >> >>> 24th, RC2 was released; on May 8th, 0.7.0 was released. These may > explain >> >>> some but not all of the long jumps... >> >>> >> >>> http://amphibian.dyndns.org/location.log.txt >> >>> http://amphibian.dyndns.org/locations.png >> >> >> >> See mine attached. This node has not been steadly 24/7 up as of late, > which >> >> can be more typical. It bounces a bit... >> >> >> > >> > How many opennet connection do you have? >> >> This node is pure opennet. So 20, I think. > > Ok, in that case each time there is long downtime it will have to reseed... > even so, seeding is directional... but your new nodes won't be the same as > your old nodes, so it may partly explain it. I thought that even in opennet, the node would try to reuse its old peers after restarting. Is this so? Still, I understand that long downtimes may cause more of your peers to vanish. From toad at amphibian.dyndns.org Fri May 16 11:23:21 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Fri, 16 May 2008 12:23:21 +0100 Subject: [Tech] Location churn: empirical data In-Reply-To: References: <200805132203.45605.toad@amphibian.dyndns.org> <200805151511.56521.toad@amphibian.dyndns.org> Message-ID: <200805161223.21562.toad@amphibian.dyndns.org> On Friday 16 May 2008 10:36, Jano wrote: > Matthew Toseland wrote: > > > On Thursday 15 May 2008 11:46, Jano wrote: > >> Daniel Cheng wrote: > >> > >> > 2008/5/14 Jano : > >> >> Matthew Toseland wrote: > >> >> > >> >>> Below are links to my node's location logs. A lot of the location churn > > is > >> >>> within roughly a slice of 10% of the keyspace, and the central signal is > >> >>> reasonably clean... but this degree of variation is going to cause > > problems > >> >>> with routing and especially data retention IMHO (on a network of several > >> >>> thousand nodes, with *at least* 1000 nodes online at a time). There were > > a > >> >>> whole bunch of resets during the week 19 April to 26 April, and on the > >> >>> 24th, RC2 was released; on May 8th, 0.7.0 was released. These may > > explain > >> >>> some but not all of the long jumps... > >> >>> > >> >>> http://amphibian.dyndns.org/location.log.txt > >> >>> http://amphibian.dyndns.org/locations.png > >> >> > >> >> See mine attached. This node has not been steadly 24/7 up as of late, > > which > >> >> can be more typical. It bounces a bit... > >> >> > >> > > >> > How many opennet connection do you have? > >> > >> This node is pure opennet. So 20, I think. > > > > Ok, in that case each time there is long downtime it will have to reseed... > > even so, seeding is directional... but your new nodes won't be the same as > > your old nodes, so it may partly explain it. > > I thought that even in opennet, the node would try to reuse its old peers after > restarting. Is this so? Still, I understand that long downtimes may cause more > of your peers to vanish. Yes, but if they are behind a NAT and not port forwarded they won't receive the packets sent by the node; they may have changed their IP if they're on a dynamic IP; and they may not want a peer. Long downtimes generally cause reseeding. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080516/d59dfd61/attachment.pgp From thelema314 at gmail.com Sat May 17 15:25:39 2008 From: thelema314 at gmail.com (Edgar Friendly) Date: Sat, 17 May 2008 10:25:39 -0500 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <200805142037.29598.toad@amphibian.dyndns.org> References: <4829BA5F.7080009@gmail.com> <200805142037.29598.toad@amphibian.dyndns.org> Message-ID: <482EF8F3.502@gmail.com> Matthew Toseland wrote: > That's completely different. Real time chat over Freenet, with both peers > anonymised, is not currently feasible (well there are prototypes with ~ 15 > second latency, but IMHO it will be slow and rely on some unreliable > mechanisms...) Hmm, I wonder if freenet would be usable for a twitter-style communication. Twitter's needs for "real-time" are much less, and the broadcast nature of twitter might fit freenet's architecture. Just a quick thought. E From m2c at nym.panta-rhei.eu.org Sat May 17 19:29:43 2008 From: m2c at nym.panta-rhei.eu.org (MyTwoCents) Date: 17 May 2008 19:29:43 -0000 Subject: [Tech] Distributed Drive and Increased Usage References: <4829BA5F.7080009@gmail.com> <200805142037.29598.toad@amphibian.dyndns.org> <482EF8F3.502@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 17 May 2008, Edgar Friendly wrote: >Matthew Toseland wrote: >> That's completely different. Real time chat over Freenet, with both peers >> anonymised, is not currently feasible (well there are prototypes with ~ 15 >> second latency, but IMHO it will be slow and rely on some unreliable >> mechanisms...) > >Hmm, I wonder if freenet would be usable for a twitter-style >communication. Twitter's needs for "real-time" are much less, and the >broadcast nature of twitter might fit freenet's architecture. > >Just a quick thought. Ah, but what would you call it? "FreeTwit"? Or better yet ... "TwitFree" (sorry, I simply couldn't resist) - -- My public keys can be found on my freenet site: SSK at TEx6TiaPeszpV4AFw3ToutDb49EPAgM/mytwocents/63//m2ckey.html (*NOTE* you must be running freenet for this link to be usefull) and on public keyservers. Key-Id: 0x92769D7E Fingerprint: 2F07D586C8D4EEA732711338CFEF46E592769D7E I can be reached either by the NiM form on the freesite or by Email: m2c AT nym.panta-rhei.eu.org Frost: MyTwoCents at Z+59LNK9NhMvxewYggENU4Ww50s On the 0.5 Freenet board -----BEGIN PGP SIGNATURE----- Version: N/A iQA/AwUBSC8VVZ5/ZUtfDwnNEQJQxgCfcc8uvitY+sEYeYJS4rcDtiY4sWcAnjYK hbeV494E2QJhBgxJREA8csL9 =xNyM -----END PGP SIGNATURE----- From toad at amphibian.dyndns.org Wed May 21 11:16:57 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 21 May 2008 12:16:57 +0100 Subject: [Tech] Distributed Drive and Increased Usage In-Reply-To: <482BFD00.8070800@gmail.com> References: <482B03BD.904@gmail.com> <200805151509.08296.toad@amphibian.dyndns.org> <482BFD00.8070800@gmail.com> Message-ID: <200805211217.03452.toad@amphibian.dyndns.org> On Thursday 15 May 2008 10:06, Dan wrote: > Matthew Toseland wrote: > > Sounds centralised, and not very freenet-ish. Note that if an ISP had a court > > order, they'd just wait for it to rotate, and then track it. > > The Server/Client model is what has made the internet and the basis of > 99% of the workings of the internet. > > A single IRC Server running 20,000 channels and serving 2 million users > is impractical under the freenet model. A centralized server becomes > then the target of any action. > > This is de-centralizing that model, using the bundled application > concept to give an ICQ type tool to the general public that gives the > user a true anonymous private and secured chat function. The return > being able to utilize them as an IP Bounce Relay and run an applet. What > applets they could possible run would depend on bandwidth and resource > availability and permissions. Yeah, a chat client that uses 10K/sec+ continually and 100MB of RAM is going to be really popular! > > With regard to the post on the file system having a hard coded category > system in the actual protocol. The best explanation of the idea is the > hard coded flags a hard drive has hidden,system,archive, read only. > > There will be abuse of this I am sure, but the idea does bring structure > and it has to be implemented at the very core level to be used and > remain being used. It will be spammed to the point of being totally unusable. Any global index system will be. Just like Frost has been. > > Further thinking on the concept of two category flags for the bit it may > be better to expand it to a 3 byte flag setting and increase each > section by a bit from 16 to 32 choices. > > The other is each section in the 2 choices needs to be independent of > the other and not rely on the first choice. > > Rather than the system seeking out the entire files list, based on > either of the 2 catagory settings a user would be requesting the return > of one of 16X2-32 lists or 32X2-64 list. > > A file being added would then appear twice, once in each category list. > > The retrieval of files would be based on what you are seeking if you are > looking for a video file them MP3 List and Books is essentially wasted > bandwidth. > > The Enqued File List Download in Frost 1500 some days and can take the > day to finish would be a nightmare with 10 Million users coming into > this, 380days left to complete type of a setting. > > .................................................. > > Hash Key form of retrieval would allow resuming and such of files > partially down from any number of sources and things like > http://bitzi.com/search/ have the hash data on some 2.7 Million Files We have used hash keys since Freenet 0.4. > > SHA1, TigerTree, ED2K, KZHASH hash's are all present and I would suggest > where possible freenet should emply all 4 along with the MHD5 hash and > also a freenet hash system Our hash keys are not compatible with other networks' hash keys. There are means to try to bridge to gap to some degree, but they have not been implemented yet. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080521/1e732d80/attachment.pgp From toad at amphibian.dyndns.org Wed May 21 11:19:13 2008 From: toad at amphibian.dyndns.org (Matthew Toseland) Date: Wed, 21 May 2008 12:19:13 +0100 Subject: [Tech] Some Questions In-Reply-To: <482C603B.4040801@gmail.com> References: <482B03BD.904@gmail.com> <200805151509.08296.toad@amphibian.dyndns.org> <482C603B.4040801@gmail.com> Message-ID: <200805211219.13166.toad@amphibian.dyndns.org> On Thursday 15 May 2008 17:09, Dan wrote: > Some Questions: > > Is there any stats with regard to how many users are on FreeNet both 0.5 > and 0.7 ? and any way for there to be a users online function / files > online with in it. A few thousand at least, at least during the /. period. > > I am having a hard time getting a clear picture of the Protocol Break > Down that FreeNet is using for net transportation, and does anyone have > some idea on the fragmentation and re - transmission rates for packets > and the sizes of the packets being resent. > > Specifics I am looking for is a byte digram showing how many bytes for > each step of the packet and then the data size. Would you please read the documentation? Start with the What is Freenet page, then move on to the FAQ, the wiki and the papers section. Then you might have more of an idea of what is going on, and should ask questions to fill in any gaps. > > I am trying to understand is the byte overhead level the packets have > with in the freenet, in the transfer process. > > I have a potential idea for a one way direction transfer needing only a > handshake to initiate there is no ack needed and requires only a request > for partial re-transmission if there is loss. It is able to withstand > some packet loss but to be of value I need to have some idea on what the > overheads are on the current system. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080521/eb4ea665/attachment.pgp From burgechris at gmail.com Sat May 24 13:59:55 2008 From: burgechris at gmail.com (Chris Burge) Date: Sat, 24 May 2008 09:59:55 -0400 Subject: [Tech] Programming Language Message-ID: <22cafe8b0805240659p7cdb330ekd9910377ac107062@mail.gmail.com> Hey! If I wanted to intreface with freenet with another application (opensource of course) which language would be best? I saw a Python API but I haven't seen any instructions for Java or Ruby. The reason I ask is that I would rather do it via an existing API rather than hacking my way around the place. Thanks, Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/tech/attachments/20080524/463d7455/attachment.htm From batosai at batosai.net Sat May 24 19:15:34 2008 From: batosai at batosai.net (Julien Cornuwel) Date: Sat, 24 May 2008 21:15:34 +0200 Subject: [Tech] Programming Language In-Reply-To: <22cafe8b0805240659p7cdb330ekd9910377ac107062@mail.gmail.com> References: <22cafe8b0805240659p7cdb330ekd9910377ac107062@mail.gmail.com> Message-ID: <48386956.30800@batosai.net> Chris Burge a ?crit : > Hey! If I wanted to intreface with freenet with another application > (opensource of course) which language would be best? I saw a Python API > but I haven't seen any instructions for Java or Ruby. The reason I ask > is that I would rather do it via an existing API rather than hacking my > way around the place. There is also jFCPlib, a Java API by Bombe. Otherwise, any language can do the trick : you just have to open a socket to the node and use the FCP specification : http://wiki.freenetproject.org/FreenetFCPSpec2Point0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080524/4fef780e/attachment.pgp From burgechris at gmail.com Sat May 24 23:06:53 2008 From: burgechris at gmail.com (Chris Burge) Date: Sat, 24 May 2008 19:06:53 -0400 Subject: [Tech] Programming Language In-Reply-To: <48386956.30800@batosai.net> References: <22cafe8b0805240659p7cdb330ekd9910377ac107062@mail.gmail.com> <48386956.30800@batosai.net> Message-ID: <22cafe8b0805241606j54d7440t9b84792e8221f752@mail.gmail.com> Thanks! Chris 2008/5/24 Julien Cornuwel : > Chris Burge a ?crit : > > Hey! If I wanted to intreface with freenet with another application > > (opensource of course) which language would be best? I saw a Python API > > but I haven't seen any instructions for Java or Ruby. The reason I ask > > is that I would rather do it via an existing API rather than hacking my > > way around the place. > > There is also jFCPlib, a Java API by Bombe. > > Otherwise, any language can do the trick : you just have to open a > socket to the node and use the FCP specification : > http://wiki.freenetproject.org/FreenetFCPSpec2Point0 > > > > _______________________________________________ > Tech mailing list > Tech at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech > -- Interested in selling your home? Ask me! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://emu.freenetproject.org/pipermail/tech/attachments/20080524/35e01d59/attachment.htm From alejandro at mosteo.com Wed May 28 11:51:05 2008 From: alejandro at mosteo.com (Jano) Date: Wed, 28 May 2008 13:51:05 +0200 Subject: [Tech] Programming Language References: <22cafe8b0805240659p7cdb330ekd9910377ac107062@mail.gmail.com> <48386956.30800@batosai.net> Message-ID: Julien Cornuwel wrote: > Chris Burge a ?crit : >> Hey! If I wanted to intreface with freenet with another application >> (opensource of course) which language would be best? I saw a Python API >> but I haven't seen any instructions for Java or Ruby. The reason I ask >> is that I would rather do it via an existing API rather than hacking my >> way around the place. > > There is also jFCPlib, a Java API by Bombe. And an Ada one. The wiki list them here: http://wiki.freenetproject.org/FreenetDevTools > Otherwise, any language can do the trick : you just have to open a > socket to the node and use the FCP specification : > http://wiki.freenetproject.org/FreenetFCPSpec2Point0 From 911freak at gmail.com Sat May 31 21:03:24 2008 From: 911freak at gmail.com (Ahmed MANSOUR) Date: Sat, 31 May 2008 21:03:24 +0000 Subject: [Tech] prototype of wiki inside freenet Message-ID: Hi, I discovered recently a wiki system called "wiki on the stick" and TiddlyWiki.com, they are both single file wiki system made in JavaScript so they run inside the browser without the need of a webserver or other scripting languages. I think this kind of wiki can be useful in the freenet: the user download the file, change it and then upload the modified single file back to the network. please try it and tell me what to you think about? http://www.tiddlywiki.com/ http://sf.net/projects/stickwiki/