[Tech] Distributed file system using routing inspired by Freenet

[Matthew Toseland]

On Wednesday 09 April 2008 09:24, Michael Rogers wrote:
> On Apr 8 2008, Matthew Toseland wrote:
> >Sure. It's not directly comparable. However, hopefully we have a higher 
> >average uptime, and the factor of 3 replication in the stores is necessary 
> >because of non-splitfile blocks.
> 
> In theory would it be possible to use FEC for all blocks, or is it wasteful 
> if you have less than one segment's worth of data?

Well the problem is what to do with single blocks... A frost post is an SSK 
plus usually a CHK for example. If a splitfile is inserted as a CHK, there 
will be a single top level block for the CHK. Granted these are more popular 
than the splitfile blocks...
> 
> >> I'm not convinced their churn model is realistic - they assume that the 
> >> nodes' uptimes are independent, but studies of Gnutella and Skype show 
> >> strong daily and weekly cycles - if each node is online 25% of the time 
> >> it doesn't follow that 25% of the nodes are online at any given time.
> >
> >True. What would the impact of this be?
> 
> There would be fewer nodes online at certain points in the cycle (eg 4am on 
> Monday morning) so you'd need higher redundancy (or at least a warning 
> saying "No glot, clom Fliday").
> 
> >> > So maybe what we need is less network level redundancy and more FEC 
> >> > level redundancy?
> >> 
> >> Sounds like a good idea, although won't it lead to higher search 
> >> overhead (each FEC block will be replicated fewer times)?
> >
> > That may not be a bad thing, but just like on Wuala, we have 
> > requestor-side healing...
> 
> True, but I was thinking of the number of hops the search will have to 
> travel: one block replicated three times can be found more cheaply than one 
> of three blocks replicated once each. (Higher FEC redundancy is still a 
> good idea IMO, I'm just thinking about the tradeoffs.)

Sure, but more hops also means more storage capacity, more reliable content. 
On the other hand it means higher bandwidth overhead - but a lot of our 
bandwidth overhead is searching for stuff we can't find at the moment.
> 
> >Perhaps... I was thinking more in terms of storing stuff on nodes with 
> >reasonable uptimes...
> 
> Good point, I guess it's a waste of bandwidth to store data on a transient 
> node.

But how would we implement that?
> 
> Cheers,
> Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080409/2ffd635a/attachment.pgp