[Tech] Distributed file system using routing inspired by Freenet

[Matthew Toseland]

On Tuesday 08 April 2008 21:09, Michael Rogers wrote:
> Matthew Toseland wrote:
> > We then multiply that by two from splitfile redundancy, to get a total 
> > redundancy of 6. Wuala works well with a factor of 5 redundancy... but 
that's 
> > entirely due to FEC.
> 
> Two FEC blocks each replicated three times aren't really comparable to 
> five FEC blocks, are they? You can't use any combination of them to 
> recover the data.

Sure. It's not directly comparable. However, hopefully we have a higher 
average uptime, and the factor of 3 replication in the stores is necessary 
because of non-splitfile blocks.
> 
> > They simulated ordinary redundancy and needed a factor 
> > of 24 to be reliable, but a factor of 5 for FEC.
> 
> I'm not convinced their churn model is realistic - they assume that the 
> nodes' uptimes are independent, but studies of Gnutella and Skype show 
> strong daily and weekly cycles - if each node is online 25% of the time 
> it doesn't follow that 25% of the nodes are online at any given time.

True. What would the impact of this be?
> 
> > So maybe what we need is less network level redundancy and more FEC level 
> > redundancy?
> 
> Sounds like a good idea, although won't it lead to higher search 
> overhead (each FEC block will be replicated fewer times)?

That may not be a bad thing, but just like on Wuala, we have requestor-side 
healing...
> 
> > Wuala's simulations assume 25% uptime, and they don't allow nodes to have 
> > extra storage unless they have at least 17% uptime. Can we implement 
> > something similar?
> 
> In theory we could reject inserts from peers that haven't been active 
> for, say, 4 of the last 24 hours, but in practice would that just drive 
> away users and decrease the amount of available content?

Perhaps... I was thinking more in terms of storing stuff on nodes with 
reasonable uptimes...
> 
> > A 
> > full blown reputation system as Wuala uses would be a lot of work and a 
lot 
> > of debugging...
> 
> Wuala has centralised identity management, Freenet doesn't. That means 
> we can't prevent Sybil attacks or whitewashing, which makes designing a 
> reputation system even harder. IMO it's a can of worms. Snakes, even. 
> Dragons!

:)
> 
> Cheers,
> Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://emu.freenetproject.org/pipermail/tech/attachments/20080408/b9dbb660/attachment.pgp