[freenet-dev] Packet size proposal

[Matthew Toseland]

On Saturday 08 March 2008 00:03, Michael Rogers wrote:
> Matthew Toseland wrote:
> >> * Makes it easier to copy TCP's good bits instead of reinventing the 
wheel
> > 
> > Much easier?
> 
> Cumulatively, yes. At the beginning of SoC 2006 I thought we could 
> design something much better suited to Freenet than TCP, but we kept 
> coming up against problems that had already been solved by TCP - you 
> found another one recently with the congestion window - and eventually I 
> realised it would have been less work, and less risky, to just copy a 
> well-engineered and well-tested design.

RFC 2861 is rather late in the day for TCP. If we had set out to copy TCP we 
would likely not have seen it.

> (Or, even better, just use it -  
> but I know you and Ian are both opposed to going back to TCP.)

Not possible. Well, maybe with transport plugins we'd have it as well as UDP, 
but our primary transport will be based on UDP for the foreseeable future, 
because of NATs.
> 
> >> * Well-defined interface between the transport layer and the FNP layer
> > 
> > In the sense that messages are never reordered, yes.
> 
> I was thinking more about modularity - we should, for example, be able 
> to change the link crypto or the congestion control without affecting 
> the FNP layer.

Why should either affect the FNP layer? The link crypto won't affect it now. 
The congestion control does, but won't with the new transport layer because 
in the new transport layer all messages will be subject to congestion control 
and output limiting.
> 
> > We won't be passing Message's to 
> > transport plugins, certainly not for most transport plugins. We will 
provide 
> > a framework that makes it easy to write transport plugins, which your 
plugin 
> > can feed its parameters into.
> 
> I can't honestly see what kind of API you could wrap around the current 
> system that wouldn't require each transport plugin to reimplement major 
> parts of the node.

Well you can think about that then! Cos we're gonna have to come up with one. 
Transport plugins are of no use if it takes six months to implement and debug 
one.

Obviously we will need several different interfaces for different classes of 
transport.
> 
> When I talk about a well-defined interface I'm thinking of something 
> like the BSD sockets API or the J2ME Connector API: pass the transport 
> plugin a string identifying an endpoint, get back a failure code or an 
> InputStream and an OutputStream.

It depends on the transport type. Admittedly the API might be simplest for a 
stream-based transport, but I don't see why we can't have reasonably 
straightforward APIs for unreliable packet based transports (with parameters 
for packet size), and reliable packet based transports. AFAICS all you need 
to do is pass in some information about the desired packet sizes, which the 
node will then try to meet on outgoing packets (just as in my proposal you 
are replying to), and it will pass packets to the plugin. When the plugin 
receives a packet, it passes it up to the node. The node implements crypto, 
acknowledgements etc at a higher layer. If what you are mimicing has its own 
acknowledgements, then implement that and tell the node you're a reliable 
packet transport. Obviously there are going to be some issues with really 
small packet sizes (e.g. Skype), and with high latencies, but the basics are 
straightforward.
> 
> >> * Transport layer problems like small MTU can be isolated from FNP
> > 
> > Messages really ought to be small. This will be the case with the changes 
I 
> > have suggested: e.g. SSK data (including headers) will be a stream.
> 
> That's great, I do think those changes are a good idea, but why stop 
> there? Why not make everything a stream?
> 
> >> * Makes it easier to write stego transports
> > 
> > Marginally. A lot of interesting stego tranports are packet based.
> 
> Right, but they'll still need retransmission and congestion control, so 
> we can either reimplement it in each plugin or implement it once on top 
> of all the plugins. With the current system it seems like each plugin 
> would have to handle encryption, authentication, congestion control, etc...

No, we will implement that. The plugin just reformats our payload into a 
packet compatible with the wrapper.
> 
> > Would you still have packet level retransmission, or some sort of range 
> > retransmit scheme? What exactly does TCP do?
> 
> AFAIK it retransmits ranges rather than packets, because the MTU can 
> decrease during path MTU discovery.

If that is true then it's a good thing.
> 
> > If we send a packet which is 
> > over the safe packet size, what happens?  Is it exactly the same as it is 
now?
> 
> Unfortunately so - I don't think there's a way to get the path MTU in 
> Java, so you basically have to guess, and the only really safe guess is 
> 512 bytes.

Well.. it's something we'll have to deal with. There are possibilities e.g. 
using JNI to set the don't-fragment flag.
> 
> > In the current scheme, we'd send the rest of the maximum window, and 
timeout 
> > and have to resend it; that's what would happen in the new scheme too. Not 
> > the congestion window, the maximum window (128 packets now, 64k with the 
new 
> > transport layer). So we can keep sending messages while we retransmit the 
> > packet repeatedly, and eventually give up on it, tell the other side that, 
> > and split it up and send the messages in smaller packets. In your scheme, 
> > what happens? This is probably the worst case in terms of increased 
latency.
> 
> Big packets will be fragmented rather than dropped, and occasionally all 
> the fragments will get through. That's the problem, in a way, because if 
> big packets were always dropped we could work out the MTU for ourselves.

No, that's not the problem. That sucks, but it doesn't suck as bad as the REAL 
problem, which is that for a lot of connections, fragmented packets are 
dropped, either more often, or more usually always, because of hardware 
firewalls along the route.
> 
> > Reasonable, yes. But it will always be more than a real out of order 
> > transport.
> 
> OK, but how much is that little bit of extra performance worth? You've 
> said yourself that Freenet isn't low-latency - unlike Tor, which uses 
> TCP. ;-)

Tor only needs 3 hops to reach its destination. Freenet usually needs a lot 
more. Strawman argument.
> 
> Cheers,
> Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://emu.freenetproject.org/pipermail/devl/attachments/20080308/ad5fc561/attachment.pgp