« The IPv6 situation | Main | auto-UDP up and running »
The UDP situation
By erdgeist | January 2, 2008
Most of the problems we have to cope with in our everyday opentracker struggle arises from operating systems having to handle several thousand tcp connection setups, reassembly and tear downs. As a mean to help ease the impact of these requests, a udp extension to the tracker protocol has been agreed upon.
However that specification page does a rather poor job – besides of the obvious html layout deficiencies – when it comes to explaining what design decissions were made and why. This lead to many client writers out there wondering what the connect packet is about.
While there are sites around with a layout that at least does not try to prevent implementation, rationale for the extension completely is absent.
1. The connect packet
Now, what is the extra connect packet all about? Wasn’t the udp extension all about saving packets hammering the server? The answer is, it was designed to prevent address spoofing. While tcp inherently prevents address spoofing with its initial handshaking – you can only finish it if the recipient of SYN+ACK is the same address that sent the SYN – udp does not provide any means of ensuring that the packet really originated from the address it claims to be.
So Olaf van der Spek – designer of that protocol extension – demands that a client first uses a connect packet to acquire a uniq-ish connection_id that it later presents to the tracker to prove listening on that very address.
What way of chosing, storing and comparing connection_ids a tracker implements is up to him, one possible way would be chosing one very long global tracker secret, and then calculate a cryptographically secure checksum of this secret appended by the clients alleged ip address. This approach would not require the tracker to store anything per client for it can be calculated everytime that client comes back.
However, when writing the specification, Olaf made the mistake of not mentioning this feature and – even worse – using the term connection_id for two completely unrelated values: A connect packet contains a static protocol identifier at the same offset where on announce and scrape packets that unique connection_id is expected. Olaf even also calls that value connection_id. Hence half of the non-connect udp packets arriving at opentracker still contain 0×41727101980 as connection_id.
To be honest, opentracker is not completely innocent. Since we could freely chose which values to put there, we just left the old connection_id. This might have given the impression that the whole connection_id thing just is a sanity feature to distinguish stray udp packets arriving on that port.
Now we’re facing the problem that enforcing connection_id checking would really hurt udp announces. This is not in our interest. As a first step we changed our code to return a different value as connection_id. Now we want to encourage all client authors supporting udp to check their implementation against the way it was meant to be. In some months from now opentracker will enforce connection_id checking.
2. Auto UDP
As you may have guessed by now we do love udp announces a lot more than the tcp stuff. So introducing ways to increase the number of those more lightweight client interactions is in our own very interest.
We do know that it is hard to encourage users to add udp:// addresses to their torrents. Still most tracker responses on earth are being handled by trackers that are capable of speaking udp (and soon also udp-v6). This means that sending udp connect packets to the corresponding udp port of a http tracker url will most likely succeed.
We hereby invite all client authors to probe the corresponding udp announce address, store the result on a per-tracker base, use udp whenever feasible (i.e. for announces and [non-full-]scrapes) and revert to tcp only if udp fails.
3. IPv6
And while you are at it: you might also want to take a look at our v6 protocol extension proposal.
As always, feel free to comment on that topic at our coders mailing list.
Topics: coding, tech | 3 Comments »
February 14th, 2008 at 12:10 pm
Sorry to ask this here, but what’s up with the MRTG stats? “Only” 36.000 Peers seem way lower that the numbers you have been reporting earlier. How come?
February 14th, 2008 at 12:18 pm
We currently have some trouble with our ISP so we delegated our traffic to one of the TPB tracker which is now handling our tracker-traffic until we cleared out the problems.
May 8th, 2008 at 9:47 am
Totaly OT, but did you guys know about Aqualab using your source for the Ono project? Any chance that will get into your branch or even on TPB? The idea itself seems pretty interesting:
http://www.aqualab.cs.northwestern.edu/projects/ono/developers.html