PDTP - The Best of Both FTP and BitTorrent? 265
ikewillis writes "For awhile I've been following the development of PDTP (Peer Distributed Transfer Protocol), which is trying to merge the concepts of FTP and BitTorrent. This sounds like it could be useful for apt-get repositories or other high demand FTP sites. It's designed to be used as part of scalable networks which could replace manual selection of FTP mirrors. It also supports a number of other nifty features like cryptographic file signatures. Isn't it about time we ditched FTP for something better?"
from the no-more-april-fools dept. (Score:4, Funny)
Next thing it'll be transmitting voice and pictures over radio waves... AS IF!
Re:from the no-more-april-fools dept. (Score:4, Funny)
How does this differ from any other day of the year?
This isn't fair... (Score:5, Insightful)
So, their chance to build a reputation is going to be damaged by the fact that anybody reading Slashdot today has already given up on finding anything useful, and will be evaluating them as a joke that they're "not getting" rather than as a proposed networking scheme.
Furthermore, the geek world is bored today by Slashdot's denial-of-normal-service throughout the day. So, once word leaks out that this is a real and normal story, they're going to get all of the pent up slashdotting force applied to their server.
Simon, you should have started your set tonight with an NY Times article or two. That would have been a suitable transition between nonsense content and factual content, since NYT operates in that murky space and has a suitable web setup to absorb a larger-than-usual slashdotting. I'm sure the people at PDTP would have not minded at all if their moment in the sun had come an hour later tonight.
Re:This isn't fair... (Score:5, Funny)
Submit it to wikipedia. http://en.wikipedia.org/wiki/Internet_slang
Re:This isn't fair... (Score:4, Funny)
SB
Re:This isn't fair... (Score:5, Funny)
No way Dude!!1 Anyone can see that they have been working on this April fool's joke since last November!!
Re:This isn't fair... (Score:3, Insightful)
Re:This isn't fair... (Score:4, Funny)
Do you know how much Andy Rooney gets paid?
Re:This isn't fair... (Score:5, Interesting)
There's a point where it's funny, but then there's a point where it's really just overdone. I haven't even read half the stories posted, but it seems like they're all fake... and if they aren't, Slashdot is really ruining the credibility of some people. Not sure if the BSD on Gameboy is real or not, but if it is, who's gonna believe it?
You don't see CNN taking the day off. "OSAMA BIN LADEN CAPTURED!!", you click the link and read a long drawn out story that COULD be true, but at the bottom: "...April Fools!"
Karma to burn. *shrugs*
Don't confuse News with Editorials (Score:2, Flamebait)
Re:This isn't fair... (Score:2)
Re:This isn't fair... (Score:4, Insightful)
Hang on...
Don't they overdo it every year and isn't that part of the joke? Maybe not an April Fool's Joke but rather an April Troll? (Which is obviously more appropriate for a geek site)
That's what I always assumed.
Cheers
Stor
Re:This isn't fair... (Score:3, Interesting)
Others have stated that the bogusness of nearly all of /.'s content today may harm the reputation of some people. BSDs on Gameboy? PDTP? Gateway shutting down all stores? Which of these is/are true, if any at all?
But what concerns me even more is this: Some people do not understand the /. community, and might not understand that much of the content here is bogus today. Therefore, there is the possibility of bogus information being propogated as correct information, an
Re:This isn't fair... (Score:3, Informative)
Re:This isn't fair... (Score:2, Funny)
BitTorrent Mod That's Neophyte Friendly (Score:4, Informative)
Re:BitTorrent Mod That's Neophyte Friendly (Score:2)
need lightweight clients, not installers (Score:5, Informative)
And unfortunately, it's windows only, and still requires installing the software, which is 3MB+.
What is needed is something along the lines of a very small, very simple java client or a browser plugin. Azureus is java, but is huge and has massive feature-bloat for the purposes of just downloading(and sharing back) one file. However, Bram and others don't seem terribly interested in expanding possibilities; a mac developer offered up numerous improvements to the BitTorrent team for the mac client(which among other things is based on 3.3a, not 3.4.1, weeks after 3.4.1 released) and was rewarded with deafening silence.
The bittorrent protocol is http based. It's extensively documented on the bitconjurer website. Cmon folks, let's at least see a mozilla plugin or something! :-)
apt-get, you say? (Score:4, Funny)
Quoth the Debian Troll [slashdot.org]:
raid (Score:4, Funny)
or at least on april fool's day....
Re:raid (Score:2)
Re: (Score:2)
Re:PDTP (Score:4, Funny)
"Heh heh heh heh...file transfers RULE!"
The concept is great, but... (Score:4, Insightful)
Unfortunately, it's (almost) impossible to mirror new files instantaneously, so mirrors are never all in sync, all the time.
Re:The concept is great, but... (Score:4, Insightful)
Sorry, pet peeve is people kvetching about something on
To fix this, perhaps they could mandate that mirrors copy a particular directory to a temporary location, then take the old directory offline for the few minutes it would take to copy the new files over. Or have a $RELEASE var that clients would ask for and get returned all files marked with that var.
or something.
Re:The concept is great, but... (Score:3, Interesting)
Re:The concept is great, but... (Score:2)
Re:The concept is great, but... (Score:2)
Re:The concept is great, but... (Score:5, Informative)
Hash trees! (Score:5, Insightful)
Bittorrent and some other file sharing networks split the file into chunks and keep metadata with the hashes of chunks. The problem with this idea is how big to make the chunks: too big and you need to download a big chunk before you can verify. Too small and the list of hashes itself takes too long to download (the hashes are what makes
I think the solution should be to use hash trees. Split the file into relatively small chunks (1k?) and calculate their hashes. Now take every two consecutive hashes and hash them. Repeat with the hash results from the previous step until you have a tree with a single hash at its root. The root hash represents the entire file just like an MD5 of SHA1 sum. The difference is that with a small amount of metadata as hints you can verify any part of the file without downloading the entire file. All you need is a short (log n) chain of hashes leading down to the root hash. The server will trickle the hash information interleaved with the download and the client will verify it on the fly and never need to write a single byte to the disk before it's cryptographically verified.
Re:Hash trees! (Score:3, Insightful)
Please don't use straight SHA1 - it requires downloading the entire file to verify.
Bittorrent and some other file sharing networks split the file into chunks and keep metadata with the hashes of chunks.
Re read the grandparent:
with clients uploading pieces to each other and verifying their integrity with MD5 or SHA1 checksums (emphasis mine, especialy on the pronoun)
ie the SHA1s are of the pieces (ie chunks) not the wh
Re:Hash trees! (Score:3, Insightful)
This sounds clever, but the percentages don't work. Sure, a
think about that sentence: (Score:5, Insightful)
Isn't it about time we ditched floppy disks for something better?
Isn't it about time we ditched IDE drives for something better?
Isn't it about time we ditched x86 for something better?
Isn't it about time we ditched Microsoft Windows for something better?
Isn't it about time we ditched CDs for something better?
Isn't it about time we ditched telnet for something better?
Isn't it about time we ditched CRTs for something better?
Isn't it about time we ditched 20-year-old TV sets for something better?
Isn't it about time we ditched COBOL for something better?
Isn't it about time we ditched BASIC for something better?
Isn't it about time we ditched SCO Unix for something better?
Isn't it about time we ditched DOS for something better?
Isn't it about time we ditched Dubya for something better?
my point is that there is a lot of very old crap out there that should be replaces, but is going to get used and keep getting used for years to come.
Re:think about that sentence: (Score:2, Troll)
Right up there with *BSD, but with less cheerleaders.
Re:think about that sentence: (Score:2, Troll)
Re:think about that sentence: (Score:2)
telnet localhost 80
GET
Re:think about that sentence: (Score:4, Insightful)
CD-RW
Isn't it about time we ditched IDE drives for something better?
SATA
Isn't it about time we ditched x86 for something better?
AMD
Isn't it about time we ditched Microsoft Windows for something better?
Linux
Isn't it about time we ditched CDs for something better?
DVDs
Isn't it about time we ditched telnet for something better?
SSH
Isn't it about time we ditched CRTs for something better?
LCDs
Isn't it about time we ditched 20-year-old TV sets for something better?
New TVs, available at your local stores.
Isn't it about time we ditched COBOL for something better?
Visual Basic.
Isn't it about time we ditched BASIC for something better?
Uhm... it's for beginners. We can't ditch the biginners...
Isn't it about time we ditched SCO Unix for something better?
Linux... we think.
Isn't it about time we ditched DOS for something better?
Windows XP
Isn't it about time we ditched Dubya for something better?
John Kerry
Re:think about that sentence: (Score:3, Funny)
CD-RW
iPod used as FireWire disk.
Isn't it about time we ditched IDE drives for something better?
SATA
Mmm... Tasty FireWire.
Isn't it about time we ditched x86 for something better?
AMD
PowerPC
Isn't it about time we ditched Microsoft Windows for something better?
Linux
Mac OS X
Isn't it about time we ditched CDs for something better?
DVDs
Depends on context. iPod fills many uses of CD's. (music storage, data backup) Not software distribu
Re:think about that sentence: (Score:3, Insightful)
>CD-RW
Yeah, no one uses floppies for small files. Or bigger ones. Many people I know, including my family, don't have CD-RW or even CD-R.
>>Isn't it about time we ditched IDE drives for something better?
>SATA
And where is this option on dell.com/apple.com etc? Can you buy this at walmart? That stuff is nice for high end servers but...
>>Isn't it about time we ditched x86 for something better?
>AMD
So we went from
Re:Nitpick(s) (Score:2, Informative)
You could try teaching them LOGO or lisp, but lisp is boring and LOGO is no better than BASIC.
Good habits early (Score:2)
How so? As long at their compiler/interpreter is catching the errors, this will in fact teach them good habits early, so they don't have to unlearn them later.
My grade 11 intro to programming teacher, bless his anal soul, wouldn't let us use any global variables. Amongst other things. And I'm a better coder for having learned the right way than the wrong way.
I still have people at work who will ask me to look at some code that's not working, and th
Re:Good habits early (Score:2)
I think Microsoft QuickBASIC is a great environment for early introduction to programming.
And Python isn't a very good language for teaching scope concepts used in C/C++/C#/Java either. But some other things give it an advantage over others for beginners, such as not having to decla
Re:Nitpick(s) (Score:2)
Heavens above. As someone who has taught beginning programmers, if they can't keep their case straight and consistant, it's going to be the least of their worries.
Re:Nitpick(s) (Score:2)
Should be spelled "consistent". I guess the thread subject works for this. ;)
But as I said in another post, I was talking about REALLY young programmers. I was in 2nd grade when I learned GW-BASIC, and there couldn't have been a better language for me at that time. Come to think of it, though, I had the caps lock key on the whole time anyway, so case sensitivity wouldn't have really mattered.
At that age, learning that everything has a type is a good way to start. Having the type be a symbol
Re:think about that sentence: (Score:4, Funny)
I'll take an overweight penguin instead of an idiotic man in a butterfly suit any day...
Re:think about that sentence: (Score:2)
Windows itself has the completely indifferent four colour paneled window thingy icon.
The BSD mascot beats all hands down though.
Re:think about that sentence: (Score:5, Funny)
Problems solved:
Isn't it about time we ditched FTP for something better? HTTPIsn't it about time we ditched floppy disks for something better? Tape drives
Isn't it about time we ditched IDE drives for something better? Cool, thin IDE cables
Isn't it about time we ditched x86 for something better? x86-64
Isn't it about time we ditched Microsoft Windows for something better? Windows XP
Isn't it about time we ditched CDs for something better? Coasters
Isn't it about time we ditched telnet for something better? Clear text passwords over HTTP
Isn't it about time we ditched CRTs for something better? Incandescent light arrays
Isn't it about time we ditched 20-year-old TV sets for something better? 19 year-old TV sets
Isn't it about time we ditched COBOL for something better? FORTRAN
Isn't it about time we ditched BASIC for something better? Power BASIC
Isn't it about time we ditched SCO Unix for something better? SCO Linux
Isn't it about time we ditched DOS for something better? Protected mode DOS
Isn't it about time we ditched Dubya for something better? Jon Stewart
Re:think about that sentence: (Score:2)
Re: (Score:2)
Re:think about that sentence: (Score:2)
Re:think about that sentence: (Score:2)
Is that like some mad scientist created mixture of Bill Clinton and George W. Bush?
Re:think about that sentence: (Score:4, Insightful)
Kerry (Score:2)
Too bad we have to pick between a moron and a charlatan.
Re:Kerry (Score:2)
Re:Kerry (Score:2)
Re:think about that sentence: (Score:2)
Isn't it about time we ditched FTP for something better? SCP, SSH (fish:/), etc.. though a P2P FTPish setup would be cool.
Isn't it about time we ditched floppy disks for something better? USB Memory Stick or perhaps CD-RW/DVD+RW
Isn't it about time we ditched IDE drives for something better? SATA
Isn't it about time we ditched x86 for something better? AMD64
Isn't it about time we ditched Microsoft Windows for something better? I like FreeBSD w/KDE
Re:think about that sentence: (Score:2, Interesting)
Ok, sure, CISC is dead, x86 is a convoluted mess, yada yada yada, but engineers have gotten around many of these problems with the instructio
piracy returns to ftp? (Score:5, Interesting)
Re:piracy returns to ftp? (Score:2)
Security? (Score:2)
about time (Score:5, Insightful)
We already have. It is called SCP [openssh.org]
Re:about time (Score:2, Informative)
Rsync. [anu.edu.au] You can even have it checksum your files. . .
Re:about time (Score:4, Informative)
Re:about time (Score:2)
Re:about time (Score:2, Interesting)
Public: In which case http/bitorrent are good choices currently. I can see that pdtp could be better than bit torrent for this.
Private: This includes transfering stuff to and from work, or for a small number of family to access photo collections. For this I currently use ssh/scp/sftp [debian.org], rsync [debian.org] and scponly [debian.org]. These tools give me reliable, and efficient methods for secure personal file transfer. "scponly" provides a
Hardy and decorative (Score:2)
P2P Research (Score:5, Informative)
Take a look at Tapestry [berkeley.edu], and Chord [mit.edu] (and read some of the papers) to understand the issues involved in providing scalable and high performance P2P services. Not only is scalable search and overlay graph connectivity an issue, but also node failure and short session times of P2P nodes.
Additionally, when you actually handle the issue of downloading data, building application-lvel multicast trees to distribute the data efficiently on a large scale is not easy. Two papers from SOSP '03 SplitStream [microsoft.com], and Bullet [rochester.edu] address that issue.
Re:P2P Research (Score:2)
Re:P2P Research (Score:2)
BitTorrent is actually pretty on-par with the current research stuff for swarming file distribution. Everything else seems like incremental improvements - many of which break things like Bittorrents "share and share alike" policy by decen
Re:P2P Research (Score:2, Informative)
The supernode/ultrapeer addition to the protocol was meant to address some of the scalability issues of the previous version of the protocol. However, the fragile nature of the original overlay network is only made worse since ultrapeers are explicitly meant to be highly connected nodes.
The problem with Gnutella is that it can't do search efficiently due to the the broadcast nat
Mirror (Score:4, Funny)
Just in case... here's a mirror. [rightbrainnetworks.com] Always glad to lend a hand.
p2p OS installls (Score:4, Insightful)
Yes, you can already do bit torrent for the ISO, but that is its own kind of wast and hassle.
Some day.
Speaking of torrents... (Score:3, Funny)
I swear, this has nothing to do with today's date.
Re:Speaking of torrents... (Score:2)
I, for one, (Score:4, Funny)
something better == sftp, or HTTP! (Score:5, Informative)
Hm... (Score:2, Interesting)
PDTP's Sketchy Liscensing (Score:4, Insightful)
Question:
"Skyfire is using a derivative of the Apache License. Doesn't that preclude linking with Qt as the Apache License is incompatible with the GPL?"
Answer:
The FAQ page [sourceforge.net]"Qt/X11 is dual licensed under both the GPL and the QPL. The Apache License, while incompatible with the GPL, is not incompatible with the QPL, so when Skyfire is linked with Qt/X11 the terms of the QPL apply. Qt Non-Commercial Edition for Windows has a separate set of license terms which apply to all Windows builds of Skyfire." (emphasis added)
Isn't this license a poor one? Aren't they breaking sourceforge.net rules by using a OSI unapproved license?
Or maybe I don't know what I am talking about. PLEASE Correct me if I am wrong.If only... (Score:3, Insightful)
QPL is OSI approved (Score:4, Informative)
You can chuck ftp but.. (Score:3, Insightful)
I think theres still a while till we ditch ftp and move onto something else completely.Torrents and other p2p stuff is good but only if you take the effort to get them.What about the masses who want to click and go?It won't happen till they can right click and it says "Save torrent as".
There's already a solution that covers this. (Score:3, Interesting)
and well-implemented (Squid) cache-tree protocols
for HTTP. This is very old stuff. FTP is just
plain obsolete. It ads *zero* value over HTTP,
and it's harder to use. Trying to bring FTP up
to the standards of HTTP is a futile effort too,
since HTTP is mature on many more dimensions,
and does not suffer from the gross defects of
the more primitive FTP such as transmission of
port numbers as stream data.
Re:There's already a solution that covers this. (Score:5, Interesting)
It's virtually trivial to mirror subparts of an FTP site, it's much harder to do that on a Website if it has any links to the parent. Especially because websites specifically aren't a filesystem. So you can't make the same heirical assumptions that you would about an FTP site. It's why I always use rsync mirrors to grab files instead of FTP or HTTP. I hate FTP, it's a stupid protocol. HTTP is nice, but there is always extra crapola that I don't want that is a part of the system (index files, icon images, other gunk). HTTP isn't a filesystem. Now, WebDAV from what I have seen, looks like it could be a real filesystem. HTTP straight up isn't.
Kirby
Re:There's already a solution that covers this. (Score:4, Insightful)
HTTP does not do the same things that FTP does.
HTTP sucks for file transfers, frankly. You need a full-fledged web-browser just to view the index of files on an HTTP server. Not to mention that automatically downloading subdirectories requires serious processing of numerous HTML sub-documents.
HTTP does not do a good job of:
handling authentication.
handling sessions.
keeping statistics
limiting connections
communicating error messages
Etc, etc, etc.
Yes, I think everyone will agree that FTP sucks in that regard, but HTTP has it's own drawbacks.
FTP would disapear quickly if something came along that had all the features of FTP, without the baggage. However, until that something comes along, we are all stuck with FTP.
Terms Of Service (Score:2)
NO SERVERS OF ANY KIND
Sorry, I'd like to participate, but I can't. I know it won't die without me, but I fear more broadband ISPs taking on equally moronic TOS. The stance isn't entirely without merit, since Joe 6pak has no business running a server on today's Internet. But there's also no way to prove competence, and even if there were, I'm sure ISPs would be eager to charge me double.
cool... (Score:2, Interesting)
BitTorrent resource-hungry? (Score:5, Interesting)
BitTorrent suffers another problem in that the only usable implementations are currently only available in Python. The primary problem with Python is its excessive resource usage
Really? I'm currently running four throttled BT downloads on a PII-350 w/64MB. Max CPU usage is 8%, load average 0.25. If you're really that bothered see here [sourceforge.net] for an alternative.
but other problems arise such as integration of the Python implementation into a native GUI frontend for a given platform
Ever heard of WxGtk [linux.de]? RPMs for most distros, if it wasn't part of your default install.
as well as the need to bundle the Python runtime with the BitTorrent client on most platforms as few deployed systems have a Python runtime available
Now this is just silly. I dont think there is a linux distro which doesn't include Python libraries and even for Windows it's a single small executable. Besides (correct me if I'm wrong) but isn't one of the reasons for using Python that it has bounds-checking on arrays and is therefore proof against the cause of most exploits - the buffer overrun?
WE already have replaced FTP (Score:2)
Upgrade yes replace no (Score:2, Interesting)
I was on the "Desert Combat" Testers team and we had to download 600-700mb patches once a week... off one ftp server.
When i mentioned the idea of using a modified BitTorrent client/server to ease the strain on the server i was told we could not use "illegal" tools.
First educate the public and then start to think about upgrading things to help the internet not crash and burn.
Phil
BT Bandwidth-saved? (Score:2, Interesting)
FTP vs TELNET/SSH (Score:3, Informative)
Yes, it is. However, SSH has been around for a significant time and still hasn't replaced telnet, even given the horrific security holes in telnet.
Re:Sigh.... (Score:4, Informative)
Re:Sigh.... (Score:5, Informative)
Re:joke? (Score:2)
Re:joke? (Score:2)
Re:It still doesn't answer a very important questi (Score:3, Insightful)
If we all contribute a little, then the cost to all of us is that much less.
Re:It still doesn't answer a very important questi (Score:4, Insightful)
Re:It still doesn't answer a very important questi (Score:5, Insightful)
So, if for example, I write this need little GPL'd app that everyone loves, and release it as opensource, I should be responsible for hosting the file server for everyone? What if hundreds of thousands of people use it everyday, and a new patch comes out. Should I have to buy a T-1 (or something bigger) that costs an arm and a leg, to provide the file patch for a free program to others with no income for me? Or should I ask others to help out with their extra bandwith, and get a few seeders out there with bittorrent and run the tracker with the DSL line i have. I could pay $20 a month for a metered tiered connection in my town, but I pay $50 for an "unlimited" (notice the quotes). I know that not everywhere has these kinds of services, but you don't have to leave the torrent open forever either, or just leave the upload at 1k/s or something. It might slow down your download, but your still going to get access to the file..
Re:It still doesn't answer a very important questi (Score:3)
Re:Forward error correction and bulk data transfer (Score:3, Interesting)
For instance, I could have a file 100 packets long, and calculate 25 coded packets, then I could receive, for example, 80 packets of the original file and 20 coded packets (80+20=100) and recalculate the entire original file. Or 90 original and 10 coded packets. Or 75 or