Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Faster Feeds Using FeedTree Peer-To-Peer

ScuttleMonkey posted more than 7 years ago | from the instant-gratification-generation dept.

109

dsandler writes "Researchers at Rice University have just released version 0.7 of FeedTree, a peer-to-peer system for distributing Web feeds faster. Instead of polling feeds independently, FeedTree users cooperate to share news updates using multicast in Pastry, a scalable p2p overlay network. FeedTree reduces the update delay for existing RSS and Atom feeds to a few minutes without putting extra stress on the webserver (anyone who's ever been temporarily banned by Slashdot's RSS feed knows this is a real concern). Feed publishers can also choose to push digitally signed updates for immediate, tamper-proof delivery to subscribers. The client software (download) runs on Linux, OS X, and Windows, and works with any desktop feed reader."

cancel ×

109 comments

Sorry! There are no comments related to the filter you selected.

So... (0)

MS_Word (877966) | more than 7 years ago | (#14762540)

I get teh free movies now right?

Re:So... (3, Insightful)

twiggy (104320) | more than 7 years ago | (#14762559)

Now how do we get a plugin for something like this mass installed into browsers like that of flash, so that a web app like google reader (but independently developed, maybe specialized to other things) can take advantage of this idea?

That would make for a real nice way of creating awesome web based feed aggregation tools...

Re:So... (2, Funny)

hackstraw (262471) | more than 7 years ago | (#14762699)

Now how do we get a plugin for something like this mass installed into browsers like that of flash, so that a web app like google reader (but independently developed, maybe specialized to other things) can take advantage of this idea?

"We" get this as soon as you 1) write it for all known platforms that people use for the web 2) integrate it into all of the known browsers for all known platforms.

I see zero need for a plugin where a standalone app can do the same thing without destroying my web experience in the process.

Remember the mid-late 90s when everybody and their mother wanted you to install a new plugin that did not work properly to view their website?

Do we want to ruin the web again with untested and proprietary extensions like plugins?

Personally, I'm much happier now that I can use one web browser to browse the web. I get a little grumpy when I go to a website that tells me my computer is not good enough for their lack of web programming abilities.

Lets stick to standards, or I'll come flying at you in the USofA at 100mph in the left lane because I feel as though that is the "better" lane to drive in. After all, that is what they use in Japan and England right?

Re:So... (1)

mikvo (587789) | more than 7 years ago | (#14762937)

Do we want to ruin the web again with untested and proprietary extensions like plugins?

When did we stop?

Re:So... (1)

twiggy (104320) | more than 7 years ago | (#14763011)

Some plugins have been worth the hassle. Love it or hate it, Flash, when not abused, is a spectacular addition to enhance the web. It now comes standard or damn close to it in every browser.

With all the buzz about "web 2.0" (again, love it or hate it) - this would be the perfect application to make a plugin.

Granted, it just so happens that such a plugin would vastly improve a project I'm working on because it would take load off of my server - but I know that many others are working on projects that would benefit from this as well.

The next step in the internet is aggregating and compartmentalizing information -- making it easier to consolidate a nice little pile of only the things you want. The problem is, the web is a mostly one-way architecture and certainly doesn't work in any sort of a p2p fashion. A plugin like this would make a whole slew of things feasible on the web that, to date, are either incredibly system intensive for the poor server on the other end, or are wholly impossible.

I don't disagree with you about plugin bloat, but done right, this could be an amazing thing.

Mozpache (1)

tepples (727027) | more than 7 years ago | (#14763742)

The problem is, the web is a mostly one-way architecture and certainly doesn't work in any sort of a p2p fashion.

As Tim Berners-Lee originally conceived the World Wide Web, each computer would run both a client that dials out on port 80 and a server that listens on port 80. In practice, only the pervasiveness of dial-up Internet connections and duopolistic residential broadband ISPs' terms of service have necessarily interfered with this vision.

animated spyware (0)

Anonymous Coward | more than 8 years ago | (#14765573)

oh, you mean the "lockup your browser once past six tabs all running flash ads" plugin?

cute, real cute and oh so cooool..not.

Naw, let's just stick to "flash sucks so hard everyone should run three plugins to STOP it"

I don't even bother installing that thing anymore, I gave it a shot for years, it fails it. 99% of the people who use it are abusive to websurfers,because 99% of the uses are for ads. C'mon, you know I'm right on this. And the ads aren't even very good and they hog resources, they can take even a decent midrange machine with half a gig of RAM down, you have little control over the phone home spyware aspect to flash, and..it just sucks, man, it sucks. It's like the blink tag on crack, steroids and red bull, it's just wrong. Some things, like curb feelers and neon fender lights are just a dumb idea.

Ok then, (1)

crhylove (205956) | more than 8 years ago | (#14766161)

I'll show you my awesome web based feed aggregation tool, if you show me yours.

Re:So... (2, Funny)

idonthack (883680) | more than 7 years ago | (#14762576)

I wonder if we'll be sued for downloading copyrighted newsfeeds.

OCILLA and caches (1)

tepples (727027) | more than 7 years ago | (#14763684)

No. Anything already made available to the public on the Web is subject to the copynorms and copyright exemptions of the Web. For example, under 17 USC 512 (enacted as a rider to the DMCA), those who operate automated caches on a computer network are not liable in any United States court for damages that result from copyright infringements performed through such caches. To learn more, read about the OCILLA at Wikipedia [wikipedia.org] .

Why? (2, Interesting)

TFGeditor (737839) | more than 7 years ago | (#14762565)

WIth Bittorrent et al firmly established, why do we need another P2P?

Re:Why? (4, Informative)

idonthack (883680) | more than 7 years ago | (#14762599)

It's not "just another p2p", it's a p2p specifically for distributing newsfeeds. Bittorrent doesn't really work to well for that because it doesn't have the infrastructure and downloading the real feed would be easier than downloading the torrent first. This software bypasses any user interaction and grabs it off its established network.

Re:Why? (1)

modecx (130548) | more than 7 years ago | (#14763519)

check out uTorrent. The beta version has the option to check RSS feeds from any site, and it will automatically add the torrets to the client. It even has the option of filtering out torrents with simple expressions, and even offers you the ability to save your files in other folders for oranization purposes.
It works really well for downloading TV shows off of some sites, for example.

Naturally, because it's bittorrent, it's great at downloading b-i-g files, whereas FeedTree sounds like it's more about distributing web pages, and perhaps small video or audio segments, or perhaps it's a sort of distributed RSS... The article isn't especially clear on how this is all supposed to work. Sounds interesting though, too bad the documentation is unresponsive to me at the moment. Maybe they should have used Coral :)

Re:Why? (1)

RPoet (20693) | more than 8 years ago | (#14766390)

No no no. FeedTree is about cooperatively distributing RSS files. Your Torrent doesn't distribute RSS files, it reads them, thus putting strain on the server hosting those files. This is what FeedTree is meant to solve.

It is unfortunate that most people think "peer-to-peer" and "file sharing" are synonymous. They're not. Peer-to-peer has many, many users outside of file sharing/distribution.

Re:Why? (2, Insightful)

gkhan1 (886823) | more than 7 years ago | (#14762603)

This is not really the same thing. This is for feed-distribution, not cooperative downloading of large files. The files are rather small in this case

Re:Why? (4, Insightful)

Jerf (17166) | more than 7 years ago | (#14762691)

As already mentioned, this doesn't compete with Bittorrent, because bittorrent isn't designed for RSS feeds. Along with the file size issue idonthack mentioned (torrents are only a win when the size of the file being transferred is much much larger than the coordination overhead, generally not the case for RSS), BitTorrent is also not designed for files to change over time; it would require a complete overhaul of the protocol because the file hashes that are the foundation of the protocol would be constantly changing.

There is room for coordination with bittorrent, though; imagine a Pastry-based P2P feed that then used RSS enclosures to tie into a (trackerless?) BitTorrent feed for a fully distributed pod-/vid-/file-casting solution that anybody could run with no fear of the bandwidth involved.

Tack in some sort of P2P web system, and in theory, you could run a massively popular podcast/blog with millions of hits a day off of your cable modem. (Although something with a bit more upstreaming oomph would be good for the rarely-requested content that falls out of the P2P; anyhow, any ol' webhost could handle this kind of bandwidth.)

I think this is a worthy goal, as if nothing else, popular websites run for fun would no longer be faced with the dilemma of advertising to cover bandwidth costs or going offline.

Re:Why? (1, Interesting)

Anonymous Coward | more than 7 years ago | (#14762724)

The particular p2p application isn't really newsworthy. The overlay network (Pastry) is. The Pastry codebase appears to be mostly sponsored by Microsoft, is written in Java, and has a 'BSD-like' license. If all that doesn't give you the shivers, then you must have been in Microsoft hell long enough to start getting comfortable.

Mark my words: Microsoft is going to attempt to co-opt the term 'p2p', and make it their own.

Re:Why? (1)

jacksonj04 (800021) | more than 7 years ago | (#14763262)

I look forward to the *AA vs. Microsoft court case.

Re:Why? (1)

PFI_Optix (936301) | more than 7 years ago | (#14762986)

Ignorance is curable

The cure: RTFA.

Re:Why? (1)

Emetophobe (878584) | more than 7 years ago | (#14763676)

My ISP, Rogers Cable, has implemented some scheme which makes bittorrent painfully slow to the point of making it not worth using at all. You'll be lucky to get 10k/sec using bittorrent, while I can get over 300k/sec from websites, ftp, etc.. Even if I try and download a legitimate torrent, my upload rate will be higher then my download rate 99% of the time. So now I'm back to using IRC and xdcc list bots for my downloading needs.

Obligatory (0)

Anonymous Coward | more than 7 years ago | (#14762595)

I for one welcome our new p2p overlords.

They just reinvented netnews (3, Insightful)

Animats (122034) | more than 7 years ago | (#14762622)

It looks like they just re-invented the netnews protocol, which works in a very similar way.

Re:They just reinvented netnews (1)

b1t r0t (216468) | more than 7 years ago | (#14762722)

Exactly. All they need now is support for posting new articles from client software.

Re:They just reinvented netnews (4, Insightful)

That's Unpossible! (722232) | more than 7 years ago | (#14762733)

That's like saying IMAP just reinvented POP3.

This is designed for USERS to help each other get the very latest RSS feeds using p2p tech.

netnews is designed to let SERVERS help each other distribute messages posted by users.

I don't really see how it is a re-invention at all.

Re:They just reinvented netnews (2, Insightful)

John Hasler (414242) | more than 7 years ago | (#14762790)

> This is designed for USERS to help each other get the very latest
> RSS feeds using p2p tech.

> netnews is designed to let SERVERS help each other distribute
> messages posted by users.

> I don't really see how it is a re-invention at all.

Usenet is a peer to peer network of "servers". This is a re-invention of the way articles propagate in Usenet.

Re:They just reinvented netnews (1)

That's Unpossible! (722232) | more than 7 years ago | (#14763000)

Usenet is a peer to peer network of "servers". This is a re-invention of the way articles propagate in Usenet.

Except that Feed Tree doesn't propagate articles for usenet, it propagates entries posted to RSS feeds. I'm really trying to understand how this is a re-invention of netnews?

Re:They just reinvented netnews (1)

k8to (9046) | more than 7 years ago | (#14763316)

The assertion is that this method of transmitting nuggets of information (news entries) is similar to an older method of transmitting nuggets of information (usenet posts). Do you really not understand that the goals here are comparable?

That said, RSS/ATOM have a single source of the truth, while usenet is a web of inserters and receivers. RSS/ATOM are uniformly linear in nature, usenet is not. RSS/ATOM are by intention very very short entries; usenet posts can be much larger. RSS/ATOM are not intended (really) to convey binary payloads, urls are used for this purpose. usenet is abused to send all kinds of things.

In short, a naive anlysis suggests that the way these things are designed and used are divergent in some significant areas. I would be surprised if the solutions to their problems were close enough to consider it a reinvention.

Does the asserter of this view have supporting analysis?

Re:They just reinvented netnews (1)

aaronl (43811) | more than 7 years ago | (#14763338)

Sounds like you're getting caught up in trees (terms) and missing the forest.

USENET is a way for articles to be propogated among coordinating servers, and then users would poll those distributed servers. RSS is a way for distributing articles with only one server, and the users query it directly. What this tech does is create a way for articles to be propogated among coordinating servers, but those servers are also the users. The users then query other users, who are acting as servers, and then become servers themselves.

So basically, this would do the same thing that USENET did, but without the network of static coordinating servers. It, instead, replaces the static servers with dynamic servers and a method of locating those dynamic servers. It's the USENET concept that has been more decentralized in the content distribution.

It isn't bad that it's basically a reinvention of USENET, but it is silly to argue that the concepts and goals aren't the same.

Re:They just reinvented netnews (1)

John Hasler (414242) | more than 7 years ago | (#14763918)

> It's the USENET concept that has been more decentralized in the
> content distribution.

Actually the decomposition of Usenet into servers and clients is a relatively recent phenomenon. Originally we read news directly from the spool using local clients. Indeed, it is still quite possible to run your own local server as a leaf node, receiving only those newsgroups you are interested in. I have been doing just that for about twenty years.

Re:They just reinvented netnews (1)

penguin-collective (932038) | more than 8 years ago | (#14765756)

So basically, this would do the same thing that USENET did, but without the network of static coordinating servers. It, instead, replaces the static servers with dynamic servers and a method of locating those dynamic servers.

That's the way USENET used to be used, and a lot of USENET software still supports that usage, including automatically locating and subscribing to newsgroups only when a user demands it. It's become more static and centralized because users preferred using it that way, not because of any limitations in USENET.

FeedTree is recreating the original, dynamic USENET, using a slightly different addressing scheme (URLs) and content format (XML). It may be worth doing that, but it's certainly not a new technology. And if it catches on, it will likely undergo the same ossification as USENET, with most "peers" ending up being servers, because people tend not to want to participate in such networks from their workstations or laptops.

Re:They just reinvented netnews (1)

penguin-collective (932038) | more than 8 years ago | (#14765728)

Except that FeedTree doesn't propagate articles for usenet, it propagates entries posted to RSS feeds.

USENET propagates news items with metadata in a tree-like fashion, overlayed over the Internet. FeedTree propagates news items with metadata in a tree-like fashion, overlayed over the Internet.

There are some minor differences in standards (MIME vs. XML) and usage (well-known article hierarchies vs. ad-hoc RSS feeds), but that doesn't make FeedTree new technology. I don't think it's even a "re-invention", unless the people who created it are so out of touch that they have never heard of USENET.

Re:They just reinvented netnews (1)

Aspirator (862748) | more than 7 years ago | (#14762756)

I'm not that familiar with netnews http://mcntp.sourceforge.net/ [sourceforge.net] ,
but a quick check of the sites shows a rather different architecture.

This seems more targeted towards RSS type feeds, and looks like one of
those rather simple and clever ideas that strike one as:

"This looks like the way it should have been done from the beginning"

It addresses a very real problem with current RSS news feeds, and has what
looks like simple (that's a complement), complete, compatible, easy to install
software for a reasonable variety of platforms.

Cudos to Daniel Sandler at Rice.

Feed Reader (1)

Kris Thalamus (555841) | more than 7 years ago | (#14762628)

What's the best OS X feed reader to use with FeedTree? I don't care for the way Safari handles RSS.

Re:Feed Reader (1, Informative)

Anonymous Coward | more than 7 years ago | (#14762839)

NewsFire [newsfirex.com] has a slick, uncluttered interface, and can be configured [feedtree.net] to poll FeedTree very frequently. NetNewsWire [ranchero.com] has been around a bit longer and is quite popular, but its minimum refresh interval is 30 min.

Does the client work on FreeBSD? (1)

fak3r (917687) | more than 7 years ago | (#14762632)

s/t - has anyone run this on FreeBSD? Perhaps it works with the Linux compat modules loaded? I'd like to try this out tonight, since I have 3 sites on my FreeBSD box that have feeds that are constantly being hit...this sounds like a solution for the long term.

Re:Does the client work on FreeBSD? (1)

dsandler (224364) | more than 7 years ago | (#14763260)

The client and publisher both run on any system with the Sun Java runtime, 1.4.2 or newer. (The networking code in Pastry requires Sun's NIO implementation.) As for the publisher helper scripts, the configurator is Python, and the run control scripts are Bourne shell.

In other words, It Ought To Work(TM) out of the box on FBSD. If not, file a ticket [feedtree.net] .

Obligitory Simpsons quote. (1, Funny)

AltGrendel (175092) | more than 7 years ago | (#14762639)

MMMmmmmmm, Pastry.

ee EE EEEE!! (0)

Anonymous Coward | more than 7 years ago | (#14762675)

too many

Multicast?!?!?!1ONEONEONE (0)

Anonymous Coward | more than 7 years ago | (#14762676)

Since when did the internet support Multicast? UDP and TCP works, IGMP definitely doesn't though. I guess accuracy is something I shouldn't expect from slashdot though.

Re:Multicast?!?!?!1ONEONEONE (1)

Wesley Felter (138342) | more than 7 years ago | (#14762925)

The Internet does not support IP multicast, so FeedTree uses application-level end-system multicast.

Already getting hit by Shrook (3, Informative)

Refried Beans (70083) | more than 7 years ago | (#14762714)

I remember seeing something like this in my logs over a year ago. I would see lines like this in my access log:

66.177.198.139 - Anonymous [04/Apr/2005:03:04:17 -0500] "GET /rdf10_xml HTTP/1.1" 200 5322 "" "Shrook/76p (Distributed; +http://www.fondantfancies.com/shrook/distfaq.php [fondantfancies.com] ) "

I haven't seen a hit from this in a while, perhaps that effort didn't gain much traction. Who knows if this one will... I never saw Shrook mentioned on Slashdot.

Re:Already getting hit by Shrook (1)

don.g (6394) | more than 7 years ago | (#14763060)

Shrook is just one shareware Mac RSS reader. Unless it ended up with a really large userbase, you wouldn't see many requests from it.

GMail RSS (4, Insightful)

jfengel (409917) | more than 7 years ago | (#14762717)

I wonder: If GMail were to incorporate an RSS reader (the way Thunderbird does), it could potentially update many, many users with a single hit of each RSS site.

I'm leaning towards using RSS as a way to do announcements rather than maintain a mailing list. Rather than tell me you want me to send you updates (and deal with being potentially a spammer, deal with your unsubscribe, your email address change, etc.), just poll my site every so often (days, for the lists I'm talking about; hours, for Slashdot) and let it show up in your mail queue.

The idea isn't quite ready for prime time; too few people use RSS. But GMail could make that happen in one fell swoop. Well, two fell swoops: you'd need some sort of browser extension to make the little orange "RSS feed" button notify GMail.

I wonder if just having GMail (and hotmail, aol, and yahoo) handle that would solve the problem to the point where we no longer needed a P2P RSS distribution system.

Alternatively, if ISPs were to cache the RSS feeds the way some do with certain web pages, that might also take a lot of the load off. People will still impolitely set their RSS readers to check the feed every 10 seconds, but at least it never gets out onto the backbone if it's cached at the ISP.

Re:GMail RSS (1)

assassinator42 (844848) | more than 7 years ago | (#14762789)

That's the way Yahoo does it, AFAIK. But I don't think they update enough. Doesn't Google also use cached RSS for their custom homepage thing? At least, for things like Slashdot.

Re:GMail RSS (2, Informative)

thing12 (45050) | more than 7 years ago | (#14762795)

Gmail has Web Clips [google.com] , that's almost to what you want. But what you really want is for Google Reader [google.com] to be integrated into Gmail. It probably won't be too long before that happens anyway.

Re:GMail RSS (1)

jfengel (409917) | more than 7 years ago | (#14762870)

Thanks; I didn't know about either of those. They have so much stuff I can't keep up. I still use my own domain, so I don't use my gmail much, but I hear they're planning to solve that problem, too.

Re:GMail RSS (0)

Anonymous Coward | more than 7 years ago | (#14763211)

You can forward your email from your personal domain to gmail account, then set up your from address to come from your personal domain.

Re:GMail RSS (1)

Wesley Felter (138342) | more than 7 years ago | (#14762801)

If GMail were to incorporate an RSS reader (the way Thunderbird does), it could potentially update many, many users with a single hit of each RSS site.

There are already several large Web-based aggregators that work this way, but for various reasons many people prefer local aggregators (just as many people prefer local mail clients instead of GMail). FeedTree solves the bandwidth and latency problem for local aggregators.

Google Reader (1)

Otto (17870) | more than 7 years ago | (#14762836)

Haven't used Google Reader [google.com] yet, have you?

They'll likely integrate this with GMail at some point. But that's just my opinion.

Re:GMail RSS (0)

Anonymous Coward | more than 7 years ago | (#14763204)

Although GMail doesn't have it, there are other clients that do this. Notably Newsgator Outlook Edition (http://newsgator.com/consumer.aspx [newsgator.com] ). It will also sync automagically to Newsgator.com, FeedDemon, and NetNewsWire.

Disclaimer: I work for Newsgator. But it's still cool software!

Wow... (3, Funny)

SoundGuyNoise (864550) | more than 7 years ago | (#14762747)

Try saying that headline 5 times fast!

Re:Wow... (1)

TacNuke (890744) | more than 7 years ago | (#14762948)

ok here it goes......"that headline 5 times fast!"

That wasn't so hard........

Rice made Pastry, too. (3, Informative)

xiphoris (839465) | more than 7 years ago | (#14762752)

As a Rice Computer Science student I would like to point out that Pastry [freepastry.org] actually originated at Rice, under Dan Sandler [rice.edu] . The first framework was in Java. You can see from his web page that he's responsible for FeedTree, too.

Microsoft Research became interested in the product and ported it to C#, effectively turning it into the form it is now. Many classes at Rice have now "backported" it, I guess you could say, and it's used for many of our classes that involve distributed networks, such as the current COMP 410 [rice.edu] class which has previously turned out distributed file and process system codename Voltron [rice.edu] .

Here's a link to the paper [rice.edu] co-authored by Sandler and others at Rice.

Re:Rice made Pastry, too. (-1, Troll)

Anonymous Coward | more than 7 years ago | (#14762816)

As a non-Rice student, I would like to point out you're all over Dan's nuts. Wipe that crap off your nose, and quit patting yourself on the back so hard for going to Rice. My university did lots of cool stuff too, but that has nothing to do with me. It's like the water boy for the Steelers bragging about his Superbowl ring.

Re:Rice made Pastry, too. (2, Insightful)

xiphoris (839465) | more than 7 years ago | (#14762847)

LOL, that was not my intention, though I guess I see how it could have come off that way. I take pride in my university, the people I have worked with, and applaud them for the things they produce.

I don't even know Dan personally, though I have researched under Drs. Wong and Wallach before. I'm just trying to point out where credit is due.

Is it wrong to want your fellow students to be praised for their hard work?

Re:Rice made Pastry, too. (-1, Offtopic)

Anonymous Coward | more than 7 years ago | (#14763099)

"I don't even know Dan personally, though I have researched under Drs. Wong"

Was I the only one to read that as:

"I don't even know Dan personally, though I have researched under Dans Wang"

Re:Rice made Pastry, too. (1)

LeonGeeste (917243) | more than 7 years ago | (#14763609)

Is it wrong to want your fellow students to be praised for their hard work?

It's wrong to define yourself by what *other people* accomplished. Those other people are not you. You have no reason to be proud that you had more physical proximity to them. You're just revealing your "us vs. them" mentality. You are an individual. You are not Rice. When you accomplish something, let us know. When people who went to the same university as you accomplish something, stay the fuck out of their limelight.

Re:Rice made Pastry, too. (1)

xiphoris (839465) | more than 8 years ago | (#14764402)

Sorry not to mention it in the first place, but I did study directly under Dr. Wallach (a security/distributed systems researcher at Rice) who is a project member listed at freepastry.org [freepastry.org] . I did not build these things in question, but I have studied their properties and used them (Pastry, specifically) to build other things.

I have worked under Dr. Wong for a couple of years at Rice, working on Pastry-related and other distributed systems. No, those people are not me, but I have worked with them, and am familiar with their research and the relevant systems, and am happy to finally see these advances make their way into the mainstream.

Why is everyone so ready to bash me for praising them? I thought freely identifying myself as being (very loosely) related to the projects and disclosing my bias was something honest people do.

I have plenty of my own projects that I hope will be worthy of praise some day, and others that have already been praised. Getting some by proximity isn't my goal here. My real goal, actually, was to talk up Rice :)

Re:Rice made Pastry, too. (2, Informative)

dsandler (224364) | more than 7 years ago | (#14762941)

Actually, my advisor, Peter Druschel [mpi-sws.mpg.de] , developed Pastry with Ant Rowstron (of Microsoft Research). Since then, a number of bright researchers from Rice and elsewhere have contributed to the project; their names and publications are listed on the official Pastry website [freepastry.org] .

There are a number of implementations of the Pastry design; FeedTree uses the Java-based FreePastry [freepastry.org] package, which is under active development by Rice and the Max Planck Institute for Software Systems [mpi-sws.mpg.de] and is available under a BSD-like license. Other interpretations include MS Pastry (C#, used in COMP 410 as you point out) and the Bamboo DHT [bamboo-dht.org] (Java, inspired by Pastry and developed at UC Berkeley).

Pastry based Squirrel seems much more exciting (1)

MCRocker (461060) | more than 7 years ago | (#14764051)

If I understand it correctly, I think that Squirrel [freepastry.org] looks like a much more exciting application.

from the site:
SQUIRREL is a fully decentralized, peer-to-peer cooperative web cache, based on the idea of enabling web browsers on desktop machines to share their local caches.


If everybody used this, then there'd be no need for mirrordot [mirrordot.org] and the slashdot effect would be a thing of the past and more people could afford to host pr0n on their personal websites ;)

Re:Rice made Pastry, too. (1)

xiphoris (839465) | more than 8 years ago | (#14764422)

Hmmm, really? I didn't know. Sorry for saying you built it. I talked to Mathias Ricken about it today (a doctoral student under Corky in the PLT group) and he told me you were largely responsible for it.

Dr. Wong also said, during my time in COMP 410, that Rice U. was entirely responsible for Pastry before MS took it.

So what's the real story? Did we make it or not? X_x

Congratulations on the /.ing, btw. Where's your office? I'll come by sometime and say hi. :) I'm an undergraduate junior from Brown.

Disturbing trend: MS Funding kills Java App for C# (1)

MCRocker (461060) | more than 7 years ago | (#14763768)

I've heard about this trend before, but it is still very disturbing to see something like this where an application that screams out for a universal client that can be run on any platform is funded by Microsoft who dictates that the language be changed to C# leaving the original Java version to languish. Although, it's nice to see that the original is still available and has an open source license, it's disappointing that MS couldn't simply fund it as it was. As well as being a waste of money to do a port where none was needed, it certainly lends creedence to the arguments of Microsoft bashers.

Re:Disturbing trend: MS Funding kills Java App for (1)

MCRocker (461060) | more than 7 years ago | (#14763919)

Hey... wait a second... I did a little more digging and it does look like there is at least a java version of the code base. The Linux version seems to run on pure java and the library contains the pastry.jar file. Even though there's a src/net tree, where much of the code seems to reside, I'm not seeing ANY C# code.

So, it seems I should've dug deeper before making my previous comment. Sorry about that folks :(

Re:Disturbing trend: MS Funding kills Java App for (2, Informative)

xiphoris (839465) | more than 8 years ago | (#14764479)

I'm not entirely sure what happened at Rice w.r.t. Pastry. What I was told by Dr. Wong [rice.edu] that Rice had a Pastry version, MS adopted it and converted it to C#, then allowed us to use it freely. All of this was part of an elective class called COMP 410 that students take. Basically, a team of 10-20 people act like a software company, self-organize, meet with a "client" (professor acting like one) and build a huge system.

And yes, we use entirely Microsoft software. But I think it's a good thing. When I took it, Microsoft gave us copies of Visual Studio 2003, SQL Server, and funding for some tablet PCs to use as part of the project. I thought it was a *superb* experience to work with so much real-world technology.

Yes, I suppose one could say that MS is stifling open-source competition... but seriously, we were building an application that used a distributed cluster of SQL Server databases, transactionally changed by Enterprise Services features with Event Queueing; all of this also used a distributed file and processing system based on Pastry (C#). Getting all that to work together with open-source in a single semester would be quite a challenge. What database would we even use? MySQL is definitely not capable of that. And Oracle isn't free.

So, in this case at least, I think Microsoft's support of us has been positive for students. We are not just a Microsoft shop -- there is even a research group at Rice called the Programming Languages Team [rice.edu] , which focuses almost exclusively on Java for research projects. I'm currently involved in improving the open-source, student-oriented Java IDE called Dr Java [sourceforge.net] , which is under the purview of PLT.

*Pant*

Well, I'm sorry this turned into a rant. I guess my point of this: Microsoft has not caused Rice to give up open-source software or anything like that. In reality, their funding has exposed us to more software and more systems than we would have otherwise. I think that is a Good Thing.

It is neither a good thing for students to be exposed solely to OSS, nor solely closed-source industry software. A university should educate well-rounded people, and much like liberal-arts universities require students to take many subjects, Rice exposes CS students to different technologies and environments in its computer science program. Otherwise, how can I ever decide which is best for a task?

[Note: I am heavily, personally in favor of Microsoft software and have accepted an internship with them in the C# Compiler group next summer. But this doesn't mean I dislike Java or OSS; I don't see why there has to be a conflict at all. Use whatever tool suits you best.

But that's just me. I'm going to do *my* best to make C#.NET the best language it can be. If you like Java, fine! We can learn from another :]

Re:Disturbing trend: MS Funding kills Java App for (1)

xiphoris (839465) | more than 8 years ago | (#14764517)

Oh, and I should also point out that C# and .NET are actually much more "free" technologies than Java is.

Java is, and always has been, a proprietary technology completely specified by Sun. Sun owns the specs and decides what language features to add. Period.

The .NET platform and C# language are fully-specified and are on their way [microsoft.com] into acceptance as international standards by the ISO. Quote:
In July 2005, Ecma submitted [the C# and .NET] TRs to ISO/IEC JTC 1 via the latter's Fast-Track process. This process usually takes 6-9 months.

So, there is nothing at all "closed" or "proprietary" about C# or .NET, especially compared to Java. The only thing you might find is people using Visual Studio for development; but there are many alternatives:
  • SharpDevelop [icsharpcode.net] is an open-source IDE very similar to Visual Studio, it can do C# and VB.NET
  • DotGNU [dotgnu.org] is a GNU-sponsored project to implement the .NET platform.
  • The Mono Project [mono-project.com] , started by Miguel de Icaza of GNOME fame, is another implementation of .NET

So, let's say you were right and Microsoft did somehow convert Pastry to C# from Java. How is this closed or proprietary at all? If anything, it's *more* open.

Sun, the company, itself owns all aspects of Java. No one owns C# or .NET -- they're on their way to becoming international standards. As much as Slashdot seems to hate FUD, the attitude people give Microsoft really seems hypocritial sometimes. And yes, I did see your correction post; but that's not what I'm addressing here. C# and .NET are Open systems in every sense of the word.

Re:Disturbing trend: MS Funding kills Java App for (1)

cakoose (460295) | more than 8 years ago | (#14765213)

C# and .Net are actually much more "free" technologies than Java is

Microsoft is in complete control over the future of the C# language and the .Net libraries and runtime. Just because they do the standards dance doesn't mean they've given up control. Do you honestly think that C# or .Net can change in a way Microsoft doesn't approve of?

The ECMA even allows the standard to be patent-encumbered as long as Microsoft provides "reasonable and non-discriminatory" licensing fees. That makes me feel completely safe.

Microsoft's policy on making changes is to solicit customer feedback and then work internally to come up with the design. Compare this to the JCP. I'm sure that, in practice, Sun has more clout than the other participants, but their control isn't total. Design-by-committee may or may not be be stupid, but it is definitely more open.

Have you seen the Java 6 [java.net] website? No, the license isn't the friendliest, but it's the production JVM. Rotor is just the research implementation.

Don't get me wrong...I think C# and .Net are open enough to allow implementations like Mono. I, personally, don't believe Microsoft will sue. I just don't understand how you can say C# and .Net are "much more" free than Java.

there's nothing at all "closed" or "proprietary" about C# or .Net [emphasis is mine]

Aside from the production implementation and the related patents, right?

The patents were released. (1)

xiphoris (839465) | more than 8 years ago | (#14765646)

Aside from the production implementation...

Um, no. .NET itself (the platform, SDK, etc.) is entirely free, just like Java. The only thing Microsoft has control over is the development tools. Microsoft's Visual Studio is not open source, but so what? In the grandparent post to this I pointed out several open-source .NET projects and one IDE. And there are plenty of popular non-open-source Java IDEs [jetbrains.com] out there too. No one has problems with them.

... and the related patents, right?

Sorry, but that's just ignorant.

Quote:
The core of the .NET Framework, and what has been patented by Microsoft falls under the ECMA/ISO submission. Jim Miller at Microsoft has made a statement on the patents covering ISO/ECMA, (he is one of the inventors listed in the patent): http://web.archive.org/web/20030609164123/ [archive.org] and http://mailserver.di.unipi.it/pipermail/dotnet-ssc li/msg00218.html [unipi.it] .

Basically a grant is given to anyone who want to implement[sic] those components for free and for any purpose. [emphasis mine]
The original comes from the Mono Project FAQ entry on patents [mono-project.com] . Please, stop the FUD.

License Terms? (1)

John Hasler (414242) | more than 7 years ago | (#14762806)

I can't find any mention of the license terms on the Web site.

Re:License Terms? (1)

Lord Satri (609291) | more than 7 years ago | (#14763069)

From the license file:
Copyright (c) 2006, Rice University
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
        this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright notice,
        this list of conditions and the following disclaimer in the documentation
        and/or other materials provided with the distribution.
    * Neither the name of Rice University (RICE) nor the names of its
        contributors may be used to endorse or promote products derived from this
        software without specific prior written permission.

This software is provided by RICE and the contributors on an "as is" basis,
without any representations or warranties of any kind, express or implied
including, but not limited to, representations or warranties of
non-infringement, merchantability or fitness for a particular purpose. In no
event shall RICE or contributors be liable for any direct, indirect,
incidental, special, exemplary, or consequential damages (including, but not
limited to, procurement of substitute goods or services; loss of use, data, or
profits; or business interruption) however caused and on any theory of
liability, whether in contract, strict liability, or tort (including
negligence or otherwise) arising in any way out of the use of this software,
even if advised of the possibility of such damage.

Re:License Terms? (1)

187807 (883881) | more than 7 years ago | (#14763088)

This? [feedtree.net]

Does /. RSS push its updates to this? (1)

Aspirator (862748) | more than 7 years ago | (#14762834)

An excellent project, it deserves to become dominant in internet
RSS news distribution.

It's nice to be able to browse the source code.

What can we do to encourage adoption of this, before some wretched
proprietary format tries to muscle in?

Web-Based News Reader (1)

niai (310235) | more than 7 years ago | (#14762842)

I personally use Bloglines [bloglines.com] - a web based news reader. This lets me check and read my subscriptions from home and work without having to read posts twice. Google Reader [google.com] is a similar application but has tagging and merges all your feeds into one.

gn4a (-1, Redundant)

Anonymous Coward | more than 7 years ago | (#14762939)

COMPREHENSIVE minutes now while I'm discussiNg posts. Due to the like they are Come mire of decay, stand anymore, won't be standing

The reason this is a issue (1)

grazzy (56382) | more than 7 years ago | (#14763027)

Is because RSS doesnt pay. There's no way to monetize the RSS-feed which often can be a large burden on a server in terms of CPU (if dynamic) and bandwidth.

Micropayments would solve this. Pay 0.001 for every reload automaticlly and you wouldn't need a solution like this. Fix that and solve thousands of small problems at once.

Re:The reason this is a issue (0)

Anonymous Coward | more than 7 years ago | (#14763103)

Lots of people "monetize" RSS by putting advertising in the feeds. That is, of course, a blight. Why do things have to "pay" to be worthwhile?

Re:The reason this is a issue (1)

grazzy (56382) | more than 7 years ago | (#14763213)

They doesn't have to. But take slashdot as a example. 100,000 people ads their RSS to recheck every 10 minutes (they want the stuff as it happens you know). Most people keep their aggregator going 24/7 - this creates a huge cost in terms of bandwidth. Even at 1kb / request (compressed, 5 item rss feed - most aggregators doesnt use mod_gzip) that'll be 100,000 x 144 = 13gb traffic every day just from RSS. Over 400 gb of traffic every month. Try finding a hosting that give you 400gb transfer for free. Dont forget the dedicated box to serve those requests aswell.

Re:The reason this is a issue (1)

DrHanser (845654) | more than 7 years ago | (#14763142)

This actually isn't true. Feedburner, for one, offers RSS advertising. I don't know if it's a pay-by-impression or pay-for-click model, though.

Re:The reason this is a issue (1)

grazzy (56382) | more than 7 years ago | (#14763197)

People put their RSS-aggregator to reload every fifth minute, if you're lucky you might get a real ad impression once a day. Getting any money from that kind of advertising is extremely hard compared to the cost of providing the service.

Re:The reason this is a issue (1)

tepples (727027) | more than 7 years ago | (#14763815)

People put their RSS-aggregator to reload every fifth minute, if you're lucky you might get a real ad impression once a day.

Then only serve one distinct ad per unique visitor per day.

What's the use from the point of view of end-user? (0)

Anonymous Coward | more than 7 years ago | (#14763070)

i don't understand how it is better than setting feed readers poll time to 10 secs? Do they think end-users care about bandwidth?

How does this relate to scribe? (1)

Anakron (899671) | more than 7 years ago | (#14763080)

Scribe [microsoft.com] [Technical paper pdf warning!] is a framework to do very similar things. Is this an application developed on top of that? Scribe works by building a multicast tree of the participants too.
One interesting thing to note is that as a participant in scribe, you'll have to pass on notifications of feeds even if you're not interested in them, because you're a part of the tree and pretty much the only path to the guys below you. How does FeedTree deal with cheating/lying nodes that refuse to pass on messages? Also, to be a part of the overlay, you need to keep sending keep-alive messages. Not a big deal, I know, but I always thought Scribe was impractical for general use, but would work great for a restricted audience (like a large geographically distributed company) that can be "trusted".

Re:How does this relate to scribe? (0)

Anonymous Coward | more than 7 years ago | (#14763172)

All the detail are explained in the technical paper [rice.edu]

A solution without a problem (2, Insightful)

DrHanser (845654) | more than 7 years ago | (#14763109)

I'm afraid I don't understand what problem this is solving. It's like a solution that's still looking for an problem to solve. As an end user, why should I care? I'm not trolling; I just don't get it.

Re:A solution without a problem (1)

nostgard (645633) | more than 7 years ago | (#14763214)

The problem is that users aren't able to retrieve RSS feeds as often as they would like. They must wait longer between updates of the feed, to save bandwidth and decrease load on the server that is hosting the content.

This solution, however, would allow one source to poll the server, and use P2P to transfer the content to the many clients as want it - and as often as they want it. No additional strain is being put on the source, and the clients are all happy.

Feedtree Technical Info (0)

Anonymous Coward | more than 7 years ago | (#14763131)

In addition to the website, there is a technical paper that descirbes the whole architecture. How is it works on top of scribe, how it can work in different models of adoption. How security is handled and all the other technical details. If your interested in the gory details of how it all works you should go here pdf [rice.edu] to see the paper.

mod 0p (-1, Redundant)

Anonymous Coward | more than 7 years ago | (#14763168)

off the play area today. It's about We'll be ayble to www.anti-slash.org Users With Large again. There are Mr. Raymond's and mortifying

Truly beneficial? (1)

RhettLivingston (544140) | more than 7 years ago | (#14763248)

I've been thinking for quite some time of utilizing this type of P2P distributed caching proxy concept with many different protocols. RSS is just one possibility amongst many that could utilize the basic technology here. Some others might include distributed file systems, distributed caching http proxies, or even a Google competitor that uses a distributed P2P implementation of the database and utilizes everyone's everyday web activity to augment the spidering (i.e. every time anyone who is part of the P2P search network hits a site, a side effect is that they update the search index with the latest data from that site).

I'm not sure though that this is truly beneficial in terms of reducing the burden that RSS places on the Internet in general. Yes, it reduces the burden on the originating web site, but I believe it increases the total number of packets that must flow across some internet connection somewhere. So, it appears to be a mechanism for shifting the cost from one at the server to a larger total one at the clients, not a mechanism for helping the internet as a whole. I would in fact be positive that this is not beneficial overall except for the fact that it may have a beneficial reduction of the peak traffic on critical network backbones. But that would only be true if the overlayed network topology is either geographically optimized or is based on something that has an accidental relationship to geography.

New game in town... (1)

trollable (928694) | more than 7 years ago | (#14763263)

The client software (download) runs on Linux, OS X, and Windows, and works with any desktop feed reader.

New game in town: never use the word Java. BTW, it doesn't run on Linux and Windows. Except if you install Java of course.

Azureus also runs on the Java platform (1)

tepples (727027) | more than 7 years ago | (#14763656)

it doesn't run on Linux and Windows. Except if you install Java of course.

A lot of people who are interested in peer-to-peer networking have installed a Java platform, even if only to run the Azureus client.

Huh? (-1, Redundant)

tacokill (531275) | more than 7 years ago | (#14763383)

So, um...they reinvented usenet? Not to be a troll, but this is *EXACTLY* how usenet works at the server level. It's P2P for the most part.

So we've "web-i-zized" the same technology for use in the go-go Web 2.0 so we can seamlessly integrate disparate mediums across a non-traditional life-cycle that leverages cross marketing synergy in order to capture positive customer sentiment.

Sweet! I am intrigued by your ideas. How can I subscribe to your newsletter? Oh, nevermind.

Frist 4sot! (-1, Offtopic)

Anonymous Coward | more than 7 years ago | (#14763505)

can connect to crisco or lube. [amazingkreskin.com] at this poin"t First, you have to fear the reaper Keep unnecessary

Why not use push? (1)

Trejkaz (615352) | more than 7 years ago | (#14763824)

I know it's a crazy suggestion, but instead of having hundreds of people polling a single RSS feed, why not have the server which hosts the RSS feed actually PUSH the updates out to the people who are interested?

We already have a nice and simple protocol (XMPP) which could be used for this, although admittedly PubSub isn't as final as it could be.

Re:Why not use push? (1)

Wesley Felter (138342) | more than 8 years ago | (#14764341)

instead of having hundreds of people polling a single RSS feed, why not have the server which hosts the RSS feed actually PUSH the updates out to the people who are interested?

FeedTree can operate in that mode, or if the server operator is too lazy to install it, FeedTree will still provide some benefit.

We already have a nice and simple protocol (XMPP) which could be used for this, although admittedly PubSub isn't as final as it could be.

Doesn't this lead to potentially high fanout (with the attendant concentrated bandwidth consumption)? One benefit of FeedTree is that it spreads the work around.

Re:Why not use push? (1)

Trejkaz (615352) | more than 8 years ago | (#14764439)

Once there is working PubSub, the work will be focused around the PubSub nodes. The site will send one message to the PubSub service, and that service would be one which is built with large scale messaging in mind.

Re:Why not use push? (1)

Wesley Felter (138342) | more than 8 years ago | (#14765036)

If this PubSub service is centralized, then it won't be free. If it's decentralized, then it's essentially similar to FeedTree.

Re:Why not use push? (1)

Trejkaz (615352) | more than 8 years ago | (#14765079)

Well, I imagine that the PubSub for each site would be centralised (either at the site, or hosted somewhere else.) But each site would probably have its own distribution node, so it's decentralised in that respect. Either way it's not free, because someone still pays for all the bandwidth, and the site still pays for its hosting. ;-)

Add encryption and IP packet morphing (0)

Anonymous Coward | more than 8 years ago | (#14764663)

Adding encryption and IP packet morphing to these P2P feeds, and censorship would be very hard to enforce.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>