Journal 2T88 Make the internet come to you, the way you want it, with RSS

Make the internet come to you, the way you want it, with RSS

by
in rss on (#2T88)
You can think of RSS as a DVR for the web! See what you want, when you want, where you want, how you want, faster, without needing broadband internet access, and never missing a single "episode". If you perhaps live with patchy, expensive, or very slow internet access, even just sometimes (like most everyone who owns any mobile computing device: whether laptop, smart phone, or similar), you'll be sorry you didn't learn about all the ins and outs of RSS, sooner.

My interest in RSS didn't come about until I got a smart phone. The tiny form-factor, primitive input method, and relatively low resources make browsing web pages slow and very cumbersome. But an RSS feed reader would take the simplified content and use the most ideal layout for your device, far more like reading an eBook than navigating a website. This was the original promise of HTML, like most mark-up language, but it has almost entirely departed from that objective and become almost as rigid as other prepress document formats.

I like version 3 of RSS Demon for Android, but the latest update has completely ruined it. I'm no great fan of the company, anyhow, (they undermine their paying customers) so I'm open to suggestions for alternatives. Another halfway decent option is the $2 FeedR RSS Reader for Android.

RSS offers many advantages, such as low bandwidth, batched updates for future offline reading, automatic tracking of which stories you've read without missing any that come in, easy methods of saving of interesting stories for future reference (think: offline bookmarks with full-text), etc.

RSS feeds can be found for practically any site. That familiar orange icon is likely hiding somewhere around the perimeter of the page. And even if its not, a search for the site name plus "feed" almost always turns up an RSS/atom/XML option available for your consumption. Several sites I was convinced didn't have a feed, I discovered work just by appending "/rss" onto the end of the site's URL. Alternatively, you can look for link rel tags in the page's source code, too.

128px-Feed-icon.svg.png

RSS isn't perfect for everything. It's not a great match for sites like Pipedot, where the comments (which come along AFTER the story is published) are desirable content. There's no practical way to include them in the RSS feed of the stories. That said, Pipedot does have a feed for ALL comments, which (while lacking the context of the web layout) works okay for a small site like this, and is a tremendous feature for those who prefer RSS.

But far more often, sites will simply abuse their RSS subscribers, and offer only a very short summary, or sometimes nothing more than a title and a link to the story, driving readers to visit the web page where they can be tracked and served up profitable advertisements.

To fix this, 3rd party services exist to combine RSS feeds with page scraping, to serve you a full article (with images), output in a standard RSS feed. All sites I've found that currently work:

Full Text RSS Expanders:

http://www.fulltextrssfeed.com/
- 6 articles (per feed, per-day) limit.
- Works great with RSSDemon
- Dead-simple URL scheme (just append the feed URL)
- Occasionally unresponsive, but that is okay for RSS.

http://feedenlarger.com/
- 10 articles limit.
- RSS output doesn't work with RSSDemon

http://fivefilters.org/content-only/
- 3 articles limit
- €5/month subscription for 10 article limit
- Works with RSSDemon
- Works on sites that block other services

http://fullcontentrss.com
- $12 "lifetime" subscription fee, but free test sample.
- 10 articles limit.
- Works with RSSDemon
- Works on sites that block other services

NOTE: All of these services are easy to block by target websites using robots.txt, htaccess based on user agent strings, or singling out server source IP addresses. Penny Arcade comics completely block their use, returning 503 errors. I return their kindness by choosing to NOT ever visit their site. YMMV. I didn't read Dilbert for years after they changed their RSS feed from including the strip to just empty items with a link to the site, but with these tricks I am back reading them again. Take note, publishers, who are considering blocking such sites/services...

In addition, you might want to compliment these services by being able to mix/aggregate/combine numerous separate feeds into a single (date-sorted) RSS feed. Keep in-mind the 3-10 daily article limit of the full text converters (listed above), so this works best for very low-volume RSS feeds, only. You might never need this, but one site I read broke-up their single article-per-day RSS feed, into THIRTEEN separate feeds, with still only a couple daily updates across them...

RSS Feed Combiner/Mixer/Aggregator:

http://rssmix.com/
- Up to 100 input feeds.
- Source feed must respond "within 5 seconds".

http://feedkiller.com/
- Maximum of 5 RSS feeds allowed!

http://chimpfeedr.com/
- Option to also resize images

http://www.blastcasta.com/mix-combine-filter-rss-feeds.aspx
- Tons of great options, but doesn't work for me

And last but not least, for sites you absolutely have to follow, but which really, honestly, totally don't have any RSS feeds at all, there are tools out there to do this, too. Just don't expect it to be very pretty:

Web Page To RSS Conversion:

http://feed43.com/
- A bit complex, but powerful & customizable
- Weak documentation and no examples makes it painful to start

http://page2rss.com/
- Quick and easy, but output can look like a mess

http://www.balluche.fr/html2rss.php
- Lots of (required) options/tweaks.

http://feedyes.com
- Registration required
- $6/month after 14 day trial
- Some tweaks necessary

http://www.rssitfor.me/
- Just for Twitter

RSS makes an admirable replacement for the now-endangered newsgroup, with similar offline reading options that have otherwise gone extinct on the internet. And while cellular deployments are keeping more and more people connected at all times, high prices are causing people to restrain themselves, or even choose much cheaper plans with little or no data and instead depending entirely on WiFi hotspots. One such service is Republic Wireless' voice&text-only $10/month plan without any cellular data, which "reigns as the most popular choice": See Republic Wireless plans

Strangely, I don't recall having the slightest interest in RSS back in my PDA days, despite frequently preferring to read on their screens over a computer monitor. I suppose that was early days for the format with very few sites using it, and I might have paid attention if I ever saw an RSS reader for my devices. Even then, the UIs would probably have been quite primitive like the horrible mobile browsers of the day. Other formats like ".mobi" were very primitive, inconvenient and cumbersome (requiring managing the software on your desktop computer), so vastly less viable. It was also very, very early days for WiFi, so PDA data connectivity was largely limited to serial tethered connections at glacial speeds. That said, a Psion5 tethered to a cell phone was a popular option for those who needed portable connectivity, and clearly the parent of modern smart phones that Apple is happy to take credit for.

A few suggested RSS feeds (for this audience) to get you started:

Recommended RSS Feeds:

Pipedot:
- http://pipedot.org/atom
- http://pipedot.org/comment/atom

XKCD (comic):
- http://xkcd.com/atom.xml

The Shortwave Listening Post:
- http://swling.com/blog/feed/

The Daily WTF (tech humor):
- http://syndication.thedailywtf.com/TheDailyWTF

National Academies News
- http://www.nationalacademies.org/rss/

Hacker News
- https://news.ycombinator.com/rss

Communications Laws:
- http://feeds.lexblog.com/CommLawBlog?format=xml

Politifact truth-o-meter:
- http://www.politifact.com/feeds/statements/truth-o-meter/

AnandTech:
- http://www.anandtech.com/rss/

RSS Feeds Search Engine:

http://www.rssmicro.com/?f=3&sd=0&p=2&q=Keyword

UPDATED 2015-10-30: Removed now-defunct Yahoo Pipes, warned of RSSDemon app regression, and a few updates to list of RSS web services.
Reply 7 comments

Anandtech (Score: 1)

by bryan@pipedot.org on 2014-11-17 21:03 (#2V3N)

Anandtech awesomely includes the entire article in their RSS feed.

Re: Anandtech (Score: 2, Informative)

by evilviper@pipedot.org on 2014-11-20 23:29 (#2V3S)

That's true of Consumer Reports, techdirt, Newsweek, VOA, National Academies, Huffington Post, Breitbart, and a decent number of others. They're a minority, but there's plenty of them. If you wanted to get all your news from sites that offer full text RSS feeds, you could do so without too much of a trade-off.

UPDATE: Had PopSci in there, but looks like they JUST NOW (in the past week) changed their full-text RSS feed to summaries only.

Good stuff (Score: 2, Informative)

by zafiro17@pipedot.org on 2014-11-18 11:49 (#2V41)

RSS is essentially my killer app for the internet, and definitely for my smartphone, and it's driving me crazy that the trend is away from RSS and towards making everyone download individual apps. The New York Times and the Economist are guilty of making their RSS feeds hard to find.

There are dozens and dozens of Android RSS readers out there, but by coincidence I settled on the same one as you and mostly like it. I hoped for a reader with cloud sync so that from one Android device to the next the app would remember which articles I'd marked as 'read' but no love. The closest I got was running an instance of ttrss on my own server and using not the official Android app but a 3rd party one. That was a lot of work and I didn't much like the interface, so I dropped it and am back to RssDemon again.

On the desktop side, huge thumbs up for the console based reader 'newsbeuter' and either rawdog or CURN. They're all somewhat obscure, but you can run newsbeuter on a shell account and then access it from work over SSH or something, and it's a lean, mean feed reader. Rawdog creates an HTML page you can use on your own site (I put one up for my own use here: http://therandymon.com/rawdogger.html via a cron script that runs every morning). CURN is a java app, which turns people off, but it's highly configurable and you can use it to create and mail to yourself an HTML email of your feeds, and do other things as well. All these solutions are better, in my opinion, than just being forced to download an app for every news source.

Then, you're only inches away from podcast technology too - highly recommend doggcatcher on Android for managing podcasts.

Re: Good stuff (Score: 1)

by evilviper@pipedot.org on 2014-11-18 14:39 (#2V42)

It's not strange we use the same app. Back when I went looking for an RSS reader, there was only a handful in the Market (now Play Store), and I used the by-far most popular and highest rated one. Just by virtue of being among the first, and still being around many years later (even if it has been a dead project for 2+ years now) they've got a lot of users to show for it. Being old and getting no updates has the secondary advantage of it still running on older phones, where newer apps often do not.

I'm not too happy with them, as I purchased a license for v1 right at the end of its life cycle, and they offered no upgrade path to v2. No idea about v2 to v3. But with no signs of life, the current release of v3 is probably the end of the line, even with the bugs with some feeds and podcast handling. It's just a question of how long until newer versions of Android break backwards compatibilty and put and end to it.

Looking through the store I don't see any other RSS readers that look obviously good. That's likely a byproduct of my now-aged phone, hiding newer and incompatible apps, as is frequently the case.

Re: Good stuff (Score: 1)

by zafiro17@pipedot.org on 2014-11-18 17:02 (#2V46)

I went to play.google.com and searched under apps for "RSS Reader" and it turned up at least 160 apps. Guesstimate that about half are either specialized ("News feed for the Syrian Revolution") or bad quality and you've still got 80 apps, all of whom seem pretty darned indistinguishable.

Ideally, get one that offers OPML import/export to make your life easier. Typing all those feeds into a new reader would be a major unpleasant activity.

RSS and Social Networks (Score: 2, Interesting)

by starbreaker@pipedot.org on 2014-11-26 16:51 (#2V9R)

One of the things that annoys me about all existing social networks is that none of them let you connect the RSS feed for your blog for auto-posting. Instead, you have to find a third-party service that provides this functionality via your preferred silo's API, and hope the API doesn't break.

RSS is the killer-app of the internet! (Score: 1)

by rpdillon@pipedot.org on 2014-11-26 17:31 (#2V9S)

I'm surprised to see TT-RSS not mentioned. With the death of Google Reader looming in December 2012, I switched to a self-hosted instance of TT-RSS, and I read it across all my machines and my phone.

It has article expanders and filtering, so there's a little bit of overlap with some of the other services you mention. Having used it for close to 2 years now, I can highly recommend it.