SimplePie: Weblog » Sorting multiple feeds by time and date

Sorting multiple feeds by time and date 23 Aug 2006

Update: With the release of SimplePie 1.0, this tutorial became obsolete. Please check out the new tutorial instead.

In this demo, I will show you how to parse multiple feeds and sort all of the posts from all feeds in reverse-chronological order (newest to oldest).

What you’ll need for this demo

The Multifeeds package
And, of course, the latest SimplePie for the RSS parsing (1.0 Beta 2 or later)

What you need to do

Unzip the multifeeds.zip file.
Drop the latest version of simplepie.inc into the resulting folder.
Edit — or don’t edit — the index.php file to your heart’s delight. (Maybe you should just let it run untouched the first time.)
Upload the whole multifeed folder to your webserver (or your localhost)
Set the cache folder to server-writable.
Load the index.php into your web browser, and voila!

Notes

The very first time you load the page will take more time than the subsequent load will because SimplePie needs to cache all of the feeds for the first time.
In the feeds you use, make sure that the items in that feed have a published or last modified date associated with them. If not, then those feed items will be shuffled out of order.
Because SimplePie Beta 3 has support for something called HTTP Conditional Get, cache handling is much improved over Beta 2, leading to better overall performance.

Live Demo

→ Multiple Feeds Demo

Posted by Ryan Parman at 9:20 am. Comments (19)

Comment by raptrex 23 Aug 2006 at 12:16 pm

yay you finally posted this!
you may or may not remember that i requested on how to do this

→ Permalink

Comment by John Hughes 23 Aug 2006 at 7:56 pm

Thank you VERY much for this effort.

First: You are distributing simplepie.inc in the multifeed folder. It is dated to a “newer” date than the beta that you say to download and add.

Second: I’ve discovered two glitches that would be nice to fix.

This feed: http://feeds.feedburner.com/tbc135 produces dates of 1 Jan 1970 | 12:00am GMT, which places all of its content at the top.

This feed: http://www.modbee.com/opinions/v-rssxml/index.rss causes these error messages:

Warning: array_slice() [function.array-slice]: The first argument should be an array in /home/www/sbedletter/multifeed/simplepie.inc on line 962

Warning: Invalid argument supplied for foreach() in /home/www/sbedletter/multifeed/index.php on line 37

Thanks again for all of the work on this project.

→ Permalink

Comment by Ryan Parman 23 Aug 2006 at 8:34 pm

John —

1) You’re right! I was using that for testing and forgot to take it back out.

2a) There are no dates in the items of the feed. In the notes portion of the page, number 2, it notes that there needs to be dates to sort the posts by.

2b) I’ll have to look into the warnings for this feed, but likewise, this feed is also missing the date information that it needs to be able to sort by.

→ Permalink

Comment by John Hughes 23 Aug 2006 at 9:26 pm

For your entertainment, here’s an example of the script at play:

http://ipsosacto.com/CaliMSM

Also, I have found a cheap work-around for the problems with the two URLs listed above.

Call the feed into Google Reader. Share the feed from Google Reader. Load the Google Reader feed.

This solves both the feedburner no date issue and the modbee errors.

The drawback: Instead of the original Headline and Title, Google turns the Title into the headline and substitues “Items labeled “modbee” by John in Google Reader”

Here’s another idea:

In simplepie.inc, in function get_date($date_format = 'j F Y, g:i a')
change the final else (around line 1136)

else { return false; }
to

else { /* no date found; fake it; -25200 (seconds) pulls the time back to Pacific Daylight Savings */ $today = getdate(); $nodate = $today[0] - 25200; return date($date_format, $nodate); }

This, again, is a limited solution. On a feed that has several day’s of content, everything is considered the same date. But since the alternative is 1970, this is an easy enough fix.

And one final point: The error I mentioned in my first post:

warning: array_slice() [function.array-slice]: The first argument should be an array in /home/www/sbedletter/multifeed/simplepie.inc on line 962.

… is caused when the feed has no content.

→ Permalink

Comment by John Hughes 25 Aug 2006 at 11:02 pm

I have no idea whether this is the best solution, but these modifications to simplepie.inc and the multifeed index.php eliminate error messages when an empty feed is encountered.

In simplepie.inc (around line 954) add a check for content in data[items]

function get_items($start = 0, $end = 0)
{
//check to be sure this isn’t empty
if (isset($this->data['items']) )
{
if ($end == 0)
{
return array_slice($this->data['items'], $start);
}
else
{
return array_slice($this->data['items'], $start, $end);
}
}
}

In the index.php for the multiple feeds (around line 37) add a check for items again.

$workitems = $feed->get_items(0, 10);
if (isset($workitems)) //make sure there’s something to do
{
foreach ($workitems as $item) {

* We’re going to take data from the $item and put it together into a single string, delimiting each chunk

* with five colons (you can delimit however you want, but it’s unlikely that you’ll come across five

* consecutive colons in the content of a feed).

// Start with a blank slate

$data = ”;

// We’re going to start with milliseconds since Unix epoch (this is the datestamp that we’ll sort by)

$data .= $item->get_date(’U') . ‘:::::’;

// We’ll choose a single timezone (GMT) to base all dates/times on.

$data .= gmdate(’j M Y | g:ia \G\M\T’, $item->get_date(’U')) . ‘:::::’;

// Get the title of the posting

$data .= $item->get_title() . ‘:::::’;

// Get the permalink for the posting

$data .= $item->get_permalink() . ‘:::::’;

// Get the description content for the posting

$data .= $item->get_description() . ‘:::::’;

// Besides $item data, we’ll also get the title of the $feed we’re pulling this from

$data .= $feed->get_feed_title() . ‘:::::’;

// Lastly, we’ll get the permalink to the $feed that we’re pulling this from.

// Since it’s the last one, we don’t need to add the delimiter to the end.

$data .= $feed->get_feed_link();

// Place this whole thing into the next available spot in the $multifeeds array.

$multifeeds[] = $data;

}

}
else
{
// no data found
$data = ‘:::::::::: Empty feed found’;
$multifeeds[] = $data;
}

// We’re done with $feed for this round of the loop, so we’ll wipe it out so we can oop back and start fresh.

unset($feed);

}

→ Permalink

Comment by feha 26 Aug 2006 at 2:08 am

Great work, thank you.
Short review:
http://www.vision.to/CMS/Home/index.php?pid=291

→ Permalink

Comment by porcupine 12 Sep 2006 at 12:09 am

Thank you for this great improvement.
Some questions:

1. Let’s say that I include 20 websites to retrieve their content. Is there anyway to show to the visitors of my website a list with the last 10 posts? In other words can I reduce the number of the posts shown in my page?

2. Is there a way to set the number of words shown in the description of a post similarly to the way the Wordpress plugin hundles the the “desc” attribute.

3. What happens to the not well formed feeds? As far as I can see I get an error message at the top of my screen. Is there any way to remove this message? Can I handle it any other way?

Thanks in advance

→ Permalink

Comment by Ben 12 Sep 2006 at 6:38 am

If a feed I’m using has its dates set to GMT and I wan’t to show them in EST, how would I go about changing the time thats shown?

→ Permalink

Comment by Simon 12 Sep 2006 at 1:02 pm

the solution suggested by john hughes above for error (Warning: Invalid argument supplied for foreach() in /home/www/sbedletter/multifeed/index.php on line 37) doesn’t seem to work for me, i’ve made the changes but, the error is still showing up.
any other suggestions?

→ Permalink

Comment by denny 10 Oct 2006 at 7:43 pm

is there a script to send email for every feed fetched?
something like rssfwd.com on my own hosting server?

→ Permalink

Comment by Boris Anthony 19 Oct 2006 at 1:07 pm

Neat, glad to see the way I did this (hadn’t seen this before) is pretty much exactly the same. Except I use an associative array and jam everything I can into it (including feed context):
$success_feed_init = $feed->init();


if ($success_feed_init && $feed->data) {

	// Feed info

	$ftitle 	= $feed->get_feed_title();

	$flink 		= $feed->get_feed_link();

	if($feed->get_image_exist()) {

		$fimg = $feed->get_image_url();

	} else { $fimg = "0"; }

	foreach($feed->get_items() as $item) {

		// restrict to within last 7 days?

		$item_datestamp 	= strtotime($item->get_date('Y-m-d H:i:s'));

		if ($item_datestamp >= $days_ago) {

			// Author info

			// B0rkd in SimplePie??

			//$item_author 	= $item->get_authors(0);

			//$iauth 			= $item_author->get_name();

			//$iauthlink 		= $item_author->get_link();

$items_array[$item->get_date('YmdHis')."-".$item->get_id()] = array( 'ftitle' => $ftitle, 'flink' => $flink, 'fimg' => $fimg, //'iauth' => $iauth, //'iauthlink' => $iauthlink, 'ititle' => $item->get_title(), 'iperma' => $item->get_permalink(), 'icontent' => $item->get_description(), 'idate' => $item->get_date('Y-m-d H:i:s'), 'idateago' => relativeDate($item->get_date('YmdHis')), 'icats' => $item->get_category(), 'idel' => $item->add_to_delicious(), 'idigg' => $item->add_to_digg(), 'ifurl' => $item->add_to_furl(), 'invine' => $item->add_to_newsvine() ); } else { break ; } } unset($ftitle); unset($flink); }
(there are references to some vars and functions I use that aren’t present in the snippet, namely vars for restriciting to X days ago and a function for making “posted X [time] ago” stamps)
(oh and yeah get_author seems to be busted… even on the demo running here at SimplePie.org. :\
Cheerio.

→ Permalink

Comment by Boris Anthony 19 Oct 2006 at 1:10 pm

oh i’m a dork.. forgot the:
krsort($items_array);
outside the while loop which loops through the array of feed URLs

→ Permalink

Comment by Abhimanyu Chirimar 4 Nov 2006 at 6:36 am

Hello,
For me all the dates get set to GMT at the time. I just want to show when they were published.

→ Permalink

Comment by TDH 15 Nov 2006 at 7:09 am

Has anyone found a solution to porcupine’s question above? Is there a way to limit the total number of feeds being displayed?

I guess you would do it here?

// Set your own configuration options as you see fit.
$feed = new SimplePie();
$feed->feed_url($url);
$feed->cache_max_minutes(30);
$feed->replace_headers(true);
$feed->init();

→ Permalink

Comment by Ryan Parman 15 Nov 2006 at 10:28 am

We don’t usually monitor the comments for questions, but this might answer TDH’s question: http://simplepie.org/support/viewtopic.php?id=297

→ Permalink

Comment by TDH 15 Nov 2006 at 12:41 pm

Many thanks Ryan, that was exactly what I was looking for. I’ll make sure to buy you something from your wishlist when this month’s payments drop in.

Cheers!

→ Permalink

Pingback by scot hacker’s foobar blog » SimplePie 8 Dec 2006 at 10:17 pm

[...] Yay SimplePie! Super-simple RSS class and mash-up tools for web services and integration. Just used it with the multifeeds package to create a mash of antiweb feeds. [...]

→ Permalink

Pingback by Gahmen Watchdog, Version 3 &#124 brennan.sg 10 Apr 2007 at 6:07 pm

[...] am working on a customised aggregation system that leverages on the power of SimplePie, around the multifeed demo which demonstrated the use of SimplePie to fetch feeds and sort them according to date. This will [...]

→ Permalink

Pingback by Finally, a feed reader that suits me | The Montoya Herald 29 Jul 2007 at 9:34 am

[...] done without SimplePie, an excellent feed parsing library for PHP. More specifically, it was the Sorting multiple feeds by time and date tutorial from the SimplePie weblog that allowed me to finish this project so quickly (and made me [...]

→ Permalink