Simple RSS Parser with Python and PHP

If you want to throw a feed into your web page, one way I have found that works well is to use the Python feed parser module. The way I do it is just put the data into a PostgreSQL database table and then truncate the table each time I pull the feed again. The python script runs on my CentOS server using cron. You can see this in action on my web site feedmud.com

One cautionary note is that some RSS providers explicitly state in their terms not to do this, so in the near future I may be removing some of these from the web site. Obviously the credentials have been removed from the code.

Here is the Python code, it’s very simple. This code was for The Drudge Report.

import feedparser
import psycopg2
#db connection
try:
    drudge_conn = psycopg2.connect("dbname='' user='' host='' password=''")
except:
    print("can't connect to the db")
drudge = feedparser.parse('http://feeds.feedburner.com/DrudgeReportFeed')
drudge_cur = drudge_conn.cursor()
#Remove old data
drudge_cur.execute("TRUNCATE TABLE drudge;")
for e in drudge.entries:
    drudge_cur.execute("INSERT INTO drudge (title,link,category,pub_date,feed_name, site_name) VALUES ('{0}','{1}','politics','{2}','DrudgeReportFeed','drudgereport.com')".format(e.title.replace("'", "''"),e.link.replace("'", "''"),e.published))
drudge_conn.commit()
drudge_cur.close()
drudge_conn.close()

import feedparser

import psycopg2

#db connection

try:

drudge_conn = psycopg2.connect("dbname='' user='' host='' password=''")

except:

print("can't connect to the db")

drudge = feedparser.parse('http://feeds.feedburner.com/DrudgeReportFeed')

drudge_cur = drudge_conn.cursor()

#Remove old data

drudge_cur.execute("TRUNCATE TABLE drudge;")

for e in drudge.entries:

drudge_cur.execute("INSERT INTO drudge (title,link,category,pub_date,feed_name, site_name) VALUES ('{0}','{1}','politics','{2}','DrudgeReportFeed','drudgereport.com')".format(e.title.replace("'", "''"),e.link.replace("'", "''"),e.published))

drudge_conn.commit()

drudge_cur.close()

drudge_conn.close()

Simple PHP database class to grab the new feeds out of the database.

<?php
class DrudgeDb
{
	function get_drudge_feed(){
		$conn = pg_pconnect("host= port=5432 dbname=user= password= ");
		$result = pg_query($conn, "SELECT title,link FROM drudge");
		if (!$result) {
			echo "An error occurred.\n";
			exit;	
		}
		pg_close($conn);
		return $result;
	}
}
?>

<?php

class DrudgeDb

{

function get_drudge_feed(){

$conn = pg_pconnect("host= port=5432 dbname=user= password= ");

$result = pg_query($conn, "SELECT title,link FROM drudge");

if (!$result) {

echo "An error occurred.\n";

exit;

}

pg_close($conn);

return $result;

}

Then pull it into your page

<?php
include 'templates/header.php';
include 'db/DrudgeDb.php';
$db = new DrudgeDb();
$result = $db->get_drudge_feed();
?>
<a name="top"></a> 
<div class="columns">
<div class="column">
    <b>Drudge Report Feed</b><br>
	<div class="content is-small">
	<?php
	while ($row = pg_fetch_row($result)) {
		echo "<a href=\"$row[1]\" target=\”_blank\”>$row[0]</a>";
		echo "<br />\n";
	}
	?>
	</div>
	<a href="#top"><b>TOP</b></a><br>
    </div>
</div>

<?php

include 'templates/header.php';

include 'db/DrudgeDb.php';

$db = new DrudgeDb();

$result = $db->get_drudge_feed();

<b>Drudge Report Feed</b><br>

<?php

while ($row = pg_fetch_row($result)) {

echo "<a href=\"$row[1]\" target=\”_blank\”>$row[0]</a>";

echo "<br />\n";

}

</div>

</div>

Here is the Python code, it’s very simple. This code was for The Drudge Report.

Simple PHP database class to grab the new feeds out of the database.

Then pull it into your page

Leave a Reply Cancel reply