can't open large RSS files

[NeXuM]

This is really weird and i really don't understand why it is happenning. I can't open large RSS feeds even if i limit vbulletin to import the first 10 items.

So i have created some test rss feeds to see at what point the bug happens.

This RSS feed will work perfectly:
http://pirate-punk.com/feed_test2.php

But if i add ONE more entry it stops working:
http://pirate-punk.com/feed_test.php

vBulletin tells me "XML error : No error at line 0" and this non-sense isn't helping me.

I'm really annoyed and i've tried everything.

Please can someone help me before i smash my keyboard on my head ? :P

I use vBulletin 3.6.4. A few years ago i have opened really large RSS files without problems but a few months ago i have moved to a new dedicated server. Maybe it is some option in php.ini preventing to open large files ?

[Zachery]

Do you have suhosin or mod_security loaded? What is the actual difference between the two files?

[NeXuM]

How can i know if i have suhosin or mod_security loaded ? i checked my phpinfo but couldn't find any match for those names

There is no big difference between the files. the first one (feed_test2.php) didn't work so i removed some <item> in the RSS file until it started working.
The only difference now is that "feed_test2.php" has one more <item> than "feed_test.php" so it must have something to do with the file size or lenght. So it is the exact same RSS feed but one has less entries.

feed_test2.php (not working) = 3536 lines (705 items in xml) php file is 69 368 bytes
feed_test.php (working) = 3531 lines (704 items in xml) php file is 69 270 bytes

My phpinfo = http://www.pirate-punk.net/phpinfo.php
XML's server phpinfo = http://www.pirate-punk.com/phpinfo.php
Both server are OVH dedicated servers

[Zachery]

suhosin is a security patch, though I don't think its loaded.
mod_security is an apache module, it has rules that filter out "problematic" things.

Anything in your php/apache error logs?

[NeXuM]

No, nothing in the error logs as far as i can tell. I have also my php.ini set to display errors and vBulletin isn't outputing any errors other than "Erreur XML : No error à la ligne 0"

[NeXuM]

I gave up and ended up writing my own RSS poster in PHP. Everything is working now if i use my own script instead of vbulletin built-in rss poster.

[tbworld]

Would be nice if you posted your fix to help others that might have the same problem. Give a little back.
Glad your up and running though, nice site.

[Zachery]

He wrote his own importer, that's not really a fix. Hard to debug this one though.

I suspect another error occurred, but you weren't being alerted to it.

[tbworld]

Originally Posted by Zachery

He wrote his own importer, that's not really a fix.

Just to be clear, I have not used the 3.6.4 version of vbulletin so I should not have used the word 'fix'. That implies that there is a bug in the vbulletin code and I have no idea if the thread-owners problem exists in that version.

My suggestion was to just help the vb.org community, and if you ask others for a solution to problems you are having: It is just nice to share your results. I mean no disrespect to the thread owner, nor vbulletin.

[NeXuM]

I'm not sure if there is a bug with vBulletin 3.6.4 RSS importer, like i said it works with small feeds but returns an error when the feed is too large. And also, i have already used the exact same version of VB to import a very large feed but that was over 3 years ago and i have moved server since then. I'm not sure what went wrong.

Anyway, here is the script i wrote to import my RSS feed into vBulletin:

<?php
header('Content-Type: text/html; charset=utf-8');
require_once('./global.php');
require_once('./includes/functions_databuild.php');
error_reporting(E_ALL & ~E_NOTICE & ~8192);

$itemnumber = "0";

$xml = file_get_contents("your_feed_url");
$sxml = simplexml_load_string($xml);

foreach($sxml->channel->item as $name => $row)
{
$itemnumber = $itemnumber + 1;
if ($itemnumber > 50) {
// uncomment if you want to limit to 50 entries
// break;
}

$forumid = 98;
$userid = "1";
$title = utf8_decode($row->title);
$pagetext = utf8_decode($row->description);


// Create a new datamanager for posting
$threaddm =& datamanager_init('Thread_FirstPost', $vbulletin, ERRTYPE_ARRAY, 'threadpost');
$allowsmilie = '0';                                                             // Are we allowing smilies in our post
$visible = '1';      // If the post visible (ie, moderated or not)

// check if the post already exists
$mtitle = addslashes(htmlspecialchars($title));
	$query = "SELECT * FROM thread WHERE title = '$mtitle' AND forumid = '$forumid' AND postuserid = '$userid'";
	$res = mysql_query($query) or die(mysql_error());
	$exist = mysql_num_rows($res);
	if ($exist == 1) {
	echo "item # $itemnumber already exists ($title)<br><br>";
	}
	if ($exist == 0) {
	echo "item # $itemnumber posted ($title)<br><br>";
		// post the shit
		$foruminfo = fetch_foruminfo($forumid);
		$threadinfo = array();
		$user = htmlspecialchars_uni( fetch_userinfo($userid) );

		$threaddm->set_info('forum', $foruminfo);
		$threaddm->set_info('thread', $threadinfo);
		$threaddm->setr('forumid', $forumid);
		$threaddm->setr('userid', $userid);
		$threaddm->setr('pagetext', $pagetext);
		$threaddm->setr('title', $title);
		$threaddm->set('allowsmilie', $allowsmilie);
		$threaddm->set('visible', $visible);

		// Lets see what happens if we save the page
		$threaddm->pre_save();
		if(count($threaddm->errors) < 1) {
			// Basically if the page will save without errors then let do it for real this time
			$threadid = $threaddm->save();
			unset($threaddm);
		} else {
			// There was errors in the practice run, so lets display them
			var_dump ($threaddm->errors);
		}
	}

}
?>