can't open large RSS files

[NeXuM]

This is really weird and i really don't understand why it is happenning. I can't open large RSS feeds even if i limit vbulletin to import the first 10 items.

So i have created some test rss feeds to see at what point the bug happens.

This RSS feed will work perfectly:
http://pirate-punk.com/feed_test2.php

But if i add ONE more entry it stops working:
http://pirate-punk.com/feed_test.php

vBulletin tells me "XML error : No error at line 0" and this non-sense isn't helping me.

I'm really annoyed and i've tried everything.

Please can someone help me before i smash my keyboard on my head ? :P

I use vBulletin 3.6.4. A few years ago i have opened really large RSS files without problems but a few months ago i have moved to a new dedicated server. Maybe it is some option in php.ini preventing to open large files ?

[Zachery]

Do you have suhosin or mod_security loaded? What is the actual difference between the two files?

[NeXuM]

How can i know if i have suhosin or mod_security loaded ? i checked my phpinfo but couldn't find any match for those names

There is no big difference between the files. the first one (feed_test2.php) didn't work so i removed some <item> in the RSS file until it started working.
The only difference now is that "feed_test2.php" has one more <item> than "feed_test.php" so it must have something to do with the file size or lenght. So it is the exact same RSS feed but one has less entries.

feed_test2.php (not working) = 3536 lines (705 items in xml) php file is 69 368 bytes
feed_test.php (working) = 3531 lines (704 items in xml) php file is 69 270 bytes

My phpinfo = http://www.pirate-punk.net/phpinfo.php
XML's server phpinfo = http://www.pirate-punk.com/phpinfo.php
Both server are OVH dedicated servers

[Zachery]

suhosin is a security patch, though I don't think its loaded.
mod_security is an apache module, it has rules that filter out "problematic" things.

Anything in your php/apache error logs?

[NeXuM]

No, nothing in the error logs as far as i can tell. I have also my php.ini set to display errors and vBulletin isn't outputing any errors other than "Erreur XML : No error à la ligne 0"

[NeXuM]

I gave up and ended up writing my own RSS poster in PHP. Everything is working now if i use my own script instead of vbulletin built-in rss poster.

[tbworld]

Would be nice if you posted your fix to help others that might have the same problem. Give a little back. :)
Glad your up and running though, nice site.

[Zachery]

He wrote his own importer, that's not really a fix. Hard to debug this one though.

I suspect another error occurred, but you weren't being alerted to it.

[tbworld]

He wrote his own importer, that's not really a fix.

Just to be clear, I have not used the 3.6.4 version of vbulletin so I should not have used the word 'fix'. That implies that there is a bug in the vbulletin code and I have no idea if the thread-owners problem exists in that version.

My suggestion was to just help the vb.org community, and if you ask others for a solution to problems you are having: It is just nice to share your results. I mean no disrespect to the thread owner, nor vbulletin.

[NeXuM]

I'm not sure if there is a bug with vBulletin 3.6.4 RSS importer, like i said it works with small feeds but returns an error when the feed is too large. And also, i have already used the exact same version of VB to import a very large feed but that was over 3 years ago and i have moved server since then. I'm not sure what went wrong.

Anyway, here is the script i wrote to import my RSS feed into vBulletin:

<?php
header('Content-Type: text/html; charset=utf-8');
require_once('./global.php');
require_once('./includes/functions_databuild.php');
error_reporting(E_ALL & ~E_NOTICE & ~8192);

$itemnumber = "0";

$xml = file_get_contents("your_feed_url");
$sxml = simplexml_load_string($xml);

foreach($sxml->channel->item as $name => $row)
{
$itemnumber = $itemnumber + 1;
if ($itemnumber > 50) {
// uncomment if you want to limit to 50 entries
// break;
}

$forumid = 98;
$userid = "1";
$title = utf8_decode($row->title);
$pagetext = utf8_decode($row->description);


// Create a new datamanager for posting
$threaddm =& datamanager_init('Thread_FirstPost', $vbulletin, ERRTYPE_ARRAY, 'threadpost');
$allowsmilie = '0'; // Are we allowing smilies in our post
$visible = '1'; // If the post visible (ie, moderated or not)

// check if the post already exists
$mtitle = addslashes(htmlspecialchars($title));
$query = "SELECT * FROM thread WHERE title = '$mtitle' AND forumid = '$forumid' AND postuserid = '$userid'";
$res = mysql_query($query) or die(mysql_error());
$exist = mysql_num_rows($res);
if ($exist == 1) {
echo "item # $itemnumber already exists ($title)<br><br>";
}
if ($exist == 0) {
echo "item # $itemnumber posted ($title)<br><br>";
// post the shit
$foruminfo = fetch_foruminfo($forumid);
$threadinfo = array();
$user = htmlspecialchars_uni( fetch_userinfo($userid) );

$threaddm->set_info('forum', $foruminfo);
$threaddm->set_info('thread', $threadinfo);
$threaddm->setr('forumid', $forumid);
$threaddm->setr('userid', $userid);
$threaddm->setr('pagetext', $pagetext);
$threaddm->setr('title', $title);
$threaddm->set('allowsmilie', $allowsmilie);
$threaddm->set('visible', $visible);

// Lets see what happens if we save the page
$threaddm->pre_save();
if(count($threaddm->errors) < 1) {
// Basically if the page will save without errors then let do it for real this time
$threadid = $threaddm->save();
unset($threaddm);
} else {
// There was errors in the practice run, so lets display them
var_dump ($threaddm->errors);
}
}

}
?>

[tbworld]

Thanks for sharing!!! That really help the community. I run vb4 so I am really not in-depth on vb3 internals, but I can appreciate the time you must have spent working things out.

Curious: What was the error when the feeds were to long?

[NeXuM]

Thanks for sharing!!! That really help the community. I run vb4 so I am really not in-depth on vb3 internals, but I can appreciate the time you must have spent working things out.

Curious: What was the error when the feeds were to long?

"XML error : No error at line 0"

That was the error.... lol

[tbworld]

"XML error : No error at line 0"

Not the most clarifying error statement. It is not one of my favorites.

[NeXuM]

Looks like everything using XML isn't working on my new server. I tried importing a new style into vbulletin and i get the same error:
"XML Error: No error at line 1"

I really need to fix it...

Must be a server problem with XML. Any idea where it comes from ?

[Zachery]

Silly question, you have XML compiled into php right?

[NeXuM]

yes, phpinfo shows libxml is enabled

[Zachery]

Can you re-link the phpinfo?

[NeXuM]

<a href="http://www.pirate-punk.net/phpinfo.php" target="_blank">http://www.pirate-punk.net/phpinfo.php</a>

[Zachery]

That is pretty weird, only red flag I see is upload_tmp_dir not being set explicitly, which can cause some issues with file processing.