vb.org Archive - can't open large RSS files

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)

- - can't open large RSS files (https://vborg.vbsupport.ru/showthread.php?t=299718)

can't open large RSS files

This is really weird and i really don't understand why it is happenning. I can't open large RSS feeds even if i limit vbulletin to import the first 10 items.

So i have created some test rss feeds to see at what point the bug happens.

This RSS feed will work perfectly:
http://pirate-punk.com/feed_test2.php

But if i add ONE more entry it stops working:
http://pirate-punk.com/feed_test.php

vBulletin tells me "XML error : No error at line 0" and this non-sense isn't helping me.

I'm really annoyed and i've tried everything.

Please can someone help me before i smash my keyboard on my head ? :P

I use vBulletin 3.6.4. A few years ago i have opened really large RSS files without problems but a few months ago i have moved to a new dedicated server. Maybe it is some option in php.ini preventing to open large files ?

Do you have suhosin or mod_security loaded? What is the actual difference between the two files?

How can i know if i have suhosin or mod_security loaded ? i checked my phpinfo but couldn't find any match for those names

There is no big difference between the files. the first one (feed_test2.php) didn't work so i removed some <item> in the RSS file until it started working.
The only difference now is that "feed_test2.php" has one more <item> than "feed_test.php" so it must have something to do with the file size or lenght. So it is the exact same RSS feed but one has less entries.

feed_test2.php (not working) = 3536 lines (705 items in xml) php file is 69 368 bytes
feed_test.php (working) = 3531 lines (704 items in xml) php file is 69 270 bytes

My phpinfo = http://www.pirate-punk.net/phpinfo.php
XML's server phpinfo = http://www.pirate-punk.com/phpinfo.php
Both server are OVH dedicated servers

suhosin is a security patch, though I don't think its loaded.
mod_security is an apache module, it has rules that filter out "problematic" things.

Anything in your php/apache error logs?

No, nothing in the error logs as far as i can tell. I have also my php.ini set to display errors and vBulletin isn't outputing any errors other than "Erreur XML : No error à la ligne 0"

I gave up and ended up writing my own RSS poster in PHP. Everything is working now if i use my own script instead of vbulletin built-in rss poster.

Would be nice if you posted your fix to help others that might have the same problem. Give a little back. :)
Glad your up and running though, nice site.

He wrote his own importer, that's not really a fix. Hard to debug this one though.

I suspect another error occurred, but you weren't being alerted to it.

Quote:

Originally Posted by Zachery (Post 2431981)

He wrote his own importer, that's not really a fix.

Just to be clear, I have not used the 3.6.4 version of vbulletin so I should not have used the word 'fix'. That implies that there is a bug in the vbulletin code and I have no idea if the thread-owners problem exists in that version.

My suggestion was to just help the vb.org community, and if you ask others for a solution to problems you are having: It is just nice to share your results. I mean no disrespect to the thread owner, nor vbulletin.

I'm not sure if there is a bug with vBulletin 3.6.4 RSS importer, like i said it works with small feeds but returns an error when the feed is too large. And also, i have already used the exact same version of VB to import a very large feed but that was over 3 years ago and i have moved server since then. I'm not sure what went wrong.

Anyway, here is the script i wrote to import my RSS feed into vBulletin:

Code:

<?php

header('Content-Type: text/html; charset=utf-8');

require_once('./global.php');

require_once('./includes/functions_databuild.php');

error_reporting(E_ALL & ~E_NOTICE & ~8192);



$itemnumber = "0";



$xml = file_get_contents("your_feed_url");

$sxml = simplexml_load_string($xml);



foreach($sxml->channel->item as $name => $row)

{

$itemnumber = $itemnumber + 1;

if ($itemnumber > 50) {

// uncomment if you want to limit to 50 entries

// break;

}



$forumid = 98;

$userid = "1";

$title = utf8_decode($row->title);

$pagetext = utf8_decode($row->description);





// Create a new datamanager for posting

$threaddm =& datamanager_init('Thread_FirstPost', $vbulletin, ERRTYPE_ARRAY, 'threadpost');

$allowsmilie = '0';                                                             // Are we allowing smilies in our post

$visible = '1';      // If the post visible (ie, moderated or not)



// check if the post already exists

$mtitle = addslashes(htmlspecialchars($title));

        $query = "SELECT * FROM thread WHERE title = '$mtitle' AND forumid = '$forumid' AND postuserid = '$userid'";

        $res = mysql_query($query) or die(mysql_error());

        $exist = mysql_num_rows($res);

        if ($exist == 1) {

        echo "item # $itemnumber already exists ($title)<br><br>";

        }

        if ($exist == 0) {

        echo "item # $itemnumber posted ($title)<br><br>";

                // post the shit

                $foruminfo = fetch_foruminfo($forumid);

                $threadinfo = array();

                $user = htmlspecialchars_uni( fetch_userinfo($userid) );



                $threaddm->set_info('forum', $foruminfo);

                $threaddm->set_info('thread', $threadinfo);

                $threaddm->setr('forumid', $forumid);

                $threaddm->setr('userid', $userid);

                $threaddm->setr('pagetext', $pagetext);

                $threaddm->setr('title', $title);

                $threaddm->set('allowsmilie', $allowsmilie);

                $threaddm->set('visible', $visible);



                // Lets see what happens if we save the page

                $threaddm->pre_save();

                if(count($threaddm->errors) < 1) {

                        // Basically if the page will save without errors then let do it for real this time

                        $threadid = $threaddm->save();

                        unset($threaddm);

                } else {

                        // There was errors in the practice run, so lets display them

                        var_dump ($threaddm->errors);

                }

        }



}

?>

Thanks for sharing!!! That really help the community. I run vb4 so I am really not in-depth on vb3 internals, but I can appreciate the time you must have spent working things out.

Curious: What was the error when the feeds were to long?

Quote:

Originally Posted by tbworld (Post 2432277)

"XML error : No error at line 0"

That was the error.... lol

Quote:

Originally Posted by NeXuM (Post 2432278)

"XML error : No error at line 0"

Not the most clarifying error statement. It is not one of my favorites.

Looks like everything using XML isn't working on my new server. I tried importing a new style into vbulletin and i get the same error:
"XML Error: No error at line 1"

I really need to fix it...

Must be a server problem with XML. Any idea where it comes from ?

Silly question, you have XML compiled into php right?

yes, phpinfo shows libxml is enabled

Can you re-link the phpinfo?

<a href="http://www.pirate-punk.net/phpinfo.php" target="_blank">http://www.pirate-punk.net/phpinfo.php</a>

That is pretty weird, only red flag I see is upload_tmp_dir not being set explicitly, which can cause some issues with file processing.

X vBulletin 3.8.12 by vBS Debug Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)bbcode_code_printable (3)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (19)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages:

Phrase Groups Available:

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.02845 seconds Memory Usage 1,762KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)bbcode_code_printable (3)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (19)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: