Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[j_86]

This hack is VB 3.05 compatiable.

[memobug]

For the past few months I have been getting the following error. I believe I have traced it back to articlebot.php

Database error in vBulletin 3.0.3:
Invalid SQL: 
UPDATE user
SET posts = posts + Array
WHERE userid = 0

mysql error: Unknown column 'Array' in 'field list'
mysql error number: 1054
Date: Thursday 09th of December 2004 05:00:03 PM
Script: http://
Referer: 
Username: Unregistered
IP Address:

The error seems to be related to this bit of code:

if (is_array($users))
		{
			foreach ($users as $userid => $user_posts)
			{
				/* +---------------------------------+
				   | Query: update article bot(s)	|
				   +---------------------------------+ */


				$DB_site->query("
					UPDATE ".TABLE_PREFIX."user
					SET posts = posts + $user_posts
					WHERE userid = $userid
				");
			}
		}

Any idea how $userposts could take on a non-numeric value? I have two robots and I believe that one of them currently has no RSS assignments.

Regards,

Matt

[mavherzog]

Originally Posted by RS25com

I am still getting this error:

There is no file located at http://washingtondc.craigslist.org/...=max&format=rss

I believe this issue has not been fully addressed yet. Can anyone shed some light or should I just ditch this and find something else?

If this is not working for known good feeds, there are probably two most common reasons:

1. PHP on the server is not compiled with XML support.
2. php.ini "allow_url_fopen" is set to off.

[ragintajin]

I got this error after uploading the newest init.php:

Warning: chdir(): No such file or directory (errno 2) in /articlebot/articlebot.php on line 42

Any ideas??

[Java624]

Im about to install this I hope it works for 3.05

[mavherzog]

Originally Posted by Java624

Im about to install this I hope it works for 3.05

I'm using it with 3.0.5 with no issues. I LIKE IT!!!

[Gandolfe]

Everything works fine, seems like would be a great hack, thats if it was syndicating with the .xml .rss feeds - I get this error from articlebot.php


Fatal error: Call to a member function on a non-object in /hsphere/local/home/williamk/cs-elite.net/forums/articlebot/articlebot.php on line 92

PHP Code:

<?php



/* +------------------------------------------------------------------+
   | articlebot.php, v1.1, 07/25/04                                   |
   +------------------------------------------------------------------+
   | Support:   www.vbulletin.org/forum/showthread.php?t=67078        |
   +------------------------------------------------------------------+
   | Author:    velocd                                                |
   |            www.animesystem.com                                   |
   +------------------------------------------------------------------+ */



/* +---------------------------------+
   | Environment                     |
   +---------------------------------+ */



error_reporting(E_ALL & ~E_NOTICE);

define('NO_REGISTER_GLOBALS', 1);

$globaltemplates = array(
    'articlebot_post'
);



/* +---------------------------------+
   | Dependencies                    |
   +---------------------------------+ */



// Provide absolute server path to forums.
// e.g. /home/cjohn/public_html/forums/


// chdir('/hsphere/local/home/williamk/cs-elite.net/forums/');
// require_once './global.php';

include_once '/hsphere/local/home/williamk/cs-elite.net/forums/articlebot/rss_fetch.inc';


// If you intend to call this script by placing a
// include_once './articlebot/articlebot.php'; in your
// phpinclude_end template, then comment out:
//
// chdir('/path/to/forums/');
// require_once './global.php';
//
// Note, recommended usage for this script is by
// crontab with this shell command in mind:
//
// e.g. /usr/local/bin/php /path/to/forums/articlebot/articlebot.php
// (substituting the paths with their correct location)
//
// If /usr/local/bin/php is not the correct path (generally it is),
// use PuTTY to SSH in and use the command "whereis php" to find
// the correct path.
//
// Explanation on crontabs:
// http://www.cpanel.net/docs/cp/cronJobs.htm
//
// By this method you are not calling an extra query on every page
// to check for ready-to-post article bots.


if ($vboptions['similarthreadsearch'])
{
    require_once './includes/functions_search.php';
}



/* +------------------------------------------+
   | Request: article bot syndication         |
   +------------------------------------------+ */



if (!$_REQUEST['do'])
{
    /* +---------------------------------+
       | Query: fetch article bots       |
       +---------------------------------+ */


    $result_articlebots = $DB_site->query("
        SELECT articlebot_rss.*, articlebot.userid,
            articlebot.articlebotid,
            articlebot.regex_title_search AS global_regex_title_search,
            articlebot.regex_title_replace AS global_regex_title_replace,
            articlebot.regex_desc_search AS global_regex_desc_search,
            articlebot.regex_desc_replace AS global_regex_desc_replace,
            ".TABLE_PREFIX."user.username
        FROM articlebot_rss
        LEFT JOIN articlebot
            USING (articlebotid)
        LEFT JOIN ".TABLE_PREFIX."user
            USING (userid)
        WHERE nextpost <= ".TIMENOW."
            AND articlebot_rss.enabled = 1
            AND articlebot.enabled = 1
    ");


    if ($DB_site->num_rows($result_articlebots))
    {
        $fetch_days = 20;
        $timecut = TIMENOW - ((60*60*24) * $fetch_days);

        /* +---------------------------------+
           | Query: fetch threads            |
           +---------------------------------+ */


        $result_threads = $DB_site->query("
            SELECT title
            FROM articlebot_rss_cache
            WHERE dateline >= $timecut
        ");


        while ($thread = $DB_site->fetch_array($result_threads))
        {
            $threads[] = trim($thread['title']);
        }

        while ($articlebot = $DB_site->fetch_array($result_articlebots))
        {
            $rss = fetch_rss($articlebot['rss_path']);

            $articles = array_reverse($rss->items);

            $count = 0;

            foreach ($articles as $article)
            {
                $article['title'] = trim($article['title']);

                if (!$article['link'] || !$article['title'])
                {
                    continue;
                }

                if ($articlebot['desc_required'] && !$article['description'])
                {
                    continue;
                }

                if (is_array($threads) && in_array($article['title'], $threads))
                {
                    continue;
                }

                if (!$articlebot['lastpost'] && (--$articlebot['initialtrim'] > 0))
                {
                    /* +---------------------------------------+
                       | Query: insert unused cache threads    |
                       +---------------------------------------+ */


                    $DB_site->query("
                        INSERT INTO articlebot_rss_cache
                            (title, dateline)
                        VALUES
                            ('".addslashes($article['title'])."', ".TIMENOW.")
                    ");


                    continue;
                }


                /* +---------------------------------+
                   | Query: insert cache thread      |
                   +---------------------------------+ */


                $DB_site->query("
                    INSERT INTO articlebot_rss_cache
                        (title, dateline)
                    VALUES
                        ('".addslashes($article['title'])."', ".TIMENOW.")
                ");


                if ($articlebot['global_regex_title_search'] || $articlebot['regex_title_search'])
                {
                    if ($articlebot['regex_title_search'])
                    {
                        $articlebot['global_regex_title_search']   = $articlebot['regex_title_search'];
                        $articlebot['global_regex_title_replace']  = $articlebot['regex_title_replace'];
                    }

                    $article['title'] = @preg_replace(
                        "$articlebot[global_regex_title_search]",
                        "$articlebot[global_regex_title_replace]",
                        $article['title']
                    );
                }

                if ($vboptions['similarthreadsearch'] && $articlebot['build_similar_threads'])
                {
                    $similarthreads = fetch_similar_threads($article['title']);
                }
                else
                {
                    $similarthreads = '';
                }

                if ($articlebot['label'])
                {
                    $article['title'] = $articlebot['label'] . ' ' . $article['title'];
                }


                /* +---------------------------------+
                   | Query: insert thread            |
                   +---------------------------------+ */


                $DB_site->query("
                    INSERT INTO ".TABLE_PREFIX."thread
                        (title, lastpost, forumid, open,
                        replycount, postusername, postuserid, lastposter, dateline, iconid,
                        visible, attach, similar, is_rss)
                    VALUES
                        ('".addslashes($article['title'])."', ".TIMENOW.", $articlebot[forumid],
                         1, 0, '".addslashes($articlebot['username']) . "', $articlebot[userid],
                         '".addslashes($articlebot['username']) . "', ".TIMENOW.", $articlebot[iconid], 1, 0,
                         '".addslashes($similarthreads)."', 1)
                ");


                $thread_id = $DB_site->insert_id();

                $article['description'] = strip_tags(html_entity_decode(trim($article['description'])));

                if ($articlebot['global_regex_desc_search'] || $articlebot['regex_desc_search'])
                {
                    if ($articlebot['regex_desc_search'])
                    {
                        $articlebot['global_regex_desc_search']   = $articlebot['regex_desc_search'];
                        $articlebot['global_regex_desc_replace']  = $articlebot['regex_desc_replace'];
                    }

                    $article['description'] = @preg_replace(
                        "$articlebot[global_regex_desc_search]",
                        "$articlebot[global_regex_desc_replace]",
                        $article['description']
                    );
                }

                eval('$articlebot_post = "' . fetch_template('articlebot_post', 0, 0) . '";');


                /* +---------------------------------+
                   | Query: insert post              |
                   +---------------------------------+ */


                $DB_site->query("
                    INSERT INTO ".TABLE_PREFIX."post
                        (threadid, parentid, title, username, userid, dateline, pagetext, allowsmilie,
                         showsignature, ipaddress, iconid, visible, attach)
                    VALUES
                        ($thread_id, 0, '".addslashes($article['title'])."',
                         '".addslashes($articlebot['username'])."', $articlebot[userid],
                         ".TIMENOW.", '" . addslashes($articlebot_post)."', 0, 1,
                         '".addslashes($_SERVER['SERVER_ADDR'])."', $articlebot[iconid], 1, 0)
                ");


                $post_id = $DB_site->insert_id();


                /* +---------------------------------+
                   | Query: update thread            |
                   +---------------------------------+ */


                $DB_site->query("
                    UPDATE ".TABLE_PREFIX."thread
                    SET firstpostid = $post_id
                    WHERE threadid = $thread_id
                ");


                require_once './includes/functions_databuild.php';

                $foruminfo['indexposts'] = 1;

                build_post_index($post_id, $foruminfo, 1);

                if ($forums[$articlebot['forumid']])
                {
                    $forum_posts = ++$forums[$articlebot['forumid']]['forum_posts'];

                    unset($forums[$articlebot['forumid']]);
                }
                else
                {
                    $forum_posts = 1;
                }

                $forums[$articlebot['forumid']] = array(
                    'lastposter'    => $articlebot['username'],
                    'lastthread'    => $article['title'],
                    'lastthreadid'  => $thread_id,
                    'lasticonid'    => $articlebot['iconid'],
                    'forum_posts'   => $forum_posts
                );

                if ($users[$articlebot['userid']])
                {
                    $users[$articlebot['userid']]++;
                }
                else
                {
                    $users[$articlebot['userid']] = 1;
                }

                if (++$count >= $articlebot['fetchlimit'])
                    break;
            }

            $nextpost = TIMENOW + ($articlebot['postinterval'] * 60);


            /* +-------------------------+
               | Query: update rss       |
               +-------------------------+ */


            $DB_site->query("
                UPDATE articlebot_rss
                SET lastpost = ".TIMENOW.",
                    nextpost = $nextpost
                WHERE rssid = $articlebot[rssid]
            ");


            unset($rss);
        }

        if (is_array($users))
        {
            foreach ($users as $userid => $user_posts)
            {
                /* +---------------------------------+
                   | Query: update article bot(s)    |
                   +---------------------------------+ */


                $DB_site->query("
                    UPDATE ".TABLE_PREFIX."user
                    SET posts = posts + $user_posts
                    WHERE userid = $userid
                ");
            }
        }

        if (is_array($forums))
        {
            foreach ($forums as $forumid => $forum)
            {
                /* +---------------------------------------+
                   | Query: update affected forum(s)       |
                   +---------------------------------------+ */


                $DB_site->query("
                    UPDATE ".TABLE_PREFIX."forum
                    SET replycount = replycount + $forum[forum_posts],
                        threadcount = threadcount + $forum[forum_posts],
                        lastpost = ".TIMENOW.",
                        lastposter = '" . addslashes($forum['lastposter']) . "',
                        lastthread = '" . addslashes($forum['lastthread']) . "',
                        lastthreadid = $forum[lastthreadid],
                        lasticonid = $forum[lasticonid]
                    WHERE forumid = $forumid
                ");
            }
        }
    }
}



?>

That's a copy of how articlebot.php is at the moment. Any ideas?

[jshafer817]

I have installed vbarticles but I cannot read a feed I really want.
Its in Atom .3 and the encoding is in utf-8
They also have an rss feel but its encoding is also utf-8

Is this possible for me to do???

Thanks for your time! I could have never gotten this far without all of yaulls help.

[TTG]

Just installed and setup a few seperate posting categories .. works like a charm.
Just gotta work out this cron job instead of manually calling articlebot/articlebot.php every couple of hours.

Clicked install .. thanks for a great hack

[James Goddard]

Originally Posted by Torqued

Find the following code in articlebot.php:

PHP Code:

SET posts = posts + $user_posts 


replace with:

PHP Code:

SET posts = posts 


Thanks!

James