3.8.7 Patch Level 3 Custom Cron Task

[schan]

Hey all,

I am working on a product that involves querying the database for, say, 100 posts and does stuff with it. I set a scheduled task via Admin Panel to run this task every minute. When the task runs, it causes our forum (front-end) to be unusable until the query is done. I can see how this can be a problem, but why do the built-in vBulletin's cron tasks do not have this problem? Are these cron tasks built using special cron functions? If so, can you point me to the right direction so I can modify my script?

Thank you in advance. I'm looking forward to your responses.

- S

[ozzy47]

From the admin popup help page:

Set it to * to have it run every minute (not really recommended!).

Also, to know why it may be doing that, we would need the code you are using in order to debug it.

[Simon Lloyd]

Maybe your query is looping but as Ozzy47 says we need to see your code in order to help with that.

[schan]

Thanks for your responses. I was able to figure it out. The code was going through too many loops.

Thanks, all!

[ozzy47]

Thanks for replying back, and glad you got it sorted.

[schan]

Sorry that I have to revive this thread. I figured it would be better to than making a new one. We host a Q/A board and want to spell correct all the posts in our database. There are approximately 3.2 million. Our cron task (query for 1000 posts + sent to our spell correct engine) takes about 30 seconds to complete one task. The problem is that when it kicks off, it renders the page unusable so the user ends up having to wait until the task is complete. There are tons of cron jobs that vBulletin also runs, but it doesn't seem to hang the page. What are some reasons why the page would become unusable? I thought the cron job was suppose to run in the background. If not, how can I make it "run in the background" like a real cron?

Thank you in advance.

-S

[Lynne]

As we said before, post your code so we can see what you are doing.

[schan]

Just a bit of background info: the goal of this function is correct the text that gets sent to it. The function utilizes regex patterns to determine what needs fixing and when.

I can't post the full code, but I will try to post enough so that it gives you guys enough information to have an idea of what could be going on. Do keep in mind that it takes about 30 seconds to complete correcting 1000 posts. The 30 secs may be normal, but since it's running as a cron job, it should not render the page unusable since it's running in the background...at least it should be. Just in case you guys are wondering where's the query, I've tested that the query completes about 1/10 of a second or even faster. The "stall" occurs when the page text gets sent to the fix_spelling function. Anyways, here's the code:

<?php

public function fix_spelling($text, $severity = 0, $exceptions = array()) {

    list($rgx_start, $rgx_end_replace) = 
        array('(?<=[ .,!?>\]\r\n]|^)', '(?=[ .,!?<\[\r\n]|$)');

    $rgx_end_delete = '( |(?=[.,!?\[\r\n]|$))';

    // $this->dictionary is a multi-dimensional array filled with ~4000 elements.
    // An element consists of multiple attributes, (corrected_word, error_weight and others)

    foreach($this->dictionary as $word=>$prop) {

    	// this line might be problematic due to in_array being called 4000 times
    	if(stripos($text, $word) !== FALSE && !in_array($word, $exceptions)) {

    		// some initialization based on value of $prop[corrected_word] takes
    		// place here ....finally, we call a preg-replace on a match

    		$text = preg_replace("variables that contain regex goes here (i.e. $rgx_start and so on)", "corrected_word goes here", $text, -1, $count);

    		if($count) {
    			// We want to highlight the corrected words so users know what was corrected
    			// Initialize highlight with another preg_replace goes here...syntax is similar to $text
    			$highlight = preg_replace("....");
    		}
    	}
    }

    // we want to capitalize i but there are some cases where we shouldn't, such as i.e.
    // do a preg_match for some special cases
    $rgx_list = preg_match("....");
    // call a preg_replace again on $text
    $text = preg_replace("....$rgx_list, ....");

    // if replacements occurred $count is set
    if($count)
    	$highlight = preg_replace(".....");

    // now we also want to have special corrections in case the post is a Math-based post
    if(preg_match("regex patterns for math keywords such as sin, pi, theta and so on")) {
    	// do something ..jsut a variable initialization
    }

    // fix single letters such as b, n, r, u, y => be, and, are, you, why respectively
    list($rgx1, $rgx_start, $rgx_end) = array('regex pattern1', 'regex pattern2', 'regex pattern3');

    // this foreach loop should only run 5 times...the length of the array holding the "b, n, r, u, y"
    foreach ($letters as $letter=>$fix) {
    	// There are two if statements which does two preg_replace each...the 2nd preg_replace will only happen if the 1st preg_replace found replacements (i.e. $count is set)

    }

    ....
    // There are a total of 8 more preg_replace calls total and 2 preg_matches
    .....
    return array(array($text, $highlight), $error_score);
}

[Simon Lloyd]

It would be prudent for you to check your server logs and see if you are using a lot of memory or cpu time when this runs, it doesn't matter that it's "running in the background" you still need the resources for the task to take place.

[schan]

Thank you for the response. I'll keep that in mind while I continue to troubleshoot.

-S