vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

y2ksw 02-04-2011 11:31 AM

Quote:

Originally Posted by Vodkaholic (Post 2158225)
Yea i did manually a few times the other day still did'nt cache them tho hmm

Please turn logging of that task on, and check the logs. If an image has been imported, you can check it clicking the link in the logs ... those images should be then stored locally.

After checking, disable the logging because it may flood your logs with a tremedous number of entries, depending on the number of pasted images ... ;)

And of course, tasks are not running until forums are on.

BirdOPrey5 02-04-2011 01:16 PM

Can you explain more about the "cache" option in the new Beta?

y2ksw 02-04-2011 06:13 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2158688)
Can you explain more about the "cache" option in the new Beta?

Not much more than it says in the options. If the cache option is on (default), all image references are loaded into memory by a single query (and N fetches), if it is off, every image is individually searched in the database. Thus, if you have a large forum and a huge amount of images, PHP may run out of memory and the mod stops apparently to work ("could not allocate N bytes" error). In this case you have 2 options: empty the iei_img table, or switch caching off.

BirdOPrey5 02-04-2011 07:28 PM

Any estimate on what you'd consider a "huge amount" of images? 100,000? 1,000.000? 10,000,000? More than that?

y2ksw 02-06-2011 08:50 AM

Quote:

Originally Posted by BirdOPrey5 (Post 2158819)
Any estimate on what you'd consider a "huge amount" of images? 100,000? 1,000.000? 10,000,000? More than that?

That depends on the memory you are using for each PHP process and the length of the URL's. When the plugin stops working from one day to the other, you probably have reached this limit.

On one site we have 120.000+ images and it continues to work, thus maybe that's not huge. But we also give unlimited memory to the PHP processes and the server can hold up to 16 GB. Maybe a million is huge, in this context :)

BirdOPrey5 02-06-2011 12:10 PM

Thanks.

BadgerDog 02-06-2011 07:04 PM

1 Attachment(s)
Went live today with 1.04 .... :up:

Crossed my fingers and opened up most of our forums to start importing 10 posts at a time from the oldest to newest .... :D

First few runs seemed quicker with this ordering (old to new) than the testing we did with (new to old)... also turned OFF logging after first few runs ...

Let's see how long it takes to get through our site to the latest ...

y2ksw and BirdOPrey5 ... thanks for all the help from both of you ... :)

Regards,
Doug

Update: Ok, I started the live execution yesterday at about 4:00 pm (same time as this original post) and checked the first few logs. Everything seemed to be working according to the logs and the earliest threads seemed to be processed. Out of curiosity to see where it was at this morning, I turned LOGGING back ON and waited until cycle for cron had past. I checked logs and there wasn't anything there at all. So, I manually ran the process and watched it (see pic attached). It's been sitting like that for almost 15 minutes, just supposedly processing 10 posts, so I suspect it's hung and I have no idea how long it hasn't been running since I started it at 4:00 pm yesterday. I also don't know if there's been any database damage or not due to the hanging, but the vBulletin error logs don't show any SQL errors and the board seems to be running fine.

I assume I should quite the processing on the screen attached and do what next? :confused:

Update to update: In checking actually file storage on the server under /imported, it appears that the last pics imported happened at about 5:05 pm, so it appears that the import mod ran just over an hour after I had originally started it before it stopped working .... and hasn't worked since.

Another update: I have tried turning OFF the cache and that didn't make any difference.

Another update: For info purposes, we are running on a dedicated private host and Safe Mode is OFF, so that shouldn't be a time out problem. y2ksw suspects the job is timing out, so I've asked him for some debug code to be built into a test version of the cron job for manual run testing, so we can see where this is failing and perhaps why.

PhilG 02-11-2011 12:05 PM

I have just tested this and it's got a serious bug.

I have set "Forum URL replacement" to http://static.domain.com and alot of images get replaced with just that domain. For example there was an image "http://www.angelfire.com/oz/gbeze/images/circuit.gif" and it got replaced with "http://static.domain.com/

Apart from that great mod.

y2ksw 02-11-2011 12:25 PM

Quote:

Originally Posted by PhilG (Post 2161337)
I have just tested this and it's got a serious bug.

I have set "Forum URL replacement" to http://static.domain.com and alot of images get replaced with just that domain. For example there was an image "http://www.angelfire.com/oz/gbeze/images/circuit.gif" and it got replaced with "http://static.domain.com/

Apart from that great mod.

Is your forum path starting with http://static.domain.com ? In other words, is your forum contained in the forum replacement or vice versa?

Could you please attach a setting with which I may reproduce this behaviour?

Thanks

sarangan 02-11-2011 06:51 PM

It would be better if there was an option to exclude forums instead of include forums. :-) Looking forward to have that option also in the feature version. A super and nice plugin. I love it.. Thanks a lot friend.. :-)


All times are GMT. The time now is 11:49 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01627 seconds
  • Memory Usage 1,747KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (4)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete