vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

y2ksw 10-11-2017 08:26 AM

Quote:

Originally Posted by VCCN (Post 2590387)
Update: seems like PB has blocked this now. Alot of PB-pictures is not changed according to log, when i check the actual post, and add ~ behind the .jpg the picture works fine :-(

Edit: checked the table and many PB-pictures is downloaded, and some is not. hmm

If some images do not download, it may be that some PB accounts are simply moved elsewhere.

VCCN 10-11-2017 08:40 AM

Quote:

Originally Posted by y2ksw (Post 2590392)
If some images do not download, it may be that some PB accounts are simply moved elsewhere.

Don't think so.
If i check a post that is "Not changed" according to log - i find the "you must upgrade your PB" picture. If i copy the URL and open it up in a browser, picture is showing up.
Not sure why, maybe because of the time out?
Anyway, how can i rerun this plugin on the whole forum? Let's say the plugin has scanned and downloaded all pictures and i want to start over - how do i do it?

y2ksw 10-12-2017 07:05 AM

Quote:

Originally Posted by VCCN (Post 2590393)
Don't think so.
If i check a post that is "Not changed" according to log - i find the "you must upgrade your PB" picture. If i copy the URL and open it up in a browser, picture is showing up.
Not sure why, maybe because of the time out?
Anyway, how can i rerun this plugin on the whole forum? Let's say the plugin has scanned and downloaded all pictures and i want to start over - how do i do it?

The point is, in any case, a server is not a browser. When simple attempts to fake a browser fail, then you have two options: you develop a software which handles that specific case and repair it every time something changes, or you say: impossible.

I do not have time to watch PB developers, so to me it's the second option :)

BGObsession 10-23-2017 02:53 PM

Quote:

Originally Posted by The_Hawk (Post 2588474)
So I've had a few messages back and forth with y2ksw, which has been brilliant.

The latest advice was the PHP might be silently crashing and wanted more memory. I've been slowly upping the max_memory (as I didn't want to go unlimited just yet) and it's running again. I'm somewhere north of 40,000 images checked so far (excluding photobucket) and I'm currently running at only about 40% of links being found. To be fair this is
2005 - 2011 so it's less of a surprise but really does ram home the need to run a script like this one if possible.

I believe I am experiencing this as well. I installed this mod in order to overcome the 'mixed content' warnings and so far it is not working for me. I see a few images have been added to the 'imported' image folder, but just a few, and I also get the 'Done' (and only 'Done') message when I manually run the cron.job.

When you say you are 'upping the max_memory' (or to set it to 'unlimited') - where are those adjustments done - is that within the vB admin panel somewhere, or are you talking about a server setting. How can I adjust this in order to get the cron job to completely run. And I'm assuming if this is working 100% correctly, that I should see the mixed content warnings disappear immediately - is that right?

y2ksw 10-24-2017 09:03 AM

Quote:

Originally Posted by BGObsession (Post 2590653)
I believe I am experiencing this as well. I installed this mod in order to overcome the 'mixed content' warnings and so far it is not working for me. I see a few images have been added to the 'imported' image folder, but just a few, and I also get the 'Done' (and only 'Done') message when I manually run the cron.job.

When you say you are 'upping the max_memory' (or to set it to 'unlimited') - where are those adjustments done - is that within the vB admin panel somewhere, or are you talking about a server setting. How can I adjust this in order to get the cron job to completely run. And I'm assuming if this is working 100% correctly, that I should see the mixed content warnings disappear immediately - is that right?

As far as I know the only place to lay hands on, is the php.ini file and physical memory.
Code:

# cat /etc/php5/apache2/php.ini | grep memory_limit

Emad ELsayed 10-25-2017 01:24 PM

my dear
Does not work on 4.2.5
php 5.6.31
mysql 5.6.37
any help ..
Best Regards

y2ksw 10-26-2017 06:20 AM

Quote:

Originally Posted by Emad ELsayed (Post 2590746)
my dear
Does not work on 4.2.5
php 5.6.31
mysql 5.6.37
any help ..
Best Regards

for me it works

Afrika 11-14-2017 08:55 AM

Is it possible to import images from a specific path from the own forum? The own forum is not imported by default

y2ksw 11-15-2017 03:45 PM

Quote:

Originally Posted by Afrika (Post 2591089)
Is it possible to import images from a specific path from the own forum? The own forum is not imported by default

Not without changing the code base.

But in reality, you may move the files directly and then change their path with a simple query in all of your posts. This is much more effective than importing and storing the images elsewhere.

Code:

UPDATE post SET pagetext=REPLACE(pagetext, 'old_url', 'new_url');
where old_url is the path as found in one in you posts now, and new_url the path as you wish to have as the new destination.

Keep in mind that the db connection may time out, but you can repeat as often as needed, since UPDATE is smart enough not to change the records which already have been changed. Best of course is to call mysql from the shell.

gerryvz 11-17-2017 02:33 PM

1 Attachment(s)
Hi Giovanni,

Great mod, and thank you so much for creating it. I am running 4.2.4 and installed the mod yesterday. I have around 7,300 users and 150,000 total posts, with a relatively small number of overall posts that would require images to be downloaded (I would say the majority of images on my forum are directly attached and residing already in the file system).

Server - Linux + Apache
vbulletin - 4.2.4
php - 5.6.31
php max post size - 64MB
php max upload size - 64MB
php memory limit - 256MB
mySQL - 5.5.7
mySQL packet size - 10MB

The mod started downloading and it got through about 4 of the 9 years of posts on my forum. Now it has frozen since last night. I turned on the logging this morning and only came across this single entry in the log since I turned it on, which is attached. The offending post has some images attached to it, using the external host home.btconnect.com

The images seem to be small in size and pixels, and I downloaded them manually and attached them to the post to see if I could have the mod continue downloading other posts. So far, I haven't been able to get it started again.

Attached is the error, as well as my settings screens.

Also, I have now received two times the mySQL error as attached, which I read appears to be the database server "timing out."

The mod is definitely enabled and running, download folder and sub-folders are 777 permissions, etc.

Does it look to you that there is an image that is too large, or do you have other ideas as to what is causing the mod to stop indexing and downloading?

EDIT: A new post that was made by one of my users today was indexed and the external images were replaced, so I know the code is working. However, it seems stalled and won't continue indexing beyond a certain point in the past. It must be hanging up on something that is keeping it from continuing past indexing. Please advise :)

SECOND EDIT: I emptied the table in the database (it has a few more than 3,000 records in it) and the code executed again, but then stopped, and it had seven new records in the database. I emptied the table again and it sent through one more cycle of downloading a few more images, and putting seven new records in the database again.

So, something related to the database and/or php memory seems to be preventing the code from executing correctly.

Any thoughts?

Thanks
Gerry







All times are GMT. The time now is 01:08 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01831 seconds
  • Memory Usage 1,757KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (7)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (2)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete