Go Back   vb.org Archive > vBulletin Modifications > vBulletin 4.x Modifications > vBulletin 4.x Add-ons
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools
Import External Images Details »»
Import External Images
Version: 1.0.5, by y2ksw y2ksw is offline
Developer Last Online: Feb 2023 Show Printable Version Email this Page

Category: Administrative and Maintenance Tools - Version: 4.x.x Rating:
Released: 11-06-2010 Last Update: 02-13-2011 Installs: 300
DB Changes Uses Plugins
Additional Files Translations  
No support by the author.

This vBulletin 4 product imports external images in your posts from external servers or paths into a specific path, normally your forum's image folder.

Differently than the previous product Image Cache for vBulletin 3.x., it parses the posts at regular intervals for new images and replaces the external references in the background. Also, instead of using only a single folder, it creates folders for years and months, making it much easier to handle large quantities of images. The source and destination image URL's are stored into database, too, avoiding unnecessary duplicates.

It moves and optionally converts images from one place to another. It is a handy tool in order to keep your images local to your forums.

Version 1.0.4 is temporarily in BETA stage until the users have confirmed the newly requested features.
Version 1.0.5 is still in BETA stage. It adds a new bunch of additional features, one of which allows you to test your mathematical understanding ... and a few other to fine-tune CURL and socket timeouts, and resizing on the fly of too large images. If you don't know what these options do, please leave them at their default values.

Download Now

File Type: zip 1.0.2.iei4.zip (80.0 KB, 477 views)
File Type: zip 1.0.4.iei4.zip (84.5 KB, 185 views)
File Type: zip 1.0.5.iei4.zip (87.5 KB, 2158 views)

Screenshots

File Type: jpg 08-11-2010-11.32.53.jpg (112.4 KB, 0 views)
File Type: jpg 02-02-2011-22.06.39.jpg (126.0 KB, 0 views)

Supporters / CoAuthors

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #542  
Old 01-07-2014, 06:51 AM
webmastersitesi webmastersitesi is offline
 
Join Date: Oct 2007
Location: Turkey
Posts: 103
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

How this system work? when will be images imported? I imported product nothing happened current images
Reply With Quote
  #543  
Old 01-07-2014, 08:39 AM
y2ksw's Avatar
y2ksw y2ksw is offline
 
Join Date: Aug 2003
Location: Italy
Posts: 1,418
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

It'll take some time to see what happens, because it will handle a number of posts each time the cronjob runs.
Reply With Quote
  #544  
Old 01-11-2014, 04:14 AM
tommythejoat's Avatar
tommythejoat tommythejoat is offline
 
Join Date: Apr 2008
Location: Boston
Posts: 155
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

We upgraded our hardware and system software and moved to 4.2.2. Since then this has stopped working.

The symptom is that the inserted image gets replaced as iei runs, but instead of the reference to the imported image we get a reference to the web root.

It has taken some time for the users to notice and complain and then it took me a while to see what they were complaining about.

The iei_img table has about 9,000 references that are mapped to web root. Perhaps that is the placeholder while they are being processed.

We also have another 45 images that map to 2014/01/1.jpg and there is not 2014 folder in the imported images. I would appreciate any insight into why the 2014 images are not coming in. Did something happen as of the first of the year?
Reply With Quote
  #545  
Old 01-12-2014, 04:18 AM
tommythejoat's Avatar
tommythejoat tommythejoat is offline
 
Join Date: Apr 2008
Location: Boston
Posts: 155
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Another possible problem is that many sites are now watermarking their images with PHP and GD dynamically and posting a URL like http://www.fhwatch.com/watermark.php?image=iPW10779a07.jpg. Will iei handle that kind of reference?

The output stream is a valid image as long as the mime type is recognized.

Reply With Quote
  #546  
Old 01-13-2014, 08:15 AM
y2ksw's Avatar
y2ksw y2ksw is offline
 
Join Date: Aug 2003
Location: Italy
Posts: 1,418
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

The links should work. Does iei create new folders on your server?
Reply With Quote
  #547  
Old 01-13-2014, 01:48 PM
tommythejoat's Avatar
tommythejoat tommythejoat is offline
 
Join Date: Apr 2008
Location: Boston
Posts: 155
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

There are folders going back to 2001 and lots of images. It stopped creating folders for January 2014. There are 65 diffferent references in iei_img to 2014/01/1.jpg so they cannot all be references to the same image. How can I find out where they were harvested from? Can I rely on them being in date order among the posts? I can search for the 2014/01/1.jpg reference and maybe they are in the same orderr as the iei_img table entries.

It is a new server, I am guessing that the cron job runs as vb itself and there might be permission issues on the parent folder /image/imports.
Reply With Quote
  #548  
Old 01-13-2014, 03:18 PM
y2ksw's Avatar
y2ksw y2ksw is offline
 
Join Date: Aug 2003
Location: Italy
Posts: 1,418
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

If it stopped creating folders in 2014 something has changed. I would suspect that the access rights of the web service can't create folders any more. Since holidays are always the moment where incisive changes take place, maybe your hosting provider has changed something in this area. Or maybe an FTP file permission slipped unintentionally through your hands

Normally, the folder containing the years subfolders should have chmod 755 or 777, depending on the server settings. If it has become for some reason 644 or similar, the directories are seen, but can't change or extend.
Reply With Quote
  #549  
Old 01-13-2014, 09:43 PM
tommythejoat's Avatar
tommythejoat tommythejoat is offline
 
Join Date: Apr 2008
Location: Boston
Posts: 155
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

All of the folders have 755 setting. We own the system at a colocation center. They do not maintain the software. We are running WHM/CPanel on Centos.

I think I am missing something about the philosophy of missing images. If the cron job cannot import an image, what does it put in the iei_img table?

It appears to check for an existing file and then increment the name.

The other thing I noticed is that we are getting image urls of the form http://mysite.net/watermark.php?file_identifier.php. Can your code fetch a file of that kind?
Reply With Quote
  #550  
Old 01-14-2014, 09:41 AM
y2ksw's Avatar
y2ksw y2ksw is offline
 
Join Date: Aug 2003
Location: Italy
Posts: 1,418
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

What it puts in place of missing images depends on the settings in the options. It may replace the image with a local image (e.g. clear.gif), or keep the link as-is (ignore).

It does check for the name and increments if already present in order to create a new set of images each month.

All links are imported. How they translate to your system however depend on the settings, and probably the *.php images have to be converted. If you decide to keep *.php "images", you will have to add a .htaccess rule for excluding php code execution for the imported files.

On my systems I have the file conversion always on (converto to JPG), because it also removes eventually present WORM's or other "crafted" images. Conversion will loose animations and transparency, but this is of much less concern. Crafted images may contain arbitrary runnable code on the client side.
Reply With Quote
  #551  
Old 01-14-2014, 01:40 PM
tommythejoat's Avatar
tommythejoat tommythejoat is offline
 
Join Date: Apr 2008
Location: Boston
Posts: 155
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

I will presume that we/I can figure out how to get images to actually upload to our server and actually fix the current problem. It may be related to problems with the image library since we have both GD and ImageMagick installed. It may have something to do with Curl since we also have PHP Soap installed.

If I delete the current repetitive local urls from iei_img and put the original url back in the posts in the order of the posts, and then set iei_parsed back to 0, will the cron job reprocess them?

I would ensure that the cron job can actually process the images and create the local files on my system first.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 11:34 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.20234 seconds
  • Memory Usage 2,358KB
  • Queries Executed 27 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (4)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (11)post_thanks_box
  • (20)post_thanks_box_bit
  • (11)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit
  • (11)post_thanks_postbit_info
  • (10)postbit
  • (5)postbit_attachment
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • fetch_musername
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • post_thanks_function_fetch_thanks_bit_start
  • post_thanks_function_show_thanks_date_start
  • post_thanks_function_show_thanks_date_end
  • post_thanks_function_fetch_thanks_bit_end
  • post_thanks_function_fetch_post_thanks_template_start
  • post_thanks_function_fetch_post_thanks_template_end
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_attachment
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete