Administrative and Maintenance Tools - Import External Images

This vBulletin 4 product imports external images in your posts from external servers or paths into a specific path, normally your forum's image folder.

Differently than the previous product Image Cache for vBulletin 3.x., it parses the posts at regular intervals for new images and replaces the external references in the background. Also, instead of using only a single folder, it creates folders for years and months, making it much easier to handle large quantities of images. The source and destination image URL's are stored into database, too, avoiding unnecessary duplicates.

It moves and optionally converts images from one place to another. It is a handy tool in order to keep your images local to your forums.

Version 1.0.4 is temporarily in BETA stage until the users have confirmed the newly requested features.
Version 1.0.5 is still in BETA stage. It adds a new bunch of additional features, one of which allows you to test your mathematical understanding ... and a few other to fine-tune CURL and socket timeouts, and resizing on the fly of too large images. If you don't know what these options do, please leave them at their default values.

[y2ksw]

Originally Posted by tommythejoat

You did notice that I said 558 copies of the same image in a total of 371 posts.

Yes it would be perfectly reasonable to see several instances of an image. It can happen when a post is quoted for example or a person might even refer to the same image in two or three different posts. When it gets to the hundreds it cannot be due to random selection by users, it can only be a bug.

If one has left the replacement image value blank as I have done, what should we expect to see? We have lots of entries in the replacement image of http://mysite/ with no file reference in the URL. I have been reading those as meaning the image was missing from the source site.

If a post is copied, it would count as a different post.

Before I step to say it's a bug, let us see what actually happens. How do you count the occurrences of a single image?

If the replacement image is empty and you have a replacement enabled, probably you will end up with a home URL only. I haven't tried that yet

[tommythejoat]

Originally Posted by y2ksw

If a post is copied, it would count as a different post.

Before I step to say it's a bug, let us see what actually happens. How do you count the occurrences of a single image?

If the replacement image is empty and you have a replacement enabled, probably you will end up with a home URL only. I haven't tried that yet

I am examining the iei_img table and count the occurrences of oldurl grouped by newurl. I included the max and min oldurl to demonstrate the fact that they are different. In fact if all are displayed, they are generally all different. It is as though the background batch process were not advancing the newurl calculation. Perhaps it would work if I were keeping the oldurl file names.

Here is the result for the first few rows of the query. Note the url references are all public and do not need to be obscured.

[smirkley]

Originally Posted by y2ksw

It is the first post with an img tag which has not the iei_parsed flag set to 1.

SELECT * FROM post WHERE iei_parsed=0 AND pagetext LIKE '%[img%' ORDER BY postid LIMIT 1

Thank you, that gave me the clue I needed.

I found the post, but what I found may be indicating a bug.

My instance of where it hung:

Put an [img ] "http://yourpic.com" [/img ] in your sig. good luck!

The above quote is the actual text in the post where it hung, and in this case it was a poster some years back giving instruction on how to use the IMG tag in the signature.

Please note the poster put an added space in the IMG open and close tag to keep it from parsing, in their example of instruction.

It appears that this mod still looks at the text "[img ]" as "[img]" instead and attempts to do a replacement on this even though it is not an actual IMG tag to be replaced.


edit to add: I cleared this problem, and it immediatly hung on the next instance of where "[img] was used in a post as text, but without the closing tag. Again, another instance of a poster giving instruction.

[tommythejoat]

To better understand what might be going on, I have been reading the iei_cron.php file.

My php reading ability is not really up to production level, but I can sort of make things out. I noticed that you were using php image functions and we have ImageMagick installed on our site and selected in the vBulletin options.

I was wondering if that might create a context that had not really been tested with this mod.

Another source of stress may be that we have been adding content to the site since 2001 and have leftover fragments from ubb, infopop and bbv2 implementations. We moved to vBulletin with 3.5 in 2008 and imported all the content we could. There are lots of old broken references to previous servers in our posts.

[smirkley]

y2ksw I was just wondering if you had a chance to checkout my last post above to see if it is a bug or something.
I know you may be busy and I can wait. Just was wondering.

tnx

[y2ksw]

Originally Posted by tommythejoat

To better understand what might be going on, I have been reading the iei_cron.php file.

My php reading ability is not really up to production level, but I can sort of make things out. I noticed that you were using php image functions and we have ImageMagick installed on our site and selected in the vBulletin options.

I was wondering if that might create a context that had not really been tested with this mod.

Another source of stress may be that we have been adding content to the site since 2001 and have leftover fragments from ubb, infopop and bbv2 implementations. We moved to vBulletin with 3.5 in 2008 and imported all the content we could. There are lots of old broken references to previous servers in our posts.

This is almost at 100% one of the problems. The image importer relies on GD only. Do you have also GD installed?

[tommythejoat]

I don't think there is a security issue in showing my config.

'./configure' '--disable-fileinfo' '--disable-pdo' '--enable-bcmath' '--enable-calendar' '--enable-exif' '--enable-ftp' '--enable-gd-native-ttf' '--enable-libxml' '--enable-magic-quotes' '--enable-mbstring' '--enable-soap' '--enable-sockets' '--prefix=/usr/local' '--with-apxs2=/usr/local/apache/bin/apxs' '--with-bz2' '--with-config-file-path=/usr/local/lib' '--with-config-file-scan-dir=/usr/local/lib/php.ini.d' '--with-curl=/opt/curlssl/' '--with-curlwrappers' '--with-freetype-dir=/usr' '--with-gd' '--with-imap=/opt/php_with_imap_client/' '--with-imap-ssl=/usr' '--with-jpeg-dir=/usr' '--with-kerberos' '--with-libdir=lib64' '--with-libexpat-dir=/usr' '--with-libxml-dir=/opt/xml2' '--with-libxml-dir=/opt/xml2/' '--with-mcrypt=/opt/libmcrypt/' '--with-mysql=/usr' '--with-mysql-sock=/var/lib/mysql/mysql.sock' '--with-mysqli=/usr/bin/mysql_config' '--with-openssl=/usr' '--with-openssl-dir=/usr' '--with-pcre-regex=/opt/pcre' '--with-pic' '--with-png-dir=/usr' '--with-tidy=/opt/tidy/' '--with-xmlrpc' '--with-xpm-dir=/usr' '--with-xsl=/opt/xslt/' '--with-zlib' '--with-zlib-dir=/usr'

Here are the GD details

gd
GD Support enabled
GD Version bundled (2.0.34 compatible)
FreeType Support enabled
FreeType Linkage with freetype
FreeType Version 2.3.11
GIF Read Support enabled
GIF Create Support enabled
JPEG Support enabled
libJPEG Version 6b
PNG Support enabled
libPNG Version 1.2.49
WBMP Support enabled
XPM Support enabled
XBM Support enabled

Directive Local Value Master Value
gd.jpeg_ignore_warning 0 0

[y2ksw]

I don'r see a problem with it, everything is in place.

[smirkley]

Yes just verified the 10 instances on my board that the mod cant recognise the difference between valid and invalid image tags in posts.

invalid meaning - [img] or [ img] used in conversation inside the post that is either not with a closing tag, or is standalone or has a space added inside the tag to keep it from parsing.

I am editing the instances where they occur on my board just to make it work.

[y2ksw]

Originally Posted by smirkley

Yes just verified the 10 instances on my board that the mod cant recognise the difference between valid and invalid image tags in posts.

invalid meaning - [img] or [ img] used in conversation inside the post that is either not with a closing tag, or is standalone or has a space added inside the tag to keep it from parsing.

I am editing the instances where they occur on my board just to make it work.

Right. These tags will not be recognized ever. They would not load an image either. There is no way to tell if they belong to an image or not.