vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

y2ksw 09-10-2013 07:28 AM

Quote:

Originally Posted by tommythejoat (Post 2444495)
You did notice that I said 558 copies of the same image in a total of 371 posts.

Yes it would be perfectly reasonable to see several instances of an image. It can happen when a post is quoted for example or a person might even refer to the same image in two or three different posts. When it gets to the hundreds it cannot be due to random selection by users, it can only be a bug.

If one has left the replacement image value blank as I have done, what should we expect to see? We have lots of entries in the replacement image of http://mysite/ with no file reference in the URL. I have been reading those as meaning the image was missing from the source site.

If a post is copied, it would count as a different post.

Before I step to say it's a bug, let us see what actually happens. How do you count the occurrences of a single image?

If the replacement image is empty and you have a replacement enabled, probably you will end up with a home URL only. I haven't tried that yet :)

tommythejoat 09-10-2013 12:50 PM

1 Attachment(s)
Quote:

Originally Posted by y2ksw (Post 2444581)
If a post is copied, it would count as a different post.

Before I step to say it's a bug, let us see what actually happens. How do you count the occurrences of a single image?

If the replacement image is empty and you have a replacement enabled, probably you will end up with a home URL only. I haven't tried that yet :)

I am examining the iei_img table and count the occurrences of oldurl grouped by newurl. I included the max and min oldurl to demonstrate the fact that they are different. In fact if all are displayed, they are generally all different. It is as though the background batch process were not advancing the newurl calculation. Perhaps it would work if I were keeping the oldurl file names.

Here is the result for the first few rows of the query. Note the url references are all public and do not need to be obscured.

smirkley 09-10-2013 03:59 PM

Quote:

Originally Posted by y2ksw (Post 2444580)
It is the first post with an img tag which has not the iei_parsed flag set to 1.

Code:

SELECT * FROM post WHERE iei_parsed=0 AND pagetext LIKE '%[img%' ORDER BY postid LIMIT 1

Thank you, that gave me the clue I needed.

I found the post, but what I found may be indicating a bug.

My instance of where it hung:

Quote:

Put an [img ] "http://yourpic.com" [/img ] in your sig. good luck!
The above quote is the actual text in the post where it hung, and in this case it was a poster some years back giving instruction on how to use the IMG tag in the signature.

Please note the poster put an added space in the IMG open and close tag to keep it from parsing, in their example of instruction.

It appears that this mod still looks at the text "[img ]" as "[img]" instead and attempts to do a replacement on this even though it is not an actual IMG tag to be replaced.


edit to add: I cleared this problem, and it immediatly hung on the next instance of where "[img] was used in a post as text, but without the closing tag. Again, another instance of a poster giving instruction.

tommythejoat 09-10-2013 07:33 PM

To better understand what might be going on, I have been reading the iei_cron.php file.

My php reading ability is not really up to production level, but I can sort of make things out. I noticed that you were using php image functions and we have ImageMagick installed on our site and selected in the vBulletin options.

I was wondering if that might create a context that had not really been tested with this mod.

Another source of stress may be that we have been adding content to the site since 2001 and have leftover fragments from ubb, infopop and bbv2 implementations. We moved to vBulletin with 3.5 in 2008 and imported all the content we could. There are lots of old broken references to previous servers in our posts.

smirkley 09-13-2013 05:08 PM

y2ksw I was just wondering if you had a chance to checkout my last post above to see if it is a bug or something.
I know you may be busy and I can wait. Just was wondering.

tnx

y2ksw 09-13-2013 07:37 PM

Quote:

Originally Posted by tommythejoat (Post 2444703)
To better understand what might be going on, I have been reading the iei_cron.php file.

My php reading ability is not really up to production level, but I can sort of make things out. I noticed that you were using php image functions and we have ImageMagick installed on our site and selected in the vBulletin options.

I was wondering if that might create a context that had not really been tested with this mod.

Another source of stress may be that we have been adding content to the site since 2001 and have leftover fragments from ubb, infopop and bbv2 implementations. We moved to vBulletin with 3.5 in 2008 and imported all the content we could. There are lots of old broken references to previous servers in our posts.

This is almost at 100% one of the problems. The image importer relies on GD only. Do you have also GD installed?

tommythejoat 09-14-2013 02:04 AM

I don't think there is a security issue in showing my config.
Quote:

'./configure' '--disable-fileinfo' '--disable-pdo' '--enable-bcmath' '--enable-calendar' '--enable-exif' '--enable-ftp' '--enable-gd-native-ttf' '--enable-libxml' '--enable-magic-quotes' '--enable-mbstring' '--enable-soap' '--enable-sockets' '--prefix=/usr/local' '--with-apxs2=/usr/local/apache/bin/apxs' '--with-bz2' '--with-config-file-path=/usr/local/lib' '--with-config-file-scan-dir=/usr/local/lib/php.ini.d' '--with-curl=/opt/curlssl/' '--with-curlwrappers' '--with-freetype-dir=/usr' '--with-gd' '--with-imap=/opt/php_with_imap_client/' '--with-imap-ssl=/usr' '--with-jpeg-dir=/usr' '--with-kerberos' '--with-libdir=lib64' '--with-libexpat-dir=/usr' '--with-libxml-dir=/opt/xml2' '--with-libxml-dir=/opt/xml2/' '--with-mcrypt=/opt/libmcrypt/' '--with-mysql=/usr' '--with-mysql-sock=/var/lib/mysql/mysql.sock' '--with-mysqli=/usr/bin/mysql_config' '--with-openssl=/usr' '--with-openssl-dir=/usr' '--with-pcre-regex=/opt/pcre' '--with-pic' '--with-png-dir=/usr' '--with-tidy=/opt/tidy/' '--with-xmlrpc' '--with-xpm-dir=/usr' '--with-xsl=/opt/xslt/' '--with-zlib' '--with-zlib-dir=/usr'
Here are the GD details
Quote:

gd
GD Support enabled
GD Version bundled (2.0.34 compatible)
FreeType Support enabled
FreeType Linkage with freetype
FreeType Version 2.3.11
GIF Read Support enabled
GIF Create Support enabled
JPEG Support enabled
libJPEG Version 6b
PNG Support enabled
libPNG Version 1.2.49
WBMP Support enabled
XPM Support enabled
XBM Support enabled

Directive Local Value Master Value
gd.jpeg_ignore_warning 0 0

y2ksw 09-14-2013 03:22 PM

I don'r see a problem with it, everything is in place.

smirkley 09-19-2013 06:36 PM

Yes just verified the 10 instances on my board that the mod cant recognise the difference between valid and invalid image tags in posts.

invalid meaning - [img] or [ img] used in conversation inside the post that is either not with a closing tag, or is standalone or has a space added inside the tag to keep it from parsing.

I am editing the instances where they occur on my board just to make it work.

y2ksw 09-20-2013 11:01 AM

Quote:

Originally Posted by smirkley (Post 2446757)
Yes just verified the 10 instances on my board that the mod cant recognise the difference between valid and invalid image tags in posts.

invalid meaning - [img] or [ img] used in conversation inside the post that is either not with a closing tag, or is standalone or has a space added inside the tag to keep it from parsing.

I am editing the instances where they occur on my board just to make it work.

Right. These tags will not be recognized ever. They would not load an image either. There is no way to tell if they belong to an image or not.

korny 10-21-2013 10:43 PM

Not sure if this was asked before, but will this work for 3.8.7? If not, do you have one developed similar to this for 3.8.7?

y2ksw 10-23-2013 06:28 PM

Quote:

Originally Posted by korny (Post 2455190)
Not sure if this was asked before, but will this work for 3.8.7? If not, do you have one developed similar to this for 3.8.7?

https://vborg.vbsupport.ru/showthread.php?t=123338

It was actually in the description :)

korny 10-24-2013 04:14 PM

Ha ha, oops. Thanks Giovanni!

That hack isn't really the same as this one is it? This one here takes the image and stores it in the file system. The other hack you linked just caches it right?

y2ksw 11-06-2013 08:39 AM

Quote:

Originally Posted by korny (Post 2455893)
Ha ha, oops. Thanks Giovanni!

That hack isn't really the same as this one is it? This one here takes the image and stores it in the file system. The other hack you linked just caches it right?

It is not exactly the same. But it still saves the images locally to your server, even though it does not create subfolders.

Morrus 11-19-2013 12:06 AM

Wow; clever idea! I wonder if it's possible to do that with files that aren't images? PDFs, .docs, zips, etc? One thing my forum has is a lot of old dead links to files that aren't there any more.

y2ksw 11-19-2013 10:44 AM

Quote:

Originally Posted by Morrus (Post 2461964)
Wow; clever idea! I wonder if it's possible to do that with files that aren't images? PDFs, .docs, zips, etc? One thing my forum has is a lot of old dead links to files that aren't there any more.

Yes, pretty easy. It would however require another product based off from this one.

gsmlover4u 11-29-2013 09:14 AM

can we change the name of images directory my images import in import directory now can i change directory name import to anyone

y2ksw 12-01-2013 10:09 PM

Quote:

Originally Posted by gsmlover4u (Post 2464493)
can we change the name of images directory my images import in import directory now can i change directory name import to anyone

You can change the name of the image directory any time. However, you can't change the name of the image directory on the server, because all the posts before the change will continue to point to the "old" folder. So if you change the directory name in the options from oldfolder to newfolder, you will have to keep both oldfolder and newfolder on the server.

fariborz khan 12-12-2013 02:19 AM

if I change my domain name what I must do?
I like this mod
mark as installed

y2ksw 12-17-2013 01:52 PM

Quote:

Originally Posted by fariborz khan (Post 2467768)
if I change my domain name what I must do?
I like this mod
mark as installed

Essentially you would have to run a query to replace the references in your posts like:
Code:

UPDATE post SET pagetext=REPLACE(pagetext, '1st.com', '2nd.com')
or fix the issue in your .htaccess file if you still have both domains pointing to the same webspace.

webmastersitesi 01-07-2014 06:51 AM

How this system work? when will be images imported? I imported product nothing happened current images

y2ksw 01-07-2014 08:39 AM

It'll take some time to see what happens, because it will handle a number of posts each time the cronjob runs.

tommythejoat 01-11-2014 04:14 AM

We upgraded our hardware and system software and moved to 4.2.2. Since then this has stopped working.

The symptom is that the inserted image gets replaced as iei runs, but instead of the reference to the imported image we get a reference to the web root.

It has taken some time for the users to notice and complain and then it took me a while to see what they were complaining about.

The iei_img table has about 9,000 references that are mapped to web root. Perhaps that is the placeholder while they are being processed.

We also have another 45 images that map to 2014/01/1.jpg and there is not 2014 folder in the imported images. I would appreciate any insight into why the 2014 images are not coming in. Did something happen as of the first of the year?

tommythejoat 01-12-2014 04:18 AM

Another possible problem is that many sites are now watermarking their images with PHP and GD dynamically and posting a URL like http://www.fhwatch.com/watermark.php?image=iPW10779a07.jpg. Will iei handle that kind of reference?

The output stream is a valid image as long as the mime type is recognized.

http://www.fhwatch.com/watermark.php...PW10779a07.jpg

y2ksw 01-13-2014 08:15 AM

The links should work. Does iei create new folders on your server?

tommythejoat 01-13-2014 01:48 PM

There are folders going back to 2001 and lots of images. It stopped creating folders for January 2014. There are 65 diffferent references in iei_img to 2014/01/1.jpg so they cannot all be references to the same image. How can I find out where they were harvested from? Can I rely on them being in date order among the posts? I can search for the 2014/01/1.jpg reference and maybe they are in the same orderr as the iei_img table entries.

It is a new server, I am guessing that the cron job runs as vb itself and there might be permission issues on the parent folder /image/imports.

y2ksw 01-13-2014 03:18 PM

If it stopped creating folders in 2014 something has changed. I would suspect that the access rights of the web service can't create folders any more. Since holidays are always the moment where incisive changes take place, maybe your hosting provider has changed something in this area. Or maybe an FTP file permission slipped unintentionally through your hands :)

Normally, the folder containing the years subfolders should have chmod 755 or 777, depending on the server settings. If it has become for some reason 644 or similar, the directories are seen, but can't change or extend.

tommythejoat 01-13-2014 09:43 PM

All of the folders have 755 setting. We own the system at a colocation center. They do not maintain the software. We are running WHM/CPanel on Centos.

I think I am missing something about the philosophy of missing images. If the cron job cannot import an image, what does it put in the iei_img table?

It appears to check for an existing file and then increment the name.

The other thing I noticed is that we are getting image urls of the form http://mysite.net/watermark.php?file_identifier.php. Can your code fetch a file of that kind?

y2ksw 01-14-2014 09:41 AM

What it puts in place of missing images depends on the settings in the options. It may replace the image with a local image (e.g. clear.gif), or keep the link as-is (ignore).

It does check for the name and increments if already present in order to create a new set of images each month.

All links are imported. How they translate to your system however depend on the settings, and probably the *.php images have to be converted. If you decide to keep *.php "images", you will have to add a .htaccess rule for excluding php code execution for the imported files.

On my systems I have the file conversion always on (converto to JPG), because it also removes eventually present WORM's or other "crafted" images. Conversion will loose animations and transparency, but this is of much less concern. Crafted images may contain arbitrary runnable code on the client side.

tommythejoat 01-14-2014 01:40 PM

I will presume that we/I can figure out how to get images to actually upload to our server and actually fix the current problem. It may be related to problems with the image library since we have both GD and ImageMagick installed. It may have something to do with Curl since we also have PHP Soap installed.

If I delete the current repetitive local urls from iei_img and put the original url back in the posts in the order of the posts, and then set iei_parsed back to 0, will the cron job reprocess them?

I would ensure that the cron job can actually process the images and create the local files on my system first.

y2ksw 01-14-2014 02:17 PM

By setting iei_parsed to 0 the cronjob will reprocess the records.

majjed2008 01-15-2014 05:15 PM

Hello

Thank you for this products i like it .
regarding resizing it is default 0 , what is it Cm or PX or what exactly so i can use it in correct way

thank you

y2ksw 01-15-2014 05:31 PM

Quote:

Originally Posted by majjed2008 (Post 2474403)
Hello

Thank you for this products i like it .
regarding resizing it is default 0 , what is it Cm or PX or what exactly so i can use it in correct way

thank you

px.

Bob_R 01-15-2014 08:55 PM

What I hate about this mod is that you cannot hotlink images.

My server people have tried (thru cPanel) and it appears it's because of this mod.

Does anyone have any comments on this?

Thanks!

EDIT: FYI - I have vBSEO installed

y2ksw 01-16-2014 08:36 AM

Quote:

Originally Posted by Bob_R (Post 2474436)
What I hate about this mod is that you cannot hotlink images.

My server people have tried (thru cPanel) and it appears it's because of this mod.

Does anyone have any comments on this?

Thanks!

EDIT: FYI - I have vBSEO installed

In fact the idea is to move entirely hotlinked images to your server. If this isn't what you want, then it is not a mod you should use :)

Bob_R 01-16-2014 01:28 PM

Quote:

Originally Posted by y2ksw (Post 2474542)
In fact the idea is to move entirely hotlinked images to your server. If this isn't what you want, then it is not a mod you should use :)

What are you talking about?

Is that supposed to represent an answer to my question?

I realize what this mod does. I use it and have for quite some time now.

So, it's YES you cannot hotlink images if you use this mod.

RichieBoy67 01-16-2014 01:45 PM

Quote:

Originally Posted by Bob_R (Post 2474436)
What I hate about this mod is that you cannot hotlink images.

My server people have tried (thru cPanel) and it appears it's because of this mod.

Does anyone have any comments on this?

Thanks!

EDIT: FYI - I have vBSEO installed

I'm a little confused as to what you mean here. Why are you going through cpanel to add links to your images?

Have you tried editing an individual post and creating the image link there?

Bob_R 01-16-2014 01:56 PM

Quote:

Originally Posted by RichieBoy67 (Post 2474621)
I'm a little confused as to what you mean here. Why are you going through cpanel to add links to your images?

Have you tried editing an individual post and creating the image link there?

No sir not image link, hotlink.

RichieBoy67 01-16-2014 03:09 PM

I understand what a hotlink is but why do you want to import the images that are served on another server and then hot link them? it sort of defeats the whole purpose of this mod unless you mean something different then what I understand hotlinking to be.

Bob_R 01-16-2014 03:56 PM

1 Attachment(s)
Quote:

Originally Posted by RichieBoy67 (Post 2474652)
I understand what a hotlink is but why do you want to import the images that are served on another server and then hot link them? it sort of defeats the whole purpose of this mod unless you mean something different then what I understand hotlinking to be.

I thought hotlinking said it all. I apologize. Maybe it's my bad. I'll explain what I'm talking about.

Someone links to a pic on my site to display it on their site. Instead of the pic they linked to they get this (see screenshot)


All times are GMT. The time now is 11:45 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02245 seconds
  • Memory Usage 1,853KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (19)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete