vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

fxdigi-cash 07-23-2013 07:36 PM

look like there is some bug on vb.4.2.1. when the image imported locally from any post, a jpg file is saved where I indicated to save any image. However, the images have small sizes 1 Kb and when I preview them, they are all blank!!!

it was working very well on vb.4.2 pl3 , but now have this weird issue...

any idea how to get this resolved??

y2ksw 07-24-2013 08:40 AM

Quote:

Originally Posted by fxdigi-cash (Post 2434996)
look like there is some bug on vb.4.2.1. when the image imported locally from any post, a jpg file is saved where I indicated to save any image. However, the images have small sizes 1 Kb and when I preview them, they are all blank!!!

it was working very well on vb.4.2 pl3 , but now have this weird issue...

any idea how to get this resolved??

Please check the options. Most probably they got changed.

If this is not the problem, check the file permissions.

And, finally check that you actually can import images to your server. If you have moved to another place or the hosting provider has incidentally changed the server settings (yes, they do it always when you do some stuff, too ;)) it may happen you can't import images any more, and if the option says to replace invalid images, they are all small and "empty".

I have this plugin running on large forums and it is running flawlessly everywhere.

fxdigi-cash 07-24-2013 02:52 PM

Quote:

Originally Posted by y2ksw (Post 2435082)
Please check the options. Most probably they got changed.

If this is not the problem, check the file permissions.

And, finally check that you actually can import images to your server. If you have moved to another place or the hosting provider has incidentally changed the server settings (yes, they do it always when you do some stuff, too ;)) it may happen you can't import images any more, and if the option says to replace invalid images, they are all small and "empty".

I have this plugin running on large forums and it is running flawlessly everywhere.

Thanks for the reply and feedback.

I'm using this version: 1.0.5.iei4

I will check the following and see what's going on:

1- mod options.
2- file permissions.
3- and whether I can import images to my server or not.

also I will check with my web host to see if there is any issue regarding importing images to the server or not.

Thanks, And be back to you with the feedback soon.

:)

Update:
1- mod options."checked"
2- file permissions."checked"
3- and whether I can import images to my server or not. "checked"

everything seems to be fine now. images can be imported, but still some images show 1kb size which means not imported. possibly the rss feed I get images from doesn't allow image import.

I guess I'm fine with this if there is not other way around this. Thanks a lot

behcet 08-05-2013 10:16 AM

pc copy/paste import pls :D

y2ksw 08-05-2013 12:02 PM

Quote:

Originally Posted by behcet (Post 2437230)
pc copy/paste import pls :D

This is impossible unless your PC is mapped world-visible. For this feature you'll probably need a pretty complex javascript.

m@hamad 08-08-2013 03:18 PM

This hack is great! Thanks

This hack works on my friend site But it does not work for me .

Both sites are on a server .

tommythejoat 08-11-2013 03:44 AM

I am using this and it is working great, but I would like the imported images to be available to use like my attachment library. I can search the post table for "pagetext like replacement url," but that is very database intensive. It would be nice if you stored the postid along with the other data in iei_img.

If you do not want to be bothered with that, I might take a shot at doing it myself. I understand this is not as simple as it sounds because of the synchronization issues with the batch job out gathering via curl.

Bob_R 08-20-2013 12:14 AM

Can I change Always Convert Images from Yes to No after installation (installed over a year ago).

The animated gifs are saved to jpg and animation is lost.

Bob_R 08-20-2013 12:23 AM

Quote:

Originally Posted by BirdOPrey5 (Post 2306386)
Mine keeps gifs as gifs... always has.

How so?

I used to love this mod. Now, with what I've discovered (post above this one) I'm not so sure.

Waiting on word if I can change settings.

y2ksw 08-20-2013 08:36 AM

Quote:

Originally Posted by Bob_R (Post 2439862)
Can I change Always Convert Images from Yes to No after installation (installed over a year ago).

The animated gifs are saved to jpg and animation is lost.

Yes, any time.

However, converting images has several benefits, one of which is the removal of eventual malware. Since some file formats allow to hide scripts and thus run arbitrary code on the client browsers before viewing an image, the file conversion makes sure that such scripts are never executed, because such images are entirely removed.

Leaving the file format intact, you eventually do copy also malicious code, depending on how those images are crafted.

If you operate on a large site with a lot of users, you run better off with converting all images, because this way your site never falls into the category where it gets banned by spiders, namingly Google.

gsmlover4u 08-27-2013 11:59 AM

i installed when i run it show me this database error

Database error in vBulletin 4.2.1:

Invalid SQL:
INSERT IGNORE INTO vb_iei_img (
oldurl,
newurl
) VALUES (
'http://img257.imageshack.us/img257/9922/3209vx4.jpg',
'http://www.******.com/clear.gif'
);

MySQL Error : MySQL server has gone away
Error Number : 2006
Request Date : Tuesday, August 27th 2013 @ 07:42:40 AM
Error Date : Tuesday, August 27th 2013 @ 07:43:40 AM
Script : http://www.*****.com/admincp/cronadm...ron&cronid=122
Referrer : http://www.****.com/admincp/cronadmin.php?do=modify
IP Address : ******.****
Username : jim
Classname : vB_Database
MySQL Version :

y2ksw 08-28-2013 09:10 AM

Translated it says, the database server did not make it in time to reply to the web request. You probably have a hosting situation in which the database timeout is too low compared to the timeout of the web server.

gsmlover4u 08-29-2013 04:38 AM

Quote:

Originally Posted by y2ksw (Post 2441647)
Translated it says, the database server did not make it in time to reply to the web request. You probably have a hosting situation in which the database timeout is too low compared to the timeout of the web server.

sir i am on shared server it mean this hack not working on shared server ?

y2ksw 08-30-2013 06:40 AM

It works perfectly well in shared hosting, but may not work on time critical or overloaded hosting.

gsmlover4u 08-30-2013 08:41 AM

sir it import images in images folder but after import 8 folder its giving this database error
but before it was working fine
is there problem in link ? cause image is removed from link

gsmlover4u 08-31-2013 03:25 PM

1 Attachment(s)
when i remove this link http://img257.imageshack.us/img257/9922/3209vx4.jpg from post hack is working fine
but why not hack convert link to clear.gif ?

imageshake server giving error see screen short

gsmlover4u 08-31-2013 07:15 PM

now database error on this link and image is removed from this link
http://img523.imageshack.us/img523/8...06efrcoea5.jpg
and hack is not working when image removed from link

y2ksw 09-01-2013 05:08 AM

Quote:

Originally Posted by gsmlover4u (Post 2441959)
sir it import images in images folder but after import 8 folder its giving this database error
but before it was working fine
is there problem in link ? cause image is removed from link

If the server dies while importing images, it should have imported already the image and saving the new Url should be really the last step. The only sotuation where I may immagine problem with this, is where a database server dies while saving the data, and this happens typically with overloaded shared hosting.

If it was working fine before, please check with your hosting the changes which were made in the mean time to the system, nothing happens for no reason, never.

And, if an image is removed in the instance where an image is downloaded and saved to your server, it may become incomplete. GIF images would show then cut in half, I don't know about others.

And, in any case all of these problems have nothing to do with this plugin. these are server issues, some of which need good systemists to fix (db issues).

y2ksw 09-01-2013 05:09 AM

Quote:

Originally Posted by gsmlover4u (Post 2442169)
when i remove this link http://img257.imageshack.us/img257/9922/3209vx4.jpg from post hack is working fine
but why not hack convert link to clear.gif ?

imageshake server giving error see screen short

There is nothing I can do about timeouts.

gsmlover4u 09-01-2013 04:29 PM

Quote:

Originally Posted by y2ksw (Post 2442273)
If the server dies while importing images, it should have imported already the image and saving the new Url should be really the last step. The only sotuation where I may immagine problem with this, is where a database server dies while saving the data, and this happens typically with overloaded shared hosting.

If it was working fine before, please check with your hosting the changes which were made in the mean time to the system, nothing happens for no reason, never.

And, if an image is removed in the instance where an image is downloaded and saved to your server, it may become incomplete. GIF images would show then cut in half, I don't know about others.

And, in any case all of these problems have nothing to do with this plugin. these are server issues, some of which need good systemists to fix (db issues).

sir my server tech reply to of your this post
i am on urljet
Quote:

Hi this is a 16core server and the load averages show
Load Averages: 5.56 4.86 4.54

The server is definitely not overloaded.
There have been no changes to our server configuration

where the developer says
"If the server dies while importing images, it should have imported already the image and saving the new Url should be really the last step."
The problem seems to me that since the link is not functional the image is never imported and its imposable to save any url.

TheLastSuperman 09-01-2013 04:41 PM

Question though... what if its a 504 Gateway timeout, not from your host but for example if they are trying to link to an image that does not exist on imageshack or similar... does the clear.gif still replace properly then? Or does the gateway timeout hinder that from happening and produce these database errors?

Basically it's not his server ultimately timing out... it is but it isn't if that makes sense, due to the fact its trying to get an image from a server where the image does not exist then timing out.

gsmlover4u 09-02-2013 05:58 PM

Quote:

Originally Posted by TheLastSuperman (Post 2442395)
Question though... what if its a 504 Gateway timeout, not from your host but for example if they are trying to link to an image that does not exist on imageshack or similar... does the clear.gif still replace properly then? Or does the gateway timeout hinder that from happening and produce these database errors?

Basically it's not his server ultimately timing out... it is but it isn't if that makes sense, due to the fact its trying to get an image from a server where the image does not exist then timing out.

sir you mean to stay that clear.gif not availeable on my server ?

smirkley 09-03-2013 03:23 AM

I have run this mod since, I think maybe a couple of weeks ago?

Everything has been running, I have set it pretty low to spread out the initial uploads over time.

I have noticed it hung somewhere, but it has stopped uploading, as in one image intensive thread,... it did the first image, but it hasnt done the followup posts in that thread. It should have gone through that date fairly quickly?

It has been days since the last upload to my image directory, and I know it isnt done.

Any idea of where to look at what hangs it up?

gsmlover4u 09-03-2013 11:39 AM

Quote:

Originally Posted by smirkley (Post 2442751)
I have run this mod since, I think maybe a couple of weeks ago?

Everything has been running, I have set it pretty low to spread out the initial uploads over time.

I have noticed it hung somewhere, but it has stopped uploading, as in one image intensive thread,... it did the first image, but it hasnt done the followup posts in that thread. It should have gone through that date fairly quickly?

It has been days since the last upload to my image directory, and I know it isnt done.

Any idea of where to look at what hangs it up?

run this hack manualy from scheduled task manager and see what happend

smirkley 09-03-2013 12:15 PM

Quote:

Originally Posted by gsmlover4u (Post 2442843)
run this hack manualy from scheduled task manager and see what happend

When I manually run the cron, all I get is a blank page with the title "vB4 Import External Images" at the top. Nothing else.

Nirjonadda 09-05-2013 06:16 PM

I have changed domain name of my Web site and now old domain imported image link are not working,How i can get link working and old domain link replace to new domain link?

y2ksw 09-05-2013 07:10 PM

Quote:

Originally Posted by TheLastSuperman (Post 2442395)
Question though... what if its a 504 Gateway timeout, not from your host but for example if they are trying to link to an image that does not exist on imageshack or similar... does the clear.gif still replace properly then? Or does the gateway timeout hinder that from happening and produce these database errors?

Basically it's not his server ultimately timing out... it is but it isn't if that makes sense, due to the fact its trying to get an image from a server where the image does not exist then timing out.

Yes, if the local server has not died by then. And if it has died, it would repeat querying. With CURL, things are slightly better, since it is possible to tune the timeouts in the options.

If database errors occur, the the db server has died before the web service, and this should never happen. In fact, in production sites the database timeout is one day, because then the connection will drop when the web server drops the page, and at no moment earlier.

y2ksw 09-05-2013 07:12 PM

Quote:

Originally Posted by smirkley (Post 2442852)
When I manually run the cron, all I get is a blank page with the title "vB4 Import External Images" at the top. Nothing else.

We have had this particular issue with another user of this plugin. In the end we worked out that extremely large images would drop PHP without error and warning, and it could be fixed only by adding more RAM to the single processes.

y2ksw 09-05-2013 07:16 PM

Quote:

Originally Posted by Nirjonadda (Post 2443498)
I have changed domain name of my Web site and now old domain imported image link are not working,How i can get link working and old domain link replace to new domain link?

UPDATE post SET pagetext=REPLACE(pagetext, 'old url', 'new url')

and similarily everywhere else you need to (signatures, blogs, cms)

Also, you should empty the postparsed table.

As always, please make a backup before issuing this query.

smirkley 09-05-2013 07:20 PM

Quote:

Originally Posted by y2ksw (Post 2443516)
We have had this particular issue with another user of this plugin. In the end we worked out that extremely large images would drop PHP without error and warning, and it could be fixed only by adding more RAM to the single processes.

Question, how do I do that?

And also, how can I find out which image it is that is causing this?

smirkley 09-05-2013 08:03 PM

I am attempting to find out where it stalled out at using phpmyadmin, when I noticed something strange...

The replacement url for missing images looks like this,.. (sometimes, not all instances)

h ttp://www.mysite.com//images/misc/BrokenImage.gif
(I added a space in http to prevent this url from parsing here)

edit... figured it out, my bad, I had added a leading / to the broken image image url in acp

now just to figure out how to correctly convert it without having to edit each and every field lol

tommythejoat 09-08-2013 09:15 PM

My moderators are reporting problems with this feature. It appears that images are being replaced with the wrong, seemingly random image.

I spent some time looking at the iei_img table and the post table with the following results.

We have 798,156 posts.
23,105 of the posts contain at least one IMG bbcode.
25,859 are marked iei_parsed.

In the iei_img table we have 34,257 records.
Of these 20,420 saved images are mapping unique external images.
We have 11,015 records with our web root (forum) directory as the saved entry (no jpg)
The big problem is 911 of the save images map multiple external images with as many as 204 references down to 2 references.

Other than restoring my backups from before I installed iei, I do not know how to recover the erroneous image mappings.

I do not really understand how the batches are managed while importing images. It appears that it is not working correctly.

We are running 4.2.1 with iei 1.0.5

smirkley 09-09-2013 03:09 AM

I too have reservations on a reverse-implentation plan. Is there a method of reversing ones way out of this mod in the event it becomes undesirable in the end?

But most importantly, I wish to make this mod fully implementable as the concept is important to me.

y2ksw 09-09-2013 02:39 PM

Quote:

Originally Posted by tommythejoat (Post 2444260)
My moderators are reporting problems with this feature. It appears that images are being replaced with the wrong, seemingly random image.

I spent some time looking at the iei_img table and the post table with the following results.

We have 798,156 posts.
23,105 of the posts contain at least one IMG bbcode.
25,859 are marked iei_parsed.

In the iei_img table we have 34,257 records.
Of these 20,420 saved images are mapping unique external images.
We have 11,015 records with our web root (forum) directory as the saved entry (no jpg)
The big problem is 911 of the save images map multiple external images with as many as 204 references down to 2 references.

Other than restoring my backups from before I installed iei, I do not know how to recover the erroneous image mappings.

I do not really understand how the batches are managed while importing images. It appears that it is not working correctly.

We are running 4.2.1 with iei 1.0.5

If the "missing/invalid" image replacement is disabled, the original links are maintained, when this product is unable to import an image. For this reason it is most probable that random images are still within the img tags, since those are unlikely to work, ever. For example, some sites don't allow hot linking, and thus put junk redirects on line. Then this product wants to import them and sees, it can't. If missing images are replaced, it will typically use clear.gif or something alike to replace the missing image. Else, it will leave the link intact, and it may work for a while, until this kind of images disappear from source.

There is nothing I can do about it, this is how Internet works.

You may reparse all or part of the posts setting the iei_parsed flag to 0. Some sites do allow hot linking after a while and so you may get them later. I've been using this product over many years now on large production sites and we have had a number of problems, which were all related to remote images. Some of which were WORM infected, some where hotlink protected and others just would not conform to the standards. But these are few exceptions out of thousands of imported images every month.

y2ksw 09-09-2013 02:46 PM

Quote:

Originally Posted by smirkley (Post 2444301)
I too have reservations on a reverse-implentation plan. Is there a method of reversing ones way out of this mod in the event it becomes undesirable in the end?

But most importantly, I wish to make this mod fully implementable as the concept is important to me.

Eventually you could parse the old/new urls from the iei_parsed table and attempt to replace with global queries the links. On large forums this could be however a very time and resource consuming operation. And in any case, it is the only one you could use, since posts may be copied to any place and it is impossible to tell if the contained links came from your site or anywhere else.

tommythejoat 09-09-2013 03:20 PM

I think perhaps you missed my point on the multiples. I just searched my forum posts for "/2007/01/1.jpg" which should yield at most a few copies of the same image in related posts. Instead it returns 558 hits in a total of 371 posts.

I don't see any way for this to occur without an indexing error in the software. I think it is at least worth a look at the arbitrary name generator and where it stores its counter.

y2ksw 09-09-2013 04:24 PM

Quote:

Originally Posted by tommythejoat (Post 2444417)
I think perhaps you missed my point on the multiples. I just searched my forum posts for "/2007/01/1.jpg" which should yield at most a few copies of the same image in related posts. Instead it returns 558 hits in a total of 371 posts.

I don't see any way for this to occur without an indexing error in the software. I think it is at least worth a look at the arbitrary name generator and where it stores its counter.

Double or more hits in a single post is possible, if for instance somebody has pasted an image N times. This happens often in my forums, where people copy their preferred smilie instead of asking me to add them to the stock items. Similarily, you'll find underlines, page separators, signatures and all kinds of graphical additions people just paste on the fly without even knowing it.

In the case of having 2 or more hits of an image in a single post, it would be interesting to see if there are actually two or more copies of that image. If it isn't, then maybe something strange is going on.

tommythejoat 09-09-2013 08:50 PM

You did notice that I said 558 copies of the same image in a total of 371 posts.

Yes it would be perfectly reasonable to see several instances of an image. It can happen when a post is quoted for example or a person might even refer to the same image in two or three different posts. When it gets to the hundreds it cannot be due to random selection by users, it can only be a bug.

If one has left the replacement image value blank as I have done, what should we expect to see? We have lots of entries in the replacement image of http://mysite/ with no file reference in the URL. I have been reading those as meaning the image was missing from the source site.

smirkley 09-10-2013 02:28 AM

I have fixed my first problem extra "/" in replacement url,... caused when I added incorrectly the leading slash in my replacement image.

Need help with my second problem.

Operation has hung and I cannot figure out how to identify the image that it is hung on.

y2ksw 09-10-2013 07:24 AM

Quote:

Originally Posted by smirkley (Post 2444536)
I have fixed my first problem extra "/" in replacement url,... caused when I added incorrectly the leading slash in my replacement image.

Need help with my second problem.

Operation has hung and I cannot figure out how to identify the image that it is hung on.

It is the first post with an img tag which has not the iei_parsed flag set to 1.

Code:

SELECT * FROM post WHERE iei_parsed=0 AND pagetext LIKE '%[img%' ORDER BY postid LIMIT 1


All times are GMT. The time now is 06:24 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02114 seconds
  • Memory Usage 1,871KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (21)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete