vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

sticky 08-12-2014 06:26 PM

Quote:

Originally Posted by y2ksw (Post 2474136)
By setting iei_parsed to 0 the cronjob will reprocess the records.


Where do I do this? I can't find iei_parsed

The product stopped working automatically for me today but when I run the cron job manually it works just fine.

sticky 08-14-2014 06:31 PM

Quote:

Originally Posted by sticky (Post 2510988)
Where do I do this? I can't find iei_parsed

The product stopped working automatically for me today but when I run the cron job manually it works just fine.

Any idea y2ksw?

fxdigi-cash 08-14-2014 08:36 PM

does this mod work??

I tried it, but the images are still linked externally from image uploaders...

is there something I'm missing here?

sticky 08-15-2014 01:42 AM

Quote:

Originally Posted by fxdigi-cash (Post 2511315)
does this mod work??

I tried it, but the images are still linked externally from image uploaders...

is there something I'm missing here?

It does work. I've used it for years without issues.

Well, until now.

fxdigi-cash 08-15-2014 02:17 AM

Quote:

Originally Posted by sticky (Post 2511360)
It does work. I've used it for years without issues.

Well, until now.

Thanks for the feedback, but still in my case it is not working!!

I guess there is something wrong here. it is either the version I'm using (v1.0.5) or a misconfiguration !!

:confused:

y2ksw 08-16-2014 08:38 AM

Quote:

Originally Posted by sticky (Post 2511309)
Any idea y2ksw?

Did you have a look in the database? :)

y2ksw 08-16-2014 08:40 AM

Quote:

Originally Posted by fxdigi-cash (Post 2511315)
does this mod work??

I tried it, but the images are still linked externally from image uploaders...

is there something I'm missing here?

If your server cannot access external servers or if external servers do not allow downloading images, they remain as is. Also, it may take some time to catch up.

vietdotseo 08-16-2014 09:37 AM

if i edit old posts, how new images in these posts ???

I have picture like
HTML Code:

https://lh5.googleusercontent.com/-meeECZ7DiCI/U9Mx7R0O4qI/AAAAAAAAA0Q/-wpqjsW6SMA/w738-h458-no/giao-an-lop-5-theo-chuong-trinh-moi.png
It don't Import

p/s: sorry my English

sticky 08-21-2014 06:31 AM

Quote:

Originally Posted by y2ksw (Post 2511496)
Did you have a look in the database? :)

Yes, but what am I looking for?

sticky 08-21-2014 06:32 AM

Quote:

Originally Posted by y2ksw (Post 2511498)
If your server cannot access external servers or if external servers do not allow downloading images, they remain as is. Also, it may take some time to catch up.

Is there only one hook? All I see is: Add and set iei_parsed field accordingly for eventual image parsing

I don't understand why when I click the cron manually it works but it just won't run automatically while all other crons work.

It's very weird.

y2ksw 08-21-2014 10:49 AM

Quote:

Originally Posted by sticky (Post 2512196)
Is there only one hook? All I see is: Add and set iei_parsed field accordingly for eventual image parsing

I don't understand why when I click the cron manually it works but it just won't run automatically while all other crons work.

It's very weird.

In fact. But it may be the product is disabled or it is scheduled in a way it never runs often enough.

The iei_parsed field is found in the post table and if it is 1 or more, it won't parse that post again.

sticky 08-21-2014 03:32 PM

Check that table and it said 1 so i set it to 0 on the posts it showed.

It still isn't working and I can't figure out what to do :(

rustyshack 08-22-2014 02:18 AM

I have had this mod installed on my one forum for years and it worked great.
Then I moved to a new server and upgraded to 4.2.2 VB
The new server has PHP 5.4 which i am thinking is the problem.

Before i discovered it wasn't working I installed this on another forum on the same brand new server, also running 4.2.2. On this forum it created a 2 folders and then stopped.
I cannot figure out what is stopping it, it seems it can't write files or something.

So i looked on my other forum and it stopped running the same day I moved it to my new server.
So there is either a problem with 4.2.2 or php 5.4
Cannot find any reason that neither will work and this is mainly the differences.

Even if I run it manually it doesn't work.

sticky 08-22-2014 11:44 PM

What would it explain it running manually but not automatically?

Delfi_r 08-23-2014 05:10 PM

Installed on 4.0x, I'm using it on 4.22pl1; php 5.3.2-1 and MySQL 5.1.63 and runs fine.

What I would ask is a version that works on blogs and their comments, too.

AndrewSimm 11-10-2014 03:08 AM

If I uninstall this can I delete the images in the imported folder? Will the original external link be back in the post?

AndrewSimm 11-10-2014 04:42 AM

I guess I know the answers to my questions. Is there a way to reverse this since the original url is stored in the DB?

y2ksw 11-10-2014 07:51 AM

Quote:

Originally Posted by AndrewSimm (Post 2522094)
I guess I know the answers to my questions. Is there a way to reverse this since the original url is stored in the DB?

It is possible to reverse, but it will need some crafting skills with queries.

I'm still unsure if you can do it with queries only, or if it requires also some scripting. If you would like to test if the source is still available, then scripts are definitively required.

addamroy 11-16-2014 01:14 PM

I have a few questions before I go ahead and use this. (And a big thanks for taking the time to make this)

1 - Does this mod have any lossy or lossless image compression features?

Would be nice if these images were compressed in some way during the import.

2 - If I install this mod, does it AUTOMATICALLY start importing images from past posts, or do I have to initiate that import manually via the mod options?

3 - Does this mod import images AS POSTS ARE MADE, or are they imported via a cron-job throughout the day?

And how often does the cron-job run?

4 - Why are there 2 maximum post count options? I notice there are 2 screenshots, each with similar options. Are both options pages accessible via the mod, or is each one a screenshot from v4 and v5 of this mod?

5 - I'm on a semi-shared server (urljet) and have somewhere around 200,000 posts. Will this mod be too server intensive for me or do I just need to lower the maximum post count options? Also, what post count per run do you recommend?

Thanks dude, this looks awesome.

y2ksw 11-16-2014 09:18 PM

Quote:

Originally Posted by addamroy (Post 2522966)
I have a few questions before I go ahead and use this. (And a big thanks for taking the time to make this)

1 - Does this mod have any lossy or lossless image compression features?

Would be nice if these images were compressed in some way during the import.


2 - If I install this mod, does it AUTOMATICALLY start importing images from past posts, or do I have to initiate that import manually via the mod options?

3 - Does this mod import images AS POSTS ARE MADE, or are they imported via a cron-job throughout the day?

And how often does the cron-job run?

4 - Why are there 2 maximum post count options? I notice there are 2 screenshots, each with similar options. Are both options pages accessible via the mod, or is each one a screenshot from v4 and v5 of this mod?

5 - I'm on a semi-shared server (urljet) and have somewhere around 200,000 posts. Will this mod be too server intensive for me or do I just need to lower the maximum post count options? Also, what post count per run do you recommend?

Thanks dude, this looks awesome.

1. If you convert all images (option) then they become compressed with the jpg standard and also filtered against embedded scripts and other strange things.
2. Normally it starts alone.
3. cron; you can decide by changing the frequency
4. they are 2 screenshots depending on the mod version old/new. No vb5 tested yet but maybe it still works ...
5. It may be very server intensive, if you have a high parsing rate and a high number of posts at a single time. I would suggest to try this first on a small test environment. I also suggest to start with a small number of images. If you have an interval of 1 minute and 10 posts at each, it will catch up in about 15 days (1440 minutes x 10 x day). It depends a lot on how much your server can hold. On a professional dedicated server (4 core 3.4G) I have run this script with 1000 posts per minute with no noticeable problems.

addamroy 11-17-2014 01:16 AM

Thanks for the reply. Quick clarification though,

Quote:

Originally Posted by y2ksw (Post 2523048)
also filtered against embedded scripts and other strange things.

What do you mean by that?

y2ksw 11-17-2014 04:33 AM

Quote:

Originally Posted by addamroy (Post 2523065)
Thanks for the reply. Quick clarification though,



What do you mean by that?

worms ;)

addamroy 11-17-2014 11:13 AM

Quote:

Originally Posted by y2ksw (Post 2523069)
worms ;)

Good because I hate worms. :)

mservis 11-24-2014 05:13 AM

1 Attachment(s)
Hello it's me not delete the images downloaded by the script, both the images and subdirectories marked out apache user and owner.

So do not let me change anything, or delete, or change attributes.

is there any solution?

akxt660 11-29-2014 02:02 PM

Hi!

This is an excellent product, thank you for offering us.

When this product will allow the user to switch between "imported images" and return to the "original links"? This is very important as it would be possible to save the images on the server and return to the external reference mode to save bandwidth and ensure the images for the future. Also, if there are problems with the operation or server, it would be possible to return to the original mode.

It would be important to allow: when return to the "imported images" mode, they are not reprocessed the rescue of those who are already on the server, only those that have not yet been redeemed (new links only and reactivate the internal reference).

I believe that after this update , this product would be perfectly safe and can even be used natively in vbulletin system.

Thank You!

y2ksw 11-29-2014 02:28 PM

Quote:

Originally Posted by akxt660 (Post 2524763)
Hi!

This is an excellent product, thank you for offering us.

When this product will allow the user to switch between "imported images" and return to the "original links"? This is very important as it would be possible to save the images on the server and return to the external reference mode to save bandwidth and ensure the images for the future. Also, if there are problems with the operation or server, it would be possible to return to the original mode.

It would be important to allow: when return to the "imported images" mode, they are not reprocessed the rescue of those who are already on the server, only those that have not yet been redeemed (new links only and reactivate the internal reference).

I believe that after this update , this product would be perfectly safe and can even be used natively in vbulletin system.

Thank You!

I do understand what you want, but it is not intended to reverse the process of imported images.

Normally, you would use this plugin to ensure to get all the images users have simply pasted and you will know beforehand the cost of the operation: bandwidth and disk space.

On very large sites (1.000+ active users), this will be about 1 GB/month or 1000 images.

mservis 11-30-2014 11:43 PM

No solution ?

https://vborg.vbsupport.ru/showpost....&postcount=624

greetings

y2ksw 12-01-2014 03:34 AM

Quote:

Originally Posted by mservis (Post 2523920)
Hello it's me not delete the images downloaded by the script, both the images and subdirectories marked out apache user and owner.

So do not let me change anything, or delete, or change attributes.

is there any solution?

There is nothing we can do about it.

This happens if the server process has different access rights than your FTP access, and must be fixed by your provider.

adwade 01-01-2015 03:48 AM

Quote:

Originally Posted by BirdOPrey5 (Post 2148409)
I installed this on a VB 3.8 test forum and it appears to be working well. :up:

Quote:

Originally Posted by y2ksw (Post 2148860)
There should be no conflict at all, since it uses a different table and an additional field to distinguish parsed posts from new or modified ones. Just disable the 3.x one and install this ... you are ready to go. Please don't forget to enable the product once you have chmodded the imported file folder and setup the options, since it installs disabled.

I disabled my installed 3.x version (i.e. Image Cache v1.0.2.0), uploaded the new cron job, created a forum/images/imported directory, imported the product-iei4.xml (v1.0.2) file per the instructions. After toggling the Always Convert Images to Yes, I then enabled the product on vB3.8

I next edited and approved a moderated thread, after adding an external image to it and the image in the thread was subsequently processed correctly. However the MOD seems to have created some month subfolders (i.e. 03, 05, etc) that appear empty.

Q. I assume the MOD is working it's way through my oldest threads, but was just curious as to why some subfolders created have no contents?

y2ksw 01-01-2015 06:04 AM

R. In a few rare cases the procedure may not be able to terminate the import process timely, or encounter another error, and will leave then an empty folder. If you care about those, from the linux system crontab you may issue the following command once a month:
Code:

find /your/import/folder/ -type d -empty -delete
Please double-check the folder name ;)

dorans 01-24-2015 11:42 AM

regards,
How to download the original image size, not thumbnails?
With postimage.org or other ...

y2ksw 01-24-2015 12:08 PM

Quote:

Originally Posted by dorans (Post 2534304)
regards,
How to download the original image size, not thumbnails?
With postimage.org or other ...

Normally, the images are imported in their original size. If you intended to download a different, attached image to a specific URL, this is not possible.

dorans 01-24-2015 02:09 PM

Postimage via script sends the thumbnail link.
IE download icon instead of the whole picture ... Or I something wrong?

y2ksw 01-25-2015 07:16 AM

Quote:

Originally Posted by dorans (Post 2534335)
Postimage via script sends the thumbnail link.
IE download icon instead of the whole picture ... Or I something wrong?

The idea behind this plugin is to save the image of the URL the user has pasted into a post. Thus, if it is the thumbnail, the thumbnail is imported.

dorans 01-25-2015 11:59 AM

thank you,
I understand the function. The problem is for technical drawings. As thumbnails are useless.
Mod works completely correctly.

y2ksw 01-25-2015 01:33 PM

Quote:

Originally Posted by dorans (Post 2534457)
thank you,
I understand the function. The problem is for technical drawings. As thumbnails are useless.
Mod works completely correctly.

If there are reproducible rules between thumbnails and the drawings, then you may eventually replace the thumbnail URL's before running this plugin's task.

dorans 02-16-2015 10:34 AM

Me again :)

After the import I have this link:

[URLl="http://postimg.org/image/txakyobrx/"]https://vborg.vbsupport.ru/[/URL]

How can I avoid this part? [URL = "http://postimg.org/image/txakyobrx/"] and [/ URL]
thanks,

y2ksw 02-16-2015 12:03 PM

Quote:

Originally Posted by dorans (Post 2537662)
Me again :)

After the import I have this link:

[URLl="http://postimg.org/image/txakyobrx/"][IMG]mysite.com/dddd/image/2014/11/108.jpg[/IMG][/URL]

How can I avoid this part? [URL = "http://postimg.org/image/txakyobrx/"] and [/ URL]
thanks,

You can't. This plugin does handles only image tags and ignores the rest. As the URL tag is surrounding the image, it is ignored. Eventually you may craft a special query which allows removing surrounding URL's, but this is far beyond of the scope here. (See MySQL REGEXP)

dorans 02-16-2015 12:41 PM

I assumed it.
The problem is that it remains a link to postimage. Gonna remove.
thanks,

BadgerDog 03-10-2015 12:18 PM

This is a great mod and it's been very useful to our forums over the past several years.. :up:

In fact, we sent y2ksw a large donation way back when to thank him for his work... ;)

The only serious problem we've ever experienced is that nagging issue of handling a picture someone uploads that's too large for the mod to handle.

Refer to old posts 167 and 168 (and older) where we discussed it and "Bird of Prey" also helped with figuring out what could be done, to some degree.

Since there wasn't an obvious solution then, we simply got into the habit of manually running the mod every morning during maintenance routines, to make sure it completed and didn't get hung on any large pics members had uploaded.

Well, that worked fine for the last 24 months and ironically, we never had one hang, except for this morning. A member must have uploaded an 8Mb pic sometime yesterday and today when we manually ran and checked completion of the mod, it simply hung at the very start. We turned on the logging to see if we could figure out where it was hung, but there was no data there, so it must have been the very first pic it tried to import at some point.

So, we began the laborious task of going through hundreds of threads for the last day or so, trying to find one with an IMG pic that was too large for the mod to handle. After a few hours we found the 8 Megabyte sized pic, downloaded and resized it, then put it back.

We then re-ran the import mod and it worked fine, completing the import of this (now smaller) pic, as well as others that had been uploaded in the last 24 hours.

So, why am I posting?

Well, it was a real pain in the a$$ to have to manually sift through all those threads looking for a single pic that cause the import mod to hang and lock up. So, I was wondering if anyone had any ideas how we could automate a search for excessively large pics, so we could manually fix them and get y2ksw's mod going again?

I gather you can't catch large pics before downloading them into the site, so it appears we have to live with the "hanging" issue, but it would be great if we could somehow find the offending pic(s) easier when that happens.

Anyone have any ideas?

Thanks..

Regards,
Doug


All times are GMT. The time now is 06:36 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02366 seconds
  • Memory Usage 1,856KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (1)bbcode_html_printable
  • (22)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete