vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Administrative and Maintenance Tools - Import External Images (https://vborg.vbsupport.ru/showthread.php?t=253309)

y2ksw 01-14-2011 09:29 AM

Quote:

Originally Posted by kether1 (Post 2148544)
Thank you, Giovanni, for answering my question! I want to use your new version on my vb4 forum instead of the vb3 version, I just wanted to know what to do before installing the vb4 version. I did not want a conflict. This has been an excellent product for me. One of my favorite modifications I use on my forum. Thank you for sharing your work and time.

Kether

There should be no conflict at all, since it uses a different table and an additional field to distinguish parsed posts from new or modified ones. Just disable the 3.x one and install this ... you are ready to go. Please don't forget to enable the product once you have chmodded the imported file folder and setup the options, since it installs disabled.

y2ksw 01-14-2011 09:33 AM

Quote:

Originally Posted by BirdOPrey5 (Post 2148568)
Can someone who has been using this give me some idea how much disk space this uses in a real world environment? Obviously it's different for everyone but just an idea- how long you had it? how many posts you get? how much disk space is the folder taking up each month?

I have this installed on a site with about 3.000 pasted images each month, which translates in about 30 MB imported images monthly.

BirdOPrey5 01-14-2011 03:42 PM

Quote:

Originally Posted by y2ksw (Post 2148862)
I have this installed on a site with about 3.000 pasted images each month, which translates in about 30 MB imported images monthly.

Thank you.

sticky 01-14-2011 04:20 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2148409)

Ever think about an option to keep the original file name rather than renaming it to a number?

That would be incredibly useful! Or to at least append #.

sticky 01-14-2011 04:21 PM

Quote:

Originally Posted by y2ksw (Post 2148491)
Yes. But under Windows, file names cannot contain ? and other strange things you may find in an image URL, some Linux versions may not love excessively + and *, and numbers are a good solution for all imaginable file systems one can come across :)

Could you specify for it just to use the letters in the title, and to avoid those characters entirely?

john h 01-14-2011 07:57 PM

When I ran this manually in the task scheduler I get this:

Quote:

Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set in [path]/includes/cron/iei_cron.php on line 148

sticky 01-14-2011 09:07 PM

Another reason to consider allowing the files to keep their names is how much it will help with google image searches driving a lot of traffic.

y2ksw 01-15-2011 02:50 PM

Quote:

Originally Posted by sticky (Post 2149022)
Could you specify for it just to use the letters in the title, and to avoid those characters entirely?

I'm not particularily interested of forging file names in a different way. Progressive numbers appear to work for everybody without additional options and international conflicts. Please see this plugin as an attempt to avoid missing images over time, and not as an alternative attachment manager.

Also, while I was reasoning about your request, occasionally duplicate file names would be then really annoying. Then I would have to use what? Numbers again? And what about the case insensitivity of Windows?

No, really, I see your point, but I can't see an easy and always working solution, which will not require me to add a lot more of code and support just to make it possible to use the same file names... ;)

y2ksw 01-15-2011 02:55 PM

Quote:

Originally Posted by john h (Post 2149136)
When I ran this manually in the task scheduler I get this:

As a warning it will proceed. Anyways, you may comment this line out. In this case, if an image URL is redirected, it will not follow and try to get the image there, but report a missing image. You may have to add the domains with redirects to your ignore list, in order to keep the original images.

y2ksw 01-15-2011 03:07 PM

Quote:

Originally Posted by sticky (Post 2149155)
Another reason to consider allowing the files to keep their names is how much it will help with google image searches driving a lot of traffic.

An efficient SEO also requires titles and image descriptions, which are not given by pasted images. A possible solution would be to import images as real attachments in an album, where eventually somebody will add a title and a description.

In real life nobody will add titles and descriptions to automatically added images, ever! The efford to correctly insert images in the old and new attachment system is much higher than just copying a file to one place, and the never-or-once-used additional feature would be overdoing this product.

I agree, this is not a perfect addition, but it is surely much better than having missing images after a few days, weeks or months :)

BirdOPrey5 01-15-2011 08:18 PM

y2ksw,

This is indeed a most awesome mod- my request for the images to retain their original names wasn't to get Google image credit since I don't allow hot-linking of my images anyway I just assume Google didn't index my images. I was asking because I can see inevitably people link to inappropriate images that I won't want on my server- indeed am not allowed to even have on my server according to my hosting account rules- and I just believed finding a real name rather than a number might make my life easier later on when I'm having to manually delete files via FTP.

There is just 1 thing preventing me from using this and it is by no means the "fault" of this mod- i just want to ask here in case anyone might have had the same issue

I have multiple domains pointing to my site. By default this mod uses my "bburl" to hard code the image URL. However a number of my users browse via alternate domains because my primary (and sometimes secondary) domains end up getting blocked by web access filters at people's jobs. SO i have gone through great pain to make sure there are no hard coded links or images on my site at all- everything is relative so if you browse via juot.net, everything is on juot.net. If you browse by juot.co, everything is on juot.co, and so on.

If I use this mod as is it will hard code every image to juot.net and anyone logging in from a place where juot.net is blocked will get broken images.

I tried to put "/forums" in the replacment URL option but the [img] bb code doesn't work with relative paths.

Curious if you or anyone have any ideas about this?

Also, if I were to edit your cron job file to keep the existing file names would you be OK if I attached it in this thread so others who might want that feature could use it?

newmomsforum 01-15-2011 11:44 PM

Hi Y2k

Firstly great mod and is just what I needed so thanks for sharing it with us :)

Quick question though.. The import worked fine at first and I have 4 folders created for 2006, 2007, 2008 and 2009 with sub folders for each month within and the images within those

The strange thing is.. No further folders have been created beyond 2009 and the import appears to have stopped... images from external sites have still been posted since 2009 and continue to be posted on a daily basis.

No memory issue and the forums running fine .. any ideas?

Cheers

Mark

y2ksw 01-16-2011 06:46 AM

Quote:

Originally Posted by BirdOPrey5 (Post 2149672)
y2ksw,

This is indeed a most awesome mod- my request for the images to retain their original names wasn't to get Google image credit since I don't allow hot-linking of my images anyway I just assume Google didn't index my images. I was asking because I can see inevitably people link to inappropriate images that I won't want on my server- indeed am not allowed to even have on my server according to my hosting account rules- and I just believed finding a real name rather than a number might make my life easier later on when I'm having to manually delete files via FTP.

There is just 1 thing preventing me from using this and it is by no means the "fault" of this mod- i just want to ask here in case anyone might have had the same issue

I have multiple domains pointing to my site. By default this mod uses my "bburl" to hard code the image URL. However a number of my users browse via alternate domains because my primary (and sometimes secondary) domains end up getting blocked by web access filters at people's jobs. SO i have gone through great pain to make sure there are no hard coded links or images on my site at all- everything is relative so if you browse via juot.net, everything is on juot.net. If you browse by juot.co, everything is on juot.co, and so on.

If I use this mod as is it will hard code every image to juot.net and anyone logging in from a place where juot.net is blocked will get broken images.

I tried to put "/forums" in the replacment URL option but the [img] bb code doesn't work with relative paths.

Curious if you or anyone have any ideas about this?

Also, if I were to edit your cron job file to keep the existing file names would you be OK if I attached it in this thread so others who might want that feature could use it?

You may find the real name and new association in the iei_img for this scope. Also I may think about a unallowed file names option, which is simple and does not change the behaviour.

As for *juot.*, please add into your exception list the partial URL, which identifies your media server(s). For example,

http://juot.

or

.juot.

could be such a string. In this way, you avoid importing images from different domains, since they are pointing to the same physical installation.

Also, you may need to add a plugin at global_bootstrap_init_start with:
PHP Code:

$vbulletin->options['bburl'] = 'http://' $_SERVER['HTTP_HOST']; 

in order to reflect the correct URL for each domain you are coming from. Please note that this change may affect other functions, depending on how your system is setup. Also, you may set the board title, keywords and description depending on the host. You will have to test this very carefully. It opens you a beautiful new door to solve many problems related to multiple domain redirects and/or thematic separation, but I don't support this feature ;)

You may need to add/remove a rule to your hotlinking in order to ignore your local or remote media servers for your forums, too.

y2ksw 01-16-2011 06:59 AM

Quote:

Originally Posted by newmomsforum (Post 2149752)
Hi Y2k

Firstly great mod and is just what I needed so thanks for sharing it with us :)

Quick question though.. The import worked fine at first and I have 4 folders created for 2006, 2007, 2008 and 2009 with sub folders for each month within and the images within those

The strange thing is.. No further folders have been created beyond 2009 and the import appears to have stopped... images from external sites have still been posted since 2009 and continue to be posted on a daily basis.

No memory issue and the forums running fine .. any ideas?

Cheers

Mark

Supposed you have (almost) unlimited disk space, maybe the folder rules have changed? They should have modification rights for the web server, such as chmod 755 or 777, and the image files 644 or 666.

Also, you may have an enormous amount of imported images, and may need to reset your lookup table. In order to see if this is the problem, please run the task manually. If there is a memory error (... could not allocate N Bytes ...), you have 2 options: reset the lookup table:
Code:

TRUNCATE iei_img
or add some memory to your PHP processes. By resetting the lookup table, a few images may get imported twice, but it is much cheaper than adding memory space to all PHP processes :)

BirdOPrey5 01-16-2011 02:31 PM

Thank you for those suggestions... I don't know that the bburl override is needed. The problem is if I do that then someone adding an image (or really, when the cron job runs) will have the alternate bburl encoded in the post- but this is still a problem because it's a hard coded domain, it won't vary based on who's viewing it but on who was viewing it when the task ran.

I seem to have solved the "problem" by making a new bb code also called img, this seems to override the built in [img] bb code and allows me to use relative urls... so by putting "/foums" in your replacment url settings all images are correctly being given a relative url such as "/forums/images/imported/2011/..." and working well.

y2ksw 01-17-2011 03:36 PM

I didn' know it was possible to redefine BB Codes :)

vietfancy 01-17-2011 11:02 PM

hangs when editting posts

vortodox 01-17-2011 11:06 PM

Is it possible to enable this hack only on articles and blogs?

y2ksw 01-18-2011 07:43 AM

Quote:

Originally Posted by vietfancy (Post 2150719)
hangs when editting posts

Try to disable the plugins at hook location: editpost_update_process

This may be due to Unicode problems and PHP/MySQL version incompatibilities. When these plugins are disabled, edited posts are not reparsed.

y2ksw 01-18-2011 07:43 AM

Quote:

Originally Posted by vortodox (Post 2150720)
Is it possible to enable this hack only on articles and blogs?

It is only for forum posts yet.

BadgerDog 01-18-2011 11:52 AM

I've downloaded the mod and using y2ksw's "patch" recommendations in posts 13-16 to limit the testing to just a single post, in order to make sure it works on our site, I was about to take a "leap of faith" and install the XML, but I just had another thought .. :D

We use Sevenskin's Image Resizer for standard IMG files that are large. It does a great job resizing them in real time down to a more viewable (no overflow) image that can be clicked upon the show the full size image.

If we install this mod, will anything negative happen when interacting with this excellent mod and what pic will be stored internally, the resized screen pic or the full size one that is pointed to in the post's IMG code itself?

Thanks for any feedback .. :)

Regards,
Doug

y2ksw 01-18-2011 03:25 PM

Nothing special should happen ... The ImageResizer seems to replace only the image attributes, not the image itself.

BirdOPrey5 01-18-2011 07:13 PM

Damn... Apparently overriding my img bbcode kills my image resizer...

Back to the drawing board... I thinking a custom plugin that runs on postbit_display_complete that does a str_replace looking for www.juot.net and replaces it with the actual domain name they are using... that should solve my issues...

BadgerDog 01-18-2011 08:10 PM

Quote:

Originally Posted by y2ksw (Post 2150975)
Nothing special should happen ... The ImageResizer seems to replace only the image attributes, not the image itself.

Thank you Sir.. :)

I'll give it a run as soon as I have a stretch of free time to focus on not screwing it up. I did make the PHP file change that you mentioned, to just affect a single post, so I think that will mitigate any mess I make.

Regards,
Doug

ps: maybe I'll pop a Prozac before I run it anyway .. :D

bada_bing 01-18-2011 08:13 PM

Will this work with VB 3.8.x ?

BirdOPrey5 01-18-2011 09:36 PM

Quote:

Originally Posted by bada_bing (Post 2151133)
Will this work with VB 3.8.x ?

It works for me on 3.8.6.

210665 01-19-2011 10:27 AM

Hello

I have installed this product now and created the folder images / imported /
The folder I have assigned the following rights 777
Use System vBulletin 4.1.0 Patch Level 2

Unfortunately, this product does not work.

English translated by Google

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxx

Hallo

Ich habe dieses Produkt jetzt installiert und den Ordner images/imported/ angelgt.
Dem Ordner habe ich folgende Rechte zugeordnet 777
Verwende System vBulletin 4.1.0 Patch Level 2

Nur leider arbeitet dieses Produkt nicht.

Englisch mit Google übersetzt

vortodox 01-23-2011 10:22 PM

Quote:

Originally Posted by y2ksw (Post 2150836)
It is only for forum posts yet.

Thanks for answer, but is it possible to restrict ths addon just for some forums?

y2ksw 01-24-2011 09:37 AM

Quote:

Originally Posted by 210665 (Post 2151378)
Hello

I have installed this product now and created the folder images / imported /
The folder I have assigned the following rights 777
Use System vBulletin 4.1.0 Patch Level 2

Unfortunately, this product does not work.

English translated by Google

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxx

Hallo

Ich habe dieses Produkt jetzt installiert und den Ordner images/imported/ angelgt.
Dem Ordner habe ich folgende Rechte zugeordnet 777
Verwende System vBulletin 4.1.0 Patch Level 2

Nur leider arbeitet dieses Produkt nicht.

Englisch mit Google ?bersetzt

If it does not work at all, probably your server cannot use sockets, CURL or is locked out for retrieving web content by a firewall.

PS: Have you enabled the product after installing?

y2ksw 01-24-2011 09:40 AM

Quote:

Originally Posted by vortodox (Post 2153532)
Thanks for answer, but is it possible to restrict ths addon just for some forums?

You may change the task code (query) in order to limit it to certain forum sections. I believe I never will add an option for selecting forum ID's since to my opinion it does not make much sense to import only images for a few sections.

sticky 01-24-2011 09:55 AM

Quote:

Originally Posted by y2ksw (Post 2153741)
You may change the task code (query) in order to limit it to certain forum sections. I believe I never will add an option for selecting forum ID's since to my opinion it does not make much sense to import only images for a few sections.

Please add this option, it would be huge for me in particular :)

vortodox 01-24-2011 10:54 AM

Quote:

Originally Posted by y2ksw (Post 2153741)
You may change the task code (query) in order to limit it to certain forum sections. I believe I never will add an option for selecting forum ID's since to my opinion it does not make much sense to import only images for a few sections.

Actually it has allot sense, in my case there are some SPAM forumus in which members post many humoristic picts which I don't whish on my server, but on the other hand there are review with pictures which i really wish to have on my server.

Anyway, how do I change task code? /noob here :erm:


Quote:

Originally Posted by sticky (Post 2153744)
Please add this option, it would be huge for me in particular :)

Seconded.

BirdOPrey5 01-24-2011 12:20 PM

Just for the info of those who may need this feature, I made this edit for it to not import images from 2 specific forums... I am releasing this for info only, I don't recommend you make this change nor will I support it.

In ie_cron.php file I changed:

Code:

"SELECT postid, dateline, pagetext, forumid
    FROM " . TABLE_PREFIX . "post join " . TABLE_PREFIX . "thread on (post.threadid = thread.threadid
    WHERE iei_parsed=0
        AND pagetext LIKE '%[/IMG]%'
    ORDER BY postid";

to:
Code:

$SQL = "SELECT postid, " . TABLE_PREFIX . "post.dateline, pagetext, forumid
    FROM " . TABLE_PREFIX . "post join " . TABLE_PREFIX . "thread on (" . TABLE_PREFIX . "post.threadid = " . TABLE_PREFIX . "thread.threadid)
    WHERE iei_parsed = 0
        AND pagetext LIKE '%[/IMG]%' AND forumid != 2 AND forumid != 4
    ORDER BY postid";

This would import images from all forums EXCEPT forum id 2 and forum id 4 which were my off-topic and a private forum on my setup.

BadgerDog 01-24-2011 01:49 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2153797)
Just for the info of those who may need this feature, I made this edit for it to not import images from 2 specific forums... I am releasing this for info only, I don't recommend you make this change nor will I support it.

Thanks, but I think you need to clarify which version you're using?

I'm using the current 1.0.2 and my ie_cron.php file shown below does not have the additional ", forumid" on end of the $SQL string to search for?

Quote:

$SQL = "SELECT postid, dateline, pagetext
FROM " . TABLE_PREFIX . "post
WHERE iei_parsed=0
AND pagetext LIKE '%[/IMG]%'
ORDER BY postid";
Regards,
Doug

BirdOPrey5 01-24-2011 05:35 PM

Quote:

Originally Posted by BadgerDog (Post 2153835)
Thanks, but I think you need to clarify which version you're using?

I'm using the current 1.0.2 and my ie_cron.php file shown below does not have the additional ", forumid" on end of the $SQL string to search for?



Regards,
Doug

I'm using the current version, whatever it is, I just downloaded it recently. I know there is no forumid in the ORIGINAL code (which I specified in my post) I ADDED the forumids and a JOIN statement in my code which is the second block of code in my post.

My point is to change the original code to the new code I specified.

BadgerDog 01-24-2011 05:45 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2153941)
I'm using the current version, whatever it is, I just downloaded it recently. I know there is no forumid in the ORIGINAL code (which I specified in my post) I ADDED the forumids and a JOIN statement in my code which is the second block of code in my post.

My point is to change the original code to the new code I specified.

Thanks... :)

I'll ignore you find part of the code search and just use the replace part .. :up:

If I use a ForumID that has sub-forums, does that mean ALL sub-forums will be ignored too?

Appreciate your efforts... :)

Regards,
Doug

BirdOPrey5 01-24-2011 06:20 PM

No it doesn't cover subforums, each forumid must be entered manually. You can add more ANDs or consult a mysql manual for a better query.

y2ksw 01-24-2011 06:31 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2153961)
No it doesn't cover subforums, each forumid must be entered manually. You can add more ANDs or consult a mysql manual for a better query.

Better query :)
Code:

$SQL = "SELECT postid, " . TABLE_PREFIX . "post.dateline, pagetext, forumid
    FROM " . TABLE_PREFIX . "post join " . TABLE_PREFIX . "thread on (" . TABLE_PREFIX . "post.threadid = " . TABLE_PREFIX . "thread.threadid)
    WHERE iei_parsed = 0
        AND pagetext LIKE '%[/IMG]%'
        AND NOT forumid IN (1, 2, 3)
    ORDER BY postid";

The IN clause holds all forums to exclude. Contrarily:
Code:

$SQL = "SELECT postid, " . TABLE_PREFIX . "post.dateline, pagetext, forumid
    FROM " . TABLE_PREFIX . "post join " . TABLE_PREFIX . "thread on (" . TABLE_PREFIX . "post.threadid = " . TABLE_PREFIX . "thread.threadid)
    WHERE iei_parsed = 0
        AND pagetext LIKE '%[/IMG]%'
        AND forumid IN (4, 5, 6)
    ORDER BY postid";

The IN clause holds all forums to include.

As always, please try first in a sandbox ;)

BirdOPrey5 01-24-2011 06:41 PM

I want to suggest one more change I've used successfully on my test forum that may be of use to people with established forums.

In that query change ORDER BY postid to ORDER BY postid DESC so that the newest threads are processed first since it might take days or weeks to process a large existing forum.

BadgerDog 01-24-2011 07:54 PM

Quote:

Originally Posted by BirdOPrey5 (Post 2153975)
I want to suggest one more change I've used successfully on my test forum that may be of use to people with established forums.

In that query change ORDER BY postid to ORDER BY postid DESC so that the newest threads are processed first since it might take days or weeks to process a large existing forum.

Wouldn't it be better to process the oldest threads first, simply because users who fail to renew their off-site storage, or their free 30 day trials run out, would get those pics moved internally first so there's less chance of ending up with missing pics when that happens? :confused:

By the way, thanks for the tips .. :)

Regards,
Doug


All times are GMT. The time now is 01:48 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01873 seconds
  • Memory Usage 1,894KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (5)bbcode_code_printable
  • (1)bbcode_php_printable
  • (27)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete