Go Back   vb.org Archive > Community Discussions > Modification Requests/Questions (Unpaid)
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Display Modes
  #11  
Old 08-22-2015, 04:58 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

If you want to stop people from scraping your site, don't put it on the internet.
Reply With Quote
  #12  
Old 08-22-2015, 08:14 PM
TheLastSuperman's Avatar
TheLastSuperman TheLastSuperman is offline
Senior Member
 
Join Date: Sep 2008
Location: North Carolina
Posts: 5,844
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Zachery View Post
If you want to stop people from scraping your site, don't put it on the internet.
I know you weren't sitting there all riled up, intentionally posting something to sound mean or rude yet I thought back to an old saying from when we were kids, most of us were taught this; "If you don't have anything nice to say, don't say anything at all" - That's not you in my opinion. Since tone is always missing I can't assume but do you ever re-read what you type and realize its not offering one bit of help sometimes? I think the OP has a valid concern and wants helpful suggestions not a reply that can't be taken any other way but being a smarty-pants.

Spamgirl,

I think Max had an excellent idea... it may take more time to review the logs for certain guests with Paul's mod but if you do it now and find who you think the culprit is, it might help! Remember though that overseas a person can unplug their modem/router and BAM instant new IP address so if they happen to be where that can happen, lets hope they only scrape content and aren't toooooo web savvy .
Reply With Quote
2 благодарности(ей) от:
ozzy47, spamgirl
  #13  
Old 08-22-2015, 08:28 PM
spamgirl spamgirl is offline
 
Join Date: Jan 2007
Posts: 57
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by TheLastSuperman View Post
Spamgirl,

I think Max had an excellent idea... it may take more time to review the logs for certain guests with Paul's mod but if you do it now and find who you think the culprit is, it might help! Remember though that overseas a person can unplug their modem/router and BAM instant new IP address so if they happen to be where that can happen, lets hope they only scrape content and aren't toooooo web savvy .
FWIW, I get what Zachary is saying, but that doesn't mean I won't try to at least stem the flow. If we sit back and don't fight, we let the monsters win, and I refuse to do that in any situation. Nothing is hopeless.

Anyhoo, I agree that Max had an excellent idea! Already three IPs are sticking out like a sore thumb, and one of them seems to be the culprit (with a scraper I didn't even know about potentially being a second problem user). Based on their shitty web design skills, I'm hopeful that means they aren't tech savvy at all. Thank you all so much for your advice!

--------------- Added [DATE]1440343872[/DATE] at [TIME]1440343872[/TIME] ---------------

I've found the IPs and tried to block them with .htaccess. I included my own IP in order to test it, but I am still able to access the forum, I just can't see the CSS or images. Here is what I did:

order allow,deny
deny from ###.#.#.
deny from ###.#.#.
deny from ###.#.#.
allow from all

Does anyone know why it would be so wonky? I put it in the main folder of my forum (html1). My site is hosted on EC2, if that matters. I tried it last week and it worked, so I don't know why it wouldn't now...
Reply With Quote
  #14  
Old 08-24-2015, 11:03 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Sometimes the truth hurts, but its important to understand the limitations of what you can do. You can ban an ip, but it will probably change and come back.

You can make it so only registered users can view content, but then your search rankings go down.

You can make some content pay only, but chances are if its stuff people want someone will steal it, and hopefully they don't do it with a stolen credit card.

I do think you should fight, just be ready for the long haul.

If they're actually stealing and rehosting your content on their site, you could try a DMCA, but it may or may not work.
Reply With Quote
  #15  
Old 08-24-2015, 11:22 PM
spamgirl spamgirl is offline
 
Join Date: Jan 2007
Posts: 57
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Zachery View Post
Sometimes the truth hurts, but its important to understand the limitations of what you can do. You can ban an ip, but it will probably change and come back.

You can make it so only registered users can view content, but then your search rankings go down.

You can make some content pay only, but chances are if its stuff people want someone will steal it, and hopefully they don't do it with a stolen credit card.

I do think you should fight, just be ready for the long haul.

If they're actually stealing and rehosting your content on their site, you could try a DMCA, but it may or may not work.
I've been doing the DMCA, but they just change hosts every day. Now I'm blocking by IP, and just redoing it constantly. I've actually found *multiple* scrapers since installing the Track Guests extension, go figure. :/ I'll just keep up the good fight and hope I annoy them into scraping someone else lol
Reply With Quote
  #16  
Old 08-28-2015, 10:00 AM
bridge2heyday's Avatar
bridge2heyday bridge2heyday is offline
 
Join Date: Aug 2014
Location: Egypt
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Is this what you are looking for ?
Limited Guest Viewing -- Motivate Guests to Register
Reply With Quote
Благодарность от:
spamgirl
  #17  
Old 08-28-2015, 10:59 AM
Dave Dave is offline
 
Join Date: May 2010
Posts: 2,583
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

It's not easy to prevent people from scraping your site.
IP's can be changed/proxies can be used and headers can be spoofed (thus making methods to detect the user-agent useless).

There may be one way to stop scrapers, that is to add a JavaScript check to your site before people are able to view your site. CloudFlare does this to prevent certain DDoS attacks. However people could simply just go to your site in a normal browser and save each file individually to their desktop.
Reply With Quote
  #18  
Old 08-28-2015, 03:33 PM
spamgirl spamgirl is offline
 
Join Date: Jan 2007
Posts: 57
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by bridge2heyday View Post
Is this what you are looking for ?
Limited Guest Viewing -- Motivate Guests to Register
That is! Thank you so much.
Reply With Quote
  #19  
Old 08-30-2015, 10:48 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by spamgirl View Post
That is! Thank you so much.
FYI, some times search engines can penalize you for this. It won't work for anyone who is blocking cookies, or who decides to use specific user agents that are generally white listed.

More often than not it just leads to:

- More Users leaving your site
- Some Users registering just to view content, but not participate.
Reply With Quote
  #20  
Old 08-31-2015, 12:30 AM
Max Taxable's Avatar
Max Taxable Max Taxable is offline
 
Join Date: Feb 2011
Posts: 3,134
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Zachery View Post
FYI, some times search engines can penalize you for this.
Pretty sure spiders are immune to it. If memory serves. I used it for awhile.

But for the rest, you're right. All it really does is irritate people.
Reply With Quote
Благодарность от:
spamgirl
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 06:31 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.08651 seconds
  • Memory Usage 2,274KB
  • Queries Executed 13 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (6)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (2)pagenav_pagelink
  • (10)post_thanks_box
  • (4)post_thanks_box_bit
  • (10)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (3)post_thanks_postbit
  • (10)post_thanks_postbit_info
  • (10)postbit
  • (10)postbit_onlinestatus
  • (10)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • fetch_musername
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • post_thanks_function_fetch_thanks_bit_start
  • post_thanks_function_show_thanks_date_start
  • post_thanks_function_show_thanks_date_end
  • post_thanks_function_fetch_thanks_bit_end
  • post_thanks_function_fetch_post_thanks_template_start
  • post_thanks_function_fetch_post_thanks_template_end
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete