Home
Forums
New posts
Search forums
What's new
New posts
New profile posts
Latest activity
Members
Current visitors
New profile posts
Search profile posts
Log in
Register
What's new
Search
Search
Search titles only
By:
New posts
Search forums
Menu
Log in
Register
Install the app
Install
Home
Forums
FreeOnes
Questions, Comments, and Suggestions
Crawler detection
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="greenmeanie" data-source="post: 1111969" data-attributes="member: 9307"><p>I think your crawler detection code needs a bit more work. I was just crawler blocked from Freeones, with the website returning the IP address of my ISPs transparent proxy cache as a crawler bot. Given just about every ISP in my part of the world is putting their broadband customers through transparent proxy caches for http requests, I am guessing there are going to be a lot of confused people getting blocked when they probably shouldn't be.</p></blockquote><p></p>
[QUOTE="greenmeanie, post: 1111969, member: 9307"] I think your crawler detection code needs a bit more work. I was just crawler blocked from Freeones, with the website returning the IP address of my ISPs transparent proxy cache as a crawler bot. Given just about every ISP in my part of the world is putting their broadband customers through transparent proxy caches for http requests, I am guessing there are going to be a lot of confused people getting blocked when they probably shouldn't be. [/QUOTE]
Insert quotes…
Name
Verification
Finish the sentence: Better late than ___
Post reply
Home
Forums
FreeOnes
Questions, Comments, and Suggestions
Crawler detection
Top
