Shout Box

Who's Online
1 registered (1 invisible), 6 Guests and 4 Spiders online.
Key: Admin, Global Mod, Mod
Page 2 of 2 < 1 2
Topic Options
#15276 - 06/07/05 07:17 PM Re: spider rate control [Re: learningtolead]
dalguard Offline
veteran

Registered: 03/22/00
Posts: 1515
Loc: CT
As I said before, I only get it here and I get it on both my home and work computer. I'm pretty sure it's gunks.com related.

Top
#15277 - 06/07/05 07:58 PM Re: spider rate control [Re: dalguard]
crackers Offline
Carpal Tunnel

Registered: 03/21/01
Posts: 3424
Loc: pdx
what?

huh?

oh, its you dawn...actually it wasnt bugging me last week but it is this week. so...but it hasnt happened for two days now.

Top
#15278 - 06/08/05 07:09 PM Re: spider rate control [Re: crackers]
webmaster Offline

veteran

Registered: 01/06/00
Posts: 1273
Loc: New Paltz (Kerhonkson, actuall...
Checked into this a bit at FutureQuest. There's not a whole hell of a lot that I can do about this problem. The software controls are for all accounts on FutureQuest, and they're not going to change them. They are designed to keep spidering (ro-)bots from overwhelming the servers. These spidering programs are sent out by sites like Google and Microsoft to help them compile what's out there on the web, for use in their search engines. The bots are apparently poorly written software when it comes to balancing the load on a server. The bots can overwhelm a network server with requests, basically shutting down a site and in my case, over-running my site's bandwidth allowance.

A few months ago, the MSbot used 3x my site's bandwidth allowance, leaving me with a extra $100 to pay to FQ so that the crappy MS-bot could run amok on Gunks.com. It took me a while but I finally found a human at MS to send an email, asking them please to stop. I eventually just blocked the MSbot from the site, so a gunks.com search on MS probably yield a lot less than Google.

But let me ask you all. What is the big inconvenience of having to wait 2.0 seconds to make your next request? Is this really a problem? Does the delay get beyond 2 seconds?

-em
_________________________
evan marks
webmaster@gunks.com
I wish I could read every post...

Top
#15279 - 06/08/05 07:23 PM Re: spider rate control [Re: webmaster]
dalguard Offline
veteran

Registered: 03/22/00
Posts: 1515
Loc: CT
Imagine you're driving down the road and you go to make a right turn and the car stalls. All you have to do is wait 2 seconds and start it again. Then you get to the next corner and it stalls again. Wait 2 seconds. Repeat. It's tedious but I suppose we'll all learn to live with it. I think it started happening when you added the new ads in. Because each ad is a file we have to load, it apparently looks like we're loading a lot of extra files now and it triggers the cutoff. Possibly cleaning up your code to load fewer gifts would help.

Top
#15280 - 06/08/05 08:13 PM Re: spider rate control [Re: dalguard]
webmaster Offline

veteran

Registered: 01/06/00
Posts: 1273
Loc: New Paltz (Kerhonkson, actuall...
good suggestion. The code is pretty sloppy right now. I can upgrade the forum software in a few months and that should clean up quite a lot. But until then, well...

-em
_________________________
evan marks
webmaster@gunks.com
I wish I could read every post...

Top
Page 2 of 2 < 1 2


Moderator:  webmaster 
Sponsored