It still is, google is the misspelling.
As for CUIL, their crawler was such a nuisance at some point that I blocked their complete network. I doubt I'm alone in that.
And if those 'twiceler' guys don't start playing nice I'll do the same to them.
As for CUIL, their crawler was such a nuisance at some point that I blocked their complete network. I doubt I'm alone in that.
And if those 'twiceler' guys don't start playing nice I'll do the same to them.
That clinches it then. One more agent for the ban list.
Apropos, I think that plenty of bad guys are masquerading as twiceler as well.
Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html
When 'CUIL' first came out their crawler was all over my servers, sometimes to the point of making them inaccessible for users, I remember having a devil of a time getting rid of them because they came from lots of different IP blocks.
Apparently their bot is called twiceler, maybe they should name it 'CuilBot' or something to that effect.
I wonder if their overzealous bots and subsequent blocking are reasons why they launched with such terrible results. I heard several reasons for that catastrophic failure:
* Overstressed subject-specific servers failed, leading to porn on hamburger searches and that kind of thing
* Crashing, random result variations due to overload
* Excessive media hype exacerbated the load issues
Another one to add to the list?