[pmwiki-users] Google local site search

Patrick R. Michaud pmichaud at pobox.com
Thu Dec 29 22:35:29 CST 2005


On Thu, Dec 29, 2005 at 08:17:56PM -0700, H. Fox wrote:
> > 3.  Tell me which robots (besides Googlebot) understand rel='nofollow'
> >     to mean "don't spider this link".
> 
> That one's over my head, although there must be some resource out
> there for that.

I've looked, but haven't been able to find such a resource anywhere.
Notably, both msnbot and Slurp (Yahoo!) are extremely quiet about
whether or not they follow rel='nofollow' links.

> I'll guess that by far the robot most people are concerned about it Google's.

Perhaps.  But so far it seems that Google is the special case in all
of this.

> > * If someone wants to use action cloaking for robots other than Google,
> >   simply use:
> >
> >     $EnableRobotCloakActions =
> >       !preg_match('/Googlebot/', @$_SERVER['HTTP_USER_AGENT']);
> >
> >   This enables action cloaking for robots other than Googlebot.
> 
> I'll do that if necessary, but I think it should be the default.

I'm not sure how to cleanly make this the default -- I suppose
we could have a $RobotCloakExcludePattern variable that identifies
the robots that shouldn't be cloaked even when $EnableRobotCloakActions
is set.  But it's starting to sound to me like we have too many
special-purpose configuration variables, one to identify robots,
one to indicate if ?action= should be cloaked, and another to
identify robots that shouldn't be cloaked....

Pm




More information about the pmwiki-users mailing list