Robots/Spiders

The Robots/Spiders Option allows you to specify which hosts and user agents should be classified as robots and spiders. This allows you to separate robots and spiders from regular web site traffic. Robot and spider traffic is displayed separately in the Robot and Spider summary and is not counted in the visit statistics. By default NetTracker is configured with 29 Robots/Spiders.

To add a Robot/Spider group click the Add button. To change the name of a Robot/Spider group highlight the Robot/Spider group in the list box and then click the Edit button. To delete a Robot/Spider group highlight the Robot/Spider group in the list box and then click the Delete button.

To add hosts or user agents to a Robot/Spider group, highlight the Robot/Spider group in the list box and click on the Members button. To add members to the Robot/Spider group click the Add button. Then select from the list boxes to create the request: Group "[user agents/hosts]" "[That start with]", enter the hostname or user agent in the next text box (e.g.: "Scooter"), and click on the Add button. Note that in the above example users can also use this option to request that NetTracker groups "[user agents/hosts]" "[That match], [That start with] or [That contain]" a hostname or user agent as a Robot/Spider.

To delete a host or user agent from a Robot/Spider group, highlight the host or user agent in the list box and click on the Delete button. To edit a host or user agent in a Robot/Spider group, highlight the host or user agent in the list box and click on the Edit button. Then make the desired changes and click on the Update button.