Yes, bots, etc. count as users. robots.txt is already in place. I have a lot of Guests not identified as bots, from Amazon AWS, for example, that are not humans. I am looking for streamlined ways to identify them -- as many Webmasters are! Other than the main search engines, I don't need any non-humans as Guests on my site.
I have a support Web site, nothing's being sold, is there any reason not to ban the traffic from Amazonaws? (I'm on IPS hosting, which charges by number of users. I don't see why I should pay more to get hits from Amazonaws.)
According to IPS Hosting, my 48-hour average active users count is 51-55. A spike can be 100 or more -- which elevates my average user count and sends me into a more expensive hosting bracket. IPS Hosting is not very forgiving of these spikes, and I don't want to pay extra to give bots a place to play.
Which free tools do you recommend, Matt? I am on IPS Hosting, which charges by number of users over a 48-hour period, so I am very concerned with keeping out bad bots, spiders, etc. Can I install something on the IPS server to help me do this?
Sorry to be obtuse, but I'm confused. I was looking for the IPBoard feedback forum and couldn't find it. A lot of things have been lost or rearranged recently. I still don't get that the Suggestions and Ideas forum, which has two major subforums, one of which is for Company Feedback and the other for Community Suite Feedback , is for IPBoard. The hierarchy doesn't make any sense: Suggestions and Ideas (aka IPBoard) Company Feedback Community Suite Feedback - IP.Blog - IP.Content - IP.Downloads - IP.Gallery - IP.Nexus This hierarchy suggests that IPBoard is the largest entity, the company (IPS) is a subunit. If this is the way IPS wants it, changing the Suggestions and Ideas forum title to IPBoard would forestall the confusion I experienced.
Like almost all admins here, I'm concerned with floods of bots, spiders, etc. visiting my site. In addition, I am on IPS Hosting, which charges according to user load measured over a 48-hour period. I have had my monthly hosting charges kicked into a higher (very expensive) bracket because of a single huge spike due to bots, spiders, etc. I have been directed by IPS Support to monitor IPB's Users Online list for suspicious activity. This is made particularly difficult because the Users Online list cannot be sorted by IP address. (Duplicates and clusters of IP addresses are highly suspicious.) IPB's Users Online list can be sorted by Member Name (exclusive of IP) and Last Click -- parameters I've never found useful -- but not by IP address, although IP address is displayed to admins. I strongly suggest the ability to sort IPB's Users Online list BY IP ADDRESS be included in the tool to enhance its usefulness. (The parens around the displayed IP addresses are unnecessary visual clutter and could be eliminated.) Given that so many admins have problems with bots, spiders, etc. and for years have been directed to IPB's Users Online list, I wonder why this hasn't been done sooner!
How can I recognize Baidu? IPS Support says Baidu is already blocked in their firewall (I am on IPS hosting). Can you recommend a third-party analytics tool? The Online list does not sort by IP, it's very hard to see duplicates and related IPs in that list.
Hello -- From time to time, my site gets a big spike in page views, most likely bots or spiders or spammers, etc. This kicks me into a higher hosting bracket on IPS, which gets pretty expensive. Naturally, I don't want to pay higher rent for bots. How can I find out where these spikes are coming from? What logs would I look at and how would I analyze them? (The Online Users list is not very helpful for this. It would be more useful if entries could be sorted by IP address.)