Feature Request: Set "successful" HTTP status codes #463
Description
Your tool is great!
When I run it, it reports a half-dozen websites return HTTP status 403. When accessed with curl, they also return status 403. But when accessed from a web browser, they return status 200. Thus, it appears these sites are blocking crawlers.
I'd like to be able to add 403 as a successful status code, to eliminate these false negatives. I'm sure there are other scenarios where it would be useful to be able to set the list of "successful" status codes.