What this checks
This reads the meta robots tag in the HTML and the X-Robots-Tag header in the response, and reports whether either one carries noindex or nofollow. Both are checked because they are set in completely different places, one in a template and one in server or CDN configuration, and a page can easily carry a directive that nobody working on the template can see. It also confirms the page actually responded with a usable status code, since a noindex on a page that returns a server error is the smaller of two problems.