robots.txt Detected

  • CWE 425
  • WASC 34

Depending on the content of the file, an attacker might discover hidden directories. Ensure you have nothing sensitive exposed within robots.txt file, such as the path of the administration panel.

Predictable Resource Location is an attack technique used to uncover hidden web site content and functionality. By making educated guesses via brute forcing an attacker can guess file and directory names not intended for public viewing. Brute forcing filenames is easy because files/paths often have common naming convention and reside in standard locations. These can include temporary files, backup files, logs, administrative site sections, configuration files, demo applications, and sample files. These files may disclose sensitive information about the website, web application internals, database information, passwords, machine names, file paths to other sensitive areas, etc...

This will not only assist with identifying site surface which may lead to additional site vulnerabilities, but also may disclose valuable information to an attacker about the environment or its users.


Do not write sensitive paths in the robots.txt, and ensure they are correctly protected by means of authentication.