Bot

Here is a minimum for a robots.txt file in a WordPress web site. Here is a more advanced list