#robots text file is used to control the access to web #site files and directories which should not be indexed # User-agent: * # #Block spidering of specific top level directories Disallow: /admin