# robots.txt # User-agent names specific robots # Disallow defines specific sub-directories to avoid User-agent: * Disallow: /