User-agent: * Disallow: /admin/ Disallow: /cont/ Disallow: /js/ Disallow: /scripts/ Disallow: /click/ Disallow: /themes/ Disallow: /xml/ Disallow: /odp/ User-agent: Yandex Disallow: / User-agent: twiceler Disallow: / User-agent: dotbot Disallow: /