User-agent: * allow: /ads.txt allow: /robots.txt disallow: / User-agent: AdsBot-Google allow: / User-agent: AdsBot-Google-Mobile-Apps allow: / User-agent: Mediapartners-Google allow: /