CrawlWitness

AI consent of the world's top websites

Which of the most visited websites allow AI training, RAG and AI search, updated every day.

15%reserve AI training
16%block GPTBot
15%block ClaudeBot
14%block Google-Extended
17%block CCBot

Daily AI-consent report →

#DomainAI trainingRAG and AI answersAI searchGPTBot
1google.comNo signalNo signalNo signalPartly blocked
2cloudflare.comNo signalNo signalNo signalAllowed
3amazonaws.comNo signalNo signalNo signalPartly blocked
5facebook.comReservedReservedReservedBlocked
6fbcdn.netReservedReservedReservedBlocked
7gstatic.comReservedReservedReservedBlocked
8googleapis.comNo signalNo signalNo signalAllowed
9microsoft.comNo signalNo signalNo signalPartly blocked
11youtube.comNo signalNo signalNo signalPartly blocked
12apple.comNo signalNo signalNo signalPartly blocked
14instagram.comReservedReservedReservedBlocked
16twitter.comReservedReservedReservedBlocked
17linkedin.comNo signalNo signalNo signalAllowed
18mail.ruReservedReservedReservedBlocked
19azure.comNo signalNo signalNo signalPartly blocked
22googlevideo.comNo signalNo signalNo signalPartly blocked
23googletagmanager.comNo signalNo signalNo signalAllowed
25triplinkintl.comNo signalNo signalNo signalPartly blocked
26dzen.ruReservedReservedReservedBlocked
28appsflyersdk.comNo signalNo signalNo signalAllowed
30office.comNo signalNo signalNo signalAllowed
31shiabank.comNo signalNo signalNo signalAllowed
32amazon.comReservedReservedReservedBlocked
33live.comNo signalNo signalNo signalAllowed
34github.comNo signalNo signalNo signalPartly blocked
38wikipedia.orgNo signalNo signalNo signalPartly blocked
39fastly.netNo signalNo signalNo signalPartly blocked
40cloud.microsoftNo signalNo signalNo signalAllowed
42whatsapp.netReservedNo signalNo signalBlocked
43netflix.comReservedReservedNo signalPartly blocked
45bing.comNo signalNo signalNo signalPartly blocked
46lsrelayaccess.comNo signalNo signalNo signalAllowed
47googleusercontent.comNo signalNo signalNo signalAllowed
48x.comReservedReservedReservedBlocked
49youtu.beNo signalNo signalNo signalPartly blocked
50doubleclick.netNo signalNo signalNo signalPartly blocked
51googlesyndication.comNo signalNo signalNo signalPartly blocked
52pinterest.comReservedReservedReservedBlocked
53gandi.netNo signalNo signalNo signalPartly blocked
56icloud.comNo signalNo signalNo signalPartly blocked
58windows.netNo signalNo signalNo signalPartly blocked
59cloudflare.netNo signalNo signalNo signalAllowed
60goo.glNo signalNo signalNo signalAllowed
61adobe.comNo signalNo signalNo signalPartly blocked
62tiktok.comReservedReservedReservedBlocked
63myfritz.netNo signalNo signalNo signalPartly blocked
64zoom.usAllowedAllowedAllowedPartly blocked
65wordpress.orgAllowedNo signalNo signalAllowed
67whatsapp.comReservedNo signalNo signalBlocked
68googledomains.comNo signalNo signalNo signalAllowed
70skype.comNo signalNo signalNo signalPartly blocked
71yahoo.comReservedNo signalNo signalBlocked
72digicert.comNo signalAllowedAllowedAllowed
73wa.meReservedReservedReservedBlocked
76opera.comNo signalNo signalNo signalPartly blocked
77spotify.comReservedAllowedAllowedBlocked
78roblox.comNo signalNo signalNo signalPartly blocked
80workers.devNo signalNo signalNo signalAllowed
82tiktokv.comNo signalNo signalNo signalAllowed
83amazonvideo.comNo signalReservedNo signalPartly blocked
84baidu.comReservedReservedReservedBlocked
85b-cdn.netNo signalNo signalNo signalAllowed
86ntp.orgNo signalNo signalNo signalAllowed
87vimeo.comNo signalNo signalNo signalPartly blocked
90qq.comNo signalNo signalNo signalAllowed
92nginx.orgNo signalNo signalNo signalPartly blocked
93msn.comReservedReservedReservedBlocked
94yandex.netNo signalNo signalNo signalPartly blocked
95mozilla.orgNo signalNo signalNo signalPartly blocked
97f5.comNo signalNo signalNo signalAllowed
98epicgames.comNo signalNo signalNo signalAllowed
99wordpress.comNo signalNo signalNo signalPartly blocked
100t.meNo signalNo signalNo signalAllowed
101nginx.comNo signalNo signalNo signalAllowed
102sentry.ioAllowedAllowedAllowedPartly blocked
104reddit.comNo signalNo signalNo signalBlocked
105lencr.orgNo signalNo signalNo signalAllowed
107yandex.ruReservedReservedReservedBlocked
108ui.comAllowedNo signalNo signalAllowed
109europa.euNo signalNo signalNo signalPartly blocked
111chatgpt.comNo signalNo signalNo signalPartly blocked
114intuit.comNo signalNo signalNo signalPartly blocked
117bit.lyNo signalNo signalNo signalAllowed
118ipify.orgNo signalNo signalNo signalAllowed
120aliyuncs.comNo signalNo signalNo signalAllowed
122samsung.comNo signalNo signalNo signalPartly blocked
124t-online.deReservedNo signalNo signalBlocked
125okcdn.ruNo signalNo signalNo signalAllowed
126skybridge.clickReservedReservedReservedBlocked
127office365.comNo signalNo signalNo signalAllowed
128registrar-servers.comReservedReservedReservedBlocked
130openai.comNo signalNo signalNo signalPartly blocked
131adobe.ioNo signalNo signalNo signalPartly blocked
133outlook.comNo signalNo signalNo signalAllowed
135github.ioNo signalNo signalNo signalAllowed
136dropbox.comNo signalNo signalNo signalPartly blocked
137android.comNo signalNo signalNo signalPartly blocked
138nic.ruNo signalNo signalNo signalAllowed
139windows.comNo signalNo signalNo signalPartly blocked
140cloudflare-dns.comNo signalNo signalNo signalAllowed

Next →