Google to explore alternatives to robots.txt in wake of generative AI and other emerging technologies

Google said it will engage with the web and AI communities with public discussions during this process.