Robot.txt for AI indexing

In the era of web crawling, we had a file ‘robot.txt’ to prevent indexing a web page into their servers. Some of them do it anyway.

Now we are in the era of AI, it indexes the whole web and train on it. We need a similar solution like robot.txt

Comment if you agree. Freedom matters.

Published by Shailendra Ahir

Programmer, Author

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: