12·
9 months agoRobots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
- 0 Posts
- 4 Comments
Joined 1 year ago
Cake day: July 2nd, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
4·11 months agoI miss my niche communities, but I don’t have my app anymore soooo. Tough for me, tougher for spez.
Niah