“Robots.txt - Jim Nielsen’s Blog”
What he said. Robots.txt is a convention based on a social contract and the industry has demonstrated that it doesn’t believe in social contracts.
... works as a web developer in Hveragerði, Iceland, and writes about the web, digital publishing, and web/product development
These are his notes
“Robots.txt - Jim Nielsen’s Blog”
What he said. Robots.txt is a convention based on a social contract and the industry has demonstrated that it doesn’t believe in social contracts.