/ Paper arXiv 2611.11306
In short
In shortReport a mistake
Key numbers
From the paperCopied from the abstract, not written by AI.
Why this ranked
MeasuredPublic numbers, not AIHow ranking works
13 / 100#37 of 1200 papers that day, from GitHub, Hugging Face, Hacker News and Semantic Scholar, looked up the morning it was ranked.
Builder evidence8 / 40
- Code released
Community attention4.8 / 30
- ▲ 2 upvotes on Hugging Face
Author track record0 / 15
- Nothing found
What the paper shows0 / 15
- Nothing found
Abstract
From the paperThe authors' own words.
AI agents deployed on the open web are becoming more capable of performing actions that may breach contractual provisions and violate computer security laws. In this paper, we introduce ScrapeBench, the first agentic benchmark for empirically evaluating the legal compliance of AI agents with website scraping policies, namely terms of service (ToS) and robots.txt directives. Across widely used closed-weight and open-weight AI agents, we find that most agents exhibit low compliance rates (< 5%) when instructed to scrape websites in our sample of 1,626 popular websites that explicitly prohibit scraping. In addition, we observe that even where AI agents actively check the applicable ToS and robots.txt, they continue to exhibit low compliance rates. Conversely, in our sample of 269 websites that do not prohibit scraping, we observe that agents sometimes refuse to scrape despite it being lawful, such as Claude Opus 4.6 exhibiting an over-refusal rate of 98.5%. Our human baseline experiments (n = 180 participants) suggest that humans exhibit compliance rates comparable to AI agents but over-refuse at much higher rates.
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.
