What if we made a search engine so good that our company name became the verb for searching? And then — get this — we replaced the search engine with a robot with a concussion? Google is innovating…
This is pure gut instinct, but I suspect this is gonna prompt an uptick in Google crawlers getting blocked. Mainly because Google just removed any real reason for people not to block them.
nah, not yet, imo. reasons: you overestimate how much & how many people care, and the complexities of doing such blocks well
I run the hosting of some small (by global standards) media setups and I’ve floated crawler defense there and had fairly little bite so far despite the issues the crawlers have caused. engagement brain is still extremely fucking prevalent
using it to source data in the meantime. I can tell you with absolute certainty that openai lies about its sourcing
This is pure gut instinct, but I suspect this is gonna prompt an uptick in Google crawlers getting blocked. Mainly because Google just removed any real reason for people not to block them.
nah, not yet, imo. reasons: you overestimate how much & how many people care, and the complexities of doing such blocks well
I run the hosting of some small (by global standards) media setups and I’ve floated crawler defense there and had fairly little bite so far despite the issues the crawlers have caused. engagement brain is still extremely fucking prevalent
using it to source data in the meantime. I can tell you with absolute certainty that openai lies about its sourcing