Written by thoughtbot, experienced designers and developers who turn your idea into the right product
You’re here for the link juice, amirite?
Search engines “crawl” and “index” web content through programs called robots (a.k.a. crawlers or spiders). Here are some approaches to blocking them in Ruby on Rails apps.
Sign up to receive a weekly recap from Giant Robots