A text file used by a website to communicate to web robots (most commonly SE bots) how to process each page of your website. Here you can set certain pages as off-limits for the bot and scan only the most useful content.
About the Author: Chad Klingensmith
Chad is a Sr. SEO Strategist for Rio SEO. He has over 10 years of experience in the digital marketing vertical. He enjoys researching and applying cutting edge marketing strategies to help clients achieve optimal online visibility.