.Google.com limits robots.txt support to four industries, clarifying its own position on unsupported regulations.Google just assists 4 certain robots.txt areas.In need of support regulations in robots.txt will definitely be actually overlooked.Think about auditing your robots.txt documents due to this improve.