WisdmLabs

Contact Us

WordPress Development Experts, Custom Plugin Solutions, Magento Ecommerce Solutions, User Experience Design and more..

Home > Archives for HttpBasicAuthentication

Wisdm Blog

Receive important updates, exclusive offers, and insider tips to get the most out of your online business.

disallow-crawlers

How to Block Crawlers, Spiders and Bots from Websites

Aug 02, 2014

Sometimes disallowing crawlers on your website using robots.txt is not sufficient. In spite of this measure crawlers are still able to come on your website. In such a situation you need to block crawlers. Read on to find how you block crawlers.

Address

428, Lodha Supremus 2, Road No 22, Wagle Estate, Thane (W) - 400604

E-mail: [email protected]

Skype ID: wisdmlabs