Your site can run slow for various reason.
1. You have a many Nameservers. ie: more then 4 or 8 to 10 name server.
2. DNS not configured properly
3. Overloaded server
4. Slow DDoS/DoS attack on your site.
If you are using wordpress then also make sure that wp-login.php and xmlrpc.php is protected by .htaccess and only accessible by your selected/whitelisted IPs.
Hope my response will help many peoples.