Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This "technique" is highly recommended for your error pages also (50x). It also helps if you set the "Retry-After" HTTP header (value in seconds), that tells google (and hopefully other crawlers) that they shouldn't bother with crawling for another X seconds. Helps with the load if you are experiencing problems with your server.

Retry-After is also usefull for "down for maintenance" pages since you usually know how long your page will be down.



Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: