WordPress duplicate content and robots.txt

How to properly block duplicate archives in WP.

 Old 05-22-2026, 03:40 PM #1
skrog
Respected Member
Join Date: May 2003
Posts: 1,890
I'm trying to clean up the duplicate content issues on my WP blog. I know I should block the category and tag archives, but what about the author archives? Is it better to disallow them in robots.txt or just add a noindex tag? What's working best for you guys right now?
 Old 05-22-2026, 04:15 PM #2
errorrr007
Senior Member
Join Date: Feb 2008
Posts: 142
Use the meta noindex, follow. If you disallow in robots.txt, the bot can't read the noindex tag anyway. Just let them crawl it once, see the tag, and drop it from the index.