There is no correct answer to the question about how many pages of your site a search engine should index. You cannot even be vague and say “all of them”, “most of them”, or “some of them” because the search engines treat each site differently. They take into consideration things like how many of your pages seem unique to them, how much indexed link value is pointed toward the site, how much of the site is accessible to crawlers, etc. If you have no links pointing to your site you can still get indexed but without the trust and PageRank-like value conferred by external links your pages may not be fully indexed or crawled. Hence, even if you see every page of your site in search results they may not be fully responsive to search queries. Website Configuration is Important If you misconfigure robots.txt you may block crawlers from important content on your site. If you misconfigure your “rel=’canonical’” assignments the search engines may not properly handle duplicate content issues. There is a really bad bug with some WordPress installations (including some of my own) where the NEXT/PREV links become confused and lead the crawlers a merry chase. You can [...]









Comments are closed.