Skip to main content

2 posts tagged with "seo"

View All Tags

· 2 min read

Based on Research Done


A 404 is not necessarily an error. If you removed the pages and you didn’t set up different URLs to which the server should redirect requests for those pages, then a 404 response is correct.

No penalty as such but indirect impact on your website is evident, when you have so much of web pages 404. 404 says the page is not found but it doesn't say the page is deleted or permanently deleted and the visitor is not going to find it again.

Google understands that. They’re reporting 404 responses, not errors. You’re not being hurt because your server responds with a 404 when Googlebot requests pages that you’ve removed. You are potentially passing up on a benefit, however. If those pages had a decent amount of link equity, that equity would be passed on to pages to which you redirected requests for the old pages. This would also make for a better user experience. Of course, all of this is based on the assumption that you’d be redirecting to pages that served to provide people looking for the old pages with the information they seek.

Waste of Crawl Budget

With 404, Google waits for 2 to 3 months to ensure the page is removed. With 404, the Other impact would be that you waste your crawl budget. The crawler spends time in crawling the periods which are not present on your website. This lead to trouble in discovering new pages on your website.

With more pages failing, Google reduces your crawl budget over time. This will reduce the timely indexing of new pages on your website.

Possible Solution?

It is better to issue 410 since it instructs that the page is permanently deleted. This helps Google to remove the URL from its index quickly. The next time Googlebot visits the page and sees the 410 status code, Google will then remove the page from it’s index within 24 hours. Google will never check again to see if the page has come back.

Here are some different types of HTTP status codes:

100s: Informational requests
200s: Successful requests
300s: Redirects
400s: Client-side errors
500s: Server-side errors

Depending on your browser, the 410 error can appear in a few different ways:

410 Gone
Error 410
HTTP Status 410

· One min read

Get started

Requires Python 3.6+, BeautifulSoup4 and urllib3.

Install by using pip

pip3 install pyseoanalyzer

Command-line Usage

If you run without a sitemap it will start crawling at the homepage.


Or you can specify the path to a sitemap to seed the urls to scan list.

seoanalyze --sitemap

HTML output can be generated from the analysis instead of json.

seoanalyze --sitemap --output-format html

Run From cloned Repository

python -f html > results.html