adsense crawler errors
To fix this issue, I’m using Robots.txt file method. Just because it is the easiest way to prevent web crawlers and other web robots from accessing all or some part of a website which are otherwise publicly viewable. So we’ll restrict some significant portions of our site like wp-admin, index.php, preview pages etc. adsense crawler errors
DIFFERENT TYPES OF CRAWLER ERRORS
1) Page Not Found adsense crawler errors
Also, know as Error 404, this is one of the most common errors seen by webmasters and page visitors alike. It merely means that the crawler attempted to crawl a page that may have been previously deleted or repositioned elsewhere on your website.
Also read : top earners from google adsense of blog
A temporary URL can cause this as well. There is a fantastic tool that can help you find these pages to avoid this type of error: Webmaster URL parameter tool.
Page Not Found Code
<style>
#error-page {
background: #007dbd;
background: url(http://4.bp.blogspot.com/–Ri7FNhfPuU/T3p0_rWUvLI/AAAAAAAAHzU/v_V5cF2O9YM/s1600/bg_body.gif)no-repeat top center,-moz-radial-gradient(center 40%, circle cover, #007dbd 0%, #003650 100%);
background: url(http://4.bp.blogspot.com/–Ri7FNhfPuU/T3p0_rWUvLI/AAAAAAAAHzU/v_V5cF2O9YM/s1600/bg_body.gif)no-repeat top center,-webkit-gradient(radial, center 40%, 0, center 40%, 800, from(#007dbd), to(#003650) );
text-align:center;
position:fixed;
top:0px;
right:0px;
bottom:0px;
left:0px;
padding-top:270px;
z-index:999999;
}
#error-page h1 {
font-size:540px!important;
position:absolute;
font-family:impact,sans serif!important;
top:70px;
left:50%;
letter-spacing: -8px;
margin-left:-502px!important;
width:960px;
z-index:-2;
color:rgba(0,0,0,.09)!important;
}
#error-page h2 {
font-family:arial black,sans serif!important;
text-transform:uppercase;
font-size:80px;
line-height:66px!important;
letter-spacing: -3px;
color:#fff!important;
margin:0!important;
padding:0!important;
}
#error-page p a,
#error-page p a:visited,
#error-page p a:hover{
font-family:arial black,sans serif!important;
text-transform:uppercase;
font-size:56px;
line-height:40px!important;
border:none;
font-weight: bold;
color:rgba(0,0,0,.5)!important;
margin:0!important;
padding:0!important;
text-decoration:none!important;
}
</style>
<div id=’error-page’>
<h1>404</h1> <h2>LINK BROKEN</h2>
<h2>PAGE NOT FOUND</h2>
<p> <a href=’/’ title=’Home Page’> Back to Home Page</a></p>
</div>
2) Robot Denied adsense crawler errors
This means that the AdSense crawler tried to crawl a page in the domain or subdomain level. Check the reports to find out where exactly.
If this happens, Google will not be able to index that specific page, and they won’t be ready to serve ads on that page.
Also read : how to create ads txt file for adsense in blogger
To resolve this, just remove these two lines from your robots.txt file to grant Google crawler access:
User-agent: Mediapartners-Google
Disallow: /
3) Content Behind a Login
There are lots of websites that require login information to get premium access to the site’s main content. It usually means that a crawler login hasn’t been set up for that premium content.
As with error #2 above, Google will not be able to access this page, and Google ads will not be served.
Imagine if you have 1000’s of premium users, how many lost impressions do you think will that be?
This is thankfully very simple to resolve. Go to your Google AdSense login page; under Access and Authorization in Settings, go to the crawler access and provide login details for the crawler to use to access your site.
You may want to check out Google’s step-by-step guide to displaying ads on login-protected pages.
4) You Have Ad Crawler Errors for a Site I Don’t Manage
This error means that someone is using your ad code on a different site without your permission. The impressions and clicks will still be counted but will not payout. Therefore, you will not be earning as this is not authorized.
Also read : adsense first payment from blog
If this ever happens, set your AdSense account to “Only allow certain sites to display ads for my account.” This is an option available under Settings > Accounts and Authorization. Expect changes to happen in 48 hours.
Crawler issues are remarkably straightforward and easy to fix, and you don’t even need a Swedish assembly guide.
Ensuring your site is fully crawlable can help you earn more revenue from your content. To make sure you’ve optimized your site for crawling, consider all of the following issues that might affect how crawlable you are. Learn more about the AdSense crawler.
Granting Google’s crawlers access in robots.txt adsense crawler errors
To ensure we can crawl your sites make sure you’ve given access to Google’s crawlers. This means enabling Google’s crawlers in your robots.txt. For instructions on how to do this, see: Give access to our crawler in your robots.txt file.
Providing access to any content behind a login.
If you have content behind a login, ensure you’ve set up a crawler login. If you haven’t provided our crawlers a login, then it’s possible that our crawlers are being redirected to a login page, which could result in a “No content” policy violation, or, that our crawlers receive a 401 (Unauthorized) or 407 (Proxy Authentication Required) error, and thus cannot crawl the content.
Page not found adsense crawler errors
If the URL sent to Google points to a page that doesn’t exist (or no longer exists) on a site, or results in a 404 (Not Found) error, Google’s crawlers will not successfully crawl your content.adsense crawler errors
Overriding URLs adsense crawler errors
If you’re overriding the page URL in ad tags, Google’s crawlers may not be able to fetch the content of the page that’s requesting an ad, especially if the overwritten page URL is malformed. Generally speaking, the page URL you send to Google in your ad request should match the actual URL of the page you’re monetizing, to ensure the right contextual information is being acted on by Google.
Also read : How to set up app-ads.txt in Admob
Name serving issues
If the name servers for your domain or subdomain are not properly directing our crawlers to your content, or have any restrictions on where requests can come from, then our crawlers may not be able to find your content.
Implementing redirects
If your site has redirects, there’s a risk that our crawler could have issues following through them. For example, if there are many redirects, and intermediate redirects fail, or if important parameters such as cookies get dropped during redirection, it could decrease the quality of crawling. Consider minimizing the use of redirects on pages with ad code, and ensuring redirects are implemented properly.
Hosting issues adsense crawler errors
Sometimes when Google’s crawlers try to access site content, the site’s servers are unable to respond in time. This can happen because the servers are down, slow or get overloaded by requests. We recommend ensuring your site is being hosted on a reliable server or by a reliable service provider.
Also read : Ad inserter review pro nulled download