Today we discuss what is Custom Robots.txt file and How To Add Custom Robots.txt File In Blogger it is very Important For SEO. You Know SEO Play important Role in Search Engine.when you start a new Blogger you need to SEO For your blog. Similar Custom Robots are Also important in SEO. Robots.txt file is a simple text.Robots.Text File plays a very important role as SEO and Crawler Your Site or Post in Search Engine Google.Bring.Yahoo and Other Search Engine.
Adding Custom Robots. Txt to Blogger
First of All, you log in to your Blogger Account.
Open the Blog Dashboard and go to Setting
After Click on Search preferences and Enable Setting and Click on Edit Option
Now Copy This Code and Paste in Box
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://yourdomain.com/atom.xml?redirect=false&start-index=1&max-results=500
And Click on Save Option. Now your Robot.txt.file Enable.if you want to See your Blog Robot.txt Then Enter you These word after domain /robots.txt.
http://www.YourDomainName.com/robots.txt
Custom robots.txt
Robots.txt Type 1
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Disallow: /b Allow: / Sitemap: https://alltechabout.blogspot.com/sitemap.xml
Robots.txt Type 2
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Disallow: /b Allow: / Sitemap: https://alltechabout.blogspot.com/feeds/posts/default?orderby=updated
Robots.txt Type 3
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Disallow: /b Allow: / Sitemap: https://alltechabout.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500 Sitemap: https://alltechabout.blogspot.com/atom.xml?redirect=false&start-index=501&max-results=500
What is Robots.txt?
The robots.txt file is part of the robots exclusion protocol (REP), a group of web standards that regulate how robots crawl the web, access, and index.
Explanation
User-agent: Mediapartners-Google:
User-agent: Media partners-Google play a good role in Robot txt .if you are using Adsense then use it if you are not using Adsense then you do not use it and remove it
User-agent: *
User-agent: * is use Because it is control And Crawlers and Spider in Search Engine and Blog.it is used For Blog labels Setting.
Disallow: /search
Disallow: /search its mean you know not to give Permission. Not Give Permission Google Crawlers to index pages Which you want to not index in Google.and not Search your blog page and post.
Allow: /
Allow: / mean Give Permission to Google Crawler to index your page it is Allow Search Engine
if you have any question regarding this post then feel free to ask me and comments.
I made a logger blog and searching about how to add robots.txt… u made my work easy.. Thanks for sharing thi article.. Keep sharing such valuable content…
Welcome Rajni
I got this post from google search. You have given a clear idea about custom robots. Thank you for the valuable information.
Thanks