fbpx
Blog

Sitemap and Robot.txt For Search Engine Optimization

Hi readers, I am Your Digital Pavan back again with Article. In the previous article, we were talking about Search Engine Optimization and we discussed some part of it in that Article, let’s continue that here.

In Search Engine Optimization, There are some processes which we have to do

  1. Sitemap
  2. Robot.txt

Let’s talk about each of them now

Sitemap :

Sitemap for Search Engine Optimization

The sitemap is nothing but giving a navigation map to the bots as well as the user. Navigation which means which page should read first and second and etc…

There are two types of Sitemaps we should give to the website for better Search Engine Optimization. They are

  1. XML Sitemap
  2. HTML Sitemap

XML Sitemap For Search Engine Optimization :

  • XML Sitemap is for the bot to understand the navigation of pages. We will Generate it for Search Engines ( Bots).

HTML Sitemap For Search Engine Optimization :

  • HTML Site map We will Generate this for Users.

Now let’s know about how to check Sitemap and how to generate.

  • To check whether the website having sitemap or not
  • Type domain/sitemap.xml and domain/sitemap.html in the search bar and press enter.
  • Google says that if you have 1000 pages give the XML sheet to craw. By giving this I will give you the ranking in Search Engine Optimization
  • You must create a Sitemap and upload it on the website.
  • To generate Sitemap we have a tool called “XML Sitemap generator”

Steps To Follow :

  • Goto “XML Sitemap Generator“  
  • Copy the URL of your website
  • Paste and enter
  • The file will be generate
  • To upload the file
  • Go to godaddy.com
  • Log in with your credentials
  • Click to manage of website.
  •  Click on manage hosting
  • Go to Cpanel admin
  • Go to files manager after login to Cpanel
  • Open the required folder
  • Click on upload
  • Choose file
  • Upload file

To link the file to the website

  • Login to your website
  • Go to dashboard
  • Select appearance
  • Go to menu
  • Select Menu from the dropdown
  • Go to a custom link
  • Link text should be sitemap
  • Give url/sitemap.html
  • Click add to the menu
  • Click on save it.

To updated page manually in XML or HTML

  • Go to server
  • Click edit
  • You add or remove
  • Sitemap.com can generate only 500 pages

What to do to add more pages?

You can go-to pro in sitemape.com or develop code or by using a plugin called sitemap you can generate a sitemap to your website.

Once you uploaded on the website you should upload in webmaster/search console too.

Webmaster tool :

To create webmaster

  • Go to google webmaster tool
  • Login and paste the URL
  • Click on add property
  • Verify by HTML
  • Copy HTML tag
  • Now Go to the dashboard in WordPress
  • Select appearance
  • Go to editor
  • Upload in Header.php
  • Paste below to the head section
  • Upload file
  • Now click verify in webmaster
  • Or if you already registered with Google analytics you can verify by using that.
  • Click on done
  • Click on sitemap
  • Paste sitemap URL in the add box
  • Click on submit

Now we will have one question that why we are submitting in the webmaster too?

  • It says how many pages indexed or how many pages not indexed. To know this we are submitting in the webmaster tool. Using sitemap and webmaster tool is one of the best practice of Search Engine Optimization

Now let’s talk about Robot.Txt

Robot.txt For Search Engine Optimization :

Robot.txt is to instruct the spider which page should allow or which pages should not allow. When we have dummy content or thank you page we will say it.

How to do that?

Take a note pad and type

User-agent : *

  Allow : /

User-agent : *

  Dis allow : /

User-agent : google bot

 Dis allow  : /page.html

User-agent : google bot

  Allow  : /folder name

  • “*” is equal to  all search engine
  • / is equal to the entire website
  • Disallow required when the website is under construction
  • Commands only for search engines
  • User can see any time of dummy content
  • When we have dummy content we stop bot using the command
  • You must welcome after getting ranking
  • To do this command
  • As I said in the above text take note pad  type command
  • Save as robot.txt name
  • Now Go to GoDaddy,
  • Go to c panel
  • Go to file manager
  • Choose the folder and open
  • Upload notepad document
  • To check type website/robot.txt

We Discussed some of SEO Factors in our Previous article. In our next article, we will talk about Schema markup and other factors and off-page optimization in search engine optimization.

So readers above all thanks for reading my article hope everyone getting knowledge with this. Keep supporting we will learn together.

Leave a Reply

Your email address will not be published. Required fields are marked *