How do I disallow specific page from robots.txt

You can also add a specific page or directory with extension in robots.txt file. Use Disallow keyword as follow:

Disallow: /index_test.html
Disallow: /products

Following are methods to disallowed index:

• using either robots.txt or
• the meta robots tag

Disallow: /index.html
Disallow: /prod

<meta name="robots" content="noindex, nofollow" />

The next time your site is indexed this will make your content drop out of the Google index.

You can also you the noarchive value – this will block caching of your page. This is Google specific:

<meta name="robots" content="noarchive" />

This tag will work <meta name="robots" content="noindex, nofollow" />

To prevent specific pages or directories from being crawled by search engine bots using robots.txt, simply add "Disallow" directives for each page or directory you want to block. For example, to block a page: Disallow: /specific-page.html, and to block a directory: Disallow: /directory/. Upload the updated robots.txt to your website's root directory.