How to use Meta Robots Tag Generator
- Choose indexing and link-following rules. Unrestricted index/follow with no other selected rule needs no tag.
- Set text, image or video preview controls as needed. Use indexifembedded with noindex only when you intend Google to consider embedded content despite the page’s noindex rule.
- Optionally set an unavailable_after date/time. Choose the crawler token and HTML meta, HTTP header or both, then inspect the updated draft.
- For your own implementation, place meta markup in the HTML head or configure the X-Robots-Tag on the HTTP response. Crawlers must be able to fetch it; this tool changes no site or server.
Example: Meta Robots Tag Generator
Keep all unrestricted defaults and confirm that no robots tag is needed.
Options
- Maximum snippet characters
- Used only when Text snippet is set to a maximum length.
- Image indexing
- noimageindex asks search engines not to index the images on this page.
- Image preview size
- Writes max-image-preview:large, max-image-preview:standard or max-image-preview:none.
- Maximum video preview seconds
- Used only when Video preview is set to a maximum.
- indexifembedded
- For Google, indexifembedded has an effect only alongside noindex: embedded content may still be indexed. It is not an authentication or access-control rule.
- unavailable_after date (optional)
- Enter a date or timestamp such as 2026-12-31 or 2026-12-31T23:59:59+00:00. Google’s unavailable_after rule requests no search-result display after that time; it does not remove the page or block visitors.
Supported inputs and limits
- Drafts noindex/nofollow, nosnippet, max-snippet 0–1,000, image indexing/preview, bounded video-preview and unavailable_after rules. indexifembedded only affects Google’s noindex handling when content is embedded.
- The builder’s date format is a supported subset.
- Other crawler tokens are emitted as declarations, without certifying those crawlers implement these directives. robots.txt blocking can prevent discovery of a noindex rule.
- X-Robots-Tag belongs in an HTTP response, not page body text.
More details
No server change, access control or guaranteed indexing/removal timing.
Where your input is processed
This tool does not upload your inputs. See the limits for anything saved on your device.
Place the rule where a crawler can read it
A robots meta tag belongs in an HTML response, while an X-Robots-Tag belongs in an HTTP response header. Google documents both forms and notes that restrictive rules only work after its crawler can reach the URL. Search engines can differ, so check the published response and the crawler-specific documentation when a rule is important.
Questions about Meta Robots Tag Generator
Do I need an index, follow tag?
No. Those are the usual default behaviors; the tool says no tag is needed when no restriction is selected.
Will noindex work on a page blocked by robots.txt?
A crawler must fetch the page to read the directive. Allow crawling and put noindex in the page or response if indexing is what you need to control.
When should I use X-Robots-Tag?
Use the response header when you need a robots rule for a non-HTML resource, such as a PDF, or when your server manages rules centrally.
Does Googlebot-only affect all search engines?
No. It targets Googlebot. Choose the general robots name when you want to address search crawlers more broadly, then test your actual setup.