Robots.txt Generator

Robots.txt Generator(Robots.txt is a plain text file placed in the root directory of the site, which specifies that search engine spiders crawl specified content, or prohibits search engine spiders from crawling part or all of the website content)

Restricted Directory:

Each path must be preceded by:"/"

Add restricted directory

Sitemap: (Leave blank for none):

Google is in XML format, Baidu is in HTML format

Add Sitemap

Default - All Robots are:

Crawl-Delay:

Chinese search engines

baidu

SOSO

Sogou

Youdao

Mainstream search engines

Google

Bing

Yahoo

Ask/Teoma

Alexa/Wayback

Cuil

MSN Search

Scrub The Web

DMOZ

GigaBlast

Special search engines (robots)

Google Image

Google Mobile

Yahoo MM

Yahoo Blogs

MSN PicSearch

Tip: Please save the following results to Notepad, name it robots.txt and upload it to the root directory of the website (click Copy to copy all results)

Generated robots results：

Robots.txt generator,robots.txt validator,Free Robots.txt Generator：Generate robots file online. The Robots.txt file should be placed in the root directory of the website and can be accessed through the Internet.

More Tools: EXR to ICO TGA to TIFF TGA to WEBP TGA to JPG RAW to AVIF RAW to PDF RAW to ICO RAW to TIFF RAW to GIF RAW to BMP RAW to WEBP RAW to PNG RAW to JPG HEIF to EXR HEIF to TGA HEIF to RAW HEIF to AVIF HEIF to JP2 HEIF to SVG HEIF to PDF HEIF to ICO HEIF to TIFF HEIF to GIF HEIF to BMP HEIF to WEBP HEIF to PNG HEIF to JPG AVIF to JP2 AVIF to ICO AVIF to TIFF avif to GIF AVIF to BMP AVIF to WEBP AVIF to PNG AVIF to JPG JP2 to EXR JP2 to TGA JP2 to RAW JP2 to HEIF JP2 to AVIF JP2 to SVG JP2 to PDF JP2 to ICO JP2 to TIFF JP2 to GIF JP2 to BMP JP2 to WEBP jp2 to PNG JP2 to JPG SVG to EXR SVG to TGA SVG to RAW SVG to HEIF SVG to JP2 SVG to PDF SVG to TIFF PDF to EXR PDF to TGA PDF to RAW PDF to HEIF PDF to AVIF PDF to JP2 PDF to ICO PDF to GIF PDF to WEBP TIFF to EXR TIFF to TGA TIFF to RAW TIFF to HEIF TIFF to AVIF TIFF to JP2 TIFF to SVG TIFF to ICO TIFF to GIF TIFF to BMP TIFF to WebP TIFF to JPG TIFF to PNG GIF to EXR GIF to TGA GIF to RAW GIF to HEIF GIF to AVIF GIF to JP2 GIF to ICO GIF to TIFF GIF to WEBP GIF to JPG GIF to PNG BMP to EXR BMP to TGA BMP to RAW BMP to HEIF BMP to AVIF BMP to JP2 BMP to ICO BMP to TIFF BMP to GIF BMP to JPG BMP to PNG WebP to EXR WEBP to TGA WebP to RAW WEBP to HEIF WebP to AVIF WEBP to JP2 WEBP to ICO WebP to TIFF WebP to BMP WebP to PNG PNG to EXR PNG to TGA PNG to RAW PNG to HEIF PNG to AVIF PNG to JP2 PNG to ICO PNG to TIFF PNG to GIF PNG to BMP PNG to WEBP ICO to EXR ICO to TGA ICO to RAW ICO to HEIF ICO to AVIF ICO to SVG ICO to JP2 ICO to TIFF ICO to GIF ICO to BMP ICO to WEBP ICO to JPG Bitmap generator ICO to PNG BMP to WEBP Image Resizer JavaScript Minifier SVG to DXF SVG to ICO SVG to BMP SVG to AVIF SVG to GIF SVG to PNG SVG to WEBP SVG to JPG JPG to SVG WEBP to SVG BMP to SVG GIF to SVG AVIF to SVG PNG to SVG WEBP to GIF WEBP to JPG JPG to WEBP MD4 Hash Generator UUID Generator Slug generator Whatsapp Link Generator UTM Link Generator File mime type checker DNS Lookup SSL Checker Hosting Checker IP Subnet Calculator Log base 2 Calculator Log Base 10 Calculator Rgb Color Picker Qr Code Decoder HTML to JavaScript Unicode code converter URL Blaster MAC Address Generator Nnix timestamp Discord timestamp generator Port Scanning Speed Test Ping Test PNG to JPG short url generator DNS Resolution