How to edit wordpress robots.txt

If you built a blog by wordpress, you must want to google spider not visit some directories, such as wp-admin,wp-contents,wp-includes and so on. If you say yes, you can edit robots.txt of your wordpress blog to realize it.

Step 1: Find content of your robots.txt

You can open url, such as:http://www.beginnerlibrary.com/robots.txt, to see contents of your wordpress blog, if the content has not contained what you want, you can edit it.
You may see a example content like:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Step 2: Check robots.txt file  exists?

In wordpress system, the robots.txt file is a virtual one, that means this file do not exist, you can not find a file called robots.txt in root directory of your website, so you can not edit it to change its content,however, you can read step 3 to edit it.On the other hand, if this file exists in root directory of your website, you can edit it directly.

Step 3: Edit virtual robots.txt

If robots.txt file is a virtual one, and you want to google spider not visit any html page of comments directory, you can edit functions.php of  your theme.
You can add code below at the end of functions.php file:

add_filter( 'robots_txt', 'robots_mod', 10, 2 );
function robots_mod( $output, $public ) {
 $output .= "Disallow: /comments/";
 return $output;
}

Then save it, open the url:http://www.beginnerlibrary.com/robots.txt, you can find the content is:

User-agent: *
 Disallow: /wp-admin/
 Disallow: /wp-includes/
 Disallow: /comments/