Filters and example code for Yoast SEO robots or WP robots.txt

custom_robots.php

<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/

/*
 * Replace Disallow with Allow Generated Robots.txt
 * Credit: Unknown
 * Last Tested: Unknown
 */

add_filter('robots_txt','custom_robots');

function custom_robots($output) {
     $public = get_option( 'blog_public' );
     if ( '0' != $public )
     return str_replace('Disallow','Allow',$output);
}

yoast_seo_robots_remove.php

<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/

/*
 * Remove meta robots from Yoast SEO
 * Credit: Yoast development team
 * Last Tested: Mar 01 2017 using Yoast SEO 4.4 on WordPress 4.7.2
 */

add_filter( 'wpseo_robots', '__return_false' );

yoast_seo_robots_remove_search.php

<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/

/*
 * Change meta robots using Yoast SEO
 * Credit: Yoast development team
 * Last Tested: Jun 19 2017 using Yoast SEO 4.9 on WordPress 4.8
 */

add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_search' );

function yoast_seo_robots_remove_search( $robots ) {
  if ( is_search() ) {
    return false;
  } else {
    return $robots;
  }
}

yoast_seo_robots_remove_single.php

<?php
/********* DO NOT COPY THE PARTS ABOVE THIS LINE *********/

/*
 * Change meta robots using Yoast SEO
 * Credit: Yoast development team
 * Last Tested: Dec 12 2017 using Yoast SEO 9.2.1 on WordPress 5.0
 *********
 * DIFFERENT POST TYPES
 * Post: Change 123456 to the post ID
 * Page: Change is_single to is_page and 123456 to the page ID
 * Custom Post Type: Change is_single to is_singular and 123456 to the 'post_type_slug'
    Example: is_singular( 'cpt_slug' )
 *********
 * MULTIPLE ITEMS
 * Multiple of the same type can use an array.
    Example: is_single( array( 123456, 234567, 345678 ) )
 * Multiple of different types can repeat the if statement
 *********
 * The return false removes the robots tag on the page
 * Or you can return index/noindex follow/nofollow like
 * return 'noindex,follow';
 * Or
 * return 'noindex,nofollow';
 */

add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_single' );

function yoast_seo_robots_remove_single( $robots ) {
  if ( is_single ( 123456 ) ) {
    return false;
  } else {
    return $robots;
  }
}

Thanks a lot, it's really useful for me....

where can i upload this php file?
BTW, thank you

Typically code snippets are added to your theme's functions.php file.

If you're not familiar with using code snippets, these articles have wonderful advice:
http://www.wpbeginner.com/plugins/how-to-easily-add-custom-code-in-wordpress-without-breaking-your-site/
http://www.wpbeginner.com/beginners-guide/beginners-guide-to-pasting-snippets-from-the-web-into-wordpress/

If I understood correctly, the first snippet removes noindex and nofollow from Yoast SEO entirely. But how can we change the noindex for “Special Pages” only, please?

I'd like to take out the noindex directive from the 404 pages, but retain the possibility to add noindex for taxonomies, for example.

Edit: I think the following snippet just did it (hopefully I didn't break anything):

add_filter( 'wpseo_robots', 'yoast_seo_robots_remove_404' );

function yoast_seo_robots_remove_404( $robots ) {
  if ( is_404() ) {
    return false;
  } else {
    return $robots;
  }
}