Remove category and tag taxonomies. Useful if you're not using the blog functionality of WordPress, including the taxonomies. Be sure to uncomment the other line if you're using Genesis to avoid errors.

remove-default-taxonomies.php

<?php
/**
 *
 * Remove default taxonomies
 *
 * @link http://w4dev.com/wp/remove-taxonomy/
 *
 */
 
add_action( 'init', 'unregister_taxonomy');
//remove_action( 'genesis_entry_footer', 'genesis_post_meta' ); // uncomment this if you're using Genesis to avoid errors
function unregister_taxonomy(){
	global $wp_taxonomies;
	$taxonomies = array( 'category', 'post_tag' );
	foreach( $taxonomies as $taxonomy ) {
		if ( taxonomy_exists( $taxonomy) )
			unset( $wp_taxonomies[$taxonomy]);
	}
}

I'm finding that this works better:

function unregister_taxonomy(){
    register_taxonomy('post_tag', array());
    register_taxonomy('category', array());
}
add_action('init', 'unregister_taxonomy');

With the earlier code, when including the 'category' in the array, I was losing the ability to paginate posts in WP 4.0. Specifically, previous_post_link() and next_post_link() stopped working altogether.

Weird long-term follow-up, but I had a similar issue with being unable to delete Media Items in 6.2.x because wp_delete_object_term_relationships( $post_id, array( 'category', 'post_tag' ) ); is hardcoded in wp-includes/post.php and causes an error.

So the register_taxonomy('', nobody) solution proposed in the first comment is the way to go.