Skip to content

Instantly share code, notes, and snippets.

@louismrose
Last active August 29, 2015 14:02
Show Gist options
  • Save louismrose/11849546efd8cf496fc2 to your computer and use it in GitHub Desktop.
Save louismrose/11849546efd8cf496fc2 to your computer and use it in GitHub Desktop.
Using mutation testing to detect bugs involving exceptions?
public class Palindrome {
// This method checks whether its input reads the same forwards
// as backwards. For example, isPalindrome("noon") should return
// true, but isPalindrome("night") should return false.
// Note that this implementation contains a deliberate bug.
public static boolean isPalindrome(String s) {
if (s.length() == 0) {
return true;
} else {
char first = s.charAt(0);
char last = s.charAt(s.length() - 1);
// Here's the bug... Imagine s is a 1-character string,
// then the assignment evaluates to:
// s.substring(1, 1-1) =>
// s.substring(1, 0) =>
// StringIndexOutOfBoundsException
String mid = s.substring(1, s.length() - 1);
return first == last && isPalindrome(mid);
}
}
}
class Palindrome
# This method checks whether its input reads the same forwards
# as backwards. For example, isPalindrome("noon") should return
# true, but isPalindrome("night") should return false.
#
# Note that this implementation contains a deliberate bug.
def is?(s)
return true if s.size == 0
first, last = s[0], s[s.size-1]
# Here's the bug... Imagine s is a 1-character string,
# then the assignment evaluates to:
# s[1, -1] =>
# nil =>
# ... causes the s.size == 0 check to fail with NoMethodError
mid = s[1,s.size-2]
first == last && is?(mid)
end
end
require "palindrome"
require "rspec"
describe Palindrome do
# The following 3 tests seem to be sufficient
# for 100% mutation coverage with mutant
# (I'm using mutant-0.5.22 and the following command
# mutant --include lib --require palindrome --use rspec Palindrome)
it "detects a palindrome" do
expect(subject.is?("noon")).to be_truthy
end
it "detects a non-palindrome" do
expect(subject.is?("midnight")).to be_falsey
end
it "detects not quite palindrome" do
expect(subject.is?("neon")).to be_falsey
end
# This additional test is needed to kill the mutant
# mid = s[1,2]
it "detects a longer palindrome" do
expect(subject.is?("hannah")).to be_truthy
end
# However the following test is needed to reveal the bug
# Is there a mutant that would cause us to discover the need
# for this test?
# it "detects odd length palindrome" do
# expect(subject.is?("radar")).to be_true
# end
# Additonally, to achieve 100% path coverage, I think that
# we'd also need the following test case. Is there a mutant
# that would cause us to discover the need for this case?
# it "detects empty string palindrome" do
# expect(subject.is?("")).to be_true
# end
end
import static org.junit.Assert.*;
import static palindrome.Palindrome.isPalindrome;
import org.junit.Test;
public class PalindromeTests {
// The following 3 tests seem to be sufficient
// for 100% mutation coverage with PIT
// (I'm using PITclipse v0.33.2.201404202230)
@Test
public void detectsPalindrome() {
assertTrue(isPalindrome("noon"));
}
@Test
public void detectsNonPalindrome() {
assertFalse(isPalindrome("midnight"));
}
@Test
public void detectsNotQuitePalindrome() {
assertFalse(isPalindrome("neon"));
}
// However the following test is needed to reveal the bug
// Is there a mutant that would cause us to discover the need
// for this test?
// @Test
// public void detectsOddPalindrome() {
// assertTrue(isPalindrome("eve"));
// }
// Additonally, to achieve 100% path coverage, I think that
// we'd also need the following test case. Is there a mutant
// that would cause us to discover the need for this case?
// @Test
// public void detectsEmptyStringPalindrome() {
// assertTrue(isPalindrome(""));
// }
}
@omgm
Copy link

omgm commented Jul 11, 2014

This requires a higher order mutant. I ran Mutator 1.7 (http://ortask.com/mutator/) to create a second order mutant and was able to easily find a live mutant that surfaces the need for the missing test. Here's the mutant:

public class Palindrome {
    public static boolean isPalindrome(String s) {
        if (s.length() == 0) {
            return true;

        } else {
            char first = s.charAt(1);               // <--- originally 0. \
                                                    //                     > a higher order mutant.
            char last = s.charAt(s.length() - 2);   // <--- originally 1. /

            String mid = s.substring(1, s.length() - 1);

            return first == last && isPalindrome(mid);
        }
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment