build.sbt

name := "plainsbt"

version := "0.1-SNAPSHOT"

libraryDependencies ++= Seq(
  "org.specs2" %% "specs2" % "2.1.1",
  "com.novocode" % "junit-interface" % "0.10" % "test" exclude("junit", "junit-dep"),
  "junit" % "junit" % "4.11" % "test"
)

testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "--ignore-runners=org.specs2.runner.JUnitRunner")

fork in Test := true

MySpec.scala

import org.specs2.mutable.Specification
import org.specs2.runner.JUnitRunner
import org.junit.runner.RunWith

@RunWith(classOf[JUnitRunner])
class MySpec extends Specification {
  "The number 1" should {
    "equal the number 2" in {
      1 must_== 2
    }
  }
}

Output when running tests

$ sbt
[info] Loading global plugins from /Users/rich/.sbt/0.13/plugins
[info] Set current project to plainsbt (in build file:/p/play/bugs/1798/plainsbt/)

> test-only MySpec
[info] MySpec
[info] The number 1 should
[info] x equal the number 2
[error]    '1' is not equal to '2' (MySpec.scala:9)
[info] Total for specification MySpec
[info] Finished in 14 ms
[info] 1 example, 1 failure, 0 error
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[success] Total time: 2 s, completed 22/10/2013 4:48:11 PM

^^^^ should be [error], notice also that no tests are recorded as failed/errors/passed in the line above

> last
[debug] Running task... Cancelable: false, check cycles: false
[debug] 
[debug] Initial source changes: 
[debug] 	removed:Set()
[debug] 	added: Set()
[debug] 	modified: Set()
[debug] Removed products: Set()
[debug] Modified external sources: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated sources: Set()
[debug] 
[debug] Sources indirectly invalidated by:
[debug] 	product: Set()
[debug] 	binary dep: Set()
[debug] 	external source: Set()
[debug] All initially invalidated sources: Set()
[debug] Copy resource mappings: 
[debug] 	
[debug] 
[debug] Initial source changes: 
[debug] 	removed:Set()
[debug] 	added: Set()
[debug] 	modified: Set()
[debug] Removed products: Set()
[debug] Modified external sources: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated sources: Set()
[debug] 
[debug] Sources indirectly invalidated by:
[debug] 	product: Set()
[debug] 	binary dep: Set()
[debug] 	external source: Set()
[debug] All initially invalidated sources: Set()
[debug] Copy resource mappings: 
[debug] 	
[debug] Framework implementation 'org.scalacheck.ScalaCheckFramework' not present.
[debug] Framework implementation 'org.specs2.runner.Specs2Framework' not present.
[debug] Framework implementation 'org.specs.runner.SpecsFramework' not present.
[debug] Framework implementation 'org.scalatest.tools.Framework' not present.
[debug] Framework implementation 'org.scalatest.tools.ScalaTestFramework' not present.
[debug] Subclass fingerprints: Stream((org.specs2.specification.SpecificationStructure,false,sbt.SubclassFingerprintWrapper@7240d0e9), ?)
[debug] Annotation fingerprints: Stream((org.junit.Test,false,sbt.AnnotatedFingerprintWrapper@77b874ea), ?)
[debug] Framework implementation 'org.specs2.runner.Specs2Framework' not present.
[info] MySpec
[info] The number 1 should
[info] x equal the number 2
[error]    '1' is not equal to '2' (MySpec.scala:9)
[info] Total for specification MySpec
[info] Finished in 14 ms
[info] 1 example, 1 failure, 0 error
[debug] Summary for specs2 not available.
[debug] Summary for JUnit not available.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[debug] Passed tests:
[debug] 	MySpec
[success] Total time: 2 s, completed 22/10/2013 4:48:11 PM
> 

Output when working properly

If I remove `testOptions`, `fork` or `@RunWith` everything works. The output below shows what happens when I remove all of them.

$ sbt
[info] Loading global plugins from /Users/rich/.sbt/0.13/plugins
[info] Set current project to plainsbt (in build file:/p/play/bugs/1798/plainsbt/)

> test-only MySpec
[info] Compiling 1 Scala source to /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes...
[info] MySpec
[info] The number 1 should
[info] x equal the number 2
[error]    '1' is not equal to '2' (MySpec.scala:9)
[info] Total for specification MySpec
[info] Finished in 16 ms
[info] 1 example, 1 failure, 0 error
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] 	MySpec
[error] (test:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 6 s, completed 22/10/2013 4:50:57 PM

^^^^ this is [error] as expected

> last
[debug] Running task... Cancelable: false, check cycles: false
[debug] 
[debug] Initial source changes: 
[debug] 	removed:Set()
[debug] 	added: Set()
[debug] 	modified: Set()
[debug] Removed products: Set()
[debug] Modified external sources: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated sources: Set()
[debug] 
[debug] Sources indirectly invalidated by:
[debug] 	product: Set()
[debug] 	binary dep: Set()
[debug] 	external source: Set()
[debug] All initially invalidated sources: Set()
[debug] Copy resource mappings: 
[debug] 	
[debug] 
[debug] Initial source changes: 
[debug] 	removed:Set()
[debug] 	added: Set()
[debug] 	modified: Set(/p/play/bugs/1798/plainsbt/src/test/scala/MySpec.scala)
[debug] Removed products: Set(/p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes/MySpec.class, /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes/MySpec$$anonfun$1$$anonfun$apply$3$$anonfun$apply$2.class, /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes/MySpec$$anonfun$1$$anonfun$apply$3.class, /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes/MySpec$$anonfun$1.class, /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes/MySpec$$anonfun$1$$anonfun$apply$3$$anonfun$apply$1.class)
[debug] Modified external sources: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated sources: Set(/p/play/bugs/1798/plainsbt/src/test/scala/MySpec.scala)
[debug] 
[debug] Sources indirectly invalidated by:
[debug] 	product: Set(/p/play/bugs/1798/plainsbt/src/test/scala/MySpec.scala)
[debug] 	binary dep: Set()
[debug] 	external source: Set()
[debug] All initially invalidated sources: Set(/p/play/bugs/1798/plainsbt/src/test/scala/MySpec.scala)
[debug] Recompiling all 1 sources: invalidated sources (1) exceeded 50.0% of all sources
[info] Compiling 1 Scala source to /p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes...
[debug] Getting compiler-interface from component compiler for Scala 2.10.2
[debug] Getting compiler-interface from component compiler for Scala 2.10.2
[debug] Running cached compiler 1aeb7308, interfacing (CompilerInterface) with Scala compiler version 2.10.2
[debug] Calling Scala compiler with arguments  (CompilerInterface):
[debug] 	-bootclasspath
[debug] 	/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/lib/JObjC.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre/classes:/Users/rich/.sbt/boot/scala-2.10.2/lib/scala-library.jar
[debug] 	-classpath
[debug] 	/p/play/bugs/1798/plainsbt/target/scala-2.10/test-classes:/p/play/bugs/1798/plainsbt/target/scala-2.10/classes:/Users/rich/.ivy2/cache/org.specs2/specs2_2.10/jars/specs2_2.10-2.1.1.jar:/Users/rich/.ivy2/cache/org.scalaz/scalaz-core_2.10/bundles/scalaz-core_2.10-7.0.2.jar:/Users/rich/.ivy2/cache/org.scalaz/scalaz-concurrent_2.10/bundles/scalaz-concurrent_2.10-7.0.2.jar:/Users/rich/.ivy2/cache/org.scalaz/scalaz-effect_2.10/bundles/scalaz-effect_2.10-7.0.2.jar:/Users/rich/.ivy2/cache/com.novocode/junit-interface/jars/junit-interface-0.10.jar:/Users/rich/.ivy2/cache/org.scala-tools.testing/test-interface/jars/test-interface-0.5.jar:/Users/rich/.ivy2/cache/junit/junit/jars/junit-4.11.jar:/Users/rich/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.3.jar
[debug] Scala compilation took 4.619985 s
[debug] Invalidated by transitive public inheritance: Set(/p/play/bugs/1798/plainsbt/src/test/scala/MySpec.scala)
[debug] Invalidated by direct dependency: Set()
[debug] New invalidations:
[debug] 	Set()
[debug] Previously invalidated, but (transitively) depend on new invalidations:
[debug] 	Set()
[debug] Copy resource mappings: 
[debug] 	
[debug] Framework implementation 'org.scalacheck.ScalaCheckFramework' not present.
[debug] Framework implementation 'org.specs2.runner.Specs2Framework' not present.
[debug] Framework implementation 'org.specs.runner.SpecsFramework' not present.
[debug] Framework implementation 'org.scalatest.tools.Framework' not present.
[debug] Framework implementation 'org.scalatest.tools.ScalaTestFramework' not present.
[debug] Subclass fingerprints: Stream((org.specs2.specification.SpecificationStructure,false,sbt.SubclassFingerprintWrapper@4aa3a403), ?)
[debug] Annotation fingerprints: Stream((org.junit.Test,false,sbt.AnnotatedFingerprintWrapper@39ae5b4e), ?)
[debug] Running TaskDef(MySpec, sbt.SubclassFingerprintWrapper@4aa3a403, false, [SuiteSelector])
[info] MySpec
[info] The number 1 should
[info] x equal the number 2
[error]    '1' is not equal to '2' (MySpec.scala:9)
[info] Total for specification MySpec
[info] Finished in 16 ms
[info] 1 example, 1 failure, 0 error
[debug] Summary for specs2 not available.
[debug] Summary for JUnit not available.
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] 	MySpec
[error] (test:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 6 s, completed 22/10/2013 4:50:57 PM
>