Testing for compiler errors with untyped macros.

noncompilation.scala

scala> import scala.language.experimental.macros
import scala.language.experimental.macros

scala> import scala.reflect.macros.{ Context, TypecheckException }
import scala.reflect.macros.{Context, TypecheckException}

scala> object NoncompilationTests {
     |   def compiles(code: _): Boolean = macro compiles_impl
     |   def compiles_impl(c: Context)(code: c.Tree) = c.literal(
     |     try {
     |       c typeCheck code
     |       true
     |     } catch {
     |       case _: TypecheckException => false
     |     }
     |   )
     | }
defined object NoncompilationTests

scala> import shapeless._
import shapeless._

scala> NoncompilationTests compiles HNil.length
res0: Boolean = true

scala> NoncompilationTests compiles HNil.head
res1: Boolean = false

I should note that this is my implementation of a suggestion by Miles Sabin in response to this question.

related: https://github.com/lrytz/efftp/blob/master/src/main/scala/scala/tools/nsc/effects/testing/TestMacros.scala

Anybody looking at this code should keep in mind that it relies on an experimental macro-paradise feature—untyped macros—that never made it into any released version of Scala (as noted by Miles Sabin). The best alternative nowadays is compile-checking code fragments represented as free-form strings. Test frameworks generally include this functionality, e.g. http://www.scalatest.org/user_guide/using_matchers#checkingThatCodeDoesNotCompile or
https://etorreborre.github.io/specs2/guide/SPECS2-3.6/org.specs2.guide.Matchers.html#typecheck-matchers