gistfile1.txt

Proposal: Default compiler/interpreter classpath in a managed environment

The compiler uses a default classpath of 'java.class.path'.  Because many
applications do not use custom class loaders, this default is often sufficient
and no classpath configuration is required.  This is because all jars- the
application and its dependencies, including the Scala compiler and library- are
in 'java.class.path' and the class loader for the application is the same as
that for Scala.

However, it does not work in more complicated setups, including various managed
environments.  People running in web containers, for example, need to explicitly
set the classpath for the jars they want to compile/interpret against.

Another case is sbt.  sbt runs, tests, and drops to the REPL in the same jvm as
sbt itself.  This means "java.class.path" is just sbt-launcher.jar and the
default compiler classpath does not work.  Additionally, the Scala class loader
is an ancestor of the application class loader.  The Interpreter.breakIf method
won't work at all without duplicating the entire method, since it only uses the
default classpath and the class loader used is the one used to load the
Interpreter class.  This class loader does not have access to the application
classes.

This proposal aims to make it possible to provide a default classpath to the
compiler with more granularity than a system property and to be able to provide
a different class loader to 'breakIf' and the REPL without much trouble.

The proposed approach is for the managed environment to put two resources on the
classpath: 'app.class.path' and 'boot.class.path'.  (These names could be
modified to reduce possible collisions with application resources.)

The compiler/interpreter reads in these resources and uses these for the
classpath and boot classpath when they are not explicitly specified.  These
would take precedence over java.class.path.

One way to implement this would be to add a method 'embeddedDefaults' to
settings.  It would take a type parameter T with an implicit Manifest[T].  This
would be used to grab a ClassLoader.  This loader would be searched for the
resources 'app.class.path' and 'boot.class.path'.  In this way, the default
values can be specified per ClassLoader.  Usage by the user looks like:

    val settings = new Settings()
    settings.embeddedDefaults[MyType]

Settings:
  def embeddedDefaults[T: Manifest] {
    val loader = implicitly[Manifest[T]].erasure.getClassLoader
    explicitParentLoader = Some(loader) // for the Interpreter parentClassLoader
    getClasspath("app", loader) foreach { classpath.value = _ }
    getClasspath("boot", loader) foreach {
       bootclasspath.value = settings.bootclasspath.value + File.separator + _
    }
  }
  private def getClasspath(id: String, loader: ClassLoader) =
      Option(loader.getResource(id + ".class.path")).map { cp =>
         Source.fromURL(cp).mkString
      }

Interpreter
  protected def parentClassLoader = settings.explicitParentLoader.getOrElse(
this.getClass.getClassLoader )

breakIf needs to be able to take a class loader for it to work in an embedded
environment:
  def breakIf[T: Manifest](assertion: => Boolean, args: DebugParam[_]*) = ...

and then use settings.embeddedDefaults[T] in the implementation.

For normal interpreter usage, something similar could be done or the user could
be required to do it manually.

Mark -- I'm open to something like this but I'm pretty much done with 2.8, certainly with "interesting" changes. Please ping me after 2.8 is shipped and/or greenhouse is open and we'll work something out.

Sounds good. Thanks for taking a look.

Good idea. We need this to contain scripts executed inside applications. I'd love to have it for the scripster.

sbt implementation and test for manual classpath configuration committed in:
http://github.com/harrah/xsbt/commit/600053bf2cc

Hey Mark - this almost works for me. I modified my project to use 0.7.5-SNAPSHOT in their build.properties (initial cmdline still starts with 0.7.4) and use the client code you have checked in...I only have one problem: I changed the scala.tools.nsc.Interpreter (which is in my project) and cannot seem to convince the sbt run or test to pick it up, it keeps referencing the old Interpreter and complains my new method is not found, even after this:

settings.classpath.value = "/home/razvanc/wsideaj/scripster/target/scala_2.8.0/classes:"+classpath("app", loader).getOrElse(error("Error: could not find application classpath"))
settings.bootclasspath.value = "/home/razvanc/wsideaj/scripster/target/scala_2.8.0/classes:" + settings.bootclasspath.value + / + classpath("boot", loader).getOrElse(error("Error: could not find boot classpath")) 

thanks

I'm very glad you guys are working this because I have no time right now. If any revisions need to take place we need to find them ASAP, because in a few days 2.8.1 will freeze and after that it's anyone's guess.

Yeah - after we make it work, you're saying that we should extract a generic/reusable class from Mark's code base and include that in 2.8.1?

I'm saying make sure you're completely happy with https://lampsvn.epfl.ch/trac/scala/changeset/22949 because in the absence of prompt intervention that's what will be in 2.8.1 and it will be months best case before it can be changed.

looks fine - I will try to test it directly. It would be nice if we'd provide an util class to set those app. and boot. resources, for use when inside J2EE or OSGI or other containers...but we can always point to this gist...I would do it, but I'm not the best person to mess with classpath (getting the classpath in different environments).

The use case I'm having trouble with here is not really normal - it's a "hack" of the Interpreter...which could also go away if you could include the PublicRequest class in 2.8.1: http://gist.github.com/570783

pretty please :)

Ok. the change is working. I tested with 2.8.1-SNAPSHOT and the following creation sequence:

// make a custom parser
def mkParser (errLogger: String => Unit) = {
if (SS.getClass.getClassLoader.getResource("app.class.path") != null) {
val settings = new Settings(errLogger)
settings embeddedDefaults getClass.getClassLoader
println (">>>>>>>>>>>>>>" + settings.classpath.value)
println (">>>>>>>>>>>>>>" + settings.bootclasspath.value)

  val p = new Interpreter (settings) {
    override protected def parentClassLoader = SS.getClass.getClassLoader // SS is just an object in scope
    }
  p.setContextClassLoader
  p
} else {
  val env = new nsc.Settings(errLogger)
  env.usejavacp.value = true

  val p = new Interpreter (env)
  p.setContextClassLoader
  p
}

}

This works for me even without overriding the parentClassLoader and without setting the context class loader, although the latter might be for something else you are doing.

It would be nice if you could set usejavacp and embeddedDefaults and have embeddedDefaults take precedence when 'app.class.path' exists, but that would be too invasive. So, I'm happy with the changeset.