Skip to content

Instantly share code, notes, and snippets.

@tabdulradi
Last active May 14, 2021 13:24
Show Gist options
  • Save tabdulradi/da4977d006689f775ee9a9dc3d1a8bdd to your computer and use it in GitHub Desktop.
Save tabdulradi/da4977d006689f775ee9a9dc3d1a8bdd to your computer and use it in GitHub Desktop.
Scala Script Runner

Scala Script Runner

Make sure you have coursier installed

chmod +x test.scala
./test.scala

First time it will download scala-compiler (and depending on your config maybe GraalVM) then cache a bootstrap runner. Next time it will run much faster using the cached runner (depending on you config can be native program!).

Understand the shebang

First line of test.scala contains all the dependencies the scala program needs followed by flags passed as is to coursier bootstrap.

  • --manifest-jar for JVM-based executable
  • --native-image for native executable (use GraalVM native image, make sure you have it installed)
  • --native for native executable (use Scala native, make sure you have it installed)
#!/usr/bin/env python3
import sys
import hashlib
import os
import subprocess
from datetime import datetime
import itertools
filename = sys.argv[-1]
with open(filename, 'r') as f:
all_code = f.read()
content_hash = hashlib.sha256(all_code.encode()).hexdigest()
scala_code = all_code[1:]
cache_path = os.path.expanduser(f"~/.ivy2/local/com.abdulradi.scala-scripts-launcher/generated/{content_hash}")
launcher_path = f'{cache_path}/launcher'
if not os.path.exists(launcher_path):
dependencies = list(itertools.takewhile(lambda x: not x.startswith('-'), sys.argv[1:-1]))
bootstrap_flags = ' '.join(itertools.dropwhile(lambda x: not x.startswith('-'), sys.argv[1:-1]))
now = datetime.now().timestamp()
os.makedirs(f'{cache_path}/jars', exist_ok=True)
os.makedirs(f'{cache_path}/ivys', exist_ok=True)
# Copy of scala code with shebang commented out as it confuses scalac. TODO: Find a better place for this file
scalacode_path = f'{cache_path}/jars/generated.scala'
with open(scalacode_path, "w") as f:
f.write(f'//{all_code}')
jar_path = f'{cache_path}/jars/generated.jar'
class_path = subprocess.check_output(['coursier', 'fetch', '--classpath'] + dependencies).decode('utf-8')
os.system(f'coursier launch scalac -- {scalacode_path} -d {jar_path} -classpath {class_path} ')
scala_version = subprocess.check_output(['coursier', 'launch', 'scalac', '--', '-version']).decode('utf-8')[23:-55] # Drop the copyrights and extract version number, probably very fragile and might break with dotty 
[scala_version_major, scala_version_minor, scala_version_patch] = scala_version.split('.')
with open(f'{cache_path}/ivys/ivy.xml', "w") as f:
f.write('<?xml version="1.0" encoding="UTF-8"?>\n')
f.write('<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">\n')
f.write(f' <info organisation="com.abdulradi.scala-scripts-launcher" module="generated" revision="{content_hash}" status="integration" publication="{now}"></info>\n')
f.write(' <configurations>\n')
f.write(' <conf name="compile" visibility="public" description=""/>\n')
f.write(' <conf name="optional" visibility="public" description=""/>\n')
f.write(' <conf name="scala-tool" visibility="private" description=""/>\n')
f.write(' </configurations>\n')
f.write(' <dependencies>\n')
f.write(f' <dependency org="org.scala-lang" name="scala-compiler" rev="{scala_version}" conf="scala-tool->default,optional(default)"/>\n')
f.write(f' <dependency org="org.scala-lang" name="scala-library" rev="{scala_version}" conf="scala-tool->default,optional(default);compile->default(compile)"/>\n')
for dependency in dependencies:
if '::' in dependency:
[org, name_rev] = dependency.split('::')
[name_unversioned, rev] = name_rev.split(':')
name = f'{name_unversioned}_{scala_version_major}.{scala_version_minor}'
else:
[org, name, rev] = dependency.split(':')
f.write(f' <dependency org="{org}" name="{name}" rev="{rev}" conf="compile->default(compile)"/>\n')
f.write(' </dependencies>\n')
f.write('</ivy-module>\n')
os.system(f'coursier bootstrap com.abdulradi.scala-scripts-launcher:generated:{content_hash} {bootstrap_flags} -o {launcher_path} --quiet')
os.system(launcher_path)
#!/usr/bin/env python3 scala-script-launcher.py io.circe::circe-generic:0.12.3 io.circe::circe-parser:0.12.3 --manifest-jar
import io.circe._, io.circe.generic.auto._, io.circe.parser._, io.circe.syntax._
sealed trait Foo
case class Bar(xs: Vector[String]) extends Foo
case class Qux(i: Int, d: Option[Double]) extends Foo
object Main {
def main(args: Array[String]): Unit = {
val foo: Foo = Qux(1, Some(14.0))
val json = foo.asJson.noSpaces
println(json)
val decodedFoo = decode[Foo](json)
println(decodedFoo)
}
}
@przemek-pokrywka
Copy link

@tabdulradi, have you had any success with compiling your example script to a native image? I've followed the steps you gave and unfortunately GraalVM was "Aborting stand-alone image build due to unsupported features" (also I needed to pass -S option to the /usr/bin/env program on Linux).

I added support for GraalVM to TSK recently and I tried to turn your example script into a native binary (by saving, chmodding +x and running the following):

// 2> /dev/null \
/*
dependencies='
  io.circe::circe-generic:0.12.3
  io.circe::circe-parser:0.12.3
'
native=true
source $( curl -L git.io/boot-tsk | sh ); run
*/

import io.circe._, io.circe.generic.auto._, io.circe.parser._, io.circe.syntax._

sealed trait Foo
case class Bar(xs: Vector[String]) extends Foo
case class Qux(i: Int, d: Option[Double]) extends Foo

object Main {
  def main(args: Array[String]): Unit = {
    val foo: Foo = Qux(1, Some(14.0))
    val json = foo.asJson.noSpaces
    println(json)
    val decodedFoo = decode[Foo](json)
    println(decodedFoo)
  }
}

but I encountered the same issues. I guess that one needs to come up with some extra GraalVM-specific descriptors that mention all uses of reflection and class loading for the ahead-of-time compilation to work.

I've recently read https://www.lightbend.com/blog/writing-kubectl-plugins-with-scala-or-java-with-fabric8-kubernetes-client-on-graalvm where Andrea mentions use of GraalVM JVM agent library in order to generate these descriptors.
Due to use of a custom "alternative main" (something that exercises the likely-problematic features of a program) this method may be not appropriate for cs bootstrap and/or for scripts, but I'm curious of your thoughts @tabdulradi, @alexarchambault

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment