Skip to content

Instantly share code, notes, and snippets.

@edsu
Created March 20, 2019 21:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save edsu/475ab8e2f3307a5854faaf7fd06a1c02 to your computer and use it in GitHub Desktop.
Save edsu/475ab8e2f3307a5854faaf7fd06a1c02 to your computer and use it in GitHub Desktop.
[edsu@r001 ~]$ spark-shell --packages "io.archivesunleashed:aut:0.17.0"Ivy Default Cache set to: /home/edsu/.ivy2/cache
The jars for the packages stored in: /home/edsu/.ivy2/jars
:: loading settings :: url = jar:file:/home/edsu/.local/lib/python2.7/site-packages/pyspark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
io.archivesunleashed#aut added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-2ae7a372-bba6-482d-8fff-910ec07724ab;1.0
confs: [default]
found io.archivesunleashed#aut;0.17.0 in local-m2-cache
:: resolution report :: resolve 234ms :: artifacts dl 13ms
:: modules in use:
io.archivesunleashed#aut;0.17.0 from local-m2-cache in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 1 | 0 | 0 | 0 || 1 | 0 |
---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-2ae7a372-bba6-482d-8fff-910ec07724ab
confs: [default]
0 artifacts copied, 1 already retrieved (0kB/8ms)
2019-03-20 17:37:59 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://r001.pvt.bridges.psc.edu:4040
Spark context available as 'sc' (master = local[*], app id = local-1553117895111).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.3.1
/_/
Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_161)
Type in expressions to have them evaluated.
Type :help for more information.
scala> import io.archivesunleashed._
import io.archivesunleashed._
scala> import io.archivesunleashed.matchbox._
import io.archivesunleashed.matchbox._
scala> val r = RecordLoader.loadArchives("example.arc.gz", sc)
java.lang.NoClassDefFoundError: net/sourceforge/cobertura/coveragedata/TouchCollector
at io.archivesunleashed.package$RecordLoader$.__cobertura_init(package.scala)
at io.archivesunleashed.package$RecordLoader$.<clinit>(package.scala)
... 53 elided
Caused by: java.lang.ClassNotFoundException: net.sourceforge.cobertura.coveragedata.TouchCollector
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 55 more
scala>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment