Skip to content

Instantly share code, notes, and snippets.

./build/mvn install ${MVN_OPT} -pl externals/kyuubi-spark-sql-engine -Pspark-3.2 -Dsuites=org.apache.kyuubi.engine.spark.operation.SparkOperationSuite
[INFO] --- scalatest-maven-plugin:2.2.0:test (test) @ kyuubi-spark-sql-engine_2.12 ---
[INFO] ScalaTest report directory: /Users/lxm/projects/kyuubi/externals/kyuubi-spark-sql-engine/target/surefire-reports
Run starting. Expected test count is: 62
SparkOperationSuite:
ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.9.3
ANTLR Runtime version 4.8 used for parser compilation does not match the current runtime version 4.9.3
ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.9.3

现在想做到的事,在主线程中使用 com.xli.Target version 1.0 在子线程中使用 com.xli.Target version 1.1

已知条件:

  1. CLASSPATH 里的 com.xli.Target 是版本1.0
  2. 有一个自定义的 classloader 用于加载 com.xli.Target 版本1.1
  • 继承自URLClassloader
  • 会优先加载自己URL里的path
URL url = new URL("file:///tmp/TargetClass-1.1.jar");
$ export MVN_OPT="-Dmaven.javadoc.skip=true -Drat.skip=true -Dscalastyle.skip=true -Dspotless.check.skip -Dorg.slf4j.simpleLogger.defaultLogLevel=info -Pjdbc-shaded -Dmaven.plugin.download.cache.path=/tmp/engine-archives"
$ ./build/mvn clean install ${MVN_OPT} -pl kyuubi-server -Pspark-3.2 -Dsuites=org.apache.kyuubi.operation.KyuubiOperationPerUserSuite > ~/mvn.log
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------< org.apache.kyuubi:kyuubi-server_2.12 >----------------
[INFO] Building Kyuubi Project Server 1.8.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ kyuubi-server_2.12 ---
@risyomei
risyomei / SparkPairedRDD.scala
Last active February 23, 2022 18:24
Dataframe Join
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions._
object SparkPairedRDD {
def main(args: Array[String]) {
val spark = SparkSession
.builder()
.appName("Spark Paired")
let timeBox = document.getElementsByClassName('c-timestamp');
let i = 0;
while(i < timeBox.length) {
    var date = new Date( timeBox[i].getAttribute('data-ts') * 1000);
    var dateFormate = date.getFullYear() + '-' + (date.getMonth()+1) + '-' + date.getDate();
    dateFormate = dateFormate + '-' + date.getHours() + ':' + date.getMinutes() + ':' + date.getSeconds();   
    var currentText = timeBox[i].getElementsByClassName('c-timestamp__label')[0].innerHTML;
    if (! currentText.includes(dateFormate)){
 timeBox[i].getElementsByClassName('c-timestamp__label')[0].innerHTML = currentText + ' (' + dateFormate + ')';
(base) sri@MBP-15UAX-030:~/projects/hadoop-mirror(HDFS-15506○) » mvn process-sources javadoc:javadoc-no-fork -pl hadoop-hdfs-project/hadoop-hdfs
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.15
[INFO] os.detected.version.major: 10
(base) sri@MBP-15UAX-030:~/projects/hadoop-mirror(HDFS-15507○) » mvn process-sources javadoc:javadoc-no-fork -pl hadoop-hdfs-project/hadoop-hdfs-client
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.15
[INFO] os.detected.version.major: 10
FILENAME=test.txt
MSG=$((ls $FILENAME) 2>&1);
ret=$?
re="name=(.*)"
if [ $ret -ne 0 ]; then
# 文件没有找到
echo $MSG >> err.log
echo "Error message written in err.log"
else
@risyomei
risyomei / karabiner.json
Created August 27, 2019 02:00
Karabiner Keyboard Setting
{
"global": {
"check_for_updates_on_startup": false,
"show_in_menu_bar": true,
"show_profile_name_in_menu_bar": false
},
"profiles": [
{
"complex_modifications": {
"parameters": {
@risyomei
risyomei / karabiner.json
Created August 27, 2019 02:00
Karabiner Keyboard Setting
{
"global": {
"check_for_updates_on_startup": false,
"show_in_menu_bar": true,
"show_profile_name_in_menu_bar": false
},
"profiles": [
{
"complex_modifications": {
"parameters": {