Tokyo Java Days 2018 notes

1. Keynote.md

Highlights

• long term roadmap for Java exists (Oracle is not reducing their investment despite shift in strategy to OSS and services)
• opening up developments
• cloud native recurring topic
• money from services
• new release model - same/better for devs, makes ent. pay
• making Java the premier choice for cloud native is recurring topic

Notable Projects:

• portola: containers (startup time, resource management)
• zgc - 10ms/sec, large heaps, EA available
• amber - language improvements (long term)
• panama - enhance jni
• valhala - value types, mem, layout

Serverless:

• oracle cloud runs containers on bare metal
• Fn automatically packages stuff in containers and deploys locally or in cloud - Linux pipeline IO (3 descriptors)
• Fn Flow is open source API to orchestrate serverless functions. Multiple languages. Not clear whether coupled to Fn.

links

• https://stackoverflow.com/questions/28327620/difference-between-java-options-java-tool-options-and-java-opts
• https://github.com/oracle/graal/tree/master/substratevm
• http://fnproject.io
• https://langserver.org
• http://www.issihosts.com/haveged/
• http://www.etalabs.net/compare_libcs.html
• https://www.musl-libc.org
• http://jqwik.net/ - PBT for Java WITH SHRINKING!

2. Java in a world of containers.md

• expecting sprawl (instance number/variability of hw&sw config)
• JVM runtime is highly adaptive to env
• Java 9+ slimmed down (jlinked) JREs are encouraged for appliances
  • smaller to download/store/start, as well as less attack service
  • typical serverside runtime down to ~50m+
• portola - minimum Linux+Java container image (Alpine+musl-libc)
  • image built on alpine linux - small, secure
  • JDK custom built with the musl libc
• application class data sharing (AppCDS)
  • libc, livjvm memory images get automatically shared by the host OS
  • class data sharing across containers - reduces footprint, speeds up startup classloading (20-50%)
• AOT - sharing jitted code
  • very experimental
• Docker features
  • JVM ergonomics - docker (cgroups&namespaces) aware (number of CPU used, scheduling, memory)
  • internal threadpools (compiler, GC, default fork/join pool, etc.)
  • https://mjg123.github.io/2018/01/10/Java-in-containers-jdk10.html
  • https://mjg123.github.io/
• serviceability (attach API, jcmd, jsa and metrics)
  • from outside containers/across containers
• Takeaways:
  • hello world container is 45mb
  • startup and footprint improved with CDS and AOT
  • ergonomics and serviceability are container aware

3. Running Java on Docker.md

• allocation: java heap -> linux eap -> vm heap -> host heap -> vmm -> hardware
• cgroups limits are not applicable to low-level APIs, so the JVM will not see the limit
• memory
  • docker will kill applications exceeding the cgroup limit, JVM does not even know
  • before Java9/Java8_131, need to add to JAVA_OPTS the cmd line and use docker run -e JAVA_OPTS=-Xmx123m --memory ...
• cpu
  • Runtime::availableProcessors always reports the host cpu count
  • JVM would over-commit threadpools
  • in docker use docker run -cpus=2 --cpuset-cpus-0,1 ...
  • alternatively ENV LD_PRELOAD=[hack] - exposes JVM_ActiveProcessorsCount(long) from env vars
  • critical to limit number of CPUs in multitenant machines
• SecureRandom is starved for entropy sources in a container (no keyboard)
  • /dev/random blocks in container/vm envs
  • /dev/urandom works
  • can change javax.security file in the JRE: securerandom.source=...
  • haveged - is a simple entropy daemon that can be used instead of urandom
• debugging inside container
  • docker run -p5005:5005 -e JAVA_OPTIONS='-Xdebug...' ...
• integration testing
  • docker-compose for app and another one for test harness
  • in integration
    • keep the data on ephemeral drives (no volume)
    • do not map ports externally, so we can run multiple tests

4. Java in serverless land.md

• eliminate infrastructure concerns, abstract as logical services (functions) -> short-lived, standardized interface
• in cloud (faas), on premises, local substrate
• event driven, stateless (external state)
• faas substrate manages resource allocation efficiently, considering num compute nodes, storage allocation, etc.
• #3 for serverless (behind Python and JS)
  • people start with languages that allow them to put up stuff quickly
  • but "when companies grow, they turn into Java shops" (James Governor)
• Grown up java development
  • simple Java (avoid invasive frameworks, emphasise interoperable API)
  • strong acceptance of Open Source (evaluating and selecting libraries is an important skill)
  • huge number of available tools and libs
  • complex long-lived applications
  • use the JVM effectively rather than reinventing existing primitives
• http://fnproject.io
  • same as the hosted version, developed in the open, ASL
  • backed by Oracle, also vailable as FAAS
  • container centric (each function is a container)
    • containers are most useful as sandboxes around individual process (Docker)
    • containers are not as good when used as VMs (LXC))
  • easily tested
  • standard POM + minimum function descriptor (yaml)
• fn-flow
  • declarative concurrency (async, scatter/gather, etc.)
  • declarative error-handling (retries, substitution, masking, etc.)
  • composition of functions, using an API (Java, JS, etc.)
  • Java API uses the CompletionSage API (another implementation is CompletableFutute)
• Needed
  • fast startup
    • lower Docker layers are shared in the disk cache, including shared libs (libs, libjava)
    • move startup activities to build time and cache results (CDS, AppCDS, AOT+JIT)
  • use smaller container images (thin Linux distribution, jlink, strip libraries with Proguard)
    • OS components - Portola (custom JVM11 build for Alpine) - not yet supported - needs adopters
    • JDK - jdeps, jlink
    • Application code - fewer and smaller libraries, sometimes copy/pasting code is justified (or go hi-tech with Proguard)
    • JVM - SubstrateVM is open source - embedded VM in static image, imposes closed world restrictions (no bytecode engineering or serialization)
  • Mind the ergonomics
    • Memory: Heap size, GC region sizes, JIT code cache size
    • CPU: assorted threadpool sizes (F/j, compile, GC), mostly depemding on Runtime.availableProcesses()
    • JDK8_131+ is sensing the cgroup limits

5. Graal - how to use it in real life.md

• better: inlining + more effective escape analysis (partial escape analysis)
• JEP295 AOT Compilation
• JEP317 Experimental Java-based JIT
• JDK9+ contains JVMCI (compiler interface) - required to plug Graal
• takes extra time to bootstrap compared to C1/C2 (which are written in native code)
• new host
  • yum install git
  • download and untar java
  • export $JAVA_HOME and add it to $PATH
  • if java --list-modules | grep jdk.internal.vm.ci ; then echo JVMCI present ; fi
• To run the tests
  • export JAVA_TOOL_OPTIONS="-XX:+UseParallelGC -Xmx512m -Xms512m" # reduce the effect of GC
  • export JAVA_TOOL_OPTIONS="$JAVA_TOOL_OPTIONS -XX:UnlockexperimentalVMOptions -XX:+EnableJVMCICompiler -XX:+UseJVMCICompiler"
  • -XX:+BootstrapJVMCI - warms up the Graal code before any client code until it is JITted
  • java -XX:+PrintFlagsFinal
• heap usage
  • Graal code is compiled with C1 (-XX:+CompileGraalWithC1Only - true by default), could be AOT-ed, but not yet
  • there is no heap isolation (yet), so Graal takes from the same heap as app
    • we need to account for Graal usage
    • most Graal usage is on startup, and then released
    • memory is used anyway - in the Java or in the native heap (if you give more Java heap for Graal, you are not using C2, so less native heap usage)
  • if Graal gets OOME, it would just stop compiling; if application gets OOME - it dies

6. Performance tuning with poor tools and cheap drinks.md

• JPDM Model
  • Actors - drive the application
    • inputs: bubble-sort is faster than quicksort for very small or almost sorted datasets
  • Application - drive the JVM
  • JVM - drive the HW
  • OS/HW (cpu, mem, disk, net, locks) - are being consumed
    • limited by capacity
• Goal: determine the dominating comsumer of system resources (App, JVM, OS, none)
  • none: liveliness issues (i.e. trying to connect to bad resource)
  • if sys>10%usr -> system profiling *stat, strace, etc.
  • usr==100% -> threaddump
  • usr<100% -> GC, algorithms -> GC dump, then live and collected heap, then profiling
  • alloc rate - rule of thumb below 1gb/sec, ideally below 300mb/sec
  • breakdown kernel, user, idle time (vmstat: us(er), sy(etem), wa(it))