Shared indexes with Coder
TLDR: Shared indexes work with Coder + JetBrains Projector. To quickly prototype your project with shared indexes, you can generate them and host them as a dev URL on your workspace, for other workspaces to try.
After testing, you'll need to upload the contents of /home/coder/indexes/generate-output to a file server and use that as the index location, or include them in the Coder image's filesystem. Dev URLs are used simply as a test. You'll need to regenerate the metadata with
cdn-layout-tool with the proper destination as well.
Generating a shared index in a Coder workspace:
Sourced from this tutorial: https://www.jetbrains.com/help/go/shared-indexes.html#configure-index-update
# For testing, we'll use a Dev URL as the "file server" for the indexes # Change stable.cdr.dev to your Coder deployment # This will not work unless your deployment has public dev URLs enabled INDEX_SERVER_URL=https://9999-$CODER_ENVIRONMENT_NAME-$CODER_USERNAME.stable.cdr.dev # Edit these with information relevant to your project REPO_URL=https://github.com/kubernetes/kubernetes.git PROJECT_NAME=kubernetes IDE=goland # Create public dev url for port 9999 coder urls create $CODER_ENVIRONMENT_NAME 9999 --access public # Clone a project git clone $REPO_URL $HOME/$PROJECT_NAME # Set up folders mkdir -p /home/coder/indexes mkdir $HOME/indexes/ide-config mkdir $HOME/indexes/ide-log mkdir $HOME/indexes/ide-system # Create shared index $IDE dump-shared-index project --output=$HOME/indexes/generate-output --tmp=$HOME/indexes/temp --project-dir=$HOME/$PROJECT_NAME --project-id=$PROJECT_NAME # Requires Java installed on the workspace # Install, if you don't have it in the image sudo apt-get install -y openjdk-11-jre # Use the cdn-layout-tool for generating metadata wget https://packages.jetbrains.team/maven/p/ij/intellij-shared-indexes-public/com/jetbrains/intellij/indexing/shared/cdn-layout-tool/0.8.60/cdn-layout-tool-0.8.60.zip sudo apt-get install -y unzip # install unzip if not present in workspace unzip cdn-layout-tool-0.8.60.zip cdn-layout-tool-0.8.60/bin/cdn-layout-tool --url=$INDEX_SERVER_URL --indexes-dir=$HOME/indexes/generate-output # Tell the IDE to use the shared index touch $HOME/$PROJECT_NAME/intellij.yaml cat <<EOF >>$HOME/$PROJECT_NAME/intellij.yaml sharedIndex: project: - url: $INDEX_SERVER_URL/project/$PROJECT_NAME EOF # Start an example http server to simulate "hosting" this shared index cd $HOME/indexes/generate-output python3 -m http.server 9999 # Ensure you have the "Shared Project Indexes" plugin installed # (GoLand and non-community JetBrains IDEs have it pre-installed) # and open the project folder. # For the Kubernetes project, # this resulted in a 56% decrease in loading time. # (22 seconds vs 50 seconds)
Testing from another workspace
- Make the dev URL for the workspace public.
- Copy the
intellij.yamlfile into another workspace before opening the project in IntelliJ. You could also commit it, but again, the Dev URL is just for testing.
Adding shared indexes into the image
The example above simulates an external "file server" hosting the indexes. When the Coder image is built, indexes could also be generated (via the script) and copied into the image. During runtime, the IDE could "download" the indexes from its own filesystem using the
During testing this led to a 60% faster indexing (versus 56% from a dev URL / server on the network). This significantly depends on the shared index size and networking speed. It does removes the need to upload the indexes anywhere :)
Note: I haven't done this in a Dockerfile yet. Here's how it might work:
# Dockerfile FROM openjdk:11 AS indexer # Generate shared index here ARG INDEXES_SERVER_URL=file:///var/tmp/indexes # e.g RUN create-shared-index.sh FROM codercom/enterprise-goland:ubuntu COPY --from=indexer /root/indexes /var/tmp/indexes
In the project's intellij.yaml file, simply reference the indexes dir with the
file:// schema. I have tested this and it works as long as the user has read permissions to the dir.
sharedIndex: project: - url: file:///var/tmp/indexes
If the image is built regularily (e.g weekly), it will lead more performant indexing as it uses a more recent copy of the project, requiring fewer updates locally.