A Globe visualization, displaying travel lines made from GPS coordinates on planet earth.
Think about the google data-globe but do-it-yourself. Display travel routes and link to content from another website. Because weaving a 2D-website and a 3D-context is a difficult task, for now there are just hyperlinks leading to other pages.
- Demo: https://weiserhei.github.io/panoglobe/
- Repository: https://github.com/weiserhei/panoglobe
I´ve started writing this post with the intent to talk about the technical details of my private project panoglobe. A behind-the-scenes of what I´m doing with three.js to create this application. But during writing I noticed I first have to pause and reflect about the general side of things.
So now this part is about the environment, about building and maintaining this project for the span of 6 years. Here is no code involved, just storytelling.
When I started this project in early 2014, a Data-Globe-template by google just became popular. If you were to follow the globe-visualization trends, you might have seen somewhere this distinctive black balls: https://experiments.withgoogle.com/chrome/globe.
I was looking for something like this, throw my GPS data at, and it would give me a nice line representation. I didn´t understand how to do any of this by looking at the readme. So I had an eye out at some other globes. Another one from Google Zeitgeist and the beautiful Arms Globe:
- Google Zeitgeist 2013 (example is dead): https://experiments.withgoogle.com/google-zeitgeist-2013-globe
- Arms Globe: https://armsglobe.chromeexperiments.com/
These examples were nice to look at and the code is not minified. Maybe because bundlers were not as popular as they are today. So its JavaScript, just view-source and copy everything? Sadly at that time I didnt understand any of what was needed to make that possible.
I had to build it myself with three.js and so panoglobe started.
In the beginning I would just download the three.js release and use the existing folder structure. Replace the examples/index.html-file with my own one and put my code inside the folders like examples/js/. The working directory was then uploaded via FTP to my webhost. Et voilà: project shipped.
The oldest working Example I could find is using three.js r67 which has been released in April, 2014. Here is an image using some mockup GPS data:
Demo: http://three.thiele-medien.de/panoglobe-r67/
By reading this you can guess my type of version control at that time: copying folders. Two years went by while I was updating librarys and working on new features. Breaking things myself or getting things broken due to big API changes. As a result I would be copying folders before updating or jumping at a new feature. I still have them:
I then created a repository and put the code on github (https://github.com/weiserhei/panoglobe). Using version control, although doing it sloppy, was very beneficial to me- even though I am the only developer in this project. Comparing an older version of this file which is now broken or coming back to my code months later and seeing where I left off saves a lot of time.
As the project was growing, more and more .js-files were created by me. I split the code over several files, basically just a collection of prototype-based classes and some functions. We didnt had all these sweet ES6 modules and features, or atleast I didnt know how to use them. On top of that, all that other librarys and plugins, external dependencies.
At some point in time my <head>-section looked like this:
<!-- THREEJS -->
<script type="text/javascript" src="js/three.js"></script>
<script type="text/javascript" src="js/Detector.js"></script>
<script type="text/javascript" src="js/libs/stats.min.js"></script>
<script type="text/javascript" src="../js/loaders/OBJLoader.js"></script>
<script type="text/javascript" src="../js/loaders/MTLLoader.js"></script>
<script type="text/javascript" src="../js/loaders/OBJMTLLoader.js"></script>
<script type="text/javascript" src="js/renderers/Projector.js"></script>
<script type="text/javascript" src="js/shaders/NormalDisplacementShader.js"></script>
<!-- GUI -->
<script type="text/javascript" src="js/libs/dat.gui.min.js"></script>
<!-- JQUERY -->
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<!-- threex -->
<script type="text/javascript" src="js/THREEx.WindowResize.js"></script>
<script type="text/javascript" src='js/threex69.domevents.js'></script>
<script type="text/javascript" src='js/threex.atmospherematerial.js'></script>
<!-- APP -->
<script type="text/javascript" src="js/controls/OrbitObjectControls.js"></script>
<script type="text/javascript" src="js/PANOUTILS.js"></script>
<script type="text/javascript" src="js/Preload.js"></script>
<script type="text/javascript" src="js/Universe.js"></script>
<script type="text/javascript" src="js/Globe.js"></script>
<script type="text/javascript" src="js/Route.js"></script>
<script type="text/javascript" src="js/RouteLine.js"></script>
<script type="text/javascript" src="js/MarkerFactory.js"></script>
<script type="text/javascript" src="js/gui_proto.js"></script>
<script type="text/javascript" src="js/makeSprite.js"></script>
<script type="text/javascript" src="js/init.js"></script>
Keeping track of which file had a dependency to another was exhausting. It was chaotic and prevented the project from growing further.
To tackle this issue I setup require.js in 02/2016. Its a JS File and Module Loader which handled all the dependencies and gave me a nice main.js bundle as output.
As a result I split the code even more, reducing the Lines of Code in each Module to keep track of whats happening.
three.js was at r74 now and getting the bundling to work with the examples, which I relied on, was difficult. It was about six Months before the famous discussion about bundlers started. You can have a look at that issue "Transform examples/js to support modules" (mrdoob/three.js#9562) with 340 replies.
Time passes and the project grows. Its 2018 now and I move on to webpack, replacing require.js. three.js has also been converted to modules by many helping hands (thank you guys!) and easily consumed now. Managing packages with npm, working with webpacks dev-server and auto-reload, and some more features felt like a big step in the right direction. Because I haven´t learned how2git in the meantime, I´ve created a new repository for this migration (https://github.com/weiserhei/panoglobe-webpack). I started off with this Boilerplate (https://github.com/paulmg/ThreeJS-Webpack-ES6-Boilerplate) and also started moving the code to ES6. Some parts of that Boilerplate are still living in my codebase.
After working with webpack for several months I became familiar with this workflow and ironed-out many of the beginner problems. I configured webpack to output a bundle of every file/image/etc. that is used in the project into a dist/-folder, which makes it a lot easier to upload and replace everything on my webhost than previously.
So now the project can be "npm-installed", built, and we get a nice single release directory containing all the relevant files. The files are all static content (dynamic content is fetched via AJAX), which means it could also be hosted on gh-pages.
Using github-workflows and some ready-made-action from the marketplace I now automatically build and deploy the dist/-folder to the gh-pages-pages branch. Doing it like this allows us to remove the release-folder from the repository and still have an up-to-date build availiable on gh-pages.
After all this is just a basic CI/CD-workflow, but everything is built-into Github and free to use.
It is 2020 now and after moving into the webpack-ecosystem, I´ve also had a look at TypeScript. Having type-safety and intellisense are huge reliefs on a growing, chaotic codebase. panoglobe is split into about 25 files, and it became difficult for me to mentally hold this model in my head. I avoided TS previously because I disliked the idea of having to compile the source. But now I am already in this bundler-workflow, so I started the migration. This time in a new branch, not another repository.
Oh wow, thats a lot of errors! //@ts-ignore to the rescue :-) Im still in the process of moving to a strict tsconfig, but so far this has allowed me to find several bugs that JavaScript did not care about. As of now, the panoglobe-webpack-repository is also merged back into the origin panoglobe-repository.
On a side note; and there is another pretty big improvement which I became familiar with at the same time. That is formatting-on-save, or just auto-formatting. Im using prettier and my dev-life got so much easier.
The project is still in active development and there is always a part that needs some work. From time to time I take a break from development to gain some distance, then coming back with new ideas.
This writeup allows me to question myself about how am I doing things and by that showing me the overall progression. Maybe you are interested how other developers in the three.js-scene are doing things or have a similar experience on another timeline. Im happy to discuss any topic and improve further.