I've opened up Github Discussions to further discuss Jailmaker.
Last active
August 27, 2024 16:40
-
-
Save Jip-Hop/4704ba4aa87c99f342b2846ed7885a5d to your computer and use it in GitHub Desktop.
Persistent Debian 'jail' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts. Without modifying the host OS at all thanks to systemd-nspawn!
That does sound crazy. I don't want to litter this gist, and can't say I fully understand your configuration, but that's definitely not what I'm seeing. Just to confirm I pulled up a Web shell session in Scale for my "IX Official" Plex container. Then did an apt-get update && apt-get install traceroute net-tools and checked traceroute and routing:
So either that was resolved (I'm on the latest Scale) or there is some other routing issue at play there.
are there PM's in github? IF you would like to take this offline / elsewhere
cos I am seeing - when I duiplicate what you have done - the same odd behaviour. I would love this to be a configuration issue
@aardvarkl Github doesn't but I'm Codelica on Discord and on the TrueNAS community there also.
and I am Confused on TN on Discord - I think I have sent you a request
At that point there is a "safe" and isolated install of Docker running which can be used
can you define safe? If you add an external interface is any network traffic to/from the Docker going through that interface or is there still shared network resources of the host?
can you define safe? If you add an external interface is any network traffic to/from the Docker going through that interface or is there still shared network resources of the host?
By safe I mean if IX decides to remove Docker from their base Scale host install (like they going to do I believe) and just keep k3s/containerd, my install of Docker (running under their system) shouldn't be affected. As it's really no different than any other custom container running. So unless they do away with all custom containers, leaving only installs from app catalogs, it should be fine. That would be extreme IMO, and even then I guess it could be done by creating a catalog and config, etc -- but would be a pain.
As far as networking goes, adding other interfaces just gives flexibility. Basically I wanted my Docker apps with their own interface (leaving NAS & Plex stuff alone on my main 10G interface). So I gave it one interface on the LAN one one that's a private network for internal service backends for other machines (dbs, message bus, etc) as I do dev work. Both show (net1, net2) in the docker container with local routes to their subnets. So local traffic from Docker apps to the LAN uses net1 for example. But by default the default internet route would be the eth0 interface that the custom container provides via k3s. But that can be changed by just changing the default route within the Docker container to point to my gateway off net1 for example.
Anyway, I have messages from you guys on Discord so we can continue there. : )
I think it makes more sense now to continue the discussion over here:
https://github.com/Jip-Hop/jailmaker/discussions
😄
Does any of you use jailmaker alongside Apps? Please let me know about your experience in this poll in order to support this pull request.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It is / was an IX bug. All traffic leaving a K3S container (from Truecharts or IX) that I tested was going to the default gateway, even if on-net and relying on the GW to redirect back to the LAN. IX declined to accept this as a bug saying that this was working correctly. We had something of an argument during which it was agreed that one of us didn't know what the other was talking about when it came to routing behaviour
A Traceroute went as follows:
As far as I am aware thats still going on - but I cannot test for it atm as I cannot find a container that has traceroute any longer (not that I have tried them all and in fact don't use many any longer) and I don't feel like collecting traffic at the firewall.
Jailmaker routes properly, allows me access to the entire hardware of the host and does so simply with less overhead as far as I can tell. Long may it continue (@Jip-Hop )