Skip to content

Instantly share code, notes, and snippets.

@Cyberboss
Last active May 25, 2023 00:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Cyberboss/4c9347ef6ba0d0df3c2a947dc56b0e22 to your computer and use it in GitHub Desktop.
Save Cyberboss/4c9347ef6ba0d0df3c2a947dc56b0e22 to your computer and use it in GitHub Desktop.
Why I won't rewrite tgstation-server in rust

Why I won't rewrite tgstation-server in rust

I love rust, debatably more than C#. But I'm never going to rewrite TGS in it. Here's why.

Systems vs Servers

INCOMPLETE

Dependency injection is a MASSIVE part of the current TGS architecture. To support such a thing in rust would involve a LOT of Box/Rc/Arc types for dyn traits. All that indirection would not be pretty to work with. I'd hope the compiler would be smart enough to devirtualize all the calls in a release build (where there is generally only one relevant implementation of an interface) but I can't even be sure of that.

Async

I almost wanted to give this a section of its own because of how big of an issue it is. Rust has a really nice async/await implementation where the runtime used to schedule tasks is interchangeable. Seems like it'd be a nice mapping to current TGS operations right?

Absolutely not.

First off, understanding stack traces are how bugs get fixed. Let's look at a current TGS stack trace that it logs.

TGS Stack Trace
2023-05-24T03:53:41.1976157Z  [03:53:20] dbg: Tgstation.Server.Host.Jobs.JobManager (Instance:1|Job:7|Request:POST /Byond|User:1|Monitor:|Bridge:|Chat:|IR:694bb806-6315-4ba0-bf3b-4b45e907d843|Node:)
2023-05-24T03:53:41.1977293Z      Job 7 exited with error!
2023-05-24T03:53:41.1977731Z  Tgstation.Server.Host.Jobs.JobException: Error downloading specified BYOND version!
2023-05-24T03:53:41.1978620Z   ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found).
2023-05-24T03:53:41.1979907Z     at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
2023-05-24T03:53:41.1981009Z     at Tgstation.Server.Host.IO.FileDownloader.DownloadFile(Uri url, String bearerToken, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/IO/FileDownloader.cs:line 56
2023-05-24T03:53:41.1982821Z     at Tgstation.Server.Host.Components.Byond.ByondManager.InstallVersionFiles(JobProgressReporter progressReporter, Version version, Stream customVersionStream, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Byond/ByondManager.cs:line 533
2023-05-24T03:53:41.1985406Z     --- End of inner exception stack trace ---
2023-05-24T03:53:41.1986913Z     at Tgstation.Server.Host.Components.Byond.ByondManager.InstallVersionFiles(JobProgressReporter progressReporter, Version version, Stream customVersionStream, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Byond/ByondManager.cs:line 569
2023-05-24T03:53:41.1989706Z     at Tgstation.Server.Host.Components.Byond.ByondManager.AssertAndLockVersion(JobProgressReporter progressReporter, Version version, Stream customVersionStream, Boolean neededForLock, Boolean allowInstallation, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Byond/ByondManager.cs:line 482
2023-05-24T03:53:41.1991986Z     at Tgstation.Server.Host.Components.Byond.ByondManager.AssertAndLockVersion(JobProgressReporter progressReporter, Version version, Stream customVersionStream, Boolean neededForLock, Boolean allowInstallation, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Byond/ByondManager.cs:line 495
2023-05-24T03:53:41.2000240Z     at Tgstation.Server.Host.Components.Byond.ByondManager.ChangeVersion(JobProgressReporter progressReporter, Version version, Stream customVersionStream, Boolean allowInstallation, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Byond/ByondManager.cs:line 149
2023-05-24T03:53:41.2001765Z     at Tgstation.Server.Host.Controllers.ByondController.<>c__DisplayClass6_1.<<Update>b__2>d.MoveNext() in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Controllers/ByondController.cs:line 230
2023-05-24T03:53:41.2002769Z  --- End of stack trace from previous location ---

It's fairly easy to understand what's going on above, right? It's simply 404'ing trying to download a BYOND version. You can clearly see the control flow from ByondController -> ByondManager -> FileDownloader. As a disclaimer, it's wrapped in a nice JobException which is hiding a lot of the nasty ThreadPool setup code that calls it. But note how this is all functional code, and yet, this is a completely async callstack.

Here's a panic from a tokio async callstack

tokio stack trace for a simple lib called async-common
thread 'main' panicked at 'explicit panic', C:\Users\ericholk\repo\backtrace-examples\async-common\src\lib.rs:10:5
stack backtrace:
 0: std::panicking::begin_panic_handler
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\/library\std\src\panicking.rs:517
 1: core::panicking::panic_fmt
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\/library\core\src\panicking.rs:101
 2: core::panicking::panic
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\/library\core\src\panicking.rs:50
 3: common::baz::generator$0
          at C:\Users\ericholk\repo\backtrace-examples\async-common\src\lib.rs:10
 4: core::future::from_generator::impl$1::poll<common::baz::generator$0>
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\core\src\future\mod.rs:80
 5: common::bar::generator$0
          at C:\Users\ericholk\repo\backtrace-examples\async-common\src\lib.rs:6
 6: core::future::from_generator::impl$1::poll<common::bar::generator$0>
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\core\src\future\mod.rs:80
 7: common::foo::generator$0
          at C:\Users\ericholk\repo\backtrace-examples\async-common\src\lib.rs:2
 8: core::future::from_generator::impl$1::poll<common::foo::generator$0>
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\core\src\future\mod.rs:80
 9: async_tokio::main::generator$0
          at .\src\main.rs:4
10: core::future::from_generator::impl$1::poll<async_tokio::main::generator$0>
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\core\src\future\mod.rs:80
11: tokio::park::thread::impl$5::block_on::closure$0<core::future::from_generator::GenFuture<async_tokio::main::generator$0> >
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\park\thread.rs:263
12: tokio::coop::with_budget::closure$0<enum$<core::task::poll::Poll<tuple$<> > >,tokio::park::thread::impl$5::block_on::closure$0>
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\coop.rs:106
13: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget> >::try_with<core::cell::Cell<tokio::coop::Budget>,tokio::coop::with_budget::closure$0,enum$<core::task::poll::Poll<tuple$<> > > >
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\std\src\thread\local.rs:399
14: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget> >::with<core::cell::Cell<tokio::coop::Budget>,tokio::coop::with_budget::closure$0,enum$<core::task::poll::Poll<tuple$<> > > >
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\std\src\thread\local.rs:375
15: tokio::coop::with_budget
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\coop.rs:99
16: tokio::coop::budget
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\coop.rs:76
17: tokio::park::thread::CachedParkThread::block_on<core::future::from_generator::GenFuture<async_tokio::main::generator$0> >
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\park\thread.rs:263
18: tokio::runtime::enter::Enter::block_on<core::future::from_generator::GenFuture<async_tokio::main::generator$0> >
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\runtime\enter.rs:151
19: tokio::runtime::thread_pool::ThreadPool::block_on<core::future::from_generator::GenFuture<async_tokio::main::generator$0> >
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\runtime\thread_pool\mod.rs:77
20: tokio::runtime::Runtime::block_on<core::future::from_generator::GenFuture<async_tokio::main::generator$0> >
          at C:\Users\ericholk\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.13.0\src\runtime\mod.rs:463
21: async_tokio::main
          at .\src\main.rs:4
22: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
          at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35\library\core\src\ops\function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Yikes.

Now you may say, "Rust uses results, not exceptions. This comparison is unfair!"

You'd be partly right however this loops back to the problem of working with a low-level language for TGS. The fact is that in rust a panic is a possibilty, in C# TGS it's literally impossible unless one of our native dependencies (of which there are currently only .NET itself and libgit2) fuck up. TGS is complex software and someone will trip a panic wire while porting it to rust. It may be in a massively obscure and hard to debug way too.

But let's play devils advocate and say our rewrite will be panic-free and handle errors gracefully with Result<T>s everywhere. Again, I say to you, "Good luck debugging errors".

What is the output of the above TGS error for a rust based equivalent? Error downloading specified BYOND version + <whatever the http library used outputs for a 404>. Okay it's not useless, we know what's going on. But suppose there was a typo in the BYOND download URL that needed to be corrected. What is clearer as to the location of the error?

Sure you could code search for the error message, but that's not a guaranteed navigation. @mothblocks has corrected me on this image

INCOMPLETE

Backwards Compatibility

INCOMPLETE

Reflection

TGS leverages .NETs built-in reflection in several places to make life easier. A few of these include:

  • API specification generation.
  • Packing additional provider data into their chat messages (like the MessageReference property used to generate replies to Discord commands).
  • API permission evaluation

INCOMPLETE

Complexity

INCOMPLETE

Time

Modern TGS has been actively developed for over four years at the time of writing (Started in 2019). A rewrite in rust to reach feature and fix parity would probably take just as long if not longer.

Ain't nobody got time for that shit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment