Skip to content

Instantly share code, notes, and snippets.

@Spottedleaf
Last active April 30, 2024 10:59
Show Gist options
  • Star 23 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Spottedleaf/6cc1acdd03a9b7ac34699bf5e8f1b85c to your computer and use it in GitHub Desktop.
Save Spottedleaf/6cc1acdd03a9b7ac34699bf5e8f1b85c to your computer and use it in GitHub Desktop.
The future of the Starlight mod

Final change of plans (March 8, 2024)

I don't see that many people are using Starlight on modern versions anymore. As such, I don't see any reason to continue to maintain the mod versions, given that they do have real mod conflict issues with other mods. Starlight is also not neccessary to use on 1.20.x anyways.

Starlight may make a return in a future project of mine, but for now I am ceasing to upload new mod versions and commit updates to the github. If you want Starlight on the server, you will need to use Paper or one of its derivatives.

Change of plans (June 6, 2023)

People have reached out to me indicating they want Starlight as a mod for whatever reason. While I personally disagree that it is still needed, I can also see that people still want it. Rather than leaving this mod in a state where people will either ship it in their mod OR upload forked versions, I will post updates to the mod but change the project descriptions to indicate that people really shouldn't be using it. I'd rather that if people are getting Starlight, it is the properly maintained version.

Light engine performance changes in 1.20 from Vanilla

As some of you may know, Mojang wrote a faster light engine for 1.20. The improvements come from implementing two core ideas from Starlight:

  1. The light engine should propagate levels, rather than updates.
  2. The sky light engine should have dedicated logic to light chunks

Level propagation

Vanilla used to operate off of one queue, an update queue, but now operates off of an increase queue and decrease queue, like Starlight. Also like Starlight, the propagation of these levels has been optimized by the following:

  • Storing whether the origin of a level change is a special shape block (reduces block reads by 2x)
  • Storing a set of directions to propagate to, used to avoid propagations in directions known not to be useful (i.e where the propagation originated from)
  • Using a simple deque for updates so that additions/removals are as fast as possible. Previously, vanilla used a linked hash set.

The full details of why propagation vs updates are faster is laid out in the Starlight technical details.

Vanilla does miss a lot of optimisations that Starlight does here, to the point where Vanilla is still 2x slower, but it is fast enough.

Sky light generation

Vanilla used to sky light chunks with the light section initialisation, but that logic was too abstract and thus was very slow. Vanilla now sky lights chunks using a technique similar (the difference is in how the height map is generated) to Starlight 1.16/1.17. However, comparisons of the 1.20 Vanilla implementation and Starlight's 1.20 implementation aren't clear on which implementation is currently faster, but they do show that lighting chunks is far faster than it needs to be so it doesn't matter (anything << 1% of time to generate chunks doesn't matter).

Is Starlight obsolete?

I would say that usage on the client is obsoleted, minus the possibility of the light section management being a problem.

Currently, Starlight's light section management is depended on for both Paper's rewrite of the chunk system (people in the modding space call this "TACS") and Folia's additional changes to that system. In Starlight, light sections are tied directly to ChunkAccess objects.

The light section management prevents most bugs (including performance ones) involving missing/absent light sections, as Starlight assumes that no updates are possible unless the chunk exists. This is why there are basically zero performance or bug related issues with the light section management in Starlight. Note that Vanilla did not change their light engine section management to adhere to Starlight's design, so this difference still exists.

This difference in light section management allows my rewrite of the chunk system (which is designed as a solid base for Folia to build on, which adds regionized multithreading to the dedicated server) to greatly simplify and optimize its chunk unload/load logic as it no longer needs to consider any light engine state as there is no light engine state.

The "stateless" property of Starlight allows the chunk system to run light updates / generation in parallel provided the scheduling is done right, see patch "Increase parallelism for neighbour writing chunk statuses" in 1.19 Folia. This is critical to allowing the chunk system to scale beyond 10 worker threads, which as you may imagine is not critical to clients but rather important for dedicated servers with more players to stress chunk generation.

With all that in mind, Starlight is still important for Paper & Folia and as such I will continue to maintain it. However, since it is not important for the client anymore, I will not be posting any updates to the mod version. I will maintain the fabric version on github solely for the purpose of assisting with updating Starlight, as is with DataConverter (as a sidenote, DataConverter is not obsoleted by the recent performance changes to DFU).

TL;DR

Mojang improved upon their own light engine using the same performance critical designs from Starlight. As a result, they have closed the performance gap, obsoleting Starlight on the client side. I am not longer posting updates to the mod on CF and Modrinth.

Mojang still has not improved the performance critical designs for things that exclusively mattered to the dedicated server, so Starlight will live on in Paper & Folia. As such, Starlight will still be maintained on GitHub for Fabric to assist in updating Paper & Folia.

Change of plans (June 6, 2023)

People have reached out to me indicating they want Starlight as a mod for whatever reason. While I personally disagree that it is still needed, I can also see that people still want it. Rather than leaving this mod in a state where people will either ship it in their mod OR upload forked versions, I will post updates to the mod but change the project descriptions to indicate that people really shouldn't be using it. I'd rather that if people are getting Starlight, it is the properly maintained version.

@PoneyClairDeLune
Copy link

A question though: What about Fabric servers that don't want any "bug fixes" from dedicated server software like Paper? More often than not, Paper's "fixes" just borks vanilla mechanics.

@vertexcubed
Copy link

If Starlight still has beneficial performance improvements on the server, then why not make the mod server side only? So that servers running fabric modpacks that can't run either paper or folia can receive the performance increase provided.

Of course if doing this is not worth the effort than that's totally understandable, just a thought I had.

@Spottedleaf
Copy link
Author

A question though: What about Fabric servers that don't want any "bug fixes" from dedicated server software like Paper? More often than not, Paper's "fixes" just borks vanilla mechanics.

Yeah the term bug fix really depends on who looks at its. A lot of owners who run public (i.e not just for you and your friends) servers tend to want the mechanics paper breaks (i.e sand dupling, bedrock breaking) to actually be broken. This does leave out a portion of users who want to play the game as broken as Mojang ships it (after all, there is fun in breaking the game).

Whenever this came up internally in Paper we really just wanted to port the performance enhancements and critical exploit fixes to a mod so that we could shove these users there and be done with this problem, but nobody has actually stepped up to do that. I considered doing that myself, but given the limited number of users and the total pain that maintaining the very large set of Mixins would take I decided that my time was better spent elsewhere. Note Starlight and DataConverter are a rare exception where 90% of the code is not actually in Mixins, but rather separated entirely so there's really little Mixin maintenance occurring. Back during 1.16.5 I made a port of a set of Tuinity patches, and still have that around, but actually creating it was endlessly frustrating as most of the work is dealing with Mixins. What is a simple single line diff in a patch can become a nightmare in a Mixin very easily and I need very good justification to work through that pain, and there just is none.

All of these details and history are important because I generally expect the light engine differences to be minimal on the server for the kinds of players Fabric would be pushing (less than 50) on 1.20. So, for actually running a server on 1.20 at those scales you're looking at different issues that far exceed light engine stability.

Unfortunately, no fabric mod actually provides help for these issues, which is why porting a significant amount of work from Paper is required. To clarify, let's review these issues: The chunk system (TACS) and exploits (i.e crashing, duping, cheating).

TACS (the chunk system) has many issues when being pushed. You can see the full list here that paper patched up to 1.19.2 (until I rewrote the chunk system). A quick highlight: Memory leaks (too many to list), issues preventing saving on crash, no sync chunk gen/load prioritization which at best cause massive lag spikes and at worse trip the watchdog and cause a crash, and improper error handling (i.e failing to convert a chunk silently discards data on disk). All of these issues were a major stability issues for larger servers. I think to this day, if you use 1.19.4 Starlight and try to pre-generate a world using Chunky the server just leaks memory and dies - which indicate how incapable Vanilla TACS is at holding a constant load.

Historically, Paper is much faster at patching things like crash exploits and dupe exploits. Maybe technical users don't care about dupes, OK. But crashes are a big deal. Crash exploits when they are easy to create means people can knock your server out for 24/7 until they're fixed. And if you're not getting a patch for months, you're having a lot of fun. An example of such an exploit I fixed is here. If you notice the date, and then look at when Mojang copied the fix here you will see that if you were running a Fabric server at the time you would absolutely have been having an awful time. I also had a coordinate exploit bug open (with a fix!) in the bug tracker that only took ~3 years to fix. I also had a collision related exploit (this one was actually really bad, because you could clip through basically anything as if you were in spectator mode) with a provided fix open on the tracker that was not resolved for about 1 year. There is also a current crash exploit that is extremely easy to create, but I'm not giving details because it's not been fixed! (I don't know if it is fixed in 1.20-pre5, but it ain't fixed in 1.19.4) But, this exploit was fixed within 1-2 days of being reported to Paper. I can't say how long it's been known for either (as it may narrow it down), but it'd be long enough to give you a headache if you were running a server dealing with it. So, both the history and current state is pretty clear: There does need to be some kind of effort to fast track exploit fixes so that people can run stable servers. Because the track record is clear, which is either have a really nice playerbase or suffer and wait until the next update if you're lucky.

For these reasons, I don't see any usecase of Starlight for servers on Fabric 1.20. At the scales you would need to run to see improvements, you would also just be swallowed by the other issues and have no choice but to use Paper or patch things yourself.

@Spottedleaf
Copy link
Author

Spottedleaf commented May 24, 2023

If Starlight still has beneficial performance improvements on the server, then why not make the mod server side only? So that servers running fabric modpacks that can't run either paper or folia can receive the performance increase provided.

Of course if doing this is not worth the effort than that's totally understandable, just a thought I had.

See the above.

@IverCoder
Copy link

IverCoder commented May 24, 2023

Vanilla does miss a lot of optimisations that Starlight does here, to the point where Vanilla is still 2x slower, but it is fast enough.

Doesn't this mean Starlight is still relevant client-side? Some people (including me) can only play on low-end hardware which means even that 2x improvement, however small, is still a very big help.

@1ByteBit
Copy link

1ByteBit commented May 25, 2023

Vanilla does miss a lot of optimisations that Starlight does here, to the point where Vanilla is still 2x slower, but it is fast enough.

Doesn't this mean Starlight is still relevant client-side? Some people (including me) can only play on low-end hardware which means even that 2x improvement, however small, is still a very big help.

Not really. If you use the chart on their modrinth page as an example, 2x slower actually sounds a lot worse then it really is. (Using the chart) Starlight is ~3 seconds (As opposed to the old default of ~106), this means that the new default is most likely somewhere in the range of ~6-7 seconds. While this isn't perfect, it is at the point of any more optimizations being practically useless in situations such as the client, even for low end hardware. Simply using the other performance mods should still make the game perfectly playable

@Spottedleaf
Copy link
Author

Vanilla does miss a lot of optimisations that Starlight does here, to the point where Vanilla is still 2x slower, but it is fast enough.

Doesn't this mean Starlight is still relevant client-side? Some people (including me) can only play on low-end hardware which means even that 2x improvement, however small, is still a very big help.

Not really. If you use the chart on their modrinth page as an example, 2x slower actually sounds a lot worse then it really is. (Using the chart) Starlight is ~3 seconds (As opposed to the old default of ~106), this means that the new default is most likely somewhere in the range of ~6-7 seconds. While this isn't perfect, it is at the point of any more optimizations being practically useless in situations such as the client, even for low end hardware. Simply using the other performance mods should still make the game perfectly playable

It's only 2x slower in the general propagation tests, not the world generation tests. So if you look at the propagation test graphs, you're looking at sub-millisecond times, unless you're at max world height then you're barely 1-2ms in. So, unless your hardware is 10 times slower...

@CuteistFox
Copy link

Realy sad about it being discuntinued, without it even post 1.20.1 there was huge visulal lag and starlight fixed most of it. Also my hardware is 10 time slower. Also while it may be invasive, all the mods i used worked. Of course if you are discountinuing it to work for mojang implement this stuff its a whole other story. I hope someone forks it. I understand not wanting to work on it do to low usership so im not annoyed at you, but i am sad. Also its realy usefull if you use the 2048 build height like some bte teams do. But i understand its hard to make mods and that there is not enoth motivation so keep doing your thing.

@Yoon0117
Copy link

TLDR; Basically results before 1.20 used to be something like
0.1ms Starlight
20ms Phosphor
30ms Vanilla

But after 1.20, it's
0.75ms Starlight
1.5ms Vanilla

So technically it's X2 faster, but the difference is so small (0.0005 second) that the developer decided it would be better to discontinue the project rather than waste hours improving the time by a fraction of a millisecond.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment