Paletted chunks & removing BlockIDs brings a few big changes to how blocks are represented. In Bedrock, Blocks used to be represented by their 8 bit ID and 4 bit data; this means that we can only represent 256 blocks and 16 variants for each block. As it happens we ran out of IDs in Update Aquatic, so we had to do something about it :)
After this change, we can represent infinite types of Blocks and infinite BlockStates, just like in Java. BlockStates are what is sent over the network as they are roughly equivalent to the old ID+data information, eg. they're all the information attached to a block.
BlockStates are serialized in two ways:
PersistentID: a PersistentID is a NBT tag containing the BlockState name and its variant number; for example
{
"name":"minecraft:log",
"val":14
}
in the future it will contain the actual properties of the BlockState explicitly like in Java. This format is pretty expensive to serialize and send over the network though, so it isn't used in the protocol yet (and possibly never) but it might still be useful information for map makers :)
RuntimeID: a RuntimeID is an alternate ID that is used to refer to a BlockState within a game session. It's an unsigned int that is assigned to each new BlockState we discover when loading a world; worlds with different Behavior packs can (in the future) represent the same BlockState with different RuntimeIDs.
Because of this, you shouldn't ever write RuntimeIDs to disk or make them persist across game sessions, because they should change.
A future version of the protocol will let the server send a GlobalPalettePacket
which will let you tell the Client which block is which and have it install new blocks.
For the time being however, RuntimeIDs are actually static ( :( ) so you will have to use a lookup table to figure out which is which! The lookup table is appended at the end.
Typically, RuntimeIDs are serialized as varint
s.
We didn't do the same ID removal for Item, because Item's ID was a short
already and so it wasn't as much of a concern; what we did was to move new blocks to negative IDs. So, every block past 256 will be -1, -2, -3 and so on.
This means that you can't compare Block IDs and Item IDs directly anymore, even if you keep Block IDs around!
This is a (hopefully exhaustive) list of packets that changed and that your server should handle to talk with this version of Minecraft.
If you're wondering where FullChunkDataPacket
is, no worries! That one still supports the old format so you don't have to implement block palettes right away. Other packets aren't retrocompatible though, so you'll have to fix those:
- UpdateBlockPacket: BlockID -> RuntimeID
- LevelSoundEventPacket: BlockID -> Data (which is a RuntimeID)
- LevelEventPacket: the Data varint now represents a RuntimeID for
ParticlesDestroyBlock
,ParticlesCrackBlock
,ParticlesCropEaten
andParticlesDestroyArmorStand
. - SetEntityDataPacket:
Endermen
andFallingBlock
use RuntimeID to represent their BlockState.
FullChunkDataPacket
FullChunkDataPacket is an exception because it still understands the old format, as we needed to implement that anyway to read old worlds. Anyhow, you should start considering moving to the new format because until you do, you won't be able to send Blocks past 256 to the client!
Also, there's a small penalty to converting old chunks to new chunks on the client, so you should do that for speed as well.
While the LevelChunk format itself is unchanged, what changed is the internal format of each SubChunk. The version is always a byte and there are a few valid SubChunkFormat
versions:
-
1.2 format (versions 0,2,3,4,5,6,7) The old format. It's just
[version:byte][16x16x16 blocks][16x16x16 data]
. There can be light information, but it will be discarded. Note how values 0,2,3,4,5,6,7 all mean this format! We had to do this because tools like MCEdit put those values in that field and we needed to keep those worlds working. -
Palettized format (version 1) This format was briefly used in the beta after palettization came in. The SubChunk just contains one block storage, so the format is
[version:byte][block storage]
-
Palettized format with storage list (version 8) This is the final Update Aquatic format, added to allow for several block storages. The additional block storage is used only for water for now, but we made the format generic.
The format is[version:byte][num_storages:byte][block storage1]...[blockStorageN]
Block Storage format
A Block Storage is now its own type of object (different from SubChunk) with its own format and version! Don't put weird numbers in the version field this time :)
- 1 bit: whether the chunk is serialized for Runtime or for Persistence: always
1
when over the network. - 7 bits: the internal format, eg. the type of palette or compression that the SubChunk is using. This one is used to select which subclass of BlockStateStorage to create. Valid types that the client understands are:
enum class Type : uint8_t {
Paletted1 = 1, // 32 blocks per word
Paletted2 = 2, // 16 blocks per word
Paletted3 = 3, // 10 blocks and 2 bits of padding per word
Paletted4 = 4, // 8 blocks per word
Paletted5 = 5, // 6 blocks and 2 bits of padding per word
Paletted6 = 6, // 5 blocks and 2 bits of padding per word
Paletted8 = 8, // 4 blocks per word
Paletted16 = 16, // 2 blocks per word
}
A word
is just a 4 byte unsigned int.
The padded formats are kind of nasty to implement but the good news are, you don't need to implement all of them (if you do, I suggest to use a template). The server can pick whatever format and the Client will understand it. Of course, if you waste space in the palettes you won't have the best compression.
4096 / blocksPerWord
words +1
optional padding word: The actual blocks. Each block takes a fixed amount of bits, plus eventual padding. The padding byte is only present if the words have padding, eg. for sizes 3,5 and 6.- 1 varint: The palette size N
- (for network) N varints: The palette entries, as RuntimeIDs. You should use the RuntimeID table to convert those to actual blocks.
- (for persistence) N NBT tags: The palette entries, as PersistentIDs. You should read the "name" and "val" fields to figure out what blocks it represents.
@oJBR in our code it's already called
BlockRuntimeID
. This doc is a spec, it doesn't say how you should name things in your code, either, other than in the packet fields.About blockstate data: we've just converted what was already there... I'm aware that that sucks, but changing that is just as big of a pain for us. Plus, that pain might not be motivated because Java tech is not developed together with us for the most part and can/will diverge on internal details like this one. So encouraging the assumption that you can just move bits over 1:1 is going to cause problems for you or a lot of work for us, eventually.
You should never expect internal implementation details like that to stay consistent because it's just not practical for us to have to consider every single data structure in the game a public part of the interface that can't be changed without people breaking.
I think the best way to do it is to just convert everything by default, and assume that 1:1 conversions are temporary.
tl;dr: if you do modding/reverse engineering, changing internals are a part of life. We do try to make it as un-painful as possible though!