Block Changes in Beta 1.2.13

blockstate_protocol.md

Block Storage & Network Protocol Changes

Paletted chunks & removing BlockIDs brings a few big changes to how blocks are represented. In Bedrock, Blocks used to be represented by their 8 bit ID and 4 bit data; this means that we can only represent 256 blocks and 16 variants for each block. As it happens we ran out of IDs in Update Aquatic, so we had to do something about it :)

After this change, we can represent infinite types of Blocks and infinite BlockStates, just like in Java. BlockStates are what is sent over the network as they are roughly equivalent to the old ID+data information, eg. they're all the information attached to a block.

BlockStates are serialized in two ways:

PersistentID: a PersistentID is a NBT tag containing the BlockState name and its variant number; for example

{
    "name":"minecraft:log",
    "val":14
}

in the future it will contain the actual properties of the BlockState explicitly like in Java. This format is pretty expensive to serialize and send over the network though, so it isn't used in the protocol yet (and possibly never) but it might still be useful information for map makers :)

RuntimeID: a RuntimeID is an alternate ID that is used to refer to a BlockState within a game session. It's an unsigned int that is assigned to each new BlockState we discover when loading a world; worlds with different Behavior packs can (in the future) represent the same BlockState with different RuntimeIDs.

Because of this, you shouldn't ever write RuntimeIDs to disk or make them persist across game sessions, because they should change. A future version of the protocol will let the server send a GlobalPalettePacket which will let you tell the Client which block is which and have it install new blocks.

For the time being however, RuntimeIDs are actually static ( :( ) so you will have to use a lookup table to figure out which is which! The lookup table is appended at the end.

Typically, RuntimeIDs are serialized as varints.

Item changes

We didn't do the same ID removal for Item, because Item's ID was a short already and so it wasn't as much of a concern; what we did was to move new blocks to negative IDs. So, every block past 256 will be -1, -2, -3 and so on.

This means that you can't compare Block IDs and Item IDs directly anymore, even if you keep Block IDs around!

Changed packets

This is a (hopefully exhaustive) list of packets that changed and that your server should handle to talk with this version of Minecraft. If you're wondering where FullChunkDataPacket is, no worries! That one still supports the old format so you don't have to implement block palettes right away. Other packets aren't retrocompatible though, so you'll have to fix those:

• UpdateBlockPacket: BlockID -> RuntimeID
• LevelSoundEventPacket: BlockID -> Data (which is a RuntimeID)
• LevelEventPacket: the Data varint now represents a RuntimeID for ParticlesDestroyBlock, ParticlesCrackBlock, ParticlesCropEaten and ParticlesDestroyArmorStand.
• SetEntityDataPacket: Endermen and FallingBlock use RuntimeID to represent their BlockState.

FullChunkDataPacket

FullChunkDataPacket is an exception because it still understands the old format, as we needed to implement that anyway to read old worlds. Anyhow, you should start considering moving to the new format because until you do, you won't be able to send Blocks past 256 to the client!

Also, there's a small penalty to converting old chunks to new chunks on the client, so you should do that for speed as well.

The new SubChunk format

While the LevelChunk format itself is unchanged, what changed is the internal format of each SubChunk. The version is always a byte and there are a few valid SubChunkFormat versions:

• 1.2 format (versions 0,2,3,4,5,6,7) The old format. It's just [version:byte][16x16x16 blocks][16x16x16 data]. There can be light information, but it will be discarded. Note how values 0,2,3,4,5,6,7 all mean this format! We had to do this because tools like MCEdit put those values in that field and we needed to keep those worlds working.

• Palettized format (version 1) This format was briefly used in the beta after palettization came in. The SubChunk just contains one block storage, so the format is [version:byte][block storage]

• Palettized format with storage list (version 8) This is the final Update Aquatic format, added to allow for several block storages. The additional block storage is used only for water for now, but we made the format generic.
  The format is [version:byte][num_storages:byte][block storage1]...[blockStorageN]

Block Storage format

A Block Storage is now its own type of object (different from SubChunk) with its own format and version! Don't put weird numbers in the version field this time :)

• 1 bit: whether the chunk is serialized for Runtime or for Persistence: always 1 when over the network.
• 7 bits: the internal format, eg. the type of palette or compression that the SubChunk is using. This one is used to select which subclass of BlockStateStorage to create. Valid types that the client understands are:

enum class Type : uint8_t {
	Paletted1 = 1,   // 32 blocks per word
	Paletted2 = 2,   // 16 blocks per word
	Paletted3 = 3,   // 10 blocks and 2 bits of padding per word
	Paletted4 = 4,   // 8 blocks per word
	Paletted5 = 5,   // 6 blocks and 2 bits of padding per word
	Paletted6 = 6,   // 5 blocks and 2 bits of padding per word
	Paletted8  = 8,  // 4 blocks per word
	Paletted16 = 16, // 2 blocks per word
}

A word is just a 4 byte unsigned int.
The padded formats are kind of nasty to implement but the good news are, you don't need to implement all of them (if you do, I suggest to use a template). The server can pick whatever format and the Client will understand it. Of course, if you waste space in the palettes you won't have the best compression.

• 4096 / blocksPerWord words + 1 optional padding word: The actual blocks. Each block takes a fixed amount of bits, plus eventual padding. The padding byte is only present if the words have padding, eg. for sizes 3,5 and 6.
• 1 varint: The palette size N
• (for network) N varints: The palette entries, as RuntimeIDs. You should use the RuntimeID table to convert those to actual blocks.
• (for persistence) N NBT tags: The palette entries, as PersistentIDs. You should read the "name" and "val" fields to figure out what blocks it represents.

@geNAZt oops you're correct, I shouldn't write these docs from memory :)
@MRG95 it's written just above, versions 2 to 7 have been blocked out because tools in the wild used them wrong. So it should have been 2, but instead it became 8. Multiple block storages are for now just to store water, which in Update Acquatic can coexist with blocks in the first layer. So yeah, layer 1 is different, and contains fluids.

Is it a bug that some of the blocks refer to an out-of-range index in the palette of the BlockStorage? I'm getting that the array of block data sometimes contains a 24 even when there are only 20 NBT tags following the block data, for instance. This is happening with the persistent (i.e. not network) format. When I look at those locations in the game itself, there is nothing unusual about them. There is usually just air or stone there. I also verified that I could correctly determine the type of block at almost every world position in a world, so there is nothing obviously wrong with my decoding logic. Furthermore, I also check that I have consumed all of the data in a subchunk after deserializing, so I am not missing any of the NBT tags.

EDIT: Never mind, I have found my bug already. My parser was working incorrectly for chunks where there were padding bits in each word.

@ocecaco I have the same problem right now... Can you elaborate a little more to what you were doing wrong? I am pulling my hair out trying to figure out why I am getting indices that exceed palette size.

@maple-shaft The problem in my case had to do with the fact that I messed up the direction in which I was reading the bits from each word. I don't remember anymore what the right ordering was, but this commit might provide some insight: ocecaco/mcworld@989bea5#diff-25d902c24283ab8cfbac54dfa101ad31 . That's the commit where I finally fixed the issue (after thinking I had fixed it before). You might notice that the shift direction has changed from <<= to >>=, so clearly it had something to do with the direction of in which the blocks are packed into a word. Another thing that can cause bugs is if you "drop" the padding bits from the wrong side of the word. If you are packing 6 5-bit words, you end up with 30 bits and 2 bits of padding. I'm not sure where the 2 bits of padding are in the block encoding anymore, however.

@ocecaco Thanks! Your code gave me a needed sanity check, I was doing this right but I did have a bug in my code where I was casting a signed byte to a signed int and it was changing the underlying bits on me. Sometimes I hate Java. Am trying to make a library that makes it easier to program ROM modules in Bedrock Redstone computers.

It looks like 1.17.30 added sub-chunk version 9 which adds an extra byte storing the sub-chunk index after the number of storages.
Would be nice to have official confirmation.
[version:byte][num_storages:byte][sub_chunk_index:byte][block storage1]...[blockStorageN]

It looks like 1.17.30 added sub-chunk version 9 which adds an extra byte storing the sub-chunk index after the number of storages. Would be nice to have official confirmation. [version:byte][num_storages:byte][sub_chunk_index:byte][block storage1]...[blockStorageN]

Thanks for the heads up! There is definitely an extra byte in version 9