shiftmca.py - shift Minecraft region files (.mca) to combine multiple worlds into one

shiftmca.py

# shiftmca.py - shift Minecraft region files (.mca) by a fixed number of regions

# Useful to combine multiple worlds into one world, with a region-level granularity
# You can't simply rename the r.#.#.mca files, because they contain embedded location
# information (which will cause "relocating" errors). This script fixes that.

# To use:
# 1. rename each r.<x>.<z>.mca adding 20 (or whatever needed to avoid overlap) to each <x>
# 2. edit the variables below appropriately to match your setup
# 3. run this script
# 4. copy the new outputted .mca files to your second world you want to merge into
#  (no files should be overwritten if shifted by the correct amount)
# 5. load the world and teleport to 10240,0,0; it should load without errors

# You now have a single unified merged world.

# Requires pymclevel - https://github.com/mcedit/pymclevel

from cStringIO import StringIO
import gzip
import itertools
import zlib
import os
import struct
import logging
logging.basicConfig(level=logging.DEBUG)

import sys
sys.path.append("..")
import pymclevel

import numpy

# amount to shift X - must be a whole number of regions
RX_SHIFT = 20               # region x coordinate eshift - 0,0 -> 20,0
CX_SHIFT = RX_SHIFT * 32    # chunk x coordinate
X_SHIFT = CX_SHIFT * 16     # world x coordinate
# TODO: option to shift Z

# pre-renamed region files (r.0.0.mca renamed to r.20.0.mca) (TODO: rename in this script)
root = "../bukkit/SERVER-beta-firstworld-anvil1.2.5/New World/region/renamed/"
# output directory
outroot = root + "../shifted/"

for filename in os.listdir(root):
    path = root + filename
    
    print filename
    _, new_rx, rz, _ = filename.split(".")
    new_rx = int(new_rx)
    rz = int(rz)
    old_rx = new_rx - RX_SHIFT

    print "Processing",(old_rx,rz),"->",(new_rx,rz)

    f = file(path, "rb")
    SECTOR_BYTES = 4096
    offsets = numpy.fromstring(f.read(SECTOR_BYTES), dtype='>u4')   # infiniteworld.py __init__
    modtimes = numpy.fromstring(f.read(SECTOR_BYTES), dtype='>u4')

    print offsets,modtimes

    def getOffset(cx, cz):
        return offsets[cx + cz * 32]

    def readChunkRaw(cx, cz):
        offset = getOffset(cx, cz)
        sectorStart = offset >> 8
        numSectors = offset & 0xff

        f.seek(sectorStart * SECTOR_BYTES)
        data = f.read(numSectors * SECTOR_BYTES)
        if data is None or len(data) == 0:
            return None
        return data

    VERSION_GZIP = 1
    VERSION_DEFLATE = 2

    def readChunk(cx, cz):
        rawData = readChunkRaw(cx, cz)
        if rawData is None:
            #print "No chunk ",cx,cz
            return None
        length = struct.unpack_from(">I", rawData)[0]
        compressionFormat = struct.unpack_from("B", rawData, 4)[0]
        compressedData = rawData[5:length + 5]
        if compressionFormat == VERSION_GZIP:
            tag = pymclevel.nbt.load(buf=pymclevel.nbt.gunzip(compressedData))
        elif compressionFormat == VERSION_DEFLATE:
            tag = pymclevel.nbt.load(buf=zlib.decompress(compressedData))
        else:
            raise Exception("Unrecognized compression format: "+compressionFormat)

        return tag

    newChunks = []

    currentSector = 2

    for cx, cz in itertools.product(range(32), range(32)):
        chunk = readChunk(cx, cz)
        if chunk is None:
            continue

        # Shift

        # Global
        chunk["Level"]["xPos"].value += CX_SHIFT

        # Entities, Pos
        entities = chunk["Level"]["Entities"]
        for entity in entities:
            entity["Pos"][0].value += X_SHIFT
            try:
                entity["TileX"].value += X_SHIFT    # paintings
            except Exception:
                pass
            print entity

        # TileEntities, x
        tes = chunk["Level"]["TileEntities"]
        for te in tes:
            te["x"].value += X_SHIFT


        # Save

        buf = StringIO()
        chunk.save(filename_or_buf=gzip.GzipFile(fileobj=buf, mode="wb", compresslevel=2))
        data = buf.getvalue()

        length = len(data)
        header = struct.pack(">I", length)
        header += struct.pack("B", VERSION_GZIP)
        data = header + data

        #print cx,cz,chunk
        CHUNK_HEADER_SIZE = 5
        sectorsNeeded = (len(data) + CHUNK_HEADER_SIZE) / SECTOR_BYTES + 1

        bytesNeeded = sectorsNeeded * SECTOR_BYTES
        excess = bytesNeeded - len(data)
        data += "\0" * excess

        offsets[cx + cz * 32] = currentSector << 8 | sectorsNeeded
        currentSector += sectorsNeeded

        newChunks.append(data)

    of = file(outroot + filename, "wb")
    of.write(offsets.tostring())
    of.write(modtimes.tostring())
    for chunkData in newChunks:
        of.write(chunkData)

    of.close()

        

I know this script is a bit old at this point, but I'm having some trouble getting it running. I'm running Python 2.7, and I've installed numpy, PyYAML, and run "python setup.py install" on pymclevel, but it is still not running correctly. Here's the traceback (I've replaced my username with "USER"):

  File "shiftmca.py", line 30, in <module>
    import pymclevel
  File "/home/USER/Applications/anaconda3/envs/py2/lib/python2.7/site-packages/pymclevel-0.1-py2.7-linux-x86_64.egg/pymclevel/__init__.py", line 2, in <module>
    from entity import Entity, TileEntity
  File "/home/USER/Applications/anaconda3/envs/py2/lib/python2.7/site-packages/pymclevel-0.1-py2.7-linux-x86_64.egg/pymclevel/entity.py", line 8, in <module>
    import nbt
  File "/home/USER/Applications/anaconda3/envs/py2/lib/python2.7/site-packages/pymclevel-0.1-py2.7-linux-x86_64.egg/pymclevel/nbt.py", line 557, in <module>
    from _nbt import (load, TAG_Byte, TAG_Short, TAG_Int, TAG_Long, TAG_Float, TAG_Double, TAG_String,
  File "_nbt.pyx", line 75, in init _nbt
    import nbt_util
  File "/home/USER/Applications/pymclevel/nbt_util.py", line 1, in <module>
    import nbt
  File "/home/USER/Applications/pymclevel/nbt.py", line 553, in <module>
    TAG_Value.__str__ = nbt_util.nested_string
AttributeError: 'module' object has no attribute 'nested_string'

I've realized that you were probably using a different version of pymclevel when you wrote this. Is it possible for you to check which commit of pymclevel you were using when you wrote this?

oh my.

I am not sure I ever got this to work how i wanted it. Did you figure it out?

I realized that the version of pymclevel you were using, as well as this script, were likely written before the maximum build height was increased to 256, so it would not be possible to use this code without rewriting all of pymclevel. I thought about using Amulet, but that currently does not support entities.

The solution we ended up going with was using the custom dimension support added in Minecraft v1.16 to link the two worlds with portals.

For people reading this in the future, here is how you can add your existing world as a custom dimension, and here is how you can make portals between your worlds. Also, if you want to bring along your nether in the "custom dimension" world, you just need to make custom portals linking them; the game handles the x8 distance multiplier for you in the /tp command.

that is awesome. I will try it a bit later!