Bluscream/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Example usage:
python merge_minecraft_servers_dat.py --ping --skip --write

Example Output:
PS P:\Python\minecraft>  p:; cd 'p:\Python\minecraft'; C:\Python311\python.exe c:\Users\blusc\.vscode\extensions\ms-python.python-2023.6.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher 49522 -- P:\Python\minecraft\servers.py --ping --skip 
Loaded 14 from B:\Timo\Quest\SideQuest\net.kdt.pojavlaunch.debug\data\2022-03-20T14-47-48.248Z\files\.minecraft\servers.dat
Loaded 14 from B:\Timo\Quest\SideQuest\net.kdt.pojavlaunch.debug\data\2022-04-23T08-44-11.327Z\files\.minecraft\servers.dat
Loaded 14 from C:\Users\blusc\AppData\Roaming\.universe-installer\repo\1.17.1\Universe-Modded\config\yosbr\servers.dat
Loaded 14 from D:\OneDrive\Games\Minecraft\servers.dat
Loaded 14 from G:\MultiMC\instances\1.19.4\.minecraft\servers.dat
Loaded 14 from G:\MultiMC\instances\Fabulously Optimized 4.7.1\.minecraft\servers.dat
Loaded 14 from G:\MultiMC\instances\Fabulously-Optimized-4.7.0-beta.2\minecraft\servers.dat
Loaded 14 from G:\MultiMC\instances\Fabulously-Optimized-4.7.1\minecraft\servers.dat
Could not load NBT file: S:\Users\blusc\Documents\My Games\Terraria\servers.dat ( -62 )
14 servers have been found in 8 files
Address                         Name                      IP                        Port    Online    Players
------------------------------  ------------------------  ------------------------  ------  --------  ---------
smp.hypercubemc.net:25565       HyperCubeMC Modded SMP    smp.hypercubemc.net       N/A     No        N/A
zockermaus124.aternos.me:25565  zockermaus124.aternos.me  zockermaus124.aternos.me  N/A     No        N/A
localhost:25565                 localhost                 localhost                 N/A     No        N/A
GrieferGames.net:25565          GrieferGames.net          GrieferGames.net          N/A     No        N/A
bluscream.minehut.gg:25565      bluscream.minehut.gg      bluscream.minehut.gg      N/A     No        N/A
empress05.minehut.gg:25565      empress05.minehut.gg      empress05.minehut.gg      N/A     No        N/A
2b2t.org:25565                  2b2t.org                  2b2t.org                  N/A     No        N/A
Bluscream.aternos.me:25565      Bluscream.aternos.me      Bluscream.aternos.me      N/A     No        N/A
play.tasmantismc.com:25565      play.tasmantismc.com      play.tasmantismc.com      N/A     No        N/A
tw1x.de:25565                   tw1x.de                   tw1x.de                   N/A     No        N/A
pfalzfurs.serv.nu:25565         pfalzfurs.serv.nu         pfalzfurs.serv.nu         N/A     No        N/A
hypixel.net:25565               hypixel.net               hypixel.net               N/A     No        N/A
mineplex.eu:25565               mineplex.eu               mineplex.eu               N/A     No        N/A
golitron.aternos.me:25565       golitron.aternos.me       golitron.aternos.me       N/A     No        N/A
Unable to ping smp.hypercubemc.net:25565: [Errno 11001] getaddrinfo failed
Unable to ping localhost:25565: timed out
Unable to ping 2b2t.org:25565: [Errno 11001] getaddrinfo failed
Unable to ping play.tasmantismc.com:25565: [Errno 11001] getaddrinfo failed
Unable to ping pfalzfurs.serv.nu:25565: timed out
Unable to ping hypixel.net:25565: timed out
Unable to ping mineplex.eu:25565: timed out
Address                         Name                      IP                        Port    Online    Players
------------------------------  ------------------------  ------------------------  ------  --------  ---------
smp.hypercubemc.net:25565       HyperCubeMC Modded SMP    smp.hypercubemc.net       N/A     No        N/A
zockermaus124.aternos.me:25565  zockermaus124.aternos.me  zockermaus124.aternos.me  N/A     Yes       0
localhost:25565                 localhost                 localhost                 N/A     No        N/A
GrieferGames.net:25565          GrieferGames.net          GrieferGames.net          N/A     Yes       1356
bluscream.minehut.gg:25565      bluscream.minehut.gg      bluscream.minehut.gg      N/A     Yes       0
empress05.minehut.gg:25565      empress05.minehut.gg      empress05.minehut.gg      N/A     Yes       0
2b2t.org:25565                  2b2t.org                  2b2t.org                  N/A     No        N/A
Bluscream.aternos.me:25565      Bluscream.aternos.me      Bluscream.aternos.me      N/A     Yes       0
play.tasmantismc.com:25565      play.tasmantismc.com      play.tasmantismc.com      N/A     No        N/A
tw1x.de:25565                   tw1x.de                   tw1x.de                   N/A     Yes       0
pfalzfurs.serv.nu:25565         pfalzfurs.serv.nu         pfalzfurs.serv.nu         N/A     No        N/A
hypixel.net:25565               hypixel.net               hypixel.net               N/A     No        N/A
mineplex.eu:25565               mineplex.eu               mineplex.eu               N/A     No        N/A
golitron.aternos.me:25565       golitron.aternos.me       golitron.aternos.me       N/A     Yes       0
Do you want to update the servers list files? (Y/N): y
Written 14 servers to B:\Timo\Quest\SideQuest\net.kdt.pojavlaunch.debug\data\2022-03-20T14-47-48.248Z\files\.minecraft\servers.dat
Written 14 servers to B:\Timo\Quest\SideQuest\net.kdt.pojavlaunch.debug\data\2022-04-23T08-44-11.327Z\files\.minecraft\servers.dat
Written 14 servers to C:\Users\blusc\AppData\Roaming\.universe-installer\repo\1.17.1\Universe-Modded\config\yosbr\servers.dat
Written 14 servers to D:\OneDrive\Games\Minecraft\servers.dat
Written 14 servers to G:\MultiMC\instances\1.19.4\.minecraft\servers.dat
Written 14 servers to G:\MultiMC\instances\Fabulously Optimized 4.7.1\.minecraft\servers.dat
Written 14 servers to G:\MultiMC\instances\Fabulously-Optimized-4.7.0-beta.2\minecraft\servers.dat
Written 14 servers to G:\MultiMC\instances\Fabulously-Optimized-4.7.1\minecraft\servers.dat
8 server.dat files updated!


## merge_minecraft_servers_dat.py
from ipaddress import IPv4Address, IPv6Address
from sys import argv
import os
import subprocess
import tabulate
import nbtlib
from mcstatus import JavaServer
from nbtlib.tag import List, Compound, String
from dataclasses import dataclass
from pathlib import Path
from nbtlib import Compound
from urllib.parse import ParseResult, urlparse
from json import dumps

@dataclass
class MinecraftServerEntry:
    original_file: Path = Path("servers.dat")
    nbt_data: Compound = None
    address: str = "localhost:25565"
    ipv4: IPv4Address = None
    ipv6: IPv6Address = None
    subdomain: str = None
    domain: str = None
    status = None # PingResult
    is_aternos = False
    is_minehut = False
    def __init__(self, _path, server, address) -> None:
        self.original_file = _path
        self.nbt_data = server
        self.address = address
        self.subdomain = urlparse("tcp://"+address).hostname
        self.domain = '.'.join(self.subdomain.split(".")[-2:])
        self.is_aternos = self.subdomain.endswith(".aternos.me")
        self.is_minehut = self.subdomain.endswith(".minehut.gg")


def get(data, keys):
    """
    Retrieves the value of nested keys from a dictionary.
    :param data: A dictionary to retrieve the value from.
    :param keys: A list of keys to traverse the nested dictionary.
    :return: The value associated with the nested keys, or "N/A" if any of the keys do not exist.
    """
    if isinstance(keys, str):
        keys = [keys]
    if not isinstance(data, dict):
        return "N/A"
    key = keys[0]
    if key not in data:
        return "N/A"
    if len(keys) == 1:
        return data[key]
    return get(data[key], keys[1:])
def get_server_row(s):
    return ["Minehut" if s.is_minehut else "Aternos" if s.is_aternos else "", s.nbt_data['name'], s.address, s.domain, f'Yes ({get(s.status,["players","online"])}/{get(s.status,["players","max"])})' if s.status else 'No']
def print_servers(data: list):
    # Step 3: Print the servers in a table format
    headers = ['Host','Name','Address','Domain', 'Online']
    table = [get_server_row(s) for s in data]
    print(tabulate.tabulate(table, headers=headers))

# Step 1: Find all "servers.dat" files on your PC using Everything CLI
proc = subprocess.Popen(['es', '-regex', '^servers.dat$'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc.communicate()
server_files = {}
for f in out.decode('utf-8').split('\n'):
     server_files[f.strip()] = None
if not server_files: raise Exception("No servers.dat files found, maybe Everything or the ES CLI is not installed?")

# Step 2: Parse the contents of each file and ping the servers
for _path in list(server_files):
    if not os.path.isfile(_path):
        del server_files[_path]
        continue
    try:
         _t = nbtlib.load(_path)
         server_files[_path] = _t
         print(f"Loaded {len(_t['servers'])} from {_path}")
    except Exception as ex:
        print("Could not load NBT file:", _path, "(",ex.args[0],")")
        del server_files[_path]

servers: dict[str, MinecraftServerEntry] = {}

for _path, _servers in server_files.items():
    for server in _servers["servers"]:
        # print(json.dumps(server))
        if not "name" in server or not server["name"].strip(): continue
        port = server["port"] if "port" in server.keys() else 25565
        address = f'{server["ip"]}:{port}'
        if "hidden" in server and server["hidden"]:
            print(f"Server {address} in file {_path} is HIDDEN!")
            server["hidden"] = False
        if address in servers.keys(): continue
        servers[address] = MinecraftServerEntry(_path, server, address)
# try: print(json.dumps(servers))
# except: pass

# for address, server in servers.items():
#     print(f"=== {address} START")
#     for k in dict(server):
#         print(f"{k}")
#     print(f"=== {address} END")

print(f"{len(servers)} servers have been found in {len(server_files)} files")
print_servers(servers.values())

confirmation = 'y' if '--ping' in argv[1:] else input(f'Do you want to ping them? (Y/N): ').lower()
if confirmation == 'y':
    for address, server in servers.items():
        server.status = None
        try:
            result = JavaServer.lookup(address, timeout=1)
            server.status = result.status().raw
        except Exception as ex: print(f"Unable to ping {address}: {ex}")

    print_servers(servers.values())

    confirmation = 'n' if '--skip' in argv[1:] else input('Do you want to remove unreachable servers? (Y/N): ').lower()
    if confirmation == 'y':
        for address in list(servers):
            if not servers[address].status: del servers[address]

        print_servers(servers.values())

if '--no-icons' in argv[1:]:
    for server in servers:
        if "icon" in server.nbt_data: del server.nbt_data["icon"]
    print("Removed icons")

new_server_items = servers.values()
if '--sort' in argv[1:]:
    new_server_items = []
    for server in sorted(servers.values(), key=lambda x: x.domain, reverse=True):
        name: str = server.nbt_data["name"]
        if any(c.isalpha() for c in name):
            if name.endswith(".aternos.me") or name.endswith(".minehut.gg"):
                server.nbt_data["name"] = String(name.removesuffix(".aternos.me").removesuffix(".minehut.gg"))
        if server.is_minehut or server.is_aternos:
            new_server_items.insert(0, server.nbt_data)
        else:
            new_server_items.append(server.nbt_data)
        print(tabulate.tabulate([get_server_row(server)]))
    print("Sorted")


print("len:", len(new_server_items))
# print(dumps(new_server_items))

# Step 4: Ask for confirmation before writing the merged data back to the files
confirmation = 'y' if '--write' in argv[1:] else input('Do you want to update the servers list files? (Y/N): ').lower()
if confirmation == 'y':
    # merged_servers = {}
    # for address, server in servers.items():
    #     if address not in merged_servers:
    #         merged_servers[server['address']] = server
    new_server_items = List[Compound](new_server_items)
    for _path, _servers in server_files.items():
        os.replace(_path, f"{_path}.bak")
        _servers["servers"] = new_server_items
        _servers.save()
        print(f"Written {len(_servers['servers'])} servers to {_path}")
    print(f'{len(server_files)} server.dat files updated!')
else:
    print('Servers list not updated.')
	from ipaddress import IPv4Address, IPv6Address
	from sys import argv
	import os
	import subprocess
	import tabulate
	import nbtlib
	from mcstatus import JavaServer
	from nbtlib.tag import List, Compound, String
	from dataclasses import dataclass
	from pathlib import Path
	from nbtlib import Compound
	from urllib.parse import ParseResult, urlparse
	from json import dumps

	@dataclass
	class MinecraftServerEntry:
	original_file: Path = Path("servers.dat")
	nbt_data: Compound = None
	address: str = "localhost:25565"
	ipv4: IPv4Address = None
	ipv6: IPv6Address = None
	subdomain: str = None
	domain: str = None
	status = None # PingResult
	is_aternos = False
	is_minehut = False
	def __init__(self, _path, server, address) -> None:
	self.original_file = _path
	self.nbt_data = server
	self.address = address
	self.subdomain = urlparse("tcp://"+address).hostname
	self.domain = '.'.join(self.subdomain.split(".")[-2:])
	self.is_aternos = self.subdomain.endswith(".aternos.me")
	self.is_minehut = self.subdomain.endswith(".minehut.gg")


	def get(data, keys):
	"""
	Retrieves the value of nested keys from a dictionary.
	:param data: A dictionary to retrieve the value from.
	:param keys: A list of keys to traverse the nested dictionary.
	:return: The value associated with the nested keys, or "N/A" if any of the keys do not exist.
	"""
	if isinstance(keys, str):
	keys = [keys]
	if not isinstance(data, dict):
	return "N/A"
	key = keys[0]
	if key not in data:
	return "N/A"
	if len(keys) == 1:
	return data[key]
	return get(data[key], keys[1:])
	def get_server_row(s):
	return ["Minehut" if s.is_minehut else "Aternos" if s.is_aternos else "", s.nbt_data['name'], s.address, s.domain, f'Yes ({get(s.status,["players","online"])}/{get(s.status,["players","max"])})' if s.status else 'No']
	def print_servers(data: list):
	# Step 3: Print the servers in a table format
	headers = ['Host','Name','Address','Domain', 'Online']
	table = [get_server_row(s) for s in data]
	print(tabulate.tabulate(table, headers=headers))

	# Step 1: Find all "servers.dat" files on your PC using Everything CLI
	proc = subprocess.Popen(['es', '-regex', '^servers.dat$'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
	out, err = proc.communicate()
	server_files = {}
	for f in out.decode('utf-8').split('\n'):
	server_files[f.strip()] = None
	if not server_files: raise Exception("No servers.dat files found, maybe Everything or the ES CLI is not installed?")

	# Step 2: Parse the contents of each file and ping the servers
	for _path in list(server_files):
	if not os.path.isfile(_path):
	del server_files[_path]
	continue
	try:
	_t = nbtlib.load(_path)
	server_files[_path] = _t
	print(f"Loaded {len(_t['servers'])} from {_path}")
	except Exception as ex:
	print("Could not load NBT file:", _path, "(",ex.args[0],")")
	del server_files[_path]

	servers: dict[str, MinecraftServerEntry] = {}

	for _path, _servers in server_files.items():
	for server in _servers["servers"]:
	# print(json.dumps(server))
	if not "name" in server or not server["name"].strip(): continue
	port = server["port"] if "port" in server.keys() else 25565
	address = f'{server["ip"]}:{port}'
	if "hidden" in server and server["hidden"]:
	print(f"Server {address} in file {_path} is HIDDEN!")
	server["hidden"] = False
	if address in servers.keys(): continue
	servers[address] = MinecraftServerEntry(_path, server, address)
	# try: print(json.dumps(servers))
	# except: pass

	# for address, server in servers.items():
	# print(f"=== {address} START")
	# for k in dict(server):
	# print(f"{k}")
	# print(f"=== {address} END")

	print(f"{len(servers)} servers have been found in {len(server_files)} files")
	print_servers(servers.values())

	confirmation = 'y' if '--ping' in argv[1:] else input(f'Do you want to ping them? (Y/N): ').lower()
	if confirmation == 'y':
	for address, server in servers.items():
	server.status = None
	try:
	result = JavaServer.lookup(address, timeout=1)
	server.status = result.status().raw
	except Exception as ex: print(f"Unable to ping {address}: {ex}")

	print_servers(servers.values())

	confirmation = 'n' if '--skip' in argv[1:] else input('Do you want to remove unreachable servers? (Y/N): ').lower()
	if confirmation == 'y':
	for address in list(servers):
	if not servers[address].status: del servers[address]

	print_servers(servers.values())

	if '--no-icons' in argv[1:]:
	for server in servers:
	if "icon" in server.nbt_data: del server.nbt_data["icon"]
	print("Removed icons")

	new_server_items = servers.values()
	if '--sort' in argv[1:]:
	new_server_items = []
	for server in sorted(servers.values(), key=lambda x: x.domain, reverse=True):
	name: str = server.nbt_data["name"]
	if any(c.isalpha() for c in name):
	if name.endswith(".aternos.me") or name.endswith(".minehut.gg"):
	server.nbt_data["name"] = String(name.removesuffix(".aternos.me").removesuffix(".minehut.gg"))
	if server.is_minehut or server.is_aternos:
	new_server_items.insert(0, server.nbt_data)
	else:
	new_server_items.append(server.nbt_data)
	print(tabulate.tabulate([get_server_row(server)]))
	print("Sorted")


	print("len:", len(new_server_items))
	# print(dumps(new_server_items))

	# Step 4: Ask for confirmation before writing the merged data back to the files
	confirmation = 'y' if '--write' in argv[1:] else input('Do you want to update the servers list files? (Y/N): ').lower()
	if confirmation == 'y':
	# merged_servers = {}
	# for address, server in servers.items():
	# if address not in merged_servers:
	# merged_servers[server['address']] = server
	new_server_items = List[Compound](new_server_items)
	for _path, _servers in server_files.items():
	os.replace(_path, f"{_path}.bak")
	_servers["servers"] = new_server_items
	_servers.save()
	print(f"Written {len(_servers['servers'])} servers to {_path}")
	print(f'{len(server_files)} server.dat files updated!')
	else:
	print('Servers list not updated.')