Skip to content

Instantly share code, notes, and snippets.

@CrendKing
Forked from mmozeiko/!README.md
Last active February 22, 2024 06:03
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save CrendKing/154abfa33200ef1cda38ddd61f4d414b to your computer and use it in GitHub Desktop.
Save CrendKing/154abfa33200ef1cda38ddd61f4d414b to your computer and use it in GitHub Desktop.
Download MSVC compiler/linker & Windows SDK without installing full Visual Studio

This downloads standalone 64-bit MSVC compiler, linker & other tools, also headers/libraries from Windows SDK into portable folder, without installing Visual Studio. Has bare minimum components - no UWP/Store/WindowsRT stuff, just files & tools for 64-bit native desktop app development.

Run python.exe portable-msvc.py and it will download output into msvc folder. By default it will download latest available MSVC & Windows SDK - currently v14.32.17.2 and v10.0.22621.0.

You can list available versions with python.exe portable-msvc.py --show-versions and then pass versions you want with --msvc-version and --sdk-version arguments.

To use cl.exe/link.exe from output folder, first run setup.bat - after that PATH/INCLUDE/LIB env variables will be setup to use all the tools as usual. You can also use clang-cl.exe with these includes & libraries.

To use clang-cl.exe without running setup.bat, pass extra /winsysroot msvc argument (msvc is folder name where output is stored).


Modifications:

  1. The original script uses system's default temporary directory, which most likely exists in the system drive (C:), and uses pathlib's Path.replace() to move extracted files to destination. Unfortunately, Path.replace() only works if the source and destination are on the same drive (it's more of a rename than move). This modified version creates temporary directory in the same directory of the script, fixing the problem.
  2. Generates a setup.ps1 for PowerShell users.
#!/usr/bin/env python3
import io
import json
import shutil
import hashlib
import zipfile
import tempfile
import argparse
import subprocess
import urllib.request
from pathlib import Path
OUTPUT = Path("msvc") # output folder
TEMP = Path(".").absolute()
# other architectures may work or may not - not really tested
HOST = "x64" # or x86
TARGET = "x64" # or x86, arm, arm64
MANIFEST_URL = "https://aka.ms/vs/17/release/channel"
def download(url):
with urllib.request.urlopen(url) as res:
return res.read()
def download_progress(url, check, name, f):
data = io.BytesIO()
with urllib.request.urlopen(url) as res:
total = int(res.headers["Content-Length"])
size = 0
while True:
block = res.read(1<<20)
if not block:
break
f.write(block)
data.write(block)
size += len(block)
perc = size * 100 // total
print(f"\r{name} ... {perc}%", end="")
print()
data = data.getvalue()
digest = hashlib.sha256(data).hexdigest()
if check.lower() != digest:
exit(f"Hash mismatch for f{pkg}")
return data
# super crappy msi format parser just to find required .cab files
def get_msi_cabs(msi):
index = 0
while True:
index = msi.find(b".cab", index+4)
if index < 0:
return
yield msi[index-32:index+4].decode("ascii")
def first(items, cond):
return next(item for item in items if cond(item))
### parse command-line arguments
ap = argparse.ArgumentParser()
ap.add_argument("--show-versions", const=True, action="store_const", help="Show available MSVC and Windows SDK versions")
ap.add_argument("--accept-license", const=True, action="store_const", help="Automatically accept license")
ap.add_argument("--msvc-version", help="Get specific MSVC version")
ap.add_argument("--sdk-version", help="Get specific Windows SDK version")
args = ap.parse_args()
### get main manifest
manifest = json.loads(download(MANIFEST_URL))
### download VS manifest
vs = first(manifest["channelItems"], lambda x: x["id"] == "Microsoft.VisualStudio.Manifests.VisualStudio")
payload = vs["payloads"][0]["url"]
vsmanifest = json.loads(download(payload))
### find MSVC & WinSDK versions
packages = {}
for p in vsmanifest["packages"]:
packages.setdefault(p["id"].lower(), []).append(p)
msvc = {}
sdk = {}
for pid,p in packages.items():
if pid.startswith("Microsoft.VisualStudio.Component.VC.".lower()) and pid.endswith(".x86.x64".lower()):
pver = ".".join(pid.split(".")[4:6])
if pver[0].isnumeric():
msvc[pver] = pid
elif pid.startswith("Microsoft.VisualStudio.Component.Windows10SDK.".lower()) or \
pid.startswith("Microsoft.VisualStudio.Component.Windows11SDK.".lower()):
pver = pid.split(".")[-1]
if pver.isnumeric():
sdk[pver] = pid
if args.show_versions:
print("MSVC versions:", " ".join(sorted(msvc.keys())))
print("Windows SDK versions:", " ".join(sorted(sdk.keys())))
exit(0)
msvc_ver = args.msvc_version or max(sorted(msvc.keys()))
sdk_ver = args.sdk_version or max(sorted(sdk.keys()))
if msvc_ver in msvc:
msvc_pid = msvc[msvc_ver]
msvc_ver = ".".join(msvc_pid.split(".")[4:-2])
else:
exit(f"Unknown MSVC version: f{args.msvc_version}")
if sdk_ver in sdk:
sdk_pid = sdk[sdk_ver]
else:
exit(f"Unknown Windows SDK version: f{args.sdk_version}")
print(f"Downloading MSVC v{msvc_ver} and Windows SDK v{sdk_ver}")
### agree to license
tools = first(manifest["channelItems"], lambda x: x["id"] == "Microsoft.VisualStudio.Product.BuildTools")
resource = first(tools["localizedResources"], lambda x: x["language"] == "en-us")
license = resource["license"]
if not args.accept_license:
accept = input(f"Do you accept Visual Studio license at {license} [Y/N] ? ")
if not accept or accept[0].lower() != "y":
exit(0)
OUTPUT.mkdir(exist_ok=True)
total_download = 0
### download MSVC
msvc_packages = [
# MSVC binaries
f"microsoft.vc.{msvc_ver}.tools.host{HOST}.target{TARGET}.base",
f"microsoft.vc.{msvc_ver}.tools.host{HOST}.target{TARGET}.res.base",
# MSVC headers
f"microsoft.vc.{msvc_ver}.crt.headers.base",
# MSVC libs
f"microsoft.vc.{msvc_ver}.crt.{TARGET}.desktop.base",
f"microsoft.vc.{msvc_ver}.crt.{TARGET}.store.base",
# MSVC runtime source
f"microsoft.vc.{msvc_ver}.crt.source.base",
# ASAN
f"microsoft.vc.{msvc_ver}.asan.headers.base",
f"microsoft.vc.{msvc_ver}.asan.{TARGET}.base",
# MSVC redist
#f"microsoft.vc.{msvc_ver}.crt.redist.x64.base",
]
for pkg in msvc_packages:
p = first(packages[pkg], lambda p: p.get("language") in (None, "en-US"))
for payload in p["payloads"]:
with tempfile.TemporaryFile(dir=TEMP) as f:
data = download_progress(payload["url"], payload["sha256"], pkg, f)
total_download += len(data)
with zipfile.ZipFile(f) as z:
for name in z.namelist():
if name.startswith("Contents/"):
out = OUTPUT / Path(name).relative_to("Contents")
out.parent.mkdir(parents=True, exist_ok=True)
out.write_bytes(z.read(name))
### download Windows SDK
sdk_packages = [
# Windows SDK tools (like rc.exe & mt.exe)
f"Windows SDK for Windows Store Apps Tools-x86_en-us.msi",
# Windows SDK headers
f"Windows SDK for Windows Store Apps Headers-x86_en-us.msi",
f"Windows SDK Desktop Headers x86-x86_en-us.msi",
# Windows SDK libs
f"Windows SDK for Windows Store Apps Libs-x86_en-us.msi",
f"Windows SDK Desktop Libs {TARGET}-x86_en-us.msi",
# CRT headers & libs
f"Universal CRT Headers Libraries and Sources-x86_en-us.msi",
# CRT redist
#"Universal CRT Redistributable-x86_en-us.msi",
]
with tempfile.TemporaryDirectory(dir=TEMP) as d:
dst = Path(d)
sdk_pkg = packages[sdk_pid][0]
sdk_pkg = packages[first(sdk_pkg["dependencies"], lambda x: True).lower()][0]
msi = []
cabs = []
# download msi files
for pkg in sdk_packages:
payload = first(sdk_pkg["payloads"], lambda p: p["fileName"] == f"Installers\\{pkg}")
msi.append(dst / pkg)
with open(dst / pkg, "wb") as f:
data = download_progress(payload["url"], payload["sha256"], pkg, f)
total_download += len(data)
cabs += list(get_msi_cabs(data))
# download .cab files
for pkg in cabs:
payload = first(sdk_pkg["payloads"], lambda p: p["fileName"] == f"Installers\\{pkg}")
with open(dst / pkg, "wb") as f:
download_progress(payload["url"], payload["sha256"], pkg, f)
print("Unpacking msi files...")
# run msi installers
for m in msi:
subprocess.check_call(["msiexec.exe", "/a", m, "/quiet", "/qn", f"TARGETDIR={OUTPUT.resolve()}"])
### versions
msvcv = list((OUTPUT / "VC/Tools/MSVC").glob("*"))[0].name
sdkv = list((OUTPUT / "Windows Kits/10/bin").glob("*"))[0].name
# place debug CRT runtime files into MSVC folder (not what real Visual Studio installer does... but is reasonable)
dst = OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{HOST}/{TARGET}"
with tempfile.TemporaryDirectory(dir=TEMP) as d:
d = Path(d)
pkg = "microsoft.visualcpp.runtimedebug.14"
dbg = first(packages[pkg], lambda p: p["chip"] == HOST)
for payload in dbg["payloads"]:
name = payload["fileName"]
with open(d / name, "wb") as f:
data = download_progress(payload["url"], payload["sha256"], f"{pkg}/{name}", f)
total_download += len(data)
msi = d / first(dbg["payloads"], lambda p: p["fileName"].endswith(".msi"))["fileName"]
with tempfile.TemporaryDirectory(dir=TEMP) as d2:
subprocess.check_call(["msiexec.exe", "/a", str(msi), "/quiet", "/qn", f"TARGETDIR={d2}"])
for f in first(Path(d2).glob("System*"), lambda x: True).iterdir():
f.replace(dst / f.name)
# download DIA SDK and put msdia140.dll file into MSVC folder
with tempfile.TemporaryDirectory(dir=TEMP) as d:
d = Path(d)
pkg = "microsoft.visualc.140.dia.sdk.msi"
dia = packages[pkg][0]
for payload in dia["payloads"]:
name = payload["fileName"]
with open(d / name, "wb") as f:
data = download_progress(payload["url"], payload["sha256"], f"{pkg}/{name}", f)
total_download += len(data)
msi = d / first(dia["payloads"], lambda p: p["fileName"].endswith(".msi"))["fileName"]
with tempfile.TemporaryDirectory(dir=TEMP) as d2:
subprocess.check_call(["msiexec.exe", "/a", str(msi), "/quiet", "/qn", f"TARGETDIR={d2}"])
if HOST == "x86": msdia = "msdia140.dll"
elif HOST == "x64": msdia = "amd64/msdia140.dll"
else: exit("unknown")
src = Path(d2) / "Program Files" / "Microsoft Visual Studio 14.0" / "DIA SDK" / "bin" / msdia
src.replace(dst / "msdia140.dll")
### cleanup
shutil.rmtree(OUTPUT / "Common7", ignore_errors=True)
for f in ["Auxiliary", f"lib/{TARGET}/store", f"lib/{TARGET}/uwp"]:
shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / f)
for f in OUTPUT.glob("*.msi"):
f.unlink()
for f in ["Catalogs", "DesignTime", f"bin/{sdkv}/chpe", f"Lib/{sdkv}/ucrt_enclave"]:
shutil.rmtree(OUTPUT / "Windows Kits/10" / f, ignore_errors=True)
for arch in ["x86", "x64", "arm", "arm64"]:
if arch != TARGET:
shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{arch}", ignore_errors=True)
shutil.rmtree(OUTPUT / "Windows Kits/10/bin" / sdkv / arch)
shutil.rmtree(OUTPUT / "Windows Kits/10/Lib" / sdkv / "ucrt" / arch)
shutil.rmtree(OUTPUT / "Windows Kits/10/Lib" / sdkv / "um" / arch)
### setup.bat
SETUP_BAT = f"""@echo off
set ROOT=%~dp0
set MSVC_VERSION={msvcv}
set MSVC_HOST=Host{HOST}
set MSVC_ARCH={TARGET}
set SDK_VERSION={sdkv}
set SDK_ARCH={TARGET}
set MSVC_ROOT=%ROOT%VC\\Tools\\MSVC\\%MSVC_VERSION%
set SDK_INCLUDE=%ROOT%Windows Kits\\10\\Include\\%SDK_VERSION%
set SDK_LIBS=%ROOT%Windows Kits\\10\\Lib\\%SDK_VERSION%
set VCToolsInstallDir=%MSVC_ROOT%\\
set PATH=%MSVC_ROOT%\\bin\\%MSVC_HOST%\\%MSVC_ARCH%;%ROOT%Windows Kits\\10\\bin\\%SDK_VERSION%\\%SDK_ARCH%;%ROOT%Windows Kits\\10\\bin\\%SDK_VERSION%\\%SDK_ARCH%\\ucrt;%PATH%
set INCLUDE=%MSVC_ROOT%\\include;%SDK_INCLUDE%\\ucrt;%SDK_INCLUDE%\\shared;%SDK_INCLUDE%\\um;%SDK_INCLUDE%\\winrt;%SDK_INCLUDE%\\cppwinrt
set LIB=%MSVC_ROOT%\\lib\\%MSVC_ARCH%;%SDK_LIBS%\\ucrt\\%SDK_ARCH%;%SDK_LIBS%\\um\\%SDK_ARCH%
"""
(OUTPUT / "setup.bat").write_text(SETUP_BAT)
### setup.ps1
SETUP_PS1 = f"""$MSVC_VERSION = '{msvcv}'
$MSVC_HOST = 'Host{HOST}'
$MSVC_ARCH = '{TARGET}'
$SDK_VERSION = '{sdkv}'
$SDK_ARCH = '{TARGET}'
$MSVC_ROOT = "${{PSScriptRoot}}\\VC\\Tools\\MSVC\\${{MSVC_VERSION}}"
$SDK_INCLUDE = "${{PSScriptRoot}}\\Windows Kits\\10\\Include\\${{SDK_VERSION}}"
$SDK_LIBS = "${{PSScriptRoot}}\\Windows Kits\\10\\Lib\\${{SDK_VERSION}}"
$env:VCToolsInstallDir = "${{MSVC_ROOT}}\\"
$env:PATH = "${{MSVC_ROOT}}\\bin\\${{MSVC_HOST}}\\${{MSVC_ARCH}};${{PSScriptRoot}}\\Windows Kits\\10\\bin\\${{SDK_VERSION}}\\${{SDK_ARCH}};${{PSScriptRoot}}\\Windows Kits\\10\\bin\\${{SDK_VERSION}}\\${{SDK_ARCH}}\\ucrt;${{env:PATH}}"
$env:INCLUDE = "${{MSVC_ROOT}}\\include;${{SDK_INCLUDE}}\\ucrt;${{SDK_INCLUDE}}\\shared;${{SDK_INCLUDE}}\\um;${{SDK_INCLUDE}}\\winrt;${{SDK_INCLUDE}}\\cppwinrt"
$env:LIB = "${{MSVC_ROOT}}\\lib\\${{MSVC_ARCH}};${{SDK_LIBS}}\\ucrt\\${{SDK_ARCH}};${{SDK_LIBS}}\\um\\${{SDK_ARCH}}"
"""
(OUTPUT / "setup.ps1").write_text(SETUP_PS1)
print(f"Total downloaded: {total_download>>20} MB")
print("Done!")
@amymor
Copy link

amymor commented Oct 26, 2023

Hello @CrendKing
Thank you for the modifications. The download and extraction were successful, you are awesome! However, there is one thing left; an uninstaller is not provided. Could you provide a batch template to uninstall it without affecting my environment variables? Here is my generated setup.bat:

@echo off

set ROOT=%~dp0

set MSVC_VERSION=14.37.32822
set MSVC_HOST=Hostx64
set MSVC_ARCH=x64
set SDK_VERSION=10.0.22621.0
set SDK_ARCH=x64

set MSVC_ROOT=%ROOT%VC\Tools\MSVC\%MSVC_VERSION%
set SDK_INCLUDE=%ROOT%Windows Kits\10\Include\%SDK_VERSION%
set SDK_LIBS=%ROOT%Windows Kits\10\Lib\%SDK_VERSION%

set VCToolsInstallDir=%MSVC_ROOT%\
set PATH=%MSVC_ROOT%\bin\%MSVC_HOST%\%MSVC_ARCH%;%ROOT%Windows Kits\10\bin\%SDK_VERSION%\%SDK_ARCH%;%ROOT%Windows Kits\10\bin\%SDK_VERSION%\%SDK_ARCH%\ucrt;%PATH%
set INCLUDE=%MSVC_ROOT%\include;%SDK_INCLUDE%\ucrt;%SDK_INCLUDE%\shared;%SDK_INCLUDE%\um;%SDK_INCLUDE%\winrt;%SDK_INCLUDE%\cppwinrt
set LIB=%MSVC_ROOT%\lib\%MSVC_ARCH%;%SDK_LIBS%\ucrt\%SDK_ARCH%;%SDK_LIBS%\um\%SDK_ARCH%

Thanks in advance. :)

@CrendKing
Copy link
Author

What kind of uninstallation do you need other than deleting the directory? All the env var changes are restricted to that terminal session.

@amymor
Copy link

amymor commented Oct 27, 2023

What kind of uninstallation do you need other than deleting the directory? All the env var changes are restricted to that terminal session.

Thank you very much.
So if I want to use certain tools, I need to place my code in the setup.bat file, right? I initially made a mistake because I thought the setup.bat was equivalent to an install file. I also use the /c switch for batch files so that the batch file automatically exits, so i thought that allows me to access the tools globally after exiting.

@CrendKing
Copy link
Author

The setup.* are equivalent to the VS Developer Command Prompt: you run it to setup the environment variables in a terminal session, so that within that session you can run VS' build commands such as cl.exe or msbuild.exe. Once you close the terminal, all the changes are gone.

@amymor
Copy link

amymor commented Oct 27, 2023

Thank you
So, I'm going to put my code in setup.bat because I don't want to use the /k switch for batch files. I personally don't use PowerShell due to its slow startup time. It used to take about 5 seconds on previous Installation of Windows, even after a clean install, a custom OS, and using some tweaks like Atlas. It's now down to about 2 seconds, but still cant beat batch files. It seems I'm very sensitive to startup times, LOL. I've created some apps using the built-in Windows VB.NET framework, and all of them have a 1-second startup delay (in my tests, a 64-bit app had 900ms while a 32-bit app had 1008ms). It seems I can't achieve the speed of VB6 in .NET. I've also tried some other apps that written in .NET (like Paint.NET) to see if they have a better startup time, but none of them had a startup delay of less than 1-2 seconds. In my opinion, PowerShell is similar to VB.NET and batch is similar to VB6. So, I'm going to learn more and more languages like Python and C++ and Java. I've also created a test form in C++ in CodeBlocks, and it started immediately like VB6. However, I heard that Java uses JVM to run, so it seems like .NET in terms of startup.
I'm also thinking about expanding setup.bat to offer more options for compiling and support drag and drop. I've already created a portable and simple GUI app to use the VB.NET framework compiler (vbc.exe), so I might create another one for the C++ compiler too.
0067

@CrendKing
Copy link
Author

CrendKing commented Oct 27, 2023

Feel free to make a fork and add whatever features you like to setup script. This fork tries to maintain consistency with VS default.

@MSK61
Copy link

MSK61 commented Jan 13, 2024

Thanks a lot. I had the same problems with the original repo that this fork fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment