Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 42 You must be signed in to star a gist
  • Fork 11 You must be signed in to fork a gist
  • Save mdPlusPlus/031ec2dac2295c9aaf1fc0b0e808e21a to your computer and use it in GitHub Desktop.
Save mdPlusPlus/031ec2dac2295c9aaf1fc0b0e808e21a to your computer and use it in GitHub Desktop.
Download, patch, compile and install the newest stable Linux kernel with the ACS override patch (Ubuntu / Debian)
#!/bin/bash
function install_dependencies() {
echo "Installing dependencies..."
sudo apt -qq update
sudo apt -qq install -y curl git wget
# sudo apt -qq install -y bison flex kernel-package libelf-dev libssl-dev
sudo apt -qq install -y bison flex libelf-dev libssl-dev
}
function init() {
echo "Get the newest version of the script here: https://gist.github.com/mdPlusPlus/031ec2dac2295c9aaf1fc0b0e808e21a"
echo "Initializing..."
kernel_config=$(ls /boot/config-* | grep generic | sort -Vr | head -n 1)
grub_config="/etc/default/grub"
current_dir=$(pwd)
working_dir=$(mktemp -d)
cd "${working_dir}" || exit
install_dependencies
}
function get_newest_versions() {
echo "Retrieving newest version information..."
stable_releases_3x_url="https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/"
stable_releases_4x_url="https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/"
stable_releases_5x_url="https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/"
stable_releases_3x_html=$(curl -s "${stable_releases_3x_url}")
stable_releases_4x_html=$(curl -s "${stable_releases_4x_url}")
stable_releases_5x_html=$(curl -s "${stable_releases_5x_url}")
stable_releases_combined_html="${stable_releases_3x_html}${stable_releases_4x_html}${stable_releases_5x_html}"
stable_version=$(echo "${stable_releases_combined_html}" | grep -E -o 'linux-([0-9]{1,}\.)+[0-9]{1,}' | sort -Vru | head -n 1 | cut -d '-' -f 2)
mainline_link=$(curl -s https://www.kernel.org/ | grep https://git.kernel.org/torvalds/t/linux- | grep -Po '(?<=href=")[^"]*')
if ! [ -z "${mainline_link}" ]
then
mainline_version=$(echo "${mainline_link}" | cut -d '-' -f 2,3 | cut -d '.' -f 1,2)
else
mainline_version="unavailable"
fi
repo_pkg=$(apt search 'linux-source-' | grep 'linux-source' | cut -d '/' -f 1 | awk -F- 'NF<=3' | sort -Vr | head -n 1)
#repo_version=$(echo "${repo_pkg}" | cut -d '-' -f 3)
repo_version=$(apt search 'linux-source-' | grep 'linux-source' | sort -Vr | head -n 1 | cut -d ' ' -f 2)
}
function stable_or_mainline_or_repo() {
get_newest_versions
echo "Newest stable version is: ${stable_version}"
echo "Mainline version is: ${mainline_version}"
echo "Repository version is: ${repo_version}"
echo -n "Do you want to get a [s]table, the newest [m]ainline release candidate or the newest kernel from your [r]epositories? [S/m/r] "
read -r s_or_m_or_r
echo -n "Do you want to apply the acs override patch? Kernels below 4.10 are not supported. [Y/n] "
read -r acso
echo -n "Do you want to apply the experimental AMD AGESA patch to fix VFIO setups on AGESA 0.0.7.2 and some others? [y/N] "
read -r agesa
# echo -n "Do you want to apply the experimental AMD Vega PCI reset patch? [y/N] "
# read -r vega
echo -n "Do you want to install the kernel and its headers after compilation? [Y/n] "
read -r install
if [[ -z "${install}" || "${install}" == "y" || "${install}" == "Y" ]]
then
echo -n "Do you want to make this kernel the new default? [Y/n] "
read -r default
fi
echo -n "Do you want to install the kernel source? [y/N] "
read -r source
if [[ -z "${s_or_m_or_r}" || "${s_or_m_or_r}" == "s" || "${s_or_m_or_r}" == "S" ]]
then
stable_preparations
else
if [[ "${s_or_m_or_r}" == "m" || "${s_or_m_or_r}" == "M" ]]
then
if [ "${mainline_version}" == "unavailable" ]
then
echo "Mainline version currently unavailable. Exiting."
exit
else
mainline_preparations
fi
else
if [[ "${s_or_m_or_r}" == "r" || "${s_or_m_or_r}" == "R" ]]
then
repo_preparations
else
echo "Not a valid option. Exiting."
exit
fi
fi
fi
}
function try_acso_patch() {
## TODO: I really should not do this manually ...
acso_patch_5_12_13="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_5_12_13.patch"
acso_patch_5_6_12="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_5_6_12.patch"
acso_patch_5_4="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_5_4.patch"
acso_patch_4_18="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_4_18.patch"
acso_patch_4_17="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_4_17.patch"
acso_patch_4_14="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_4_14.patch"
acso_patch_4_10="https://raw.githubusercontent.com/mdPlusPlus/linux-acs-override/master/patches/acso_4_10.patch"
acso_patch_4_18_ubuntu="https://gist.github.com/mdPlusPlus/bb3df6248ffc7c6b3772ae0901659966/raw/acso_4_18_ubuntu.patch"
echo "Trying to apply acs override patch for 5.12.13+."
wget -O ../acso_5_12_13.patch "${acso_patch_5_12_13}"
if $(git apply --check ../acso_5_12_13.patch)
then
echo "Applying acs override patch for 5.12.13+."
git apply ../acso_5_12_13.patch
else
echo "Trying to apply acs override patch for 5.6.12+."
wget -O ../acso_5_6_12.patch "${acso_patch_5_6_12}"
if $(git apply --check ../acso_5_6_12.patch)
then
echo "Applying acs override patch for 5.6.12+."
git apply ../acso_5_6_12.patch
else
echo "Trying to apply acs override patch for 5.4+."
wget -O ../acso_5_4.patch "${acso_patch_5_4}"
if $(git apply --check ../acso_5_4.patch)
then
echo "Applying acs override patch for 5.4+."
git apply ../acso_5_4.patch
else
echo "Trying to apply acs override patch for 4.18+."
wget -O ../acso_4_18.patch "${acso_patch_4_18}"
if $(git apply --check ../acso_4_18.patch)
then
echo "Applying acs override patch for 4.18+."
git apply ../acso_4_18.patch
else
echo "Trying to apply acs override patch for 4.17+."
wget -O ../acso_4_17.patch "${acso_patch_4_17}"
if $(git apply --check ../acso_4_17.patch)
then
echo "Applying acs override patch for 4.17+."
git apply ../acso_4_17.patch
else
echo "Trying to apply acs override patch for 4.14+."
wget -O ../acso_4_14.patch "${acso_patch_4_14}"
if $(git apply --check ../acso_4_14.patch)
then
echo "Applying acs override patch for 4.14+."
git apply ../acso_4_14.patch
else
echo "Trying to apply acs override patch for 4.10+."
wget -O ../acso_4_10.patch "${acso_patch_4_10}"
if $(git apply --check ../acso_4_10.patch)
then
echo "Applying acs override patch for 4.10+."
git apply ../acso_4_10.patch
else
echo "Last resort: Trying to apply acs override patch for Ubuntu repository kernel source."
wget -O ../acso_4_18_ubuntu.patch "${acso_patch_4_18_ubuntu}"
if $(git apply --check ../acso_4_18_ubuntu.patch)
then
echo "Applying acs override patch for Ubuntu repository kernel source."
git apply ../acso_4_18_ubuntu.patch
else
echo "[ERROR]: Failed to apply acs override patch. Exiting."
exit
fi
rm -f ../acso_4_18_ubuntu.patch
fi
rm -f ../acso_4_10.patch
fi
rm -f ../acso_4_14.patch
fi
rm -f ../acso_4_17.patch
fi
rm -f ../acso_4_18.patch
fi
rm -f ../acso_5_4.patch
fi
rm -f ../acso_5_6_12.patch
fi
rm -f ../acso_5_2_13.patch
kernel_localversion+="-acso"
}
function try_agesa_patch() {
## by reddit user https://www.reddit.com/user/hansmoman/
## https://www.reddit.com/r/VFIO/comments/bqeixd/apparently_the_latest_bios_on_asrockmsi_boards/eo4neta
agesa_patch="https://clbin.com/VCiYJ"
agesa_patch_filename="agesa.patch"
echo "Trying to apply AMD AGESA patch."
wget -O ../${agesa_patch_filename} "${agesa_patch}"
if $(git apply --check ../${agesa_patch_filename})
then
echo "Applying AMD AGESA patch."
git apply ../${agesa_patch_filename}
else
echo "[ERROR]: Failed to apply AMD AGESA patch. Exiting."
exit
fi
rm -f ../${agesa_patch_filename}
kernel_localversion+="-agesa"
}
#function try_vega_patch() {
# ## Original by github user https://gist.github.com/numinit
#
# vega_patch_new="https://gist.github.com/mdPlusPlus/1754d82b53269ead770133829874fbfd/raw/"
# vega_patch="https://gist.githubusercontent.com/numinit/1bbabff521e0451e5470d740e0eb82fd/raw/0ec757c30cec041c3d0553bb246b5bc327d67abb/fix-vega-reset.patch"
# vega_patch_filename="fix-vega-reset.patch"
#
# echo "Trying to apply AMD Vega reset patch."
# wget -O ../${vega_patch_filename} "${vega_patch_new}"
#
# if $(git apply --check ../${vega_patch_filename})
# then
# echo "Applying AMD Vega reset patch."
# git apply ../${vega_patch_filename}
# else
# rm -f ../${vega_patch_filename}
# wget -O ../${vega_patch_filename} "${vega_patch}"
#
# if $(git apply --check ../${vega_patch_filename})
# then
# echo "Applying AMD Vega reset patch."
# git apply ../${vega_patch_filename}
# else
# echo "[ERROR]: Failed to apply AMD Vega reset patch. Exiting."
# exit
# fi
# fi
#
# rm -f ../${vega_patch_filename}
#
# kernel_localversion+="-vega"
#}
function stable_preparations() {
echo "The newest available stable kernel version is ${stable_version}. Kernels below 4.10 are not supported."
echo -n "Which version do you want to download? [${stable_version}] "
read -r user_version
if ! [ -z "${user_version}" ]
then
stable_version="${user_version}"
fi
kernel_version="${stable_version}"
kernel_name="linux-${kernel_version}"
## Test whether the version string starts with 3, 4, or 5 and sets the URL accordingly
if [[ $stable_version == 5* ]]
then
stable_link="${stable_releases_5x_url}linux-${stable_version}.tar.xz"
else
if [[ $stable_version == 4* ]]
then
stable_link="${stable_releases_4x_url}linux-${stable_version}.tar.xz"
else
if [[ $stable_version == 3* ]]
then
stable_link="${stable_releases_3x_url}linux-${stable_version}.tar.xz"
else
echo "${stable_version} is not a vaild version number. Exiting."
exit
fi
fi
fi
wget "${stable_link}"
tar xf "${kernel_name}.tar.xz"
cd "${kernel_name}" || exit
independent_code
}
function mainline_preparations() {
kernel_version="${mainline_version}"
kernel_name="linux-${kernel_version}"
wget "${mainline_link}"
tar xf "${kernel_name}.tar.gz"
cd "${kernel_name}" || exit
independent_code
}
function repo_preparations() {
kernel_name="${repo_pkg}"
sudo apt install "${repo_pkg}"
tar xf "/usr/src/${kernel_name}.tar.bz2"
cd "${kernel_name}" || exit
makefile_version=$(grep "^VERSION" Makefile | tr -d '[:space:]' | cut -d '=' -f 2)
makefile_patchlevel=$(grep "^PATCHLEVEL" Makefile| tr -d '[:space:]' | cut -d '=' -f 2)
makefile_sublevel=$(grep "^SUBLEVEL" Makefile | tr -d '[:space:]' | cut -d '=' -f 2)
#makefile_extraversion=$(grep "^EXTRAVERSION" Makefile| tr -d '[:space:]' | cut -d '=' -f 2)
if ! [ -z "${makefile_version}" ]
then
kernel_version="${makefile_version}"
if ! [ -z "${makefile_patchlevel}" ]
then
kernel_version="${makefile_version}.${makefile_patchlevel}"
if ! [ -z "${makefile_sublevel}" ]
then
kernel_version="${makefile_version}.${makefile_patchlevel}.${makefile_sublevel}"
fi
fi
fi
## linux-hwe (5.0.0-25.26~18.04.1) bionic; urgency=medium
## ->
## 5.0.0-25.26~18.04.1
#kernel_version=$(head -n 1 debian/changelog | cut -d '(' -f 2 | cut -d ')' -f 1)
## This somehow comes out empty when run in the script, but is successful when issued manually
independent_code
}
function independent_code() {
kernel_localversion=""
if [[ -z "${acso}" || ( "${acso}" != "n" && "${acso}" != "N" ) ]]
then
try_acso_patch
fi
if [[ "${agesa}" == "y" || "${agesa}" == "Y" ]]
then
try_agesa_patch
fi
# if [[ "${vega}" == "y" || "${vega}" == "Y" ]]
# then
# try_vega_patch
# fi
cp "${kernel_config}" .config
yes '' | make oldconfig
##set xhci_hcd to load as a module instead of including it directly into the kernel
##results in vfio_pci being able to grab usb controllers before xhci_hcd is able to
sed -i -e 's/^CONFIG_USB_XHCI_HCD=y/CONFIG_USB_XHCI_HCD=m/g' .config
##Enable AMD's HSA driver for ROCm support. Thanks at https://github.com/jfturcot
sed -i -e 's/^#\ CONFIG_HSA_AMD\ is\ not\ set/CONFIG_HSA_AMD=y/g' .config
# ## Disable debug builds
# sed -i -e 's/^CONFIG_DEBUG_INFO=y/CONFIG_DEBUG_INFO=n/g' .config
# sed -i -e 's/^CONFIG_DEBUG_KERNEL=y/CONFIG_DEBUG_KERNEL=n/g' .config
## check for versions containing only a single dot instead of two
## but only when not choosing the repo source
if [[ "${s_or_m_or_r}" != "r" && "${s_or_m_or_r}" != "R" ]]
then
dots=$(echo "${kernel_version}" | awk -F "." '{print NF-1}')
dashes=$(echo "${kernel_version}" | awk -F "-" '{print NF-1}')
if [ "${dots}" -lt 2 ]
then
if [ "${dashes}" -lt 1 ]
then
##5.1 -> 5.1.0
kernel_version="${kernel_version}.0"
else
##5.1-rc1 -> 5.1.0-rc1
kernel_version="$(echo "${kernel_version}" | sed 's/-/.0-/')"
fi
kernel_name="linux-${kernel_version}"
fi
fi
## Install kernel source, thanks https://github.com/meberlein
if [[ "${source}" == "y" || "${source}" == "Y" ]]
then
source_dir="/usr/src/linux-source-${kernel_version}${kernel_localversion}"
sudo mkdir -p "${source_dir}"
sudo cp -a "${working_dir}/${kernel_name}/." "${source_dir}"
fi
make clean
make -j "$(nproc)" bindeb-pkg LOCALVERSION="${kernel_localversion}" || exit
if [[ -z "${install}" || "${install}" == "y" || "${install}" == "Y" ]]
then
sudo dpkg -i ../linux-image-${kernel_version}${kernel_localversion}*.deb ../linux-headers-${kernel_version}${kernel_localversion}*.deb
if [[ -z "${default}" || "${default}" == "y" || "${default}" == "Y" ]]
then
if [ "$(lsb_release -s -d | cut -d ' ' -f 1)" == "Ubuntu" ]
then
## removing previous commented line
sudo sed -i -e 's/^#GRUB_DEFAULT=.*//g' "${grub_config}"
## commenting current line
sudo sed -i 's/^GRUB_DEFAULT=/#GRUB_DEFAULT=/' "${grub_config}"
## adding line
## TODO: multilanguage! only works for en-US (and other en-*) right now!
grub_line="GRUB_DEFAULT=\"Advanced options for Ubuntu>Ubuntu, with Linux ${kernel_version}${kernel_localversion}\""
sudo sed -i -e "s/^#GRUB_DEFAULT=.*/\0\n${grub_line}/" "${grub_config}"
sudo update-grub
fi
fi
fi
cd "${current_dir}" || exit
rm -rf "${working_dir}"
echo "Done."
echo "Make sure to check out https://github.com/gnif/vendor-reset"
}
## actual logic
init
stable_or_mainline_or_repo
@Steve1150
Copy link

trying to apply stable gives me the error:
Not a valid option. Exiting.

Doesn't matter what options i provide...

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Jun 26, 2020

@dglb99: Did you start your kernel with pcie_acs_override=downstream, pcie_acs_override=multifunction or pcie_acs_override=downstream,multifunction? Because that's how you tell the kernel to actually make use of the ACSO feature.
Change your /etc/default/grub file so the line starting with GRUB_CMDLINE_LINUX_DEFAULT= looks similar to this: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on pcie_acs_override=downstream,multifunction".
Then run sudo update-grub.

Take a look at the actual patch to understand what these options mean.

@Steve1150: Are you sure you're not running a really outdated version of this script? Btw, it expects the answers to be lowercase (or empty).

@mdPlusPlus
Copy link
Author

@dglb99: I've just found an error in my script which led to the kernel not actually getting patched with the newest patch. Oops.
Should be fixed now. Please try again.

I've also added support for uppercase answers.

@chirmstream
Copy link

chirmstream commented Jun 27, 2020

@mdPlusPlus, I redownloaded the script with the changes you just made and was able to compile the 5.7.6 kernel. I then edited the kernel start to include pcie_acs_override=downstream,multifunction. 20.04 PopOS no longer uses grub (I think the older versions used grub) but now uses systemd. Same concept though and I was able to figure it out. IOMMU groups are now separated correctly! Thank you so much for all your help, I don't think I would have been able to figure this all out on my own.
Now I just need the guest GPU to use the VirtIO drivers...

EDIT: I was able to get the vfio drivers to bind to the gpu, was not that bad

@meberlein
Copy link

@mdPlusPlus Thank you for the script. I have been using it for a while and it's been working great.

Only thing I added to my fork was for saving off the kernel source which became necessary for me to get the Nvidia driver installed on kernel 5.8.+ which may be something you could pull into you main version as well.

@mdPlusPlus
Copy link
Author

@meberlein Thanks, I'm going to look into it.

@mdPlusPlus
Copy link
Author

Vega reset patch is now obsolete. Check out https://github.com/gnif/vendor-reset

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Nov 21, 2020

@meberlein Do you really need the kernel source for that? Shouldn't that be covered by the kernel headers (which already get installed along the kernel)?

@DDzwiedziu
Copy link

DDzwiedziu commented Dec 2, 2020

Why the fsck the script deletes the whole temporary environment? I've selected to not install the kernel packages right away, the script happily patched*, compiled, build the packages, and equally happily deleted the whole environment, along with those packages.

* I had to amend the 5.6 patch to work on 5.8, by deleting the documentation patch, as a dirty workaround.

@mdPlusPlus
Copy link
Author

Yeah, that was kind of an oversight on my side. I'm going to change this behaviour next time I update the script.

@meberlein
Copy link

@meberlein Do you really need the kernel source for that? Shouldn't that be covered by the kernel headers (which already get installed along the kernel)?

My apologies you are absolutely correct. I tested again with kernel 5.9.14 and the new NVIDIA package that supports 5.9. Pointing to the header location worked as well. Not sure why the nvidia run package didn't find them at their default location but either way it works.

@Alexitachi
Copy link

Alexitachi commented Jul 13, 2021

will it work with kernel 5.13.1 .?

I tried with 5.13.1 ND got FAILED TO APPLY ACS

I tried with 5.7.6 bcz it was suppose to work with..but now I'm getting
make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 3
make: *** [Makefile:1463: bindeb-pkg] Error 2

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Jul 13, 2021

@Alexitachi The only thing I can tell you is that I'm still using the newest available ACS patch from https://gitlab.com/Queuecumber/linux-acs-override/-/tree/master/workspaces for this script.

Edit: Oh wow, I actually don't! Sorry, I will fix this as soon as I find time, probably tomorrow.
Edit2: Nevermind that new patch is identical to the old one.

@Alexitachi
Copy link

thanx..prob solved.

@stefanleh
Copy link

thanx..prob solved.

How did you solve it?
For me the ACS patch is broken since 5.12.10 on Fedora Server 33.

Best regards, Stefan

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Jul 27, 2021

Okay, so I updated the script to use the newest ACS override patch. (Sorry it took me so long!)
I didn't have any time testing it so far, so .. here be dragons. Please let me know if it works for you.

Edit: Newer kernels (5.11+? 5.12.13+) apparently do not work with the newest ACSO patch. Hopefully Queuecumber will release an update soon.

Edit2: https://gitlab.com/Queuecumber/linux-acs-override/-/issues/12
There won't be any updates to the linux-acs-override repository anymore.

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Jul 28, 2021

Update: I released a modified patch for all current kernels (5.12.13 to 5.13-rc4). All patches can be found in my new repository (https://github.com/mdPlusPlus/linux-acs-override). Eventually this script will probably move to this repository, too.

For some reason there's currently no kernel-package for current Ubuntu or Debian releases. I'm not quite sure what to make of it yet, but there's probably a new way to compile kernels on those platforms. As soon as I've figured out what to replace kernel-package with, I'm going to update the script.
This script isn't even using kernel-package anymore, my bad. Something is still off.

@stefanleh
Copy link

stefanleh commented Aug 3, 2021

Update: I released a modified patch for all current kernels (5.12.13 to 5.13-rc4). All patches can be found in my new repository (https://github.com/mdPlusPlus/linux-acs-override). Eventually this script will probably move to this repository, too.

Thank you for updating the patch. It works great with my docker based fedora ACS kernel build: https://hub.docker.com/r/stefanlehmann/acs_fedora

@meberlein
Copy link

meberlein commented Sep 18, 2021

I think there is a typo for kernels 5.12.13+, it downloads the 5_12_13 patch but then tries to apply the 5.12.6+ patch and errors.

@mdPlusPlus
Copy link
Author

mdPlusPlus commented Sep 18, 2021

@meberlein You're right! I should really replace that whole construct with something that doesn't need manual attention every now and then...

Edit: Fixed.

@TrickBit
Copy link

TrickBit commented Oct 2, 2021

@mousemat86
Copy link

How can I use this script for 5.13.0-21-generic? Is there a workaround?

@DDzwiedziu
Copy link

@mousemat86: I had 100% success rate (mind you 3 out of 3) when I've removed the doc part of the patch. But then I comment out half of the script and change the logic of other half (don't clean, don't download the patch if it's there, skip questions, etc.).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment