Skip to content

Instantly share code, notes, and snippets.

@pdxjohnny
Last active July 10, 2020 22:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pdxjohnny/aed9d495e606a816f4dfa51db6c2ebd4 to your computer and use it in GitHub Desktop.
Save pdxjohnny/aed9d495e606a816f4dfa51db6c2ebd4 to your computer and use it in GitHub Desktop.
Linux build and running tips and tricks

Linux Kernel Tips And Tricks

Random things that I forget and can't find on the interent again.

Debugging

Core dumps and kgdb: https://elinux.org/images/f/f0/Bingham.pdf

Exploitation

SMEP disable via ROP, KASLR bypasss via DMESG

Hacking With VIM

https://stackoverflow.com/questions/33676829/vim-configuration-for-linux-kernel-development

Build In Tree Modules

make -j $(($(nproc)*4)) M=arch/x86/kvm/ modules

Install Kernel To chroot

Here's how you install a kernel to a chroot. Or tar it up and send it over to another system for use there.

# Edit your config
make olddefconfig
export INSTALL_MOD_PATH=/path/to/your/chroot
export INSTALL_PATH=${INSTALL_MOD_PATH}/boot
mkdir -p "${INSTALL_PATH}"
# Use number of cores times 4, this usually is about how many
# threads your system can run
make -j $(($(nproc)*4))
make install
make modules_install -j $(($(nproc)*4))

Chroot

Instead of doing a regular chroot use systemd-nspawn which gives you a more fully featured chroot.

https://wiki.archlinux.org/index.php/Systemd-nspawn

Debugging With GDB

If you are lucky enough to be working on something thats in a VM you can get GDB working!

You need to run the kernel without KASLR (on the cmdline thats nokaslr).

If you don't disable KASLR gdb can't set breakpoints or display the source to you

Then you run QEMU with -s -S.

  • -s says enable GDB on port 1234
  • -S says wait to start the VM until GDB connects.

Run GDB with gdb ./vmlinux

Now run

(gdb) target remote 127.0.0.1:1234

To view the source code, type layout src

To focus back on the typey type window, type focus cmd

   ┌──arch/x86/kernel/machine_kexec_64.c─────────────────────────────────────────────────────────────────┐
   │353                 if (result)                                                                      │
   │354                         return result;                                                           │
   │355                                                                                                  │
   │356                 /* update purgatory as needed */                                                 │
   │357                 result = arch_update_purgatory(image);                                           │
   │358                 if (result)                                                                      │
   │359                         return result;                                                           │
   │360                                                                                                  │
   │361                 return 0;                                                                        │
   │362         }                                                                                        │
   │363                                                                                                  │
   │364         void machine_kexec_cleanup(struct kimage *image)                                         │
   │365         {                                                                                        │
   │366                 free_transition_pgtable(image);                                                  │
   │367         }                                                                                        │
   │368                                                                                                  │
   │369         /*                                                                                       │
   │370          * Do not allocate memory (or fail in any way) in machine_kexec().                       │
   │371          * We are past the point of no return, committed to rebooting now.                       │
   │372          */                                                                                      │
   │373         void machine_kexec(struct kimage *image)                                                 │
B+>374         {                                                                                        │
   │375                 unsigned long page_list[PAGES_NR];                                               │
   │376                 void *control_page;                                                              │
   │377                 int save_ftrace_enabled;                                                         │
   │378                                                                                                  │
   │379         #ifdef CONFIG_KEXEC_JUMP                                                                 │
   │380                 if (image->preserve_context)                                                     │
   │381                         save_processor_state();                                                  │
   │382         #endif                                                                                   │
   │383                                                                                                  │
   │384                 save_ftrace_enabled = __ftrace_enabled_save();                                   │
   │385                                                                                                  │
   │386                 /* Interrupts aren't acceptable while we reboot */                               │
   │387                 local_irq_disable();                                                             │
   │388                 hw_breakpoint_disable();                                                         │
   │389                                                                                                  │
   │390                 if (image->preserve_context) {                                                   │
   │391         #ifdef CONFIG_X86_IO_APIC                                                                │
   │392                         /*                                                                       │
   │393                          * We need to put APICs in legacy mode so that we can                    │
   │394                          * get timer interrupts in second kernel. kexec/kdump                    │
   │395                          * paths already have calls to restore_boot_irq_mode()                   │
   │396                          * in one form or other. kexec jump path also need one.                  │
   └─────────────────────────────────────────────────────────────────────────────────────────────────────┘
remote Thread 1.1 In: machine_kexec                                          L374  PC: 0xffffffff810635f0
(gdb) b *machine_kexec
Breakpoint 1 at 0xffffffff810635f0: file arch/x86/kernel/machine_kexec_64.c, line 374.
(gdb) c
Continuing.

Breakpoint 1, machine_kexec (image=0xffff888006f96800) at arch/x86/kernel/machine_kexec_64.c:374
(gdb)

KVM

https://lwn.net/Articles/658511/

Nested and VMCS

https://web.archive.org/web/20191105205408/http://events19.linuxfoundation.org/wp-content/uploads/2017/12/Improving-KVM-x86-Nested-Virtualization-Liran-Alon-Oracle.pdf

Contributions

In your patches, write all about what you're doing. The kernel doesn't seem to like comments very much. Instead they rely on people writing insanely detailed commit messages describing what they are changing.

Every time you change the kernel you're changing something that has been working fine for someone for X amount of long time. As such, you need to be convincing for why your change should be accepted! Make your coverletter and commit messages very detailed. The kernel is a big place. It may have been a long time since someone reading your patch series has looked at the place you're working. Be courtious and remind them of all the moving peices involved in what you're doing, and why and how you're changing them.

Etiquette

  • You can say patch or patchset in your coverletter. Just not in the commit messages themselves.

KVM

VMX

When a VM enters VMX root mode (aka "I'm gonna run some VMs mode") using the VMXON instruction, it sets up what's called a VMCS for each VM / virtualized processor it wants to run. All VMX related instructions operate on whatever VMCS we pointed to via the VMPTRLD instruction.

Nested

  • L0 == The host

  • L1 == The guest

  • L2 == The guest of the guest

vmx/nested.c is responsible for emulating VMX for L1. When L1 does a VMLAUNCH or VMRESUME, enter_guest_mode is called, which means that is_guest_mode will now return true.

Assembly

What is jmp 1f?

https://stackoverflow.com/a/27353169/12310488

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment