cwshu/linux_kvm_armv8.md

## linux_kvm_armv8.md

      
    Raw
  

              linux_kvm_armv8.md
            
          
    linux guest on linux-kvm-arm host on armv8 FoundationModel

all you need


FoundationModel armv8 simulator
linux-kvm-arm kernel (including dtb(device tree blob))

kernel config


linux root file system: host.img && guest.img
userspace virtualization tool: qemu/kvm-tool
FoundationModel boot wrapper

all files

    downloads/ (download from virtualopensystem)
    ├── cookies.txt           
    ├── Image                
    ├── foundation-v8.dtb
    ├── .config
    ├── linaro-image-minimal-genericarmv8-20130526-319.rootfs.tar.gz (download from linaro)
    ├── lkvm-static
    └── qemu-system-aarch64

    boot-wrapper/             (clone from github:virtualopensystem/boot-wrapper)
    linux-kvm-arm/            (clone from github:virtualopensystem/linux-kvm-arm)
    qemu/                     (clone from github:virtualopensystem/qemu)
    
    result/
    ├── Image
    ├── foundation-v8.dtb
        (host.img use directory shared by NFS: /srv/nfsroot)
    ├── guest.img
    ├── qemu-system-aarch64
    └── lkvm-static
        (boot wrapper in build directory: boot-wrapper/image-foundation_NFS.axf)

download resources

    # Download rootfs from linaro
    wget http://releases.linaro.org/13.05/openembedded/aarch64/linaro-image-minimal-genericarmv8-20130526-319.rootfs.tar.gz

    # Download things from virtual-open-system, login first and save cookie
    wget --save-cookies cookies.txt --post-data 'username=downloader&password=uploading1' \
        http://www.virtualopensystems.com/logintrml/

    # Download kernel config
    wget --load-cookies cookies.txt http://www.virtualopensystems.com/downloads/guides/kvm_on_armv8/config_kvm_arm64 -O .config
    # Download kernel 
    wget --load-cookies cookies.txt http://www.virtualopensystems.com/downloads/guides/kvm_on_armv8/Image
    # Download kernel device tree blob for foundation-v8
    wget --load-cookies cookies.txt http://www.virtualopensystems.com/downloads/guides/kvm_on_armv8/foundation-v8.dtb

    # Download kvm tool
    wget --load-cookies cookies.txt http://www.virtualopensystems.com/downloads/guides/kvm_on_armv8/lkvm-static
    # Download qemu
    wget --load-cookies cookies.txt http://www.virtualopensystems.com/downloads/guides/kvm_on_armv8/qemu-system-aarch64
installation steps


FoundationModel

login and download from http://www.arm.com/products/tools/models/fast-models/foundation-model.php
tar -xf to extract and executable in ./Foundation_Platformpkg/models/Linux64_GCC-4.1/Foundation_Platform


build kernel


     # get kernel source(kvm-arm64 branch)
     git clone git://github.com/virtualopensystems/linux-kvm-arm.git -b kvm-arm64-stable

     # get kernel config from virtualopensystems
     cd linux-kvm-arm 
     cp ../downloads/.config .

     # build by aarch64-linux-gnu-gcc
     ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j4

     # target file: Image and dtb(device tree blob)
     cp arch/arm64/boot/Image result/
     cp arch/arm64/boot/dts/foundation-v8.dtb ../result/


rootfs: host and guest image


download from linaro:


host img: use NFS to share /srv/nfsroot directory


apt-get install nfs-kernel-server nfs-common


initial /srv/nfsroot
sudo mkdir -p /srv/nfsroot/
sudo tar -xf downloads/linaro-image-minimal-genericarmv8-20130526-319.rootfs.tar.gz \
  -C /srv/nfsroot/


/etc/exports add one mountpoint
/srv/nfsroot 192.168.1.10/24(rw,sync,no_root_squash,no_subtree_check,insecure)


note: NFS IP address is used in boot-wrapper compiler arguments and VM boot time in FoundationModel.


guest img


create 512MB empty image, format file system as ext3, mount as loop device, dump android userland files to image.
qemu-img create -f raw ./result/guest.img 512M
mkfs.ext3 ./result/guest.img
mkdir mnt
sudo mount -o loop ./result/guest.img mnt
sudo tar -xf downloads/linaro-image-minimal-genericarmv8-20130526-319.rootfs.tar.gz \
     -C mnt/
sudo umount mnt


Tool of userspace emulation, kvm-tool and qemu


kvmtool
# kvm-tool source is also linux-kernel, use virtualopensystems' repo for diff branch: kvmtool
# in linux-kvm-arm
git checkout -b kvmtool

# build by aarch64-linux-gnu-gcc
cd tools/kvm/
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make lkvm-static

# target file (tools/kvm/lkvm-static)
cp lkvm-static ../../../result/


qemu


build schroot environment for cross compilation
build qemu 需要 ARM64 的 package, 這邊選擇把 build environment 放在 schroot 裡.
schroot 使用 Ubuntu raring(13.04), 因為有人做 raring 的 ARM64 packages repo: https://people.debian.org/~wookey/bootstrap.html
(note: this repo is unstable.)


install schroot
sudo apt-get install sbuild
sudo sbuild-update --keygen
sudo sbuild-adduser <your-username>


create raring schroot


Ubuntu raring is EOL, so official repo change to http://old-releases.ubuntu.com/ubuntu/


create at /srv/chroots/raring
sudo sbuild-createchroot --components=main,universe raring \       
  /srv/chroots/raring http://old-releases.ubuntu.com/ubuntu/


initailize raring schroot by installation and configuring


enter raring schroot: sudo sbuild-shell raring-amd64-sbuild


install common tools: apt-get install nano vim wget sudo pkgbinarymangler git


install apt-transport-https, apt-utils in schroot: apt-get install apt-transport-https apt-utils


configure apt/dpkg settings


same as virtualopensystem's tutorial, only modify
# file /etc/apt/sources.list
# use EOL repo url (http://archive.ubuntu.com/ubuntu/ => http://old-releases.ubuntu.com/ubuntu/)

# file /etc/apt/sources.list.d/bootstrap.list (https)
deb https://people.debian.org/~wookey/bootstrap/ubunturepo/ raring-bootstrap main universe
deb-src https://people.debian.org/~wookey/bootstrap/ubunturepo/ raring-bootstrap main universe


install key
wget http://people.debian.org/~wookey/bootstrap/bootstrap-archive.key
apt-key add bootstrap-archive.key


install all required AArch64 packages when compiling qemu::
sudo dpkg --add-architecture arm64
sudo apt-get update
sudo apt-get install libglib2.0-dev:arm64 zlib1g-dev:arm64
sudo apt-get install libpixman-1-dev:arm64 crossbuild-essential-arm64
sudo apt-get install python


build qemu
# target file: qemu-system-aarch64
cp aarch64-softmmu/qemu-system-aarch64 ../result/


boot wrapper (NFS or VirtIO)


soft link linux-kvm-arm's thing(kernel, dtb ...) to boot-wrapper directory
ln -sf ../linux-kvm-arm/ LINUX
ln -sf LINUX/scripts/dtc/dtc
ln -sf LINUX/arch/arm64/boot/Image
ln -sf LINUX/arch/arm64/boot/dts/foundation-v8.dts
ln -sf LINUX/arch/arm64/boot/dts/rtsm_ve-aemv8a.dts
ln -sf LINUX/arch/arm64/boot/dts/skeleton.dtsi


build boot-wrapper


host.img using NFS
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
BOOTARGS='"root=/dev/nfs nfsroot=192.168.1.10:/srv/nfsroot/ \
           ip=dhcp rw console=ttyAMA0 maxcpus=1"' \
FDT_SRC=foundation-v8.dts IMAGE=image-foundation_NFS.axf


host.img using VirtIO


running


run linux-kvm on FoundationModel
run_FoundationModel.sh:
#!/bin/sh
~/workspace/FoundationModel/Foundation_Platformpkg/models/Linux64_GCC-4.1/Foundation_Platform \
    --image image-foundation_NFS.axf \
    --network=nat --network-nat-ports="2222=22"
note: linux-kvm has running ssh server by default, so redirect ssh port for remote login.


run linux guest by qemu on linux-kvm
run_qemu.sh:
#!/bin/sh
./qemu-system-aarch64 \
-nographic \
-enable-kvm \
-kernel Image \
-m 512 -M virt -cpu host \
-drive file=guest.img,id=fs,if=none \
-device virtio-blk-device,drive=fs \
-append "earlyprintk
         console=ttyAMA0 mem=512M rootwait root=/dev/vda rw"


reference


virtualopensystem armv8 guide


FoundationModel (ARM infomation center)

command line options