Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Build a custom kernel for Android emulator
git clone https://android.googlesource.com/kernel/goldfish/ -b android-goldfish-3.18
git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9
cd goldfish
export CROSS_COMPILE=x86_64-linux-android-
export ARCH=x86_64
export PATH=$PATH:/path/to/x86_64-linux-android-4.9/bin
make x86_64_ranchu_defconfig
make menuconfig  # enable overlayfs and namespaces support here
make -j8

Then create an Android virtual device via android avd. Here I use x86-64 image for API 24 (7.0 Nougat). Don't forget to use a larger internal memory size when creating the virtual device. (The default is 200MB and I use 2000MB) After that run the emulator with:

emulator -avd (avd_name) -kernel /path/to/goldfish/arch/x86/boot/bzImage -show-kernel
@pshp25

This comment has been minimized.

Copy link

@pshp25 pshp25 commented Jul 27, 2017

what does this line do?
make menuconfig # enable overlayfs and namespaces support here

and another question, I did what you said but aftar i used this command:
./emulator -avd Nexus_5X_API_23 -kernel ~/Desktop/goldfish/arch/x86/boot/bzImage -show-kernel

nothing is print on terminal and screen of mobile is black. can you help to solve this?

@yan12125

This comment has been minimized.

Copy link
Owner Author

@yan12125 yan12125 commented Jul 28, 2017

what does this line do?

I need some kernel features that are not enabled in normal Android emulators' kernels, so I enable it with the kernel's config tool.

nothing is print on terminal and screen of mobile is black. can you help to solve this?

A wild guess is that the compiled kernel is not compatible with your AVD. Maybe wrong architecture or wrong version.

@pshp25

This comment has been minimized.

Copy link

@pshp25 pshp25 commented Jul 28, 2017

Yes, you are right i change my avd (to android 7.0 x86-64 google api level 24) and now the boot animation is shown on screen but there is another problem!

I exactly do what you said in this page but after
./emulator -avd Nexus_5_API_24 -kernel ~/Desktop/goldfish/arch/x86/boot/bzImage -show-kernel
these texts is coming on terminal (come and again come) and screen of mobile is showing boot animation and never changes.

[ 270.896420] init: Service 'audioserver' is being killed...
[ 270.897184] init: Service 'cameraserver' is being killed...
[ 270.897986] init: Service 'media' is being killed...
[ 270.898725] init: Service 'netd' is being killed... [ 270.899573] init: Service 'audioserver' (pid 6354) killed by signal 9
[ 270.900441] init: Service 'audioserver' (pid 6354) killing any children in process group
[ 270.901661] init: Service 'cameraserver' (pid 6355) killed by signal 9 [ 270.902456] init: Service 'cameraserver' (pid 6355) killing any children in process group
[ 270.903456] init: Service 'media' (pid 6356) killed by signal 9
[ 270.904071] init: Service 'media' (pid 6356) killing any children in process group [ 270.904909] init: Service 'netd' (pid 6357) killed by signal 9
[ 270.905518] init: Service 'netd' (pid 6357) killing any children in process group
[ 270.915485] init: Service 'zygote_secondary' (pid 6587) killed by signal 11 [ 270.916367] init: Service 'zygote_secondary' (pid 6587) killing any children in process group
[ 271.918795] init: Starting service 'audioserver'...
[ 271.919822] init: Starting service 'cameraserver'... [ 271.920826] type=1400 audit(1501255007.918:530): avc: denied { getattr } for pid=6598 comm="audioserver" path="/vendor" dev="rootfs" ino=6328 scontext=u:r:audioserver:s0 tcontext=u:object_r:rootfs:s0 tclass=lnk_file permissive=0 [ 271.920848] init: couldn't write 6599 to /dev/cpuset/camera-daemon/tasks: No such file or directory
[ 271.920871] init: Starting service 'media'...
[ 271.921227] init: Starting service 'netd'... [ 271.927633] type=1400 audit(1501255007.918:531): avc: denied { getattr } for pid=6599 comm="cameraserver" path="/vendor" dev="rootfs" ino=6328 scontext=u:r:cameraserver:s0 tcontext=u:object_r:rootfs:s0 tclass=lnk_file permissive=0 [ 272.523185] type=1400 audit(1501255008.518:532): avc: denied { module_request } for pid=6601 comm="netd" kmod="netdev-dummy0" scontext=u:r:netd:s0 tcontext=u:r:kernel:s0 tclass=system permissive=0 [ 272.524725] type=1400 audit(1501255008.518:533): avc: denied { module_request } for pid=6601 comm="netd" kmod="net-pf-16-proto-5" scontext=u:r:netd:s0 tcontext=u:r:kernel:s0 tclass=system permissive=0 [ 275.926349] init: Starting service 'ipv6proxy'...
[ 275.927882] init: Starting service 'emu_hostapd'...
[ 275.929091] init: Starting service 'zygote'...
[ 275.930412] init: Starting service 'zygote_secondary'...

can you solve this problem please.

@yan12125

This comment has been minimized.

Copy link
Owner Author

@yan12125 yan12125 commented Jul 28, 2017

Sorry I have no idea what happened. Maybe logcat can provide more information.

@pshp25

This comment has been minimized.

Copy link

@pshp25 pshp25 commented Jul 28, 2017

I think this is logcat:
log
@yan12125

@pshp25

This comment has been minimized.

Copy link

@pshp25 pshp25 commented Jul 28, 2017

Can you upload your bzImage here?

@yan12125

This comment has been minimized.

Copy link
Owner Author

@yan12125 yan12125 commented Jul 29, 2017

Seems related: anbox/anbox#337. I'm not sure whether the latest system images includes that fix or not.

I don't have time to build a custom kernel now. Will post updates if any.

@harishvk27

This comment has been minimized.

Copy link

@harishvk27 harishvk27 commented Oct 11, 2017

Latest system images don't yet include the fix..

you have to build ASOP and get this fix to thread.cc added. I tried it worked.

I will have a blog get each and every step. But for now.. if you get the fix included things work.

@ppoffice

This comment has been minimized.

Copy link

@ppoffice ppoffice commented Oct 11, 2017

I tried x86, x86_64 and arm architecture in 3.18 and they all have this problem. But the 3.10 branch runs perfectly.

@harishvk27

This comment has been minimized.

Copy link

@harishvk27 harishvk27 commented Oct 12, 2017

With 3.10 branch.. what configuration is required while building AVD ? which API ?
If you can list some instructions that would be helpful for only kernel developers who don't want to build custom ASOP.

For me.. to get the thread.cc fix in custom ASOP has to be build.

@harishvk27

This comment has been minimized.

Copy link

@harishvk27 harishvk27 commented Oct 12, 2017

Steps to follow

  1. SYSTEM REQUIREMENTS
    I had ubuntu 16.04 with 16GB RAM and 1TB disk

also install
openjdk-8
git

  1. BASIC:
    =====
    mkdir /bin
    PATH=
    /bin:$PATH
    curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo

  2. GET ANDROID_SOURCE CODE and BUILD same
    ===================

mkdir ~/WORKING_DIRECTORY
cd ~/WORKING_DIRECTORY
repo init -u https://android.googlesource.com/platform/manifest -b android-7.0.0_r3
repo sync
source ./build/envsetup.sh
lunch asop_x86_64-eng

GET THE FIX into thread.cc ( in simple way: remove lines prefixed with - and add lines prefixed with + )

  • // Read every page from the high address to the low.
  • volatile uint8_t dont_optimize_this;
  • UNUSED(dont_optimize_this);
  • for (uint8_t* p = stack_top; p >= pregion; p -= kPageSize) {
  • dont_optimize_this = *p;
  • }
  • struct RecurseDownStack {
  • // This function has an intentionally large stack size.
    +#pragma GCC diagnostic push
    +#pragma GCC diagnostic ignored "-Wframe-larger-than="
  • NO_INLINE
  • static void Touch(uintptr_t target) {
  •  volatile size_t zero = 0;
    
  •  // Use a large local volatile array to ensure a large frame size. Do not use anything close
    
  •  // to a full page for ASAN. It would be nice to ensure the frame size is at most a page, but
    
  •  // there is no pragma support for this.
    
  •  volatile char space[kPageSize - 256];
    
  •  char sink ATTRIBUTE_UNUSED = space[zero];
    
  •  if (reinterpret_cast<uintptr_t>(space) >= target + kPageSize) {
    
  •    Touch(target);
    
  •  }
    
  •  zero *= 2;  // Try to avoid tail recursion.
    
  • }
    +#pragma GCC diagnostic pop
  • };
  • RecurseDownStack::Touch(reinterpret_cast<uintptr_t>(pregion));

make -j8

  1. DOWNLOAD KERNEL AND BUILD
    =====================

git clone https://android.googlesource.com/kernel/goldfish/ -b android-goldfish-3.18
cd goldfish
export CROSS_COMPILE=x86_64-linux-android-
export ARCH=x86_64
export PATH=$PATH:~/WORKING_DIRECTORY/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin
make x86_64_ranchu_defconfig
make menuconfig # enable overlayfs and namespaces support here
make -j8

  1. now it's time to lauch emulator:
    =====================

./emulator -mem 2048 ./arch/x86/boot/bzImage

To figure out all these took me some time.. as i had to build android also.. which i think could be avoided.

Any way welcome to word of android system programming, wish you all the best.

-thanks
harish

@ppoffice

This comment has been minimized.

Copy link

@ppoffice ppoffice commented Oct 12, 2017

@harishvk27 Thanks. I built emulator images from the Android source master branch and booted with my custom kernel, and it worked out. It seems that they have fixed it in thread.cc.

I also used 3.10 kernel branch with the same configurations @yan12125 posted and booted it with Android N SDK 24.

@harishvk27

This comment has been minimized.

Copy link

@harishvk27 harishvk27 commented Oct 12, 2017

One thing i am currently on is.. how to add network support..

Normal kernel which is part of ASOP when loaded on emulator.. internet is working fine.

But with custom kernel i don't see it working.

Is there a place in ASOP where i get to know the .config file ( menuconfig ) with which ASOP kernels are built?.

-thanks
harish

@ghost

This comment has been minimized.

Copy link

@ghost ghost commented Nov 27, 2017

Hello,

@harishvk27: With the AOSP if you did not specify a TARGET_PREBUILT_KERNEL the common kernel is used by default.
You can export TARGET_PREBUILT_KERNEL=/your/default/goldfish/arch/$ARCH/boot/zImage

  • In BoardConfig.mk
    • Remove TARGET_NO_KERNEL if set
  • In your device Makefile, have something like
    ifeq ($(TARGET_PREBUILT_KERNEL),)
    LOCAL_KERNEL:= device/ti/panda/kernel
    else
    LOCAL_KERNEL:=$(TARGET_PREBUILT_KERNEL)
    endif
    PRODUCT_COPY_FILES := $(LOCAL_KERNEL):kernel

If that doesn't work you can replace the x86_64_defconfig config in the kernel folder ($AOSP/kernel/) with your own config and recompile the Kernel, you need also to remove the common Kernel that is used by default.

Best regards,

@poooooooooooooi

This comment has been minimized.

Copy link

@poooooooooooooi poooooooooooooi commented Jan 3, 2018

hello,
I run "adb devices",but it show "emulator 5554 offline"
emulator is use of AOSP
make aosp_arm64-eng
kernel 3.10 arm64
how to solve the problem?

@palratnesh05

This comment has been minimized.

Copy link

@palratnesh05 palratnesh05 commented Mar 29, 2019

Hi I did not find #enable overlayfs and namespaces support here in menu

rudra@rudra-ThinkPad-E470:~/aosp7/goldfish$ make menuconfig
HOSTCC scripts/kconfig/mconf.o
HOSTCC scripts/kconfig/lxdialog/checklist.o
HOSTCC scripts/kconfig/lxdialog/util.o
HOSTCC scripts/kconfig/lxdialog/inputbox.o
HOSTCC scripts/kconfig/lxdialog/textbox.o
HOSTCC scripts/kconfig/lxdialog/yesno.o
HOSTCC scripts/kconfig/lxdialog/menubox.o
HOSTLD scripts/kconfig/mconf
scripts/kconfig/mconf Kconfig

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

make completed successfully (02:23 (mm:ss))

rudra@rudra-ThinkPad-E470:~/aosp7/goldfish$ make -j4
make: x86_64-linux-android-gcc: Command not found
scripts/kconfig/conf --silentoldconfig Kconfig
make: x86_64-linux-android-gcc: Command not found
SYSHDR arch/x86/syscalls/../include/generated/asm/unistd_32_ia32.h
SYSTBL arch/x86/syscalls/../include/generated/asm/syscalls_32.h
HOSTCC scripts/basic/bin2c
CHK include/config/kernel.release
UPD include/config/kernel.release
SYSHDR arch/x86/syscalls/../include/generated/asm/unistd_64_x32.h
SYSTBL arch/x86/syscalls/../include/generated/asm/syscalls_64.h
WRAP arch/x86/include/generated/asm/clkdev.h
WRAP arch/x86/include/generated/asm/cputime.h
WRAP arch/x86/include/generated/asm/dma-contiguous.h
WRAP arch/x86/include/generated/asm/early_ioremap.h
WRAP arch/x86/include/generated/asm/mcs_spinlock.h
WRAP arch/x86/include/generated/asm/scatterlist.h
SYSHDR arch/x86/syscalls/../include/generated/uapi/asm/unistd_32.h
CHK include/generated/uapi/linux/version.h
UPD include/generated/uapi/linux/version.h
SYSHDR arch/x86/syscalls/../include/generated/uapi/asm/unistd_64.h
CHK include/generated/utsrelease.h
HOSTCC scripts/genksyms/genksyms.o
UPD include/generated/utsrelease.h
SHIPPED scripts/genksyms/parse.tab.c
SYSHDR arch/x86/syscalls/../include/generated/uapi/asm/unistd_x32.h
CC scripts/mod/empty.o
/bin/sh: 1: x86_64-linux-android-gcc: not found
scripts/Makefile.build:257: recipe for target 'scripts/mod/empty.o' failed
make[2]: *** [scripts/mod/empty.o] Error 127
scripts/Makefile.build:402: recipe for target 'scripts/mod' failed
make[1]: *** [scripts/mod] Error 2
make[1]: *** Waiting for unfinished jobs....
SHIPPED scripts/genksyms/lex.lex.c
SHIPPED scripts/genksyms/keywords.hash.c
SHIPPED scripts/genksyms/parse.tab.h
HOSTCC scripts/genksyms/parse.tab.o
kernel/Makefile:145: *** No X.509 certificates found ***
HOSTCC scripts/genksyms/lex.lex.o
HOSTLD scripts/genksyms/genksyms
Makefile:556: recipe for target 'scripts' failed
make: *** [scripts] Error 2
make: *** Waiting for unfinished jobs....

make failed to build some targets (2 seconds)

please help me how to resolve this problem?

@PGoski

This comment has been minimized.

Copy link

@PGoski PGoski commented Jan 19, 2020

rudra@rudra-ThinkPad-E470:~/aosp7/goldfish$ make -j4
make: x86_64-linux-android-gcc: Command not found

checkout the correct branch in x86_64-linux-android-4.9 repository:

  1. cd x86_64-linux-android-4.9
  2. git checkout pie-b4s4-release

now kernel should start compiling

@johnnyboy18

This comment has been minimized.

Copy link

@johnnyboy18 johnnyboy18 commented Jan 23, 2020

rudra@rudra-ThinkPad-E470:~/aosp7/goldfish$ make -j4
make: x86_64-linux-android-gcc: Command not found

checkout the correct branch in x86_64-linux-android-4.9 repository:

1. cd x86_64-linux-android-4.9

2. git checkout pie-b4s4-release

now kernel should start compiling

Thank You!

@ysun

This comment has been minimized.

Copy link

@ysun ysun commented May 6, 2020

Hi, as Android kernel is moving on to v5.4, do you have guide to boot Android emulator with customized kernel v5.4 ?
As I know, latest kernel use clang instead of gcc to compile. And android 5.4 have dozens of modules (.ko) packaged in ramdisk, which make it difficult to customize kernel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.