rdump/Android Upgrade when OTA fails.txt

## Android Upgrade when OTA fails.txt
2016-03-19

Here are a couple of options for getting to latest desired Android release when OTA fails on your Google Nexus.

Option 1 (manual OTA chain) may allow preservation of root access.
Option 2 (flashing new OS but not userdata) will require re-rooting if root access is needed.

Each of these options presumes you have installed and in your $PATH 'android', 'adb', 'fastboot', and other parts they need from the "Android SDK Tools" and "Android SDK Platform-tools" packages at http://developer.android.com/sdk/installing/index.html?pkg=tools and http://developer.android.com/sdk/index.html#Other
Also, you want to be relatively current on the release versions per http://tools.android.com/recent/updatingsdkfromcommand-line
$ android list sdk
Once you have the list of available updates, look particularly for new versions of "Android SDK Tools" and "Android SDK Platform-tools", they'll typically be items 1 and 2 if there's an update
$ android update sdk --no-ui --filter 1,2	# Change the numbers as needed, or skip this command if there's no update necessary

=======
1) Follow the OTA chain manually
-------
You need a working recovery to sideload the OTA archives.  But attempting an OTA upgrade can damage the recovery past usability if you have CWM or TWRP recovery, perhaps due to the "error" in the fingerprint (below).  If this has happened on your device, fix recovery by:
Using TWRP Recovery Manager.apk to reinstall from within Android, or
Obtaining the current TWRP for your device (http://teamw.in/devices/) and using fastboot to reflash it with
$ adb reboot bootloader
$ fastboot flash recovery twrp-#.#.#.#-devicecodename.img

Download the OTAs from links provided via @jduck's DroidSec http://www.droidsec.org/ and gists https://gist.github.com/jduck (or for older versions archive lists on on xda-dev threads like http://forum.xda-developers.com/nexus-6/general/ref-nexus-6-stock-ota-urls-t2906493 [there are other threads for other devices]).
There may be a multi-step OTA chain to follow from the phone's current release through to the desired release. Be sure to get all the archives necessary to follow these steps, following along by release code to the next release code (e.g., when "LMY47Z from LMY47D" is available but you want to get to LMY48I, you need as well "LMY48I from LMY47Z"). You'll apply these in the step order.

If you have rooted or otherwise modified your system partition, it's become more difficult to follow OTAs. In Android 5, there was a rudimentary check on build fingerprint, which fingerprinted from RAM rather than from /system.  In Android 6, there's a more detailed checksum check because the entire system partition is no longer included in the OTA; it's a binary patch of a known system partition. This will break if you have modified your system partition with root, etc.

More recent TWRP recoveries will ask you if you want to keep system read-only; this is likely necessary if you're going to follow OTAs, even manually.

First check for system build fingerprint:

"You can no longer flash OTA updates using a custom recovery without modifying the updater-script file. There's an "error" in the script which, rather than checking the build fingerprint from /system, it checks it from RAM. So when you are booted in TWRP, it's actually reading the version that TWRP is based on, not your ROM.
You just have to delete (or modify if you want to keep the check in place) one line in the updater-script file:
	| getprop("ro.build.fingerprint") == "google/shamu/shamu:5.0.1/LRX22C/1602158:user/release-keys" ||
	|    getprop("ro.build.fingerprint") == "google/shamu/shamu:5.1/LMY47D/1743759:user/release-keys" ||
	|    abort("Package expects build fingerprint of google/shamu/shamu:5.0.1/LRX22C/1602158:user/release-keys or google/shamu/shamu:5.1/LMY47D/1743759:user/release-keys; this device has " + getprop("ro.build.fingerprint") + ".");
If you delete that, it should install using TWRP."

Newer check for piecemeal integrity of system, which will fail and likely render the device unbootable if bypassed on a device with a modified system:

In Android 6 signed OTAs on rooted devices where system was modified, we'll encounter mismatches on a check of system image integrity.  This check is done because the OTAs are binary patches of existing image, and no longer contain a full system image for overwriting what was on the phone.  The updater-script as patched above will say, when sideloaded: "EMMC:/dev/block/platform/msm_sdcc.1/by-name/boot:###:$sum:####:$sum" has unexpected contents". Deleting the check in updater-script will just get us into trouble. This is because during- and post-install checksums of the system partition will then fail, leaving a partially updated and likely unusable/unbootable system partition.  Recover with option 2 system partition overwrite with stock, then re-root.

Note that some re-zip operations of the OTA will lose metadata on META-INF/com/google/android/update-binary, rendering it unexecutable. Editing the updater-script in-place with an editor that works on zip contents (e.g., BBEdit on Mac OS X) is one way to avoid the problem.

If you have a match on the OTA system check, then chaining is possible:

Each OTA $archive in the chain of steps for the upgrade will be something like this: c46173017156abb27e0d39aee700c1392e32f224.signed-shamu-LMY47Z-from-LMY47D-fullradio-fix-superblock.zip

For each OTA $archive, unzip it, make the above edit to remove the fingerprint check (or adjust it to match the TWRP fingerprint), and if on Mac OX for safety's sake kill any junk dotfile inserts with something like this:
$ find $archiveextractdirectory -name .DS_Store -exec rm {} \;
Then rezip it, and sideload it as follows
$ adb reboot bootloader
Select recovery
	or
$ adb reboot recovery
Then select sideload (on the advanced page for TWRP).
If given the option, don't select to wipe caches (Dalvik, etc.) on the sideload page.
When the phone indicates ready, run the sideload for the current OTP step $archive
$ adb sideload $archive

Then reboot into the new release. Android may do some app optimization and other post-processing. If you are rooted and have apps that attempt to preserve root for you, they may succeed.

Repeat for each $archive in the OTA chain you've downloaded and fingerprint-fixed, until you're at the final desired release.

If a sideload fails along the way because you killed a needed cache, the step was mislabeled and you ended up with a bad file, or other reasons, then you can still do the upgrade the second way, below.


=======
2) Flash up directly
-------
You can take the parts you need from a release, and flash them, while deliberately leaving your userdata like installed apps, settings, media contents, &c. intact (e.g., by not doing a fastboot -w update... from the release's flash-all.sh)

Download the target release from https://developers.google.com/android/nexus/images
Decompress it, and pull out the radio & bootloader file if present:
	radio-$devicecodename-{varies}.img
	bootloader-$devicecodename-{varies}.img
Also decompress the interior zip to pull out the following needed files:
	boot.img
	cache.img
	recovery.img	# Skip this if you're using/going to use TWRP
	system.img
Download the TWRP recovery, if you want to install it rather than the stock recovery, from http://teamw.in/devices/

Boot to the bootloader:
$ adb reboot bootloader # or fastboot reboot-bootloader
Once the bootloader is running on the device, fastboot flash the new bootloader, cache, recovery, radio and system:
$ fastboot flash bootloader bootloader-<variable>.img
$ fastboot reboot-bootloader
$ fastboot flash cache cache.img
$ fastboot reboot-bootloader
$ fastboot flash recovery recovery.img	# Skip this if you're using/going to use TWRP
$ fastboot reboot-bootloader
$ fastboot flash radio radio-<variable>.img
$ fastboot reboot-bootloader
$ fastboot flash system system.img	# This will take a while due to size

If you're doing TWRP recovery, restart the Bootloader, and flash TWRP recovery.
$ fastboot flash recovery twrp-#.#.#.#-shamu.img

Then reboot into your new release.  At first boot, Android will probably optimize apps for the new release.

Following this, any of your apps that have been added to /system (BusyBox manager, etc.) will need to have that done again, if they don't do it automagically when you run them.  If you want root, you'll almost certainly need to re-root the device.
	2016-03-19

	Here are a couple of options for getting to latest desired Android release when OTA fails on your Google Nexus.

	Option 1 (manual OTA chain) may allow preservation of root access.
	Option 2 (flashing new OS but not userdata) will require re-rooting if root access is needed.

	Each of these options presumes you have installed and in your $PATH 'android', 'adb', 'fastboot', and other parts they need from the "Android SDK Tools" and "Android SDK Platform-tools" packages at http://developer.android.com/sdk/installing/index.html?pkg=tools and http://developer.android.com/sdk/index.html#Other
	Also, you want to be relatively current on the release versions per http://tools.android.com/recent/updatingsdkfromcommand-line
	$ android list sdk
	Once you have the list of available updates, look particularly for new versions of "Android SDK Tools" and "Android SDK Platform-tools", they'll typically be items 1 and 2 if there's an update
	$ android update sdk --no-ui --filter 1,2 # Change the numbers as needed, or skip this command if there's no update necessary

	=======
	1) Follow the OTA chain manually
	-------
	You need a working recovery to sideload the OTA archives. But attempting an OTA upgrade can damage the recovery past usability if you have CWM or TWRP recovery, perhaps due to the "error" in the fingerprint (below). If this has happened on your device, fix recovery by:
	Using TWRP Recovery Manager.apk to reinstall from within Android, or
	Obtaining the current TWRP for your device (http://teamw.in/devices/) and using fastboot to reflash it with
	$ adb reboot bootloader
	$ fastboot flash recovery twrp-#.#.#.#-devicecodename.img

	Download the OTAs from links provided via @jduck's DroidSec http://www.droidsec.org/ and gists https://gist.github.com/jduck (or for older versions archive lists on on xda-dev threads like http://forum.xda-developers.com/nexus-6/general/ref-nexus-6-stock-ota-urls-t2906493 [there are other threads for other devices]).
	There may be a multi-step OTA chain to follow from the phone's current release through to the desired release. Be sure to get all the archives necessary to follow these steps, following along by release code to the next release code (e.g., when "LMY47Z from LMY47D" is available but you want to get to LMY48I, you need as well "LMY48I from LMY47Z"). You'll apply these in the step order.

	If you have rooted or otherwise modified your system partition, it's become more difficult to follow OTAs. In Android 5, there was a rudimentary check on build fingerprint, which fingerprinted from RAM rather than from /system. In Android 6, there's a more detailed checksum check because the entire system partition is no longer included in the OTA; it's a binary patch of a known system partition. This will break if you have modified your system partition with root, etc.

	More recent TWRP recoveries will ask you if you want to keep system read-only; this is likely necessary if you're going to follow OTAs, even manually.

	First check for system build fingerprint:

	"You can no longer flash OTA updates using a custom recovery without modifying the updater-script file. There's an "error" in the script which, rather than checking the build fingerprint from /system, it checks it from RAM. So when you are booted in TWRP, it's actually reading the version that TWRP is based on, not your ROM.
	You just have to delete (or modify if you want to keep the check in place) one line in the updater-script file:
	\| getprop("ro.build.fingerprint") == "google/shamu/shamu:5.0.1/LRX22C/1602158:user/release-keys" \|\|
	\| getprop("ro.build.fingerprint") == "google/shamu/shamu:5.1/LMY47D/1743759:user/release-keys" \|\|
	\| abort("Package expects build fingerprint of google/shamu/shamu:5.0.1/LRX22C/1602158:user/release-keys or google/shamu/shamu:5.1/LMY47D/1743759:user/release-keys; this device has " + getprop("ro.build.fingerprint") + ".");
	If you delete that, it should install using TWRP."

	Newer check for piecemeal integrity of system, which will fail and likely render the device unbootable if bypassed on a device with a modified system:

	In Android 6 signed OTAs on rooted devices where system was modified, we'll encounter mismatches on a check of system image integrity. This check is done because the OTAs are binary patches of existing image, and no longer contain a full system image for overwriting what was on the phone. The updater-script as patched above will say, when sideloaded: "EMMC:/dev/block/platform/msm_sdcc.1/by-name/boot:###:$sum:####:$sum" has unexpected contents". Deleting the check in updater-script will just get us into trouble. This is because during- and post-install checksums of the system partition will then fail, leaving a partially updated and likely unusable/unbootable system partition. Recover with option 2 system partition overwrite with stock, then re-root.

	Note that some re-zip operations of the OTA will lose metadata on META-INF/com/google/android/update-binary, rendering it unexecutable. Editing the updater-script in-place with an editor that works on zip contents (e.g., BBEdit on Mac OS X) is one way to avoid the problem.

	If you have a match on the OTA system check, then chaining is possible:

	Each OTA $archive in the chain of steps for the upgrade will be something like this: c46173017156abb27e0d39aee700c1392e32f224.signed-shamu-LMY47Z-from-LMY47D-fullradio-fix-superblock.zip

	For each OTA $archive, unzip it, make the above edit to remove the fingerprint check (or adjust it to match the TWRP fingerprint), and if on Mac OX for safety's sake kill any junk dotfile inserts with something like this:
	$ find $archiveextractdirectory -name .DS_Store -exec rm {} \;
	Then rezip it, and sideload it as follows
	$ adb reboot bootloader
	Select recovery
	or
	$ adb reboot recovery
	Then select sideload (on the advanced page for TWRP).
	If given the option, don't select to wipe caches (Dalvik, etc.) on the sideload page.
	When the phone indicates ready, run the sideload for the current OTP step $archive
	$ adb sideload $archive

	Then reboot into the new release. Android may do some app optimization and other post-processing. If you are rooted and have apps that attempt to preserve root for you, they may succeed.

	Repeat for each $archive in the OTA chain you've downloaded and fingerprint-fixed, until you're at the final desired release.

	If a sideload fails along the way because you killed a needed cache, the step was mislabeled and you ended up with a bad file, or other reasons, then you can still do the upgrade the second way, below.


	=======
	2) Flash up directly
	-------
	You can take the parts you need from a release, and flash them, while deliberately leaving your userdata like installed apps, settings, media contents, &c. intact (e.g., by not doing a fastboot -w update... from the release's flash-all.sh)

	Download the target release from https://developers.google.com/android/nexus/images
	Decompress it, and pull out the radio & bootloader file if present:
	radio-$devicecodename-{varies}.img
	bootloader-$devicecodename-{varies}.img
	Also decompress the interior zip to pull out the following needed files:
	boot.img
	cache.img
	recovery.img # Skip this if you're using/going to use TWRP
	system.img
	Download the TWRP recovery, if you want to install it rather than the stock recovery, from http://teamw.in/devices/

	Boot to the bootloader:
	$ adb reboot bootloader # or fastboot reboot-bootloader
	Once the bootloader is running on the device, fastboot flash the new bootloader, cache, recovery, radio and system:
	$ fastboot flash bootloader bootloader-<variable>.img
	$ fastboot reboot-bootloader
	$ fastboot flash cache cache.img
	$ fastboot reboot-bootloader
	$ fastboot flash recovery recovery.img # Skip this if you're using/going to use TWRP
	$ fastboot reboot-bootloader
	$ fastboot flash radio radio-<variable>.img
	$ fastboot reboot-bootloader
	$ fastboot flash system system.img # This will take a while due to size

	If you're doing TWRP recovery, restart the Bootloader, and flash TWRP recovery.
	$ fastboot flash recovery twrp-#.#.#.#-shamu.img

	Then reboot into your new release. At first boot, Android will probably optimize apps for the new release.

	Following this, any of your apps that have been added to /system (BusyBox manager, etc.) will need to have that done again, if they don't do it automagically when you run them. If you want root, you'll almost certainly need to re-root the device.