< Index

NixOS on OnePlus 6 with Extra Steps, or the Diary of my Descent into Madness

Did you know you can run NixOS on phones? I certainly did. I have experience making NixOS run on Oracle VPS (I made it use a custom partition scheme, which Oracle normally doesn't provide, not for free, anyway), on various ARM boards (it's nice when the boards support UEFI, but often they don't). I'm running NixOS on my laptop (x86_64), on my router (Banana Pi BPI-R3, requires U-Boot and a custom kernel config; the router config used to run on an x86_64 laptop), on my server/NAS (Radxa Rock 5A, luckily it provides UEFI and almost works well on the mainline kernel with default config; the server config used to run first on an x86 Oracle VPS, then on an arm64 Oracle VPS, then on the same laptop that was my router), and my phone is the last missing piece in this chain, especially since I wanted to ditch Android even before I became a Nix cultist (or a communist, for that matter).

I didn't want to use a Pinephone{, Pro} as a daily driver. I have hands-on experience with the original Pinephone. First of all, it seemed pretty sluggish. I know that theoretically it can change, and I'll be using a WM instead of a DE either way, but it still felt quite annoying. Worse yet, the battery life is pretty bad, and while suspend probably does help, if I were to use my phone for listening to music or playing Youtube videos in the background, suspend would be useless and it'd still die pretty quickly.

Pinephone Pro does solve the first problem, which led me to buy it in early 2022. While I got really lucky with the timing, as Visa/MasterCard stopped working in Russia shortly after, DHL unfortunately misdelivered the package. Worse yet, Pine64 was completely unresponsive, they took so long to get to this case that the DHL refund period ended, and only over a year later did I get a 50% refund from Pine64.

This situation, along with the fact Pinephone Pro's battery life is still pretty bad (I wanted to solve this with the keyboard case, but I heard that doesn't help that much either), led me to seek alternatives. The only other phone "well supported" by Mobile NixOS is OnePlus 6. OnePlus 6T has more or less the same support in the broader mobile Linux ecosystem, so I could simply contribute to Mobile NixOS to improve OnePlus 6T support - after all, it's a small upgrade over OnePlus 6 with an OLED screen, but it doesn't have a headphone jack - so it was a non-option for me, especially on mobile Linux where USB adapter compatibility is dubious (I haven't used Bluetooth ever since school days when we sent files to each other, and I don't plan to).

The first OnePlus 6 I bought ended up having broken Wi-Fi, so I had to return it. After I ordered the second one from a different seller, they said it's not in stock (!) and the order just timed out in a few weeks. Nothing went right with this phone, even when I tried buying a case, they sent me a case for a different phone. That got me pretty demotivated, at first I even tried buying a second-hand OnePlus 6 (at least if it's used it means it's usable, unlike the one with broken Wi-Fi), but on the third time, finally, I got a new (allegedly produced in 2022, I have no idea whether that's true or why they would still produce a 5 year old phone except maybe for spare parts) OnePlus 6 to tinker with, and I have full intentions of making it my daily driver. After all, I've been working on rofi-menu-stack and other components of my future hypothetical mobile UI stack for exactly that purpose. My current phone (Redmi K30 5G) is one year newer, supports 5G and is 120Hz, but it's a small sacrifice for running mainline Linux with the familiar userspace.

So the first thing I have to do to make the phone work is install an OS on it. How do I do it? Mobile NixOS has me covered, it should be quick and simple to build an image using its tooling and flash it, right?

Wrong! It actually uses a premade partition scheme (not sure whether it's configurable/how configurable it is), but I want full disk encryption, and it definitely doesn't have built-in support for generating LUKS images. And overall, I like to have full control and understanding of my system - that's why the distro I used before NixOS was Arch.

So, what do I do? Obviously, sidestep the entire thing and install NixOS with UEFI instead! Luckily, there's a UEFI implementation for OnePlus 6. In theory, it may support booting NixOS from a USB OTG drive, or a partition on the device itself! Let's check that theory.

  1. Day 1 - Flashing the Stock ROM
  2. Day 2 - Bootloader Shenanigans
  3. Day 3 - Configuring the Kernel
  4. Day 4 - Building the NixOS Config
  5. Day 5 - Flashing NixOS
  6. Day 6 - Unlocking LUKS
  7. Day 7 - Final Touchup
  8. Credits

Day 1 - Flashing the Stock ROM

...But first - the "new" phone from China came with a weird English ROM that says it's "OxygenOS" (OnePlus's official global version of Android), but doesn't want to OTA (or manually) update to the latest version. Yes, Chinese sellers love to flash dubious global versions when selling phones overseas - I'd really rather they didn't, but what's done is done, now I have to find a way to update it, because generally all custom flashing comes after updating your phone to make sure you get the latest firmware and etc.

No problem - all I have to do is just download the original ROM from Random People on the Internet, flash it to my to-be primary communications device (what could possibly go wrong), and we're good. I will use edl, which is a very useful Linux alternative to Windows GUI programs for interacting with Qualcomm devices in EDL mode, and oppo-decrypt, which is necessary for decrypting the ROM files before flashing - both are written by the same author, Bjoern Kerler!

The factory ROM contains a bunch of metadata, the proprietary Windows tool for flashing it, and the ROM itself - enchilada_22_J.50_210121.ops. First, I have to convert the .ops image into something I can flash with EDL. The first step is obviously decrypting it - python3 opscrypto.py <image>.ops --extractdir=../out (surprisingly, extractdir seems to be relative to the image file). Now we got a directory with a bunch of raw image files, the .elf loader binary used for interacting with the device (I don't need it since edl has a built-in loader for OnePlus 6T, which works for OnePlus 6 as well), a bunch of UFS provisioning files like provision_samsung.xml (they state that "provisioning UFS is an irrecoverable one time operation", so I decided not to inquire further), and most importantly settings.xml, which contains the actual info about what partitions go where. Here's a small sample:

<?xml version="1.0" encoding="utf-8" ?>
<Setting>
    <!-- snip -->
    <Program0>
        <program SECTOR_SIZE_IN_BYTES="4096" file_sector_offset="0" filename="" label="ssd" num_partition_sectors="2" partofsingleimage="false" physical_partition_number="0" readbackverify="false" size_in_KB="8.0" sparse="false" start_byte_hex="0x6000" start_sector="6" FileOffsetInSrc="0" SizeInSectorInSrc="0" SizeInByteInSrc="0" Sha256="0" />
        <program SECTOR_SIZE_IN_BYTES="4096" file_sector_offset="0" filename="persist.img" label="persist" num_partition_sectors="8192" partofsingleimage="false" physical_partition_number="0" readbackverify="true" size_in_KB="32768.0" sparse="true" start_byte_hex="0x8000" start_sector="8" FileOffsetInSrc="1377" SizeInSectorInSrc="65536" SizeInByteInSrc="33554432" Sha256="" />
        <program SECTOR_SIZE_IN_BYTES="4096" file_sector_offset="0" filename="" label="misc" num_partition_sectors="256" partofsingleimage="false" physical_partition_number="0" readbackverify="false" size_in_KB="1024.0" sparse="false" start_byte_hex="0x2008000" start_sector="8200" FileOffsetInSrc="0" SizeInSectorInSrc="0" SizeInByteInSrc="0" Sha256="0" force_erase="true" />
        <!-- snip -->
    </Program0>
    <Patch0>
        <patch SECTOR_SIZE_IN_BYTES="4096" byte_offset="2088" filename="gpt_main0.bin" physical_partition_number="0" size_in_bytes="8" start_sector="2" value="NUM_DISK_SECTORS-6." what="Update last partition 17 &apos;userdata&apos; with actual size in Primary Header." />
        <patch SECTOR_SIZE_IN_BYTES="4096" byte_offset="2088" filename="DISK" physical_partition_number="0" size_in_bytes="8" start_sector="2" value="NUM_DISK_SECTORS-6." what="Update last partition 17 &apos;userdata&apos; with actual size in Primary Header." />
        <!-- snip -->
    </Patch0>
    <!-- snip -->
</Setting>

The edl tool doesn't support this file format, but you know what it does support? The QFIL file format, with rawprogram and patch XML files! Luckily, I had a QFIL flash for a different Qualcomm phone lying around, and using it as reference I made the following Python script for converting settings.xml to the QFIL format:

with open('settings.xml', 'rt') as f:
    xml = f.read()
for intag, outtag, out in (('Program', 'data', 'rawprogram'), ('Patch', 'patches', 'patch')):
    for pr in xml.split(f'<{intag}')[1:]:
        num, data = pr.split('>', 1)
        lines = filter(lambda x: x, map(lambda x: x.strip(), data.split(f'</{intag}')[0].split('\n')))
        with open(f'{out}{num}.xml', 'wt') as f:
            print(f'<?xml version="1.0" ?>\n<{outtag}>', file=f)
            for line in lines:
                print(' ', line, file=f)
            print(f'</{outtag}>', file=f)

After running this I got 6 rawprogram+patch file pairs, which I simply flashed with edl qfil rawprogram<num>.xml patch<num>.xml . (after booting the phone in EDL mode of course). A few hours of 90% single-core CPU load (no idea why edl needs that, but whatever) later, I finally got the official ROM installed... it's still on Android 10, but that's only one OTA update away from the "latest and greatest" Android 11 ROM.

Some 15 hours after getting the phone, after utilizing lots of domain specific knowledge, we've reached the starting point. Isn't Android just wonderful?

Now that we have successfully installed the latest stock ROM through blood, sweat and tears (preferably to both A and B slots, luckily the OnePlus update UI offers local zip installation, so we can tell it to install the latest ROM again manually), we can proceed to uninstall this useless Google-infested crap (plus as I updated it from Android to 8 to 10 to 11, I could see how progressively worse the UI got; though this is mostly OnePlus's fault as the AOSP UI didn't change that much). In my case, I want to run UEFI (prerably I want to try running GRUB or systemd-boot). Luckily, there's a guide on the postmarketOS wiki. I don't need dualbooting with Android, so I just have to follow the "Erasing unused partitions/Custom formatting" section... Let's give it a go!

(Pretending I didn't just flash the new stock OS) first, I have to unlock the bootloader by enabling OEM unlocking in developer settings and running fastboot flashing unlock_critical and fastboot flashing unlock in fastboot (this allows flashing all partitions, and I may just need it, who knows).

Now onto the actual flashing - partitioning requires a decent recovery; TWRP is my usual go-to because it has a good feature set and I'm familiar with it.

In the recovery, I removed the partitions 13-17 (system_{a,b}, odm_{a,b}, userdata) and created two partitions (boot and root). I forgot to unmount userdata before flashing, so gdisk printed some errors, but surely it will be fine.

Now, before flashing UEFI, let's make sure it works via fastboot boot uefi.img... what? "Failed to load/authenticate boot image: Load Error"? Let's try booting the recovery... it doesn't boot either? Ugh, what went wrong? PMOS wiki does mention "on oneplus 6t you can only can remove /dev/sda17. Removing /dev/sda13-16 will cause the bootloader cant boot anything", but I have the normal OnePlus 6!

Fine, let's experiment. First, I have to reinstall the stock OS via EDL... but let's drop userdata from the xml files, I don't want to flash an empty 120GB partition.... uh? edl fails with DeviceClass - USBError(5, 'Input/Output Error'). Maybe switching the loader will help? Nope, the one bundled with the firmware doesn't work either... We're off to a great start. That's enough for today...

Day 2 - Bootloader Shenanigans

After booting up a Windows VM and passing the phone through to it, I was successfully able to use MSM Flash Tool - guess edl still has some bugs. After unbricking the phone, let's repeat the process, but step by step, making sure everything works after every step.

So, first, let's install the OTA update again (twice, i.e. in both slots) (now that I think of it, I should've tried updating from Android 8 straight to 11, not from 10 to 11, maybe that's why it didn't work the first time)... Now, after using edl before, bootloader lock state was preserved, but for some reason the MSM Flash Tool locked the bootloader, so let's unlock it again.

Now I'll remove partitions step by step, starting with userdata...

  1. After removing userdata (/dev/sda17), everything works fine
  2. After removing odm_b (/dev/sda16), everything still works fine
  3. After removing odm_a (/dev/sda15), everything continues to work fine
  4. After removing system_b (/dev/sda14), everything keeps working fine
  5. After removing system_a (/dev/sda13)... oh finally, it broke this time. Maybe it only broke now because the slot A is the currently selected slot?

So, with more info on what can and can not be done (and no clues regarding the reasons behind that), I do what I realized I should've tried, and download the oldest image I've found to flash with edl, to then update with the OTA update. After all, whoever wrote the article on PMOS wiki managed to do it somehow, right? If that doesn't work, I can try deleting the partitions after installing Android 10 or Android 9.

Um... this time MSM Flash Tool doesn't work either? this is... surprising. Just to be sure, what about edl? Still no? Okay, that's to be expected... Let's pray and try again! Maybe waiting after plugging the phone will help? Nope, edl just hangs! But hanging is not crashing, so this is a new reaction - let's try again... It worked! And immediately started spamming DeviceClass - USBError(19, 'No such device (it may have been disconnected)'). Well, seems like the USB cable is at fault here, not the software! I retract all my statements about edl having bugs (lol). Let's try switching from the official cable to some other... it works!

Android 8 is flashed... "System update installation failed"? I see, so I have to flash Android 8 -> 9 -> 11. That works for me.

Now, for science, let's see if it boots without system_b when slot B is selected... okay, the answer is "No". I see! Now let's try booting without the system partition after flashing an older Android version.

Doesn't work on Android 10 (this is a single line in a blog post, but it took a long time to check...). As for Android 9... let's actually change the method, I don't want to use old firmware. Instead, let's progressively remove files from the system partition until it doesn't boot anymore