This week in Anaconda #7

Posted on December 3, 2010 by Chris Lumens in work.

This week’s entry covers installation to a disk image, storage testing, and disks with 4k sector sizes.

Image installation

There are a whole lot of tools that take some input file (often but not always kickstart) and generate a disk image that’s got some filesystems and packages in it. The livecd creator stack of stuff is a prime example of this. You may also recognize this as basically what anaconda does. It stands to follow, then, that anaconda could do the livecd creator job as well. The reason we don’t is largely historic: for the longest time, anaconda was not really easy for anything else to make use of. It wasn’t installed like a real Python module and even if you managed to get it imported, the code wasn’t really organized in a way at all easy to make fun of. Essentially, anaconda was a ton of code off in its own world.

It’s taken us a couple years, but we’ve gotten to a point where anaconda is something you can import and use like any other Python module (albeit a pretty strange one). First, there was that whole storage rewrite that made the storage code much more modular and easy to add crazy new stuff to. Second, we reorganized the source tree and package layout. Third, for all my storage test code I worked on making it really importable outside the special anaconda environment.

With all that done, dlehman was able to crank out a long series of patches allowing image installs in just about a week. The way it works is that you first create some disk images (using dd, qemu-img, whatever) and then you pass them to anaconda as arguments. anaconda will ignore every other disk on the system and only use what you pass.

For now, you’d do something like this:

LIVECMD="anaconda --image=/opt/disk1.img --image=/opt/disk2.img \
         -m http://cannonball/install/rawhide/20101110/x86_64/os/" liveinst

If you’re doing a kickstart install, you may also want to use your “disks” as –ondisk= arguments. Instead of having to know what loopback device anaconda will assign, you can use an alternate syntax when starting anaconda that allows you to give the images whatever name you want, then refer to that in the kickstart.

None of this is really documented yet, and I haven’t given it a test run. However it’s looking promising so far and I’m hoping we can soon start to work it in as the guts of some of these image installation tools.

Storage testing

A huge part of testing any release is testing the installer. And a huge part of testing the installer is testing the partitioning code. This is probably the single most intensely tested area of the entire distribution. Unfortunately, this is also mostly manual labor.

If we could automate this testing, there are tons of benefits. First, we could run the tests after every git commit, or every night, or every week, or continuously. It doesn’t have to wait for someone to have free time before a release. We’d also be starting from a known on-disk state which means problems are reproducible. With clever enough tests, we could figure out problems in seldom-used code paths. And if run frequently enough, it could help us make sure our storage-related commits weren’t breaking things elsewhere. All told, it seems incredibly useful.

Well, turns out there IS a way to do it and I have it working as a part of autoqa. It’s pretty complicated, but the basic idea is that each test case first creates whatever disk images it needs in whatever state it needs them (pre-existing partitions? encrypted disk? garbage RAID metadata?), then starts up a VM with those disk images and runs a small program that runs a kickstart file through anaconda’s storage code. Then, it examines the final state of the disks and compares it to what’s expected and reports the results.

It’s really into the finishing touches stage now. Except for resizing, I have the partitioning section of the Fedora test matrix automated plus some more complicated tests. jlaska has been busy chewing through his list of related autoqa tasks to get this merged into master. In addition, I’ve put out a call for further test ideas and signed up to lead a training session at FUDCon Tempe in January.

4k sector disks

Speaking of storage, pjones has been hard at work on adding support for installing and booting from hard drives with 4k sector sizes. Currently, just about every single shipping hard drive has a 512 byte sector size. However, we expect this to change over time as disks with different sector sizes slowly come onto the market.

The first part of making this work is of course making sure the bootloader can recognize and read the disk so it can find the kernel. Much to no one’s surprise, the first part also ended up being the hardest part. The second part would have been making sure the kernel and user space tools (parted and anaconda, most importantly) worked. However, they’d all either been updated earlier or required nothing, so everything just worked.

Along the way, he also discovered that booting from a single 3 TB large drive works fine, though making a filesystem that large takes long enough to let you go grab lunch and the 3 TB vs. 3 TiB difference is now about 200 GB in size, which itself is a pretty large hard drive.