Submitted by admin on
Some notes on partitioned disk images - how to create, initialize and mount them. This information can be quite handy when working with Virtual Machines.
Creating a blank file system
This part is easy and relatively well documented - so you've probably seen this before:
Use the dd and mkfs.ext3 utilities to create large empty file space. Then, use the mount utility to mount the file space to a convenient directory.
- Use the /dev/zero device as the input source for dd so that the output file will be full of zero's.
- Set the block size to a convenient number. I like 1MB blocks - but anything will do.
- Set the output block count to the number of blocks that you want to write out. If you want, for example, a 4GB file and you are using 1MB blocks: You will need a block count of 4K.
- After the file has been created, use mkfs.ext3 to initialize the file system in the image.
- Finally, you can mount the image to a local directory and start copying files into it.
Putting all this together:
$ dd if=/dev/zero bs=1M count=4K of=mypartition.img 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 145.067 seconds, 29.6 MB/s $ sudo /sbin/mkfs.ext3 mypartition.img mke2fs 1.39 (29-May-2006) mypartition.img is not a block special device. Proceed anyway? (y,n) Y Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 524288 inodes, 1048576 blocks 52428 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1073741824 32 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. $ mkdir ./fs $ sudo mount -o loop mypartition.img ./fs $ df -h Filesystem Size Used Avail Use% Mounted on ... mydisk.img 4.0G 137M 3.7G 4% ./fs
Now you can copy files to the ./fs directory and they will be written to the file system in the mypartition.img file. Later, when you mount that file system on a virtual machine, the files will be readable as expected.
If you need a different type of file system - fat32, for example - check the man page for the mkfs utility. A vast array of file systems are supported by the major Linux distros.
Creating a Partitioned Disk Image
Creating partitioned disk images is a little more tricky. The procedure is basically similar to the above procedure - but the image file must first be partitioned - so there's more work involved. The easy way to get the job done is:
- Use the dd utility as above to create the blank image file.
- After the file has been created, use a utility such
as fdisk to partition the image file. Calculate first the
number of cylinders that are in the image because fdisk needs to
know:
- Fdisk will automatically select 255 heads per cylinder and 63
sectors track. Therefore, we can calculate the total number of cylinders by
multiplying the block size that we used to create the image (1mb) by the block
count (4k) and dividing this by 255 heads, 63 sectors and 512 bytes
per sector, like this:
$ echo $(( 1*1024*1024 * 4*1024 / (255*63*512) ))
- (Note also that fdisk will complain that it doesn't know how to handle disk images - but you can ignore that complaint as it manages very nicely.)
- Fdisk will automatically select 255 heads per cylinder and 63
sectors track. Therefore, we can calculate the total number of cylinders by
multiplying the block size that we used to create the image (1mb) by the block
count (4k) and dividing this by 255 heads, 63 sectors and 512 bytes
per sector, like this:
- Use the kpartx utility to connect the image file to the mapper device.
- Use the mkfs utility to initialize each partition through the loopback devices provided by the mapper.
- Finally, you can mount each partition to a local directory and start copying files into it.
Putting all this together:
$ dd if=/dev/zero bs=1M count=4K of=mydisk.img 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 145.067 seconds, 29.6 MB/s $ echo $(( 1*1024*1024 * 4*1024 / (255*63*512) )) 522 $ /sbin/fdisk mydisk.img last_lba(): I don't know how to handle files with mode 81b4 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. You must set cylinders. You can do this from the extra functions menu. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): x Expert command (m for help): c Number of cylinders (1-1048576): 522 Expert command (m for help): r Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-522, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-522, default 522): +128M Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (18-522, default 18): Using default value 18 Last cylinder or +size or +sizeM or +sizeK (18-522, default 522): Using default value 522 Command (m for help): p Disk mydisk.img: 0 MB, 0 bytes 255 heads, 63 sectors/track, 522 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System mydisk.img1 1 17 136521 83 Linux mydisk.img2 18 522 4056412+ 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 25: Inappropriate ioctl for device. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. $ sudo /sbin/kpartx -a -v mydisk.img add map loop0p1 : 0 273042 linear /dev/loop0 63 add map loop0p2 : 0 8112825 linear /dev/loop0 273105 $ sudo /sbin/mkfs.ext3 /dev/mapper/loop0p1 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 34136 inodes, 136520 blocks 6826 blocks (5.00%) reserved for the super user First data block=1 Maximum filesystem blocks=67371008 17 block groups 8192 blocks per group, 8192 fragments per group 2008 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. $ sudo /sbin/mkfs.ext3 /dev/mapper/loop0p2 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 507904 inodes, 1014103 blocks 50705 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1040187392 31 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 37 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. $ mkdir ./boot ./root $ sudo mount -o loop /dev/mapper/loop0p1 ./boot $ sudo mount -o loop /dev/mapper/loop0p2 ./root $ df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/loop0p1 130M 5.6M 117M 5% ./boot /dev/mapper/loop0p2 3.9G 72M 3.6G 2% ./root
Notes
- Package names and directory paths are most probably going to work on any Fedora-derived distro. If you use a different distro you will have to figure out what packages you need to install and how to call the utilities from the shell. (Log into the root account and use the which command to figure out which directory to use.)
- You might not have the kpartx utility installed. It's not hard to find, though, as it's in a package called kpartx.
- Fdisk will complain if you try to fdisk an image which has more than 1024 cylinders. This is due to some very old BIOS software that does not recognize more than 1024 cylinders. If you are building a VM for an older operating system you should try to keep your disks under 8GB or so in size. Newer systems support more than 1024 cylinders.
- It is possible to initialize the file systems and mount them
without using the kpartx utility. It's far more complicated - but,
in case you need to know, here are some notes below. Please note that the following was written using an 8GB disk image with three partitions (one each for boot, root and swap.)
- Use the losetup utility (in the util-linux package on Fedora-derived distros,) to connect parts of the disk image to loopback devices.
- To do this we will need
to calculate some offsets to pass to the losetup utility. An easy
way to find the information we need is to use the sfdisk utility's
--dump option, like this:
$ sfdisk --dump mydisk.img last_lba(): I don't know how to handle files with mode 81b4 # partition table of mydisk.img unit: sectors mydisk.img1 : start= 63, size= 273042, Id=83 mydisk.img2 : start= 273105, size= 2008125, Id=83 mydisk.img3 : start= 2281230, size= 14169330, Id=83 mydisk.img4 : start= 0, size= 0, Id= 0
Here the units are described as sectors - which are normally 512 bytes each. So the first partition is 63 * 512 bytes offset from the start of the disk image, the second is 273105 * 512 and the third is 2281230 * 512 bytes from the start of the disk image. We will also need to know the size of each partition. The unit of measure will be file system blocks. I will use 4096 bytes per block for my needs - you can choose a value that is appropriate to you (512, 1024, 2048 or 4096 bytes are the usual file system block sizes.) Putting all this together, we get:
Partition Purpose Offset Blocks 1 /boot 63 *512 273042 *512/4096 2 (swap) 273105 *512 2008125 *512/4096 3 / 2281230 *512 14169330 *512/4096 Now we can connect the loopback devices to the disk image, as follows:
$ sudo /sbin/losetup -o $((63*512)) -f mydisk.img Password: $ sudo /sbin/losetup -o $((273105*512)) -f mydisk.img $ sudo /sbin/losetup -o $((2281230*512)) -f mydisk.img $ sudo /sbin/losetup -a /dev/loop0: [fd07]:10299306 (...) /dev/loop1: [fd08]:30635847 (...) /dev/loop2: [fd08]:10780676 (mydisk.img), offset 32256 /dev/loop3: [fd08]:10780676 (mydisk.img), offset 139829760 /dev/loop4: [fd08]:10780676 (mydisk.img), offset 1167989760
Note that losetup must have root permissions to operate. Note also that I used the -f option to choose the first free loop devices for each call. The last call to losetup, where the -a option is passed, displays a list of all the assigned loopback devices. As you can see my system was already using the first two - so the three partitions were mounted on /dev/loop2, 3 and 4.
Next we need to initialize the partitions. This is done using the usual mkfs utilities - but we need to specify the number of blocks with each call. Otherwise the utility will try to use the entire disk image for each file system. Here are the commands that I used:
$ sudo /sbin/mkfs.ext3 -b 4096 /dev/loop2 $((273042*512/4096)) mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 34176 inodes, 34130 blocks 1706 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=37748736 2 block groups 32768 blocks per group, 32768 fragments per group 17088 inodes per group Superblock backups stored on blocks: 32768 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. $ sudo /sbin/mkswap -p 4096 /dev/loop3 $((2008125*512/4096)) Setting up swapspace version 1, size = 257032 kB $ sudo /sbin/mkfs.ext3 -b 4096 /dev/loop2 $((14169330*512/4096)) mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 887040 inodes, 1771166 blocks 88558 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1816133632 55 block groups 32768 blocks per group, 32768 fragments per group 16128 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
At this point we have three partitions mounted on three loopback devices. One of the partitions is swap space - so there's nothing remaining to be done with it. We can disconnect it from the loopback device like this:
$ sudo /sbin/losetup -d /dev/loop3
The other two partitions can be mounted directly:
$ mkdir ./boot ./root $ sudo mount -o loop /dev/loop2 ./boot $ sudo mount -o loop /dev/loop4 ./root
If you have XEN installed you might want to use the lomount utility instead of mounting, as above, a looback device on top of a loopback device. If you don't have XEN installed you can pass an offset parameter to the mount utility (as lomount does.) Start by disconnecting the loopback devices that remain in use:
$ sudo /sbin/losetup -d /dev/loop2 $ sudo /sbin/losetup -d /dev/loop4
Next, mount the partitions using the lomount utility (which simply calculates the offset to pass to mount:)
$ sudo lomount -verbose -t ext3 -diskimage mydisk.img -partition 1 boot mount -oloop,offset=32256 mydisk.img -t ext3 boot $ sudo lomount -verbose -t ext3 -diskimage mydisk.img -partition 3 root mount -oloop,offset=1167989760 mydisk.img -t ext3 root $ df -h Filesystem Size Used Avail Use% Mounted on ... mydisk.img 130M 17M 107M 14% boot mydisk.img 6.7G 144M 6.2G 3% root
Now it's possible to simply copy any files that need to be copied to the boot and root partitions in the disk image. Once you umount the file systems you can attach them to your virtual machines.