KVM-QEMU, QCOW2, QEMU-IMG and Snapshots

During the past few weeks I found myself doing lots of work with Windows VMs. I finally started reading-up on the use of snapshots.

The QCOW2 format is the most powerful format supported by KVM-QEMU. It does not pre-allocate disk space, it offers the ability to create snapshots and, later, you can create new images that use a previous image as a base to dramatically speed-up the setup of a future VM.

In this article we will look at some of the advanced features available in the QCOW2 format through the QEMU Image tool. We will create a QCOW2 image, create snapshots within the image, create an image that is based on a backing file and copy an image to get rid of the reference to the backing file.

The QEMU Image tool

At the time of this writing there are no features in Virt-Manager to simplify the use of the advanced QCOW2 features. The only way to use these features (that I know of - let me know if you find a better way,) is to execute QEMU-IMG commands in a shell.

Before we start I suggest that you check the qemu-img tool man page to find out about all the nifty things you can do with this tool. Here, though, is a summary of the few commands that we will actually discuss in this article:

  • qemu-img info <imagename>
    Dump some basic information about the image file. Includes the size on disk of the file, the name of any backing file referenced and a list of the snapshots available.
     
  • qemu-img create -f qcow2 <imagename> <max-storage>
    Create a simple QCOW2 image file. The disk space used for this file will be relatively small but the maximim storage capacity will be max-storage.
     
  • qemu-img create -b <imagename1> -f qcow2 -l <imagename2>
    Create a QCOW2 image file named imagename2 which is based on a backing image, imagename1. The new image file will reference the backing file. Any clusters written by the VM will be written to imagename2 so that the backing file remains unchanged. The backing file can be referenced by many future images but must not be changed by any of them. Warning: If any process modifies the backing file the image file(s) will be corrupted.
     
  • qemu-img snapshot -l <imagename>
    List all the snapshots in the specified imagename file.
     
  • qemu-img snapshot -c <snapshot-name> <imagename>
    Create a snapshot and name it snapshot-name. This snapshot is a simple picture of the VM image state at the time the snapshot is created.
     
  • qemu-img snapshot -a <snapshot-name> <imagename>
    Apply a snapshot named snapshot-name. This function simply restores the clusters that were saved when the snapshot, snapshot-name, was created. It has the effect of returning the VM image to the state it was in at that time.
     
  • qemu-img snapshot -d <snapshot-name> <imagename>
    Delete the snapshot named snapshot-name from the specified image file, imagename. Snapshots can gobble-up a significant amount of disk space. The delete command does not actually release any disk space allocated to the image file but it does release the associated clusters - effectively making them available to the VM for future storage.
     
  • qemu-img convert -p -f qcow2 <imagename1> -O qcow2 <imagename2>
    The convert command, when converting to and from the same QCOW2 format, acts as a copy command that only copies the current state of the VM image to the output file. The -p option displays progress information during the copy operation - which is often quite time consuming. The output file, imagename2, will contain all the clusters of a backing file that were referenced in the original image, imagename1. It will not copy any snapshot information. This has the effect of creating a standalone image with no references to any backing image.

All of the commands above are fairly simple to use after you have tried them a couple of times. Let's start with the simple Create command.

Creating a simple QCOW2 image file

Unfortunately Virt-Manager does not yet create QCOW2 images. Before you create a new VM you will want to create an image file. It's easy: Just open a shell and change to the directory you will be using for your VM images.

For this example I will be working on a Windows 7 demo VM in the /kvm/ folder - so I'll call my image: /kvm/win7demo-kvm.qcow2.

The QEMU image create command is very simple:

qemu-img create -f qcow2 <imagename> <max-storage>

  • The create option tells qemu-img that we want to create an image file.
  • The -f option specifies the format that we want to use; in this case it's QCOW2 because we want to use the advanced functionality supported by the format.
  • The imagename is whatever disk file name you want to use for the QCOW2 image.
  • The max-storage parameter represents the maximum amount of storage space you want to have available for this VM. The QCOW2 format does not pre-allocate disk space so the actual image file will be very small. It will stop growing, though, when (or if) it ever hits the specified size.

Let's put it all together and see what happens. Below is the command that I used to create my QCOW2 image and the output from that command.

$ qemu-img create -f qcow2 win7demo-kvm.qcow2 256G
Formatting 'win7demo-kvm.qcow2', fmt=qcow2 size=274877906944 encryption=off cluster_size=65536 lazy_refcounts=off 

Use the sudo command if you are creating an image in the standard virt-manager images folder as you will not have permission to write files in there.

Note the cluster_size value that is used, by default, for the QCOW2 images. There is a command line parameter allowing you to change that value. I never tried it as I assume the large clusters are intended to improve performance (and, in some cases, the performance can be excellent.)

You might like to know, though, that Windows writes to temporary files and various log files as it is running. Every little character that is written during the boot process therefore results in a 64k update to the VM image. If you are using snapshots you will notice huge amounts of disk space being gobbled-up by such tasks. In general, though, this is not important. We don't really care about disk space when we're doing work that benefits from snapshots. On the other hand it's a bit surprising when you first see it. Now you know that it's because of the huge default cluster size.

Saving a Snapshot

At some point in my VM Setup I had installed Service Pack 1 and all performance and security updates. This was an important milestone for me because I had Internet Explorer 9 on a secure and stable VM - a state which could be useful in the future. So I decided to take a snapshot.

QEMU snapshots on QCOW2 images are pretty much what they sound like: A picture of the VM at the time of the snapshot. These snapshots essentially represent a collection of clusters that have been saved for future use.

To take the shot we use the QEMU Image tool to Create the snapshot using an ID that we provide. Later on, when we need to return the VM to the state it was in at the time of the snapshot, we use the QEMU Image tool to Apply that snapshot using the ID that we used when we created it. There is a list command that dumps a list of the snapshots in the image. All of this uses a great deal of disk space so the QEMU Image tool includes a Delete command for snapshots that we no longer need.

Warning: The QEMU Image tool works much like a camera. It takes simple pictures - that's all. If you forgot to create a snapshot before trying to apply a previous snapshot the unsaved image state cannot be recovered.

The QEMU image snapshot create command is, as you might expect, also simple:

qemu-img snapshot -c <snapshot-name> <imagename>

  • The snapshot option tells qemu-img that we want to work with snapshots
  • The -c option tells qemu-img that we want to create a snapshot of the current VM image state.
  • The snapshot-name is the ID that we want to assign to this state of the VM image.
  • The imagename is the disk file name for the VM image with which we are working.

Putting it all together: Open a shell and change to the directory in which you have your image files. In the following example I am working on the same VM image that I created above and now I'm saving the image state using the name base-win7pro-winupdates-ie9.

As you can see there is no output from the snapshot create operation so I follow that up with an info request. Take a look (though it's slightly edited.)

$ qemu-img snapshot -c base-win7pro-winupdates-ie9 win7demo-kvm.qcow2

$ qemu-img info win7demo-kvm.qcow2 
image: 
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 38G
cluster_size: 65536
Snapshot list:
ID        TAG                 
1         base-win7pro-winupdates-ie9

Again, please use the sudo command if you are creating an image in the standard virt-manager images folder as you will not have permission to write files in there.

As I write this I'm also running Windows Update and installing all the security and performance updates for IE11. So far I've got the snapshot above and I've also got IE10 with all of its updates. For this reason you see that I've applied the Strike attribute to the disk size info above. If memory serves the value, at that time, was smaller - I'm guessing about 29gb - I don't recall exactly.

Here's the info dump of my VM as it stands now (slightly edited.)

$ qemu-img info win7demo-kvm.qcow2 
image: win7demo-kvm.qcow2
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 38G
cluster_size: 65536
Snapshot list:
ID        TAG
1         base-win7pro-winupdates-ie9
2         win7pro-sp1-ie9
3         win7pro-sp1-ie10

I still have Windows Updates to install before IE11 will be fully up to date. The disk space value you see above, 38gb, represents the state of the image as the updates are being installed (for IE11.) When Windows Update is finished I'll create a snapshot called win7pro-sp1-ie11.

Applying different Snapshots

Let's say that somebody calls to complain that one of my web pages is not displaying correctly IE9. How do I restore the state of the VM as it was after I had installed IE9?

First, now that I've got IE11 installed with all security patches and feature updates I should make sure that I've taken a snapshot of the current state. This is the only way to make sure that I can get back to this state in the future.

After that I need to Apply the snapshot that I took when IE9 was up-to-date on the VM.

Fortunately this is really easy to do. The snapshot Apply command is almost identical to the Create command that we used above:

qemu-img snapshot -a <snapshot-name> <imagename>

  • The snapshot option tells qemu-img that we want to work with snapshots
  • The -a option tells qemu-img that we want to apply a previously created snapshot of the VM image state - ie: we want to apply that snapshot so that it becomes the current image state.
  • The snapshot-name is the ID of the previously created snapshot.
  • The imagename is the disk file name for the VM image with which we are working.

Putting it all together: I created the IE11 snapshot as soon as Windows Update was finished. It was created exactly the same way as the previous snapshot. Now I'll simply apply the IE9 snapshot. To do this: Open a shell and change to the directory in which you have your image files. In the following example I am working on the same VM image that I created above and I'm applying the image state that was created under the name base-win7pro-winupdates-ie9.

As you can see there is no output from the snapshot apply operation so I follow that up with an info request. In practice there's nothing new to see:

$ qemu-img snapshot -a base-win7pro-winupdates-ie9 win7demo-kvm.qcow2

$ qemu-img info win7demo-kvm.qcow2 
image: win7demo-kvm.qcow2
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 40G
cluster_size: 65536
Snapshot list:
ID        TAG
1         base-win7pro-winupdates-ie9
2         win7pro-sp1-ie9
3         win7pro-sp1-ie10
4         win7pro-sp1-ie11

Now I can boot my VM and I will find IE9 when I click on the icon for Internet Explorer.

Note that the snapshot apply operation takes a few seconds but runs fairly quickly. When I'm done I can apply the snapshot for IE11. Note that I probably made extensive changes to the VM state after applying the IE9 snapshot - but these will all be lost when I apply the IE11 snapshot. If I want to save them I must create a new snapshot before applying a previous snapshot.

Creating a new VM based on an existing VM

At this point I have a very basic Windows 7 VM with IE11 and all Windows Updates installed. In practice this took me quite a bit of time to produce and there's no way that I ever want to have to do all that work again. Because I took snapshots of my VM at every stage of completion that was important to me (when Internet Explorer 9, 10 and 11 were fully up-to-date,) I can apply those snapshots at any time. If I need, for example, to test a web page with IE9 I can just apply the IE9 snapshot, boot my VM and there it is, IE9!

Let's say, though, that I want to create a whole bunch of Windows VMs for testing purposes. In the past I would have made copies of the image files. That is still a workable solution but the image files are huge; they take a long time to copy.

Today, though, with the QCOW2 format, we have a better way: We can create a new image file based on an existing image file. This involves creating an image that contains a reference to a Backing Image which will be used by the VM for all read operations.

To do this we will repeat the QEMU image create command that we used before. The only difference is the reference to the backing image file:

qemu-img create -f qcow2 -b <backing-image-name> <imagename>

  • The backing-image-name is the base image that we want to use for our new VM image.
  • The imagename is the disk file name for the new VM image we want to create.

Note that we don't have to specify the maximum size of the new image - it will be the same as the backing image. All cluster updates, though, will be written to the new imagename file and the backing-image-name file will never be touched. As a matter of fact the new image will be corrupted if any process makes a change to the backing image file.

Putting it all together: Open a shell and change to the directory in which you have your image files. In this example I will create a new image, win7demo2, based on the VM image that I created above.

$ qemu-img create -f qcow2 -b win7demo-kvm.qcow2 win7demo2-kvm.qcow2
Formatting 'win7demo2-kvm.qcow2', fmt=qcow2 size=274877906944 backing_file='win7demo-kvm.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off 

$ qemu-img info win7demo2-kvm.qcow2 
image: win7demo2-kvm.qcow2
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 196K
cluster_size: 65536
backing file: win7demo-kvm.qcow2

$ qemu-img info win7demo-kvm.qcow2 
image: win7demo-kvm.qcow2
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 40G
cluster_size: 65536
Snapshot list:
ID        TAG
1         base-win7pro-winupdates-ie9
2         win7pro-sp1-ie9
3         win7pro-sp1-ie10
4         win7pro-sp1-ie11

Notice that the snapshots in the backing image are not visible in the new VM image. Notice also that the new VM uses very little disk space but, because QEMU references the contents of the current state of the backing image, it is fully functional. It took less than a second to create this new VM so it's easy to create as many of them as I need as often as I need them.

The instant that one of these images is created I can go ahead and boot it. Any changes that I make to it will be recorded it its own disk file (ie: for the example above the system would always write to the Win7Demo2 disk file.) Once again, though, I have to warn that any attempt to boot the original Win7Demo VM or otherwise do anything that might write to the Win7Demo VM image will result in the corruption of the Win7Demo2 VM image! Don't do that.

Note that Virt-Manager will assign a new MAC address to the new Ethernet device on Win7Demo2. Windows will eventually detect this change and it will ask that each new VM be activated. Note also that the Computer Name must be changed manually for each new Windows VM.

Merging Images

The ability to make snapshots is very handy when we are repeating a collection of tests, writing documentation or otherwise trying something new. For working VMs, especially production VMs that are being used in a business environment, it's useful to review the issues and consider the options:

  • If several VMs are referencing the same backing image file they all share the risk of damage to that file. Any bitrot, for example, on that backing image will result in all the associated VMs being corrupted.
  • Windows Update, software upgrades and editing of files will slowly result in many system files being updated over time. If an image is based on a backing image the disk space savings will slowly evaporate on a production VM. (On the plus side this reduces the severity of the previous point.)

In general, though, once you have finished your testing and you decide to create a production VM you no longer benefit from the snapshots and the backing images - so you will want to get rid of them.

This can be done easily: Qemu-img has a copy function called Convert. It understands the image file structure so it correctly reads the current state of an image file, ignoring the snapshots that may be in that image file and any associated backing image file.

As you might expect: The convert command simply takes, as parameters, an input and output file name:

qemu-img convert -p -f qcow2 <imagename1> -O qcow2 <imagename2>

  • The convert option tells qemu-img that we want to copy an existing image file to a new image file.
  • The -p option tells the qemu-img tool that I want to know about the progress of the copy operation.
  • The -f option specifies the format of the original input file.
  • The imagename1 parameter is the disk name of the input file.
  • The -O option specifies the format that we want to use for the output file.
  • The imagename2 parameter is the disk name of the output.

Let's put it all together and see what happens. Below is the command that I used to merge the original Win7Demo2 image into Win7Demo2-Merged. Keep in mind that these files can be huge so the copy operation will probably take a bit of time.

$ /usr/bin/time -v qemu-img convert -p -f qcow2 win7demo2-kvm.qcow2 -O qcow2 win7demo2-merge-kvm.qcow2
    (100.00/100%)
        Command being timed: "qemu-img convert -p -f qcow2 win7demo2-kvm.qcow2 -O qcow2 win7demo2-merge-kvm.qcow2"
        User time (seconds): 55.07
        System time (seconds): 98.29
        Percent of CPU this job got: 7%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 34:10.36
        ...

$ qemu-img info win7demo2-merge-kvm.qcow2 
image: win7demo2-merge-kvm.qcow2
file format: qcow2
virtual size: 256G (274877906944 bytes)
disk size: 29G
cluster_size: 65536

If you watched carefully the progress info as the merge operation was running you would have noticed that the progress is not linear. This is because the QEMU Image utility does not need to read the disk space used by all the snapshots. It only needs to read the current state of the VM.

For the example above the Win7Demo image was 40gb and the Win7Demo2 image was a bit less than 1gb. After the merge operation the resulting Win7Demo2-Merged image file was only 29gb in size. There should be no real improvement in performace after the merge operation - QEMU handles these images in an efficient way - but all the extraneous data from the backing file and the various snapshots was not copied to the output file. This resulted in a savings of about 12gb for unused snapshots.

Going further

Once you have taken the time to play with the QEMU Image utility you will find it easy to use the Snapshot List and Delete commands that we didn't cover above.

While we covered above the main features of interest just now, there is still more for future needs. It's a good idea to review the man page for the QEMU Image utility and keep in mind the other functions that it offers such as image integrity checking.

As always I invite you to send me your questions and comments. I look forward to hearing from you.

Tags: