How to resize a qcow2 virtual disk image

Not sure why I haven't written this down, since I did it countless times back in my earliest days with IBM in 2010 or 2011. Today I need to do it again, growing a 20GB qcow2 disk image to, say, 80GB. Here is how, with self explanatory comments.

root@fabv1:/var/lib/libvirt/images# virsh list --all | grep tony1                                # make sure the virtual machine has been shut down
 -     tony1                          shut off
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# mv tony1.qcow2 tony1.small.qcow2                             # we don't want to accidentally ruin any data
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# qemu-img convert -p -O raw tony1.small.qcow2 tony1.large.raw # parted and resize2fs can only operate on a raw disk
    (100.00/100%)
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# qemu-img info tony1.large.raw
image: tony1.large.raw
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 20G
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# qemu-img resize tony1.large.raw 80G                          # first enlarge the virtual disk size
Image resized.
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# qemu-img info tony1.large.raw
image: tony1.large.raw
file format: raw
virtual size: 80G (85899345920 bytes)
disk size: 20G
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# parted tony1.large.raw -- print
Model:  (file)
Disk /var/lib/libvirt/images/tony1.large.raw: 85.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  21.5GB  21.5GB  primary  ext4         boot

root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# parted tony1.large.raw -- resizepart 1 -1                    # and then modify the partition table
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# parted tony1.large.raw -- print
Model:  (file)
Disk /var/lib/libvirt/images/tony1.large.raw: 85.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  85.9GB  85.9GB  primary  ext4         boot

root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# kpartx -av tony1.large.raw
add map loop0p1 (252:0): 0 167768160 linear /dev/loop0 2048
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# e2fsck -f /dev/mapper/loop0p1                                # a must before resize2fs
e2fsck 1.42.9 (4-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/loop0p1: 636755/1310720 files (0.4% non-contiguous), 3724312/5242368 blocks
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# resize2fs -p /dev/mapper/loop0p1                             # finally resize the filesystem
resize2fs 1.42.9 (4-Feb-2014)
Resizing the filesystem on /dev/mapper/loop0p1 to 20971020 (4k) blocks.
The filesystem on /dev/mapper/loop0p1 is now 20971020 blocks long.

root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# kpartx -d tony1.large.raw
loop deleted : /dev/loop0
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# losetup -a                                                   # run "losetup -d /dev/loop0" if it shows in the output
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# qemu-img convert -p -O qcow2 tony1.large.raw tony1.qcow2
    (100.00/100%)
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# ll | grep tony1
-rw-r--r-- 1 root         root  85899345920 Apr  1 02:00 tony1.large.raw
-rw-r--r-- 1 root         root  21268922368 Apr  1 02:03 tony1.qcow2
-rw-r--r-- 1 libvirt-qemu kvm   21319450624 Mar 31 00:18 tony1.small.qcow2
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# chown libvirt-qemu:kvm tony1.qcow2                           # we've been manipulating the files as root so far
root@fabv1:/var/lib/libvirt/images#
root@fabv1:/var/lib/libvirt/images# ll | grep tony1
-rw-r--r-- 1 root         root  85899345920 Apr  1 02:00 tony1.large.raw
-rw-r--r-- 1 libvirt-qemu kvm   21268922368 Apr  1 02:03 tony1.qcow2
-rw-r--r-- 1 libvirt-qemu kvm   21319450624 Mar 31 00:18 tony1.small.qcow2
root@fabv1:/var/lib/libvirt/images#

A side note: nowadays people tend to use the high level command virt-resize combined with qemu-img resize, which makes you less concerned about all these underlying details. I haven't tried it yet though.

Building Hyperledger Fabric v1.1.0 on Ubuntu 14.04

TLDR - Don't do that. Build it on Ubuntu 16.04 and it'll work like a breeze.


Rationale -

Here by building Hyperledger Fabric, I mean calling make docker against the code and running through the e2e_cli example that demonstrates a proof-of-concept blockchain network. If you've done this before, you'll know that in order to do this, you'll need the following 2 base docker images that will be used to further build Fabric images like peer, orderer, etc.

root@tony2:/opt/gopath/src/github.com/hyperledger/fabric# docker images | grep base
hyperledger/fabric-baseimage   x86_64-0.4.7        390ac2e95bc7        37 hours ago        1.41GB
hyperledger/fabric-baseos      x86_64-0.4.7        c0e784934c4e        37 hours ago        152MB

I know that it's the 0.4.6 version base images that go with the official 1.1.0 release. I've just started migrating from 1.0.0 since yesterday, when the 0.4.7 version has rolled out. So I figured why not giving it a try? Simply modify BASEIMAGE_RELEASE in the Makefile should do the trick.

These images provide the runtime environment for the Fabric binary executables. When building these executables, there were actually a bunch of warnings that go something like:

Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking.

I obviously didn't give enough attention to these warnings when I first saw them. And later when I tried the e2e_cli example it failed when trying to install a chaincode, complaining about unexpected signal during runtime execution with a bunch of call stack traces. It was the 2nd or 3rd time during the trial and error process that I gave the warnings some serious thoughts, because some of the function names mentioned were similar to that in the call stack.

This is what I believe happened: the binaries were built on Ubuntu 14.04 with glibc 2.19, and the warning message suggested that the runtime environment should also be based on glibc 2.19, which is not the case since the baseimage was an Ubuntu 16.04 with glibc 2.23, hence the error and call stacks.

Figuring this all out I then upgraded my development machine to Ubuntu 16.04 (a command as simple as do-release-upgrade did it nicely), re-built the Fabric images and successfully ran through the e2e_cli example.

A side note: I believe upgrading glibc alone would also work, but there doesn't seem to be an easy way to do that except for building it from source, as Canonical doesn't ship in the 14.04 package repo a newer version.

Side note Number Two: I revisited my Fabric 1.0.0 environment and realized the baseimage v0.3.1, the official one for Fabric 1.0.0, was already an Ubuntu 16.04 with glibc 2.19, and I'd successfully run countless times of e2e_cli example with Fabric images built on Ubuntu 14.04. I'd only assume back in the 1.0.0 days Fabric code didn't call the incompatible functions. But now that we've learned this was a dangerous loophole.