Adding an SSD cache to an existing encrypted filesystem with dm-cache

Overview

I wanted to try SSD caching in Fedora and dm-cache seemed like the logical choice to use on existing data. I had an existing home filesystem encrypted using cryptsetup/LUKS and I wanted to cache the encrypted blocks so as not to leave unencrypted data on the SSD. This meant that I had to insert the dm-cache device between the logical volume and the encrypted filesystem.

My 32GB OCX Vertex was pressed into service as the cache. I primarily followed the excellent write-up here: SSD Caching Using dm-cache Tutorial with changes neccesary for my environment.

Nuts & Bolts

My home filesystem is on is /dev/benwayvg/ehome or /dev/mapper/benwayvg-ehome. /etc/crypttab contained:

ehome   /dev/mapper/benwayvg-ehome

I found it easier to comment out or delete /etc/crypttab, comment out the entry for /home in /etc/fstab and simply reboot to set this up, but you could do it all manually by killing user processes, umounting /home, etc. After logging in as root I partitioned my SSD. I created a 2GB partition (too large I think) for metadata and the rest of the the SSD for cached blocks. I determined the size of the LV containing my home partition using: blockdev --getsz /dev/benwayvg/ehome which gave me a result of 209715200.

The real work is done by dmsetup:

/sbin/dmsetup create cehome --table '0 209715200 cache /dev/disk/by-id/ata-OCZ-VERTEX_FPRTEF1A3C31XA69I1XB-part1 /dev/disk/by-id/ata-OCZ-VERTEX_FPRTEF1A3C31XA69I1XB-part2 /dev/mapper/benwayvg-ehome 512 1 writeback default 0'

The magic above is explained well in the article I linked and I am basically copying it verbatim substituting for my partitions and sizes. I chose to use real partitions and not use the linear target.

The next step is to use cryptsetup to create the device we’ll actually mount: cryptsetup luksOpen /dev/mapper/cehome ehome

This creates /dev/mapper/ehome which can be mounted as a filesystem. All my data was preserved I am now simply accessing it through the cache.

Automatically run dmsetup at boot

I wanted this to all happen automatically. To achieve that I needed to get Fedora to create the cache device, and do so before it runs cryptsetup. To that end I created /etc/systemd/system/dmsetup-dm-cache.service with these contents:

[Unit]
Description=Initialize dm-cache device
DefaultDependencies=no
Conflicts=shutdown.target
After=fedora-storage-init.service
Before=cryptsetup.service

[Service]
ExecStart=/sbin/dmsetup create cehome --table '0 209715200 cache /dev/disk/by-id/ata-OCZ-VERTEX_FPRTEF1A3C31XA69I1XB-part1 /dev/disk/by-id/ata-OCZ-VERTEX_FPRTEF1A3C31XA69I1XB-part2 /dev/mapper/benwayvg-ehome 512 1 writeback default 0'
Type=oneshot
TimeoutSec=0
RemainAfterExit=yes

To be honest, I am not sure how many of the options above are needed, but it works! I also needed to ensure that the “service” gets started by making a link in /usr/lib/systemd/system/local-fs.target.wants to our service above:

ln -s /etc/systemd/system/dmsetup-dm-cache.service /usr/lib/systemd/system/local-fs.target.wants/

The last step to ensure this all works at boot is to update /etc/crypttab:

ehome /dev/mapper/cehome

Device summary

To summarize, all the devices in my setup:

/dev/mapper/benwayvg-ehome         luks block device
/dev/mapper/cehome                 cache device from dmsetup
/dev/mapper/ehome                  opened luks device

More to do…

Now I am looking at tuning and performance. So far, very little is actually getting cached, but I’ll comment on that some other time.