[libvirt-users] VM running with storage image file deleted

[Michal Privoznik]

On 1/10/19 11:19 AM, Roosen, Nicolas wrote:
> Hello, I have a strange issue I'd like to have some insights on.
> 
> 
> We have 2 running VM, the storage pool is over NFS. Unfortunately, the backend storage folder has been deleted (so the image file in it as well, obviously) ...
> 
> 
> Yet, the 2 VM are still up and running, with their root filesystem being mounted "rw".
> 
> 
> - how does that work? I was expecting that with the file image being deleted, the VM would not work at all.
> 
> - since the VM is still running, is there a way to create a snapshot of it in order to re-create the image file on another storage pool ? I'm currently exploring "virsh snapshot-create-as ...", but if access to the image file is needed, it will fail.
> 

On Linux a file is not deleted right away if there's a process that has
the file opened, which is your case. The deletion is deferred until the
last process closed the file.

In order to rescue the file you could get the PID of qemu that still has
the disk open, and then find the FD corresponding to the file and with
some magic you should be able to recover the file:

# pgrep qemu
221472

# ls -l /proc/221472/fd/ | grep deleted
lr-x------ 1 root root 64 Jan 10 11:31 29 ->
/var/lib/libvirt/images/fd.img (deleted)

(Here, 29 is the FD we're looking for)

# dd if=/proc/221472/fd/29 of=blah.img

snapshot-create-* won't work because that will instruct qemu to open the
file again which will fail because the filename is gone.

Michal