Test suite: filesystem shares vs. snapshots
Filesystem shares cannot (due to QEMU limitations) be added to an active VM, and cannot (due to QEMU limitations) be active (i.e. mounted) during a snapshot save. Hence unmounting before save and remounting after restore them seems like a good idea.
9p* modules seem to get into a broken state during a save + restore (the "tags" used as
mount source cannot be found), so unloading before save and reload after restore comes to mind. But loading
9pnet_virtio fails after a restore with
probe of virtio2 failed with error -2 (in
dmesg) and the shares remain unmountable.
We should research this further, and determine whether we’re just doing something wrong, or if this is an upstream bug.
Given we don’t use 9p shares at all, do we care more than what’s needed to just quickly report this to Debian and be done with it?
Actually we do:
git grep "I setup a filesystem share". This bug is what forces us to "drop" the background snapshot in the
MAT can clean a PDF filescenario. But it’s perhaps not a big issue. I’m fine with closing this bug and living with this as a limitation, I just want someone else’s oppinion. After all, we have more important stuff to do.
|Feature #6224: Reproduce the 9p reload bug with a recent kernel||Resolved||
|Feature #6225: Report the 9p kernel bug||Rejected||
|Feature #6226: Ask about FS shares vs. snapshots on libvirt list||Resolved||
|Bug #10214: Get 9p live migration support into QEMU||Rejected||
#1 Updated by intrigeri 2013-08-08 13:58:25
- Starter set to No
The August dev meeting said that this is no practical issue right now, since we only use FS shares for things that don’t care about these bugs:
- MAT: should be moved to a dedicated feature anyway
usb_install: doesn’t use background snapshots
… but a limitation in what we can do in the future, when we want to
have libvirt fs shares work even when scenarios are using
a background snapshot.
So, we want to report this bug where it should be.
#15 Updated by anonym 2015-09-13 11:48:08
I have some pretty good news! Live migration of (p9) shared filesystems can be fixed with this patch series ([Qemu-devel] [PATCH 0/2] Virtio-9p live migration patchset). So, yeah, this is a QEMU problem, not a libvirt one. It is pretty obvious to me now, years later when I have a much better understanding of our virtualization stack, but I am still apparently brain-dead enough to not reconsider what I have been reading in these tickets since then. Oh well…
So, the patches apply cleanly on Jessie’s QEMU, 2.1+dfsg-12+deb8u2, and I have attached a Debian source patch that applies against that version. With the patched QEMU, the filesystem shares work after restoring a snapshot without any extra work (e.g. no unmounting + remounting). Woo! For us to use this we’d need to rethink how we use the shares completely, but that’s a later question.
So how do we proceed? It seems to me that the current children tickets are irrelevant. New ones should be created about communicating our need for this on qemu-devel@, and possibly try to refresh the patches for QEMU’s current master (on patch doesn’t apply cleanly any more). What do you think?
#18 Updated by intrigeri 2015-09-15 11:49:51
> So how do we proceed? It seems to me that the current children tickets are irrelevant. New ones should be created about communicating our need for this on qemu-devel@, and possibly try to refresh the patches for QEMU’s current master (on patch doesn’t apply cleanly any more). What do you think?
I say close the obsolete tickets and create new ones.
Any estimate wrt. how close to be applied upstream these patches are?
#19 Updated by anonym 2015-09-16 16:38:23
- Type of work changed from Code to Wait
> > So how do we proceed? It seems to me that the current children tickets are irrelevant. New ones should be created about communicating our need for this on qemu-devel@, and possibly try to refresh the patches for QEMU’s current master (on patch doesn’t apply cleanly any more). What do you think?
> I say close the obsolete tickets and create new ones.
> Any estimate wrt. how close to be applied upstream these patches are?
Nope. The last incarnation of the patch series was sent almost a year ago, and apparently it dropped one of the original patches that fixed a “theoretical potential bug”. I’ve emailed both the original and second submitter to ask if any of them are interested in pushing for it again. => Wait
#25 Updated by intrigeri 2015-10-01 09:04:49
- Deliverable for deleted (
This is about an internal commitment. What was promised internally has been completed already. The new subtasks are new, bonus stuff. I don’t know how to translate that into Redmine and have no time to look into it further right now, just to try and lower a bit the level of stress in passing.
#27 Updated by anonym 2015-10-14 08:22:05
Hm. I’ve just seen that one can share folders via SPICE but it requires spice-webdavd which is not packaged in Debian. I couldn’t get it to compile inside Tails (neither on Wheezy (too old
libsoup) or Jessie (some weird linking error)). Any way, it’s an alternative I guess…
#39 Updated by anonym 2016-11-04 13:29:04
- Assignee changed from anonym to intrigeri
- QA Check set to Info Needed
Given the complexity of making filesystem shares working with snapshots, I think we should just stop using filesystem shares. We just need a fast channel to transfer data from the host to the guest filesystem (e.g. SPICE, or the remote shell if we solve
Feature #11887 + Feature #11888) and then I think we can replace steps like this:
Given I setup a filesystem share containing the Tails ISO
Given I plug a USB drive containing the Tails ISO
The only drawbacks (compared to a filesystem share) I can see is that this will need additional space and time for these transferred files. Whatever. At the moment we reboot when we need this, which at least takes a lot more time, and I think we can leave the peak disk usage as-is by just reordering features depending on this (for large files, e.g. the ISO) to run early.
What do you think?
#42 Updated by anonym 2016-11-08 19:20:52
- Status changed from Confirmed to In Progress
- Feature Branch set to test/5571-no-more-filesystem-shares
> I just realized that we don’t need the SPICE/remote shell, but can use guestfs to copy in the files to the virtual disk. So this is not blocked at all by anything. Woo!
Implemented! Let’s see what Jenkins thinks about it.
#48 Updated by anonym 2016-12-27 18:40:22
- Assignee changed from anonym to intrigeri
- QA Check changed from Dev Needed to Ready for QA
Jenkins finally seems to have accepted this branch. Please review and merge!
PS: For this branch I tried out a workflow where I only relied on Jenkins’ test suite runs in order to get an idea of how easy it would be to contribute test suite work without being able to run it locally. As I expected, anything beside trivial image bumps ends up with many back-and-fourths making this an awkward experience, and the long feedback loop makes this useless for learning how to write tests.
#50 Updated by intrigeri 2016-12-28 08:40:50
- Assignee deleted (
- QA Check changed from Ready for QA to Pass
I’ve pushed a few fixes on top of the branch and merged into stable and devel. Then I’ve merged into feature/stretch, and resolved conflicts (including one caused by having dogtail’ified the same code both on feature/stretch and on this branch, but in different ways; meh) the best I could. Cool!