Bug #10232

Investigate vagrant-lxc for our build system

Added by intrigeri 2015-09-23 03:08:59 . Updated 2016-12-18 10:00:20 .

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Build system
Target version:
Start date:
2015-09-23
Due date:
% Done:

100%

Feature Branch:
Type of work:
Research
Blueprint:

Starter:
Affected tool:
Deliverable for:
289

Description

It’s in Debian Stretch and could be an easier way to have something that works on our target platforms, including on our infrastructure (as opposed to vagrant-libvirt).

One challenge here is to avoid wasting too much dev+maintenance time on supporting both vagrant-libvirt and vagrant-lxc: that effort can easily cost more than the hardware that could simply use vagrant-libvirt (and nested KVM).


Subtasks


Related issues

Has duplicate Tails - Feature #6787: Investigate vagrant-lxc for our build system Duplicate 2014-02-27

History

#1 Updated by intrigeri 2015-09-23 03:09:34

  • Target version changed from Hole in the Roof to 2016
  • Parent task set to Feature #7526

#2 Updated by intrigeri 2015-09-23 03:10:05

  • Assignee set to kytv

#3 Updated by intrigeri 2015-09-23 03:10:43

  • blocks Feature #7580: Decide mid-term strategy regarding easy, self-contained build environment added

#4 Updated by anonym 2015-09-27 06:58:19

  • Assignee changed from kytv to anonym

#5 Updated by anonym 2015-09-27 14:39:34

Interesting link: http://fabiorehm.com/blog/2013/07/18/crafting-your-own-vagrant-lxc-base-box/

If this works we can skip distributing a basebox and make the creation of the basebox automatic and transparent for users.

#6 Updated by anonym 2015-11-05 07:00:21

  • % Done changed from 0 to 10

anonym wrote:
> Interesting link: http://fabiorehm.com/blog/2013/07/18/crafting-your-own-vagrant-lxc-base-box/
>
> If this works we can skip distributing a basebox and make the creation of the basebox automatic and transparent for users.

I followed the instructions and it seems a couple of things are missing:

# Before the network is used, e.g. apt-get
sudo cp /etc/resolv.conf ${ROOTFS}/etc

# When creating the .box
cat > metadata.json <<EOF
{
  "provider": "lxc",
  "version":  "1.0.0",
  "built-on": "$(date -u)"
}
EOF
cp /usr/share/vagrant-plugins/vagrant-lxc/scripts/lxc-template .
wget -O lxc-config https://raw.githubusercontent.com/fgrehm/vagrant-lxc-base-boxes/master/conf/debian

# This is important because lxc-template untar's with --strip-components=2
cd /var/lib/lxc
tar --numeric-owner -czf /tmp/vagrant-lxc-${RELEASE}/rootfs.tar.gz ${RELEASE}-base/rootfs

I had to bang my head for quite a while to realize that the docs are wrong regarding the .box’s name for the lxc configuration file; the docs say lxc.conf but it actually is lxc-config (realized after unpacking an official .box file… yay).

Next I had to add this to the Vagrantfile (created by vagrant init jessie-lxc /path/to/vagrant-lxc-jessie.box):

  config.vm.provider :lxc do |lxc|
    lxc.backingstore = 'dir'
  end


otherwise it assumed that there was an LVM group called lxc to create the container in…

And even after all that it fails with:

$ VAGRANT_LOG=DEBUG vagrant up --provider=lxc
[...]
ERROR warden: Error occurred: There was an error executing ["sudo", "/usr/bin/env", "lxc-attach", "--name", "vagrant-test_default_1446729822656_82678", "--namespaces", "NETWORK|MOUNT", "--", "/sbin/ip", "-4", "addr", "show", "scope", "global", "eth0"]
[... and the error is repeated many times ...]


because eth0 is missing (according to sudo lxc-attach -n jessie-lxc -- ip link). I have no idea why.

In hope that it’s just my box that is incompatible with modern vagrant lxc I tried generating one with vagrant-lxc-base-boxes, but I also hit trouble:

$ make jessie
[sudo] password for anonym: 
==> [debian-jessie] Building box to 'output/2015-11-05/vagrant-lxc-jessie-amd64.box'...
    [debian-jessie] Creating container...
    [debian-jessie] Container created!
    [debian-jessie] Adding ipv6 allhosts entry to container's /etc/hosts
    [debian-jessie] Running [/usr/sbin/update-rc.d -f checkroot-bootclean.sh remove] inside 'vagrant-base-jessie-amd64' container...
    [debian-jessie] Running [/usr/sbin/update-rc.d -f mountall-bootclean.sh remove] inside 'vagrant-base-jessie-amd64' container...
    [debian-jessie] Running [/usr/sbin/update-rc.d -f mountnfs-bootclean.sh remove] inside 'vagrant-base-jessie-amd64' container...
    [debian-jessie] Running [/usr/sbin/locale-gen en_US.UTF-8] inside 'vagrant-base-jessie-amd64' container...
    [debian-jessie] Running [update-locale LANG=en_US.UTF-8] inside 'vagrant-base-jessie-amd64' container...
==> [debian-jessie] Installing extra packages and upgrading
    [debian-jessie] Sleeping for 5 seconds...
    [debian-jessie] Running [apt-get update] inside 'vagrant-base-jessie-amd64' container...
    [debian-jessie] Running [apt-get install curl nfs-common wget python-software-properties ca-certificates sudo -y --force-yes] inside 'vagrant-base-jessie-amd64' container...
Makefile:28: recipe for target 'jessie' failed
make: *** [jessie] Error 100


So I debugged it with:

sudo lxc-attach -n vagrant-base-jessie-amd64 -- ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00


i.e. no network device here wither. Note that now I git network issues even earlier than in my previous attempt; when manually creating the basebox the networking was never an issue. The networking issue came only later, in vagrant up.

Adding all this as notes for the next time I’ll have a look (if any…).

As a conclusion I must say that the lack of robustness in this stack so far isn’t encouraging at all. I really hope I’m just doing all this completely wrong. Also, it doesn’t seem like a safe way to isolate the build system from the host system; I use the Xkb ctrl:nocaps option, and it’s unset on my host when Vagrant manages to start the lxc container (and then failing due to no network). I guess some tuning of the lxc config can fix that, but again, not very encouraging.

#7 Updated by intrigeri 2016-02-21 21:40:51

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • Target version changed from 2016 to 284

It would be nice for our infrastructure, so that we don’t have to do nested KVM (we have weird problems with it). But for our personal systems, some of us want more separation, and then Feature #6354 seems more fitting.

Let’s re-evaluate in a year how much we want it.

#8 Updated by intrigeri 2016-02-21 21:42:44

  • blocked by deleted (Feature #7580: Decide mid-term strategy regarding easy, self-contained build environment)

#9 Updated by intrigeri 2016-02-21 21:47:40

  • Subject changed from Evaluate vagrant-lxc to Investigate vagrant-lxc for our build system

#10 Updated by intrigeri 2016-02-21 21:47:54

  • has duplicate Feature #6787: Investigate vagrant-lxc for our build system added

#11 Updated by intrigeri 2016-06-08 07:44:17

  • Subject changed from Investigate vagrant-lxc for our build system to Investigate vagrant-lxc for our build system
  • Description updated

#12 Updated by intrigeri 2016-06-08 07:48:01

That’s not blocking Feature #7526 any more, really.

#13 Updated by intrigeri 2016-11-20 11:16:56

  • Assignee changed from intrigeri to bertagaz
  • Target version changed from 284 to Tails_2.9.1
  • QA Check set to Info Needed

We’ll likely drop this idea entirely, once bertagaz’ experiments with vagrant-libvirt are deemed successful enough.

#14 Updated by intrigeri 2016-11-20 12:44:11

  • Deliverable for set to 289

#15 Updated by anonym 2016-12-14 20:11:16

  • Target version changed from Tails_2.9.1 to Tails 2.10

#16 Updated by intrigeri 2016-12-18 10:00:20

  • Status changed from In Progress to Rejected
  • Assignee deleted (bertagaz)
  • % Done changed from 10 to 100