Tails

#2 Updated by ioerror 2014-07-25 17:24:30

I removed the a965b45c-0500-48de-9e31-c7999cdf19fa device from the box.ovf file and the build continued. This may be a virtualbox bug where at one point, I had a DVD drive attached to the computer and it is no longer available. Perhaps virtualbox is trying to be smart and then fails when an expected device isn’t present? Bizzare.

The boot is now progressing and I’m waiting for the machine to boot. I’ll update my progress with this process a little later tonight.

#3 Updated by ioerror 2014-07-25 17:31:08

Upon revisiting this - it appears to fail again:

$ time rake build
No HTTP proxy set.

This is the first time that the Tails builder virtual machine is
started. The virtual machine template is about 300 MB to download,
so the process might take some time.

Please remember to shut the virtual machine down once your work on
Tails is done:

    $ rake vm:halt

Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'tails-builder-20140709'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Running 'pre-boot' VM customizations...
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
rake aborted!
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period. This can
mean a number of things.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.

Tasks: TOP => build => vm:up
(See full trace by running task with --trace)

real    5m32.610s
user    0m10.245s
sys 0m8.473s

#4 Updated by intrigeri 2014-07-25 18:11:54

> The package list likely needs to be updated from: […]

Fixed in e3e18ed, thanks!

> I would also suggest that a default Wheezy system needs to be modified to include the following lines inside of /etc/apt/sources.list:

I’m not sure it would make sense to go into such details, in the context of this documentation. I expect that most people who will read it, intending to modify the Tails source code, already know about the basics of APT.

So, the following nitpicking of mine is probably off-topic and useless, but just in case it can be helpful:

• Pinning security over stable feels wrong: security updates are superseded in stable point-releases.
• I think testing should be pinned to 500 instead of 750, so that no package is pulled from testing without asking explicitly, but package pulled from testing can still be automatically upgraded.

> The rake build of downloading the .box file (tails-builder-20140709.box) surprised me a bit. How is this verified?

See BOX_CHECKSUM in vagrant/lib/tails_build_settings.rb, and verification code in vagrant/lib/vagrant_verified_download.rb.

For the rest, I’ll leave it for anonym, as I don’t use Vagrant myself.

#5 Updated by ioerror 2014-07-26 12:29:59

intrigeri wrote:
> > The package list likely needs to be updated from: […]
>
> Fixed in e3e18ed, thanks!
>
> > I would also suggest that a default Wheezy system needs to be modified to include the following lines inside of /etc/apt/sources.list:
>
> I’m not sure it would make sense to go into such details, in the context of this documentation. I expect that most people who will read it, intending to modify the Tails source code, already know about the basics of APT.

Well, it isn’t so much about the basics of APT but about ensuring that all differences from a stock Wheezy install are documented. As it stands, I can’t tell if my issues are because of a difference of my system, missing documentation for a small set of details, a requirement that isn’t listed, a mismatched package or something else. :(

>
> So, the following nitpicking of mine is probably off-topic and useless, but just in case it can be helpful:
>
> * Pinning security over stable feels wrong: security updates are superseded in stable point-releases.

Yes, that seems like a good point.

> * I think testing should be pinned to 500 instead of 750, so that no package is pulled from testing without asking explicitly, but package pulled from testing can still be automatically upgraded.

That seems fine, yes.

>
> > The rake build of downloading the .box file (tails-builder-20140709.box) surprised me a bit. How is this verified?
>
> See BOX_CHECKSUM in vagrant/lib/tails_build_settings.rb, and verification code in vagrant/lib/vagrant_verified_download.rb.
>

Great! I’ll take a look and send feedback if I find anything odd.

> For the rest, I’ll leave it for anonym, as I don’t use Vagrant myself.

I’m a bit stuck, so I’d love to know what I’m missing - if Wheezy is supported, it feels like this should be working.

With that said - how do you build Tails if not with Vagrant? Is it documented somewhere?

#6 Updated by intrigeri 2014-07-26 12:43:16

> Well, it isn’t so much about the basics of APT but about ensuring that all
> differences from a stock Wheezy install are documented. As it stands, I can’t tell if
> my issues are because of a difference of my system, missing documentation for a small
> set of details, a requirement that isn’t listed, a mismatched package or something
> else. :(

I’ve improved things a bit in commit 95b7013, pushing. Good enough?

>> For the rest, I’ll leave it for anonym, as I don’t use Vagrant myself.

> I’m a bit stuck, so I’d love to know what I’m missing - if Wheezy is supported, it feels like this should be working.

I think Julien Voisin has tested this new Vagrant setup. No idea if that was on Wheezy. He’s jvoisin on IRC.

> With that said - how do you build Tails if not with Vagrant? Is it documented somewhere?

https://tails.boum.org/contribute/build/#manual

#7 Updated by ioerror 2014-07-26 13:21:49

intrigeri wrote:
> > Well, it isn’t so much about the basics of APT but about ensuring that all
> > differences from a stock Wheezy install are documented. As it stands, I can’t tell if
> > my issues are because of a difference of my system, missing documentation for a small
> > set of details, a requirement that isn’t listed, a mismatched package or something
> > else. :(
>
> I’ve improved things a bit in commit 95b7013, pushing. Good enough?

It looks a bit better - though the formatting seems strange. Also it isn’t clear that a user will pin in the way that you’d expect.

>
> >> For the rest, I’ll leave it for anonym, as I don’t use Vagrant myself.
>
> > I’m a bit stuck, so I’d love to know what I’m missing - if Wheezy is supported, it feels like this should be working.
>
> I think Julien Voisin has tested this new Vagrant setup. No idea if that was on Wheezy. He’s jvoisin on IRC.

Ok. I suspect this is some kind of Virtualbox issue but I’m unclear on how it would fail (no firewall, etc).

>
> > With that said - how do you build Tails if not with Vagrant? Is it documented somewhere?
>
> https://tails.boum.org/contribute/build/#manual

Ok. I may give up on Vagrant for a bit and try a manual build.

#8 Updated by intrigeri 2014-07-26 15:00:05

> It looks a bit better - though the formatting seems strange.

I’ve improved the formatting a bit. It’s still a bit strange — oh well, CSS.

> Also it isn’t clear that a user will pin in the way that you’d expect.

I’ve tried to clarified the wording a bit. If “Pin all packages from Debian unstable at 500 (apt_preferences(5)).” is not enough, well, I don’t know what to say. But maybe I’m fully wrong about the skills I should expect from anyone following these instructions.

#9 Updated by ioerror 2014-07-26 16:52:18

intrigeri wrote:
> > It looks a bit better - though the formatting seems strange.
>
> I’ve improved the formatting a bit. It’s still a bit strange — oh well, CSS.
>

That looks great!

> > Also it isn’t clear that a user will pin in the way that you’d expect.
>
> I’ve tried to clarified the wording a bit. If “Pin all packages from Debian unstable at 500 (apt_preferences(5)).” is not enough, well, I don’t know what to say. But maybe I’m fully wrong about the skills I should expect from anyone following these instructions.

I think that explicit is always better than implicit, regardless of skill set - this makes it possible to report bugs in documentation or note when processes do not work as expected. That isn’t about the skillset, I think. Rather I would say it is hoping/assuming/realizing that people are so smart and so skilled that they’ll do it in some really amazing way and it should work but well, it won’t. :)

#10 Updated by ioerror 2014-07-26 17:12:52

OK, I’ve reset everything - disabled ufw (which may or may not have been on at last build time) and I’ve had this error again:

$ time rake build --trace
** Invoke build (first_time)
** Invoke parse_build_options (first_time)
** Execute parse_build_options
** Invoke ensure_clean_repository (first_time)
** Execute ensure_clean_repository
** Invoke validate_http_proxy (first_time)
** Execute validate_http_proxy
No HTTP proxy set.
** Invoke vm:up (first_time)
** Invoke parse_build_options 
** Invoke validate_http_proxy 
** Execute vm:up

This is the first time that the Tails builder virtual machine is
started. The virtual machine template is about 300 MB to download,
so the process might take some time.

Please remember to shut the virtual machine down once your work on
Tails is done:

    $ rake vm:halt

Bringing machine 'default' up with 'virtualbox' provider...
[default] Box 'tails-builder-20140709' was not found. Fetching box from specified URL for
the provider 'virtualbox'. Note that if the URL does not have
a box for this provider, you should interrupt Vagrant now and add
the box yourself. Otherwise Vagrant will attempt to download the
full box prior to discovering this error.
Downloading box from URL: http://dl.amnesia.boum.org/tails/project/vagrant/tails-builder-20140709.box
Extracting box...te: 2328k/s, Estimated time remaining: --:--:--)
Successfully added box 'tails-builder-20140709' with provider 'virtualbox'!
[default] Importing base box 'tails-builder-20140709'...
rake aborted!
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["import", "/home/error/.vagrant.d/boxes/tails-builder-20140709/virtualbox/box.ovf"]

Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting /home/error/.vagrant.d/boxes/tails-builder-20140709/virtualbox/box.ovf...
OK.
0%...
Progress state: NS_ERROR_FAILURE
VBoxManage: error: <vbox:Machine> element in OVF contains a medium attachment for the disk image a965b45c-0500-48de-9e31-c7999cdf19fa but the OVF describes no such image
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component Appliance, interface IAppliance, callee 
Context: "ImportAppliance" at line 793 of file VBoxManageAppliance.cpp
/usr/share/vagrant/plugins/providers/virtualbox/driver/base.rb:342:in `block in execute'
/usr/lib/ruby/vendor_ruby/vagrant/util/retryable.rb:17:in `retryable'
/usr/share/vagrant/plugins/providers/virtualbox/driver/base.rb:300:in `execute'
/usr/share/vagrant/plugins/providers/virtualbox/driver/version_4_1.rb:172:in `import'
/usr/share/vagrant/plugins/providers/virtualbox/action/import.rb:15:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/customize.rb:40:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/check_accessible.rb:18:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:95:in `block in finalize_action'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:51:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/config_validate.rb:25:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:57:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/check_virtualbox.rb:17:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/machine.rb:147:in `action'
/usr/lib/ruby/vendor_ruby/vagrant/batch_action.rb:63:in `block (2 levels) in run'
Tasks: TOP => build => vm:up

real    2m31.531s
user    0m7.028s
sys 0m5.820s

I went and removed the two DVD drive stanzas - this allowed the “Importing base box” - it would then boot the VM and wait. During this time I see the following with lsof -ni:

ruby      14557       error   45r  IPv4  42075      0t0  TCP 127.0.0.1:54838->127.0.0.1:2222 (ESTABLISHED)
VBoxHeadl 14934       error   29u  IPv4  39104      0t0  TCP 127.0.0.1:2222 (LISTEN)
VBoxHeadl 14934       error   31u  IPv4  39955      0t0  TCP 127.0.0.1:2222->127.0.0.1:54833 (CLOSE_WAIT)
VBoxHeadl 14934       error   32u  IPv4  40232      0t0  TCP 127.0.0.1:2222->127.0.0.1:54834 (CLOSE_WAIT)
VBoxHeadl 14934       error   33u  IPv4  41752      0t0  TCP 127.0.0.1:2222->127.0.0.1:54835 (CLOSE_WAIT)
VBoxHeadl 14934       error   34u  IPv4  41790      0t0  TCP 127.0.0.1:2222->127.0.0.1:54836 (CLOSE_WAIT)
VBoxHeadl 14934       error   35u  IPv4  41912      0t0  TCP 127.0.0.1:2222->127.0.0.1:54837 (CLOSE_WAIT)
VBoxHeadl 14934       error   36u  IPv4  40883      0t0  TCP 127.0.0.1:2222->127.0.0.1:54838 (ESTABLISHED)

This has the same timeout as previous attempts:

$ time rake build --trace
** Invoke build (first_time)
** Invoke parse_build_options (first_time)
** Execute parse_build_options
** Invoke ensure_clean_repository (first_time)
** Execute ensure_clean_repository
** Invoke validate_http_proxy (first_time)
** Execute validate_http_proxy
No HTTP proxy set.
** Invoke vm:up (first_time)
** Invoke parse_build_options 
** Invoke validate_http_proxy 
** Execute vm:up

This is the first time that the Tails builder virtual machine is
started. The virtual machine template is about 300 MB to download,
so the process might take some time.

Please remember to shut the virtual machine down once your work on
Tails is done:

    $ rake vm:halt

Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'tails-builder-20140709'...
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Running 'pre-boot' VM customizations...
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
rake aborted!
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period. This can
mean a number of things.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/wait_for_communicator.rb:68:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/customize.rb:40:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/boot.rb:18:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/customize.rb:40:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/sane_defaults.rb:49:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/set_hostname.rb:16:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/forward_ports.rb:31:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/network.rb:112:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/clear_network_interfaces.rb:26:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/prepare_nfs_settings.rb:15:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/synced_folders.rb:68:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/synced_folder_cleanup.rb:28:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/prepare_nfs_valid_ids.rb:12:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/handle_forwarded_port_collisions.rb:118:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/prepare_forwarded_port_collision_params.rb:30:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/env_set.rb:19:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/provision.rb:52:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/clear_forwarded_ports.rb:13:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/set_name.rb:19:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/clean_machine_folder.rb:17:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/check_accessible.rb:18:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:95:in `block in finalize_action'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:51:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:95:in `block in finalize_action'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:51:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:95:in `block in finalize_action'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:51:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/config_validate.rb:25:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/check_virtualbox.rb:17:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:57:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/config_validate.rb:25:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builtin/call.rb:57:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/share/vagrant/plugins/providers/virtualbox/action/check_virtualbox.rb:17:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `block in run'
/usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
/usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:69:in `run'
/usr/lib/ruby/vendor_ruby/vagrant/machine.rb:147:in `action'
/usr/lib/ruby/vendor_ruby/vagrant/batch_action.rb:63:in `block (2 levels) in run'
Tasks: TOP => build => vm:up

real    5m33.590s
user    0m10.197s
sys 0m8.121s

#11 Updated by ioerror 2014-07-26 17:19:07

I tried this with git branch origin/stable and had a similar failure:

$ git checkout origin/stable
Note: checking out 'origin/stable'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at b099744... Merge branch 'feature/vagrant-wheezy-basebox' into stable

$ time rake build --trace
** Invoke build (first_time)
** Invoke parse_build_options (first_time)
** Execute parse_build_options
rake aborted!
Vagrant timed out while attempting to connect via SSH. This usually
means that the VM booted, but there are issues with the SSH configuration
or network connectivity issues. Please try to `vagrant reload` or
`vagrant up` again.
/usr/share/vagrant/plugins/communicators/ssh/communicator.rb:241:in `rescue in connect'
/usr/share/vagrant/plugins/communicators/ssh/communicator.rb:184:in `connect'
/usr/share/vagrant/plugins/communicators/ssh/communicator.rb:64:in `execute'
/home/error/Documents/tails/Rakefile:109:in `free_vm_memory'
/home/error/Documents/tails/Rakefile:115:in `enough_free_vm_memory_for_ram_build?'
/home/error/Documents/tails/Rakefile:120:in `enough_free_memory_for_ram_build?'
/home/error/Documents/tails/Rakefile:146:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/rake/task.rb:205:in `call'
/usr/lib/ruby/vendor_ruby/rake/task.rb:205:in `block in execute'
/usr/lib/ruby/vendor_ruby/rake/task.rb:200:in `each'
/usr/lib/ruby/vendor_ruby/rake/task.rb:200:in `execute'
/usr/lib/ruby/vendor_ruby/rake/task.rb:158:in `block in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/usr/lib/ruby/vendor_ruby/rake/task.rb:151:in `invoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:176:in `block in invoke_prerequisites'
/usr/lib/ruby/vendor_ruby/rake/task.rb:174:in `each'
/usr/lib/ruby/vendor_ruby/rake/task.rb:174:in `invoke_prerequisites'
/usr/lib/ruby/vendor_ruby/rake/task.rb:157:in `block in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/usr/lib/ruby/vendor_ruby/rake/task.rb:151:in `invoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:144:in `invoke'
/usr/lib/ruby/vendor_ruby/rake/application.rb:116:in `invoke_task'
/usr/lib/ruby/vendor_ruby/rake/application.rb:94:in `block (2 levels) in top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:94:in `each'
/usr/lib/ruby/vendor_ruby/rake/application.rb:94:in `block in top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:133:in `standard_exception_handling'
/usr/lib/ruby/vendor_ruby/rake/application.rb:88:in `top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:66:in `block in run'
/usr/lib/ruby/vendor_ruby/rake/application.rb:133:in `standard_exception_handling'
/usr/lib/ruby/vendor_ruby/rake/application.rb:63:in `run'
/usr/bin/rake:27:in `<main>'
Tasks: TOP => build => parse_build_options

real    1m16.296s
user    0m0.920s
sys 0m0.192s

#12 Updated by dawuud 2014-07-26 23:12:03

I got quite far along in the build process with ioerror’s suggested apt pinning…

Here’s my patch I used to remove references to absolete debian packages:

http://vbydyg7rcotccilo.onion/uri/URI:DIR2-RO:zoy2g44valzyhe7cqmazx6scgy:hffifgnbq6hlmjivcilvs3nokpm75bp5onnekrbq2fu5fp66seeq/tails_diff1

My build ended with these errors… the chroot inside the vm has no /etc/apt/sources…
I am building on the “stable” branch. Should I be using the “devel” branch like the
online documentation says? :

Err http://ftp.us.debian.org wheezy-backports Release.gpg
Unable to connect to 127.0.0.1:8118:
Err http://deb.tails.boum.org stable Release.gpg
Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.torproject.org wheezy Release.gpg
Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.torproject.org sid Release.gpg
Unable to connect to 127.0.0.1:8118:
Reading package lists…
W: Failed to fetch http://ftp.de.debian.org/debian/dists/wheezy/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://security.debian.org/dists/wheezy/updates/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://mozilla.debian.net/dists/wheezy-backports/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/experimental/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/unstable/Release.gpg Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.tails.boum.org/dists/stable/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/testing/Release.gpg Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/wheezy/Release.gpg Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/sid/Release.gpg Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy-backports/Release.gpg Unable to connect to 127.0.0.1:8118:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists…
Building dependency tree…
Reading state information…
E: You must put some ‘source’ URIs in your sources.list
E: config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem failed (exit non-zero). You should check for errors.
P: Begin unmounting filesystems…

real 74m26.649s
user 50m12.036s
sys 5m48.106s
lb build failed (1).
human@computer:~/tails$

#13 Updated by ioerror 2014-07-27 00:08:23

I decided that I was frustrated with vagrant on Wheezy as these instructions do not appear to work as expected. I decided to follow the manual build process as outlined on the build page: https://tails.boum.org/contribute/build/#index2h1

I added the following to sources.list:

# Tails
deb http://deb.tails.boum.org/ builder-wheezy main
deb http://http.debian.net/debian wheezy-backports main

I downloaded the correct GPG key as outlined on the page:

gpg --recv-key 221F9A3C6FA3E09E182E060BC7988EA7A358D82E | sudo apt-get add -

I updated apt:

apt-get update

I installed all the required packages:

apt-get install live-build/builder-wheezy syslinux eatmydata time whois ikiwiki/wheezy-backports libyaml-perl libyaml-libyaml-perl po4a perlmagick libyaml-syck-perl dpkg-dev intltool 

I configured the live build system:

mkdir /etc/live
cat << 'EOF' >> /etc/live/build.conf
LB_PARENT_MIRROR_BINARY="http://ftp.de.debian.org/debian/"
LB_MIRROR_BINARY="http://ftp.de.debian.org/debian/"
EOF

I switched to the devel branch from the tails git repo:

$ git checkout devel
Previous HEAD position was b099744... Merge branch 'feature/vagrant-wheezy-basebox' into stable
Switched to branch 'devel'

I became root:

su -

I started the build:

lb clean --all && lb config && lb build

Around ten minutes later I had a build failure:

Processing triggers for python-support ...
Processing triggers for ca-certificates ...
Updating certificates in /etc/ssl/certs... 158 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
Processing triggers for sgml-base ...
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
Reading package lists...
Building dependency tree...
Reading state information...
P: Begin queueing installation of packages lists...
P: Begin queueing installation of local packages lists...
P: Begin queueing installation of packages...
P: Begin queueing installation of local packages...
P: Begin installing packages...
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
Reading package lists...
Building dependency tree...
Reading state information...
Package linux-image-3.14-1-486 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package linux-image-3.14-1-amd64 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'linux-image-3.14-1-486' has no installation candidate
E: Package 'linux-image-3.14-1-amd64' has no installation candidate
P: Begin unmounting filesystems...

real    10m58.901s
user    3m46.038s
sys 0m21.865s
lb build failed (1).

It appears that the devel branch isn’t working with Wheezy with vagrant or the manual build process as documented. Am I doing something completely incorrectly?

#14 Updated by ioerror 2014-07-27 01:18:45

I decided that the last thing to do was to try a Vagrant build with Jessie. I upgraded my previous Wheezy system (which had working VirtualBox kernel modules, I might add) and the rest of this comment is attempting to build Tails on the Jessie system.

It appears that VirtualBox isn’t happy:

$ time rake build --trace
** Invoke build (first_time)
** Invoke parse_build_options (first_time)
** Execute parse_build_options
rake aborted!
VirtualBox is complaining that the installation is incomplete. Please
run `VBoxManage --version` to see the error message which should contain
instructions on how to fix this error.
/usr/share/vagrant/plugins/providers/virtualbox/driver/meta.rb:142:in `read_version'
/usr/share/vagrant/plugins/providers/virtualbox/driver/meta.rb:34:in `initialize'
/usr/share/vagrant/plugins/providers/virtualbox/provider.rb:34:in `new'
/usr/share/vagrant/plugins/providers/virtualbox/provider.rb:34:in `machine_id_changed'
/usr/share/vagrant/plugins/providers/virtualbox/provider.rb:14:in `initialize'
/usr/lib/ruby/vendor_ruby/vagrant/machine.rb:116:in `new'
/usr/lib/ruby/vendor_ruby/vagrant/machine.rb:116:in `initialize'
/usr/lib/ruby/vendor_ruby/vagrant/environment.rb:434:in `new'
/usr/lib/ruby/vendor_ruby/vagrant/environment.rb:434:in `machine'
/home/error/Documents/tails/Rakefile:51:in `primary_vm'
/home/error/Documents/tails/Rakefile:59:in `primary_vm_state'
/home/error/Documents/tails/Rakefile:94:in `vm_running?'
/home/error/Documents/tails/Rakefile:119:in `enough_free_memory_for_ram_build?'
/home/error/Documents/tails/Rakefile:146:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/rake/task.rb:236:in `call'
/usr/lib/ruby/vendor_ruby/rake/task.rb:236:in `block in execute'
/usr/lib/ruby/vendor_ruby/rake/task.rb:231:in `each'
/usr/lib/ruby/vendor_ruby/rake/task.rb:231:in `execute'
/usr/lib/ruby/vendor_ruby/rake/task.rb:175:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/usr/lib/ruby/vendor_ruby/rake/task.rb:168:in `invoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:197:in `block in invoke_prerequisites'
/usr/lib/ruby/vendor_ruby/rake/task.rb:195:in `each'
/usr/lib/ruby/vendor_ruby/rake/task.rb:195:in `invoke_prerequisites'
/usr/lib/ruby/vendor_ruby/rake/task.rb:174:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/usr/lib/ruby/vendor_ruby/rake/task.rb:168:in `invoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:161:in `invoke'
/usr/lib/ruby/vendor_ruby/rake/application.rb:149:in `invoke_task'
/usr/lib/ruby/vendor_ruby/rake/application.rb:106:in `block (2 levels) in top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:106:in `each'
/usr/lib/ruby/vendor_ruby/rake/application.rb:106:in `block in top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:115:in `run_with_threads'
/usr/lib/ruby/vendor_ruby/rake/application.rb:100:in `top_level'
/usr/lib/ruby/vendor_ruby/rake/application.rb:78:in `block in run'
/usr/lib/ruby/vendor_ruby/rake/application.rb:165:in `standard_exception_handling'
/usr/lib/ruby/vendor_ruby/rake/application.rb:75:in `run'
/usr/bin/rake:27:in `<main>'
Tasks: TOP => build => parse_build_options

real    0m1.269s
user    0m0.676s
sys 0m0.080s

This appears to be a problem where VirtualBox needs something more than the listed packages:

WARNING: The character device /dev/vboxdrv does not exist.
     Please install the virtualbox-ose-dkms package and the appropriate
     headers, most likely linux-headers-686.

     You will not be able to start VMs until this problem is fixed.
4.1.18_Debianr78361

Naturally, those packages do not work as they have “no installation candidate” in jessie. apt-get helpfully instructed me to use the virtualbox-ose package - which was already installed. All in all - the issue is simple - the vboxdrv kernel module is not available in jessie with the listed packages.

I installed module-assistant and attempted to use it to help with the virtualbox module:

apt-get install module-assistant
m-a prepare

I still didn’t find myself with vboxdrv so I tried to reconfigure the package:

dpkg-reconfigure virtualbox-dkms

The result was a failure:

------------------------------
Deleting module version: 4.1.18
completely from the DKMS tree.
------------------------------
Done.
Loading new virtualbox-4.1.18 DKMS files...
Building only for 3.14-1-686-pae
Building initial module for 3.14-1-686-pae
Error! Bad return status for module build on kernel: 3.14-1-686-pae (i686)
Consult /var/lib/dkms/virtualbox/4.1.18/build/make.log for more information.
Job for virtualbox.service failed. See 'systemctl status virtualbox.service' and 'journalctl -xn' for details.
invoke-rc.d: initscript virtualbox, action "restart" failed.

The error is clear:

cat /var/lib/dkms/virtualbox/4.1.18/build/make.log
DKMS make.log for virtualbox-4.1.18 for kernel 3.14-1-686-pae (i686)
Sun Jul 27 03:11:52 CEST 2014
make: Entering directory '/usr/src/linux-headers-3.14-1-686-pae'
  LD      /var/lib/dkms/virtualbox/4.1.18/build/built-in.o
  LD      /var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/built-in.o
  CC [M]  /var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.o
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c: In function ‘vboxdrvLinuxUid’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:226:5: error: incompatible types when returning type ‘kuid_t’ but ‘RTUID’ was expected
     return current->cred->uid;
     ^
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c: In function ‘vboxdrvLinuxGid’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:235:5: error: incompatible types when returning type ‘kgid_t’ but ‘RTGID’ was expected
     return current->cred->gid;
     ^
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c: In function ‘vboxdrvLinuxEuid’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:244:5: error: incompatible types when returning type ‘kuid_t’ but ‘RTUID’ was expected
     return current->cred->euid;
     ^
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:248:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c: In function ‘vboxdrvLinuxUid’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:230:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c: In function ‘vboxdrvLinuxGid’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.c:239:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
/usr/src/linux-headers-3.14-1-common/scripts/Makefile.build:308: recipe for target '/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.o' failed
make[4]: *** [/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/linux/SUPDrv-linux.o] Error 1
/usr/src/linux-headers-3.14-1-common/scripts/Makefile.build:455: recipe for target '/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv' failed
make[3]: *** [/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv] Error 2
/usr/src/linux-headers-3.14-1-common/Makefile:1291: recipe for target '_module_/var/lib/dkms/virtualbox/4.1.18/build' failed
make[2]: *** [_module_/var/lib/dkms/virtualbox/4.1.18/build] Error 2
Makefile:133: recipe for target 'sub-make' failed
make[1]: *** [sub-make] Error 2
Makefile:8: recipe for target 'all' failed
make: *** [all] Error 2
make: Leaving directory '/usr/src/linux-headers-3.14-1-686-pae'

Has anyone actually built Tails with Vagrant on jessie?

#16 Updated by ioerror 2014-07-27 03:18:39

My guess is that the Vagrant builds are somehow especially broken on this X60 with a fresh Wheezy install. Perhaps it may be that the CPU in the X60 doesn’t support some specific hardware virtualization feature? VirtualBox seems to not complain but perhaps we’re dealing with such an issue? It seems unclear why it would hang for five minutes or more.

After a little digging, I am hopeful that the manual builds will work once Feature #7656 is fixed. I’ll start grinding away on a manual build and see about fixing that specific build issue.

#17 Updated by cypherpunks 2014-07-27 04:36:41

I think I’ve managed to build a Tails iso with the following patch applied to git master using the manual process:

diff --git a/auto/config b/auto/config
index f614c9b..293f3fd 100755
--- a/auto/config
+++ b/auto/config
@@ -35,7 +35,7 @@ $RUN_LB_CONFIG \
    --memtest none \
    --packages-lists="standard" \
    --tasks="standard" \
-   --linux-packages="linux-image-3.14-1" \
+   --linux-packages="linux-image-3.14-2" \
    --syslinux-menu vesamenu \
    --syslinux-splash data/splash.png \
    --syslinux-timeout 4 \
diff --git a/config/chroot_apt/preferences b/config/chroot_apt/preferences
index 18f3a9f..e64f0bc 100644
--- a/config/chroot_apt/preferences
+++ b/config/chroot_apt/preferences
@@ -114,19 +114,19 @@ Package: linux-headers-amd64
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-headers-3.14-1-common
+Package: linux-headers-3.14-2-common
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-headers-3.14-1-486
+Package: linux-headers-3.14-2-486
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-headers-3.14-1-686-pae
+Package: linux-headers-3.14-2-686-pae
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-headers-3.14-1-amd64
+Package: linux-headers-3.14-2-amd64
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

@@ -142,15 +142,15 @@ Package: linux-image-amd64
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-image-3.14-1-486
+Package: linux-image-3.14-2-486
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-image-3.14-1-686-pae
+Package: linux-image-3.14-2-686-pae
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

-Package: linux-image-3.14-1-amd64
+Package: linux-image-3.14-2-amd64
 Pin: release o=Debian,a=unstable
 Pin-Priority: 999

I have some feedback on the manual build process but as of now (with the above patch) it appears to actually have not failed with an obvious error.

#18 Updated by dawuud 2014-07-27 08:49:05

I built a Tails iso using the vagrant method on a wheezy system with ioeror’s suggested apt pinning priorities…

Here’s my patch:
https://github.com/david415/tails/commit/8c1fca97185df2d3e96ccb34db4ab7a0ca7c23f7

I wonder if this build process will work for me with cypherpunk’s patch… which is a smaller patch.
I might have a few lines in my patch that are not needed.

At the time of my successful build I was also running a squid proxy on 127.0.0.1:8118
https://tails.boum.org/contribute/build/squid-deb-proxy/

#19 Updated by intrigeri 2014-07-27 09:25:03

> Package linux-image-3.14-1-486 is not available, but is referred to by another package.
> This may mean that the package is missing, has been obsoleted, or
> is only available from another source

Yes, I told about this on tails-dev@, that’s Feature #7656.

#20 Updated by intrigeri 2014-07-27 09:25:50

> Naturally, those packages do not work as they have “no installation candidate” in jessie.

Indeed, it was removed from testing due to RC bugs. I would try the one from sid.

#21 Updated by intrigeri 2014-07-27 09:26:11

> I think I’ve managed to build a Tails iso with the following patch applied to git master using the manual process:

See Feature #7656.

#23 Updated by cypherpunks 2014-07-27 13:59:43

intrigeri wrote:
> > I think I’ve managed to build a Tails iso with the following patch applied to git master using the manual process:
>
> See Feature #7656.

I was wrong - even when upgrading to 3.14-2 it still appears to fail:

time lb build
P: Executing auto/build script.
aktualisiere Wiki..
fertig
refresh-translations failed (3).

real    0m3.373s
user    0m2.928s
sys 0m0.208s

#24 Updated by dawuud 2014-07-27 14:47:40

My patch mentioned in comment #18
https://labs.riseup.net/code/issues/7661#note-18

It causes the vagrant build to succeed… but the manual build
still fails on the translation stuff.

#25 Updated by intrigeri 2014-07-27 15:16:31

>

time lb build
> P: Executing auto/build script.
> aktualisiere Wiki..
> fertig
> refresh-translations failed (3).

> real  0m3.373s
> user  0m2.928s
> sys   0m0.208s
> 

• What branch are you building from?
• May you please git clean -fdx and retry?
• What does ./build-wiki in a clean Git tree do?

#26 Updated by cypherpunks 2014-07-27 15:26:52

intrigeri wrote:
> > […]
>
> * What branch are you building from?
> * May you please git clean -fdx and retry?
> * What does ./build-wiki in a clean Git tree do?

I’m building from master with the branch feature/linux-3.14-2 from Feature #7656 - I was only able to keep the manual build running by making refresh-translations a no-op (eg: I made it immediately return 0) - I did this just to see if I ran into any other issues. Once I know that this is the only blocking issue, I’ll figure out how to actually solve it cleanly - clearly though the translation stuff doesn’t work out of the box for me with the current Manual build documentation on Wheezy. I’m sure I made a mistake though - so I’ll try your suggestion once this current build finishes (with a failure or not, probably).

#27 Updated by kytv 2014-07-27 15:32:26

intrigeri wrote:
> > […]
>
> * What branch are you building from?
> * May you please git clean -fdx and retry?
> * What does ./build-wiki in a clean Git tree do?

I think—but am not sure—that I resolved this in the past with either

`git checkout po`

or

`git checkout wiki`

I can say for sure that I was able to build in a clean checkout, leading me to the conclusion that it was a problem with the workspace itself.

#29 Updated by cypherpunks 2014-07-27 17:43:28

After hacking around the po translate issue, I have tried to build as so:

LB_PARENT_MIRROR_BINARY="http://ftp.de.debian.org/debian/"
LB_MIRROR_BINARY="http://ftp.de.debian.org/debian/"
export TAILS_BUILD_OPTIONS="noproxy"
time lb build

This had an error:

# time lb build
P: Executing auto/build script.
aktualisiere Wiki..
fertig
Building iso image tails-i386-feature_linux-3.14-2-1.1.1-20140727...
P: Setting up cleanup function
P: Begin caching bootstrap stage...
W: skipping bootstrap_cache.restore
P: Begin bootstrapping system...
W: skipping bootstrap
P: Begin caching bootstrap stage...
W: skipping bootstrap_cache.save
P: Begin unmounting filesystems...
P: Setting up cleanup function
P: Begin caching chroot stage...
P: Begin mounting /dev/pts...
W: skipping chroot_devpts
P: Begin mounting /proc...
W: skipping chroot_proc
P: Begin mounting /sys...
W: skipping chroot_sysfs
P: Configuring file /etc/debian_chroot
P: Configuring file /sbin/start-stop-daemon
P: Configuring file /usr/sbin/policy-rc.d
W: skipping chroot_sysv-rc
P: Configuring file /usr/sbin/initctl
W: skipping chroot_upstart
P: Configuring file /etc/hosts
W: skipping chroot_hosts
P: Configuring file /etc/resolv.conf
W: skipping chroot_resolv
P: Configuring file /etc/hostname
W: skipping chroot_hostname
P: Configuring file /etc/apt/apt.conf
W: skipping chroot_apt
P: Configuring file /etc/apt/sources.list
W: skipping chroot_sources
P: Configuring file /etc/kernel-img.conf
P: Begin executing preseed...
P: Begin executing local preseeds...
W: skipping chroot_local-preseed
P: Begin installing tasks...
W: skipping chroot_tasks
P: Begin queueing installation of packages lists...
W: skipping chroot_packageslists
P: Begin queueing installation of local packages lists...
W: skipping chroot_local-packageslists
P: Begin queueing installation of packages...
P: Begin queueing installation of local packages...
P: Begin installing packages...
W: skipping chroot_install-packages
P: Begin installing localization packages...
W: skipping chroot_localization
P: Begin copying chroot local includes...
W: skipping chroot_local-includes
P: Begin applying chroot local patches...
W: skipping chroot_local-patches
P: Configuring package sysvinit
P: Begin executing local hooks...
W: skipping chroot_local-hooks
P: Begin executing hooks...
P: Begin executing hacks...
W: skipping chroot_hacks
P: Deconfiguring file /etc/kernel-img.conf
P: Deconfiguring file /etc/apt/sources.list
P: Deconfiguring file /etc/apt/apt.conf
P: Deconfiguring file /etc/hostname
P: Deconfiguring file /bin/hostname
P: Deconfiguring file /etc/resolv.conf
P: Deconfiguring file /etc/hosts
P: Deconfiguring file /usr/sbin/policy-rc.d
P: Deconfiguring file /usr/sbin/initctl
P: Deconfiguring file /sbin/start-stop-daemon
P: Deconfiguring file /etc/debian_chroot
P: Begin unmounting /sys...
P: Begin unmounting /selinux...
P: Begin unmounting /proc...
P: Begin unmounting /dev/pts...
P: Begin caching chroot stage...
P: Begin unmounting filesystems...
P: Setting up cleanup function
P: Begin copying chroot...
W: skipping binary_chroot
P: Begin mounting /dev/pts...
P: Begin mounting /proc...
P: Begin mounting /sys...
P: Configuring file /etc/hosts
P: Configuring file /etc/resolv.conf
P: Configuring file /etc/hostname
P: Configuring file /bin/hostname
P: Configuring file /usr/sbin/policy-rc.d
P: Configuring file /usr/sbin/initctl
P: Configuring file /etc/apt/apt.conf
P: Configuring file /etc/apt/sources.list
OK
OK
OK
Err http://ftp.de.debian.org wheezy Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://security.debian.org wheezy/updates Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://mozilla.debian.net wheezy-backports Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://ftp.us.debian.org experimental Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://ftp.us.debian.org unstable Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://ftp.us.debian.org testing Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://ftp.us.debian.org wheezy-backports Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://deb.tails.boum.org devel Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.tails.boum.org feature-linux-3.14-2 Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://deb.torproject.org wheezy Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.torproject.org sid Release.gpg
  Unable to connect to 127.0.0.1:8118:
Reading package lists...
W: Failed to fetch http://ftp.de.debian.org/debian/dists/wheezy/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://security.debian.org/dists/wheezy/updates/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://mozilla.debian.net/dists/wheezy-backports/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/experimental/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/unstable/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.tails.boum.org/dists/devel/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.tails.boum.org/dists/feature-linux-3.14-2/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/testing/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/wheezy/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/sid/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy-backports/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
debian-archive-keyring is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Err http://security.debian.org wheezy/updates Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.tails.boum.org devel Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.tails.boum.org feature-linux-3.14-2 Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://deb.torproject.org wheezy Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://deb.torproject.org sid Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://ftp.de.debian.org wheezy Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://ftp.us.debian.org experimental Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Err http://ftp.us.debian.org unstable Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://ftp.us.debian.org testing Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://ftp.us.debian.org wheezy-backports Release.gpg
  Unable to connect to 127.0.0.1:8118:
Err http://mozilla.debian.net wheezy-backports Release.gpg
  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)
Reading package lists...
W: Failed to fetch http://ftp.de.debian.org/debian/dists/wheezy/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://security.debian.org/dists/wheezy/updates/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://mozilla.debian.net/dists/wheezy-backports/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/experimental/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://ftp.us.debian.org/debian/dists/unstable/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.tails.boum.org/dists/devel/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.tails.boum.org/dists/feature-linux-3.14-2/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/testing/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/wheezy/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/sid/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy-backports/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
P: Begin building root filesystem image...
W: skipping binary_rootfs
P: Begin creating manifest...
W: skipping binary_manifest
P: Begin installing local packages lists...
P: Begin install linux-image...
W: skipping binary_linux-image
P: Begin installing syslinux...
W: skipping binary_syslinux
P: Begin installing disk information...
W: skipping binary_disk
P: Begin copying binary local includes...
W: skipping binary_local-includes
P: Begin executing local hooks...
P: customize buggy syslinux help
P: customize syslinux menu
rm: Entfernen von „binary/isolinux/exithelp.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
rm: Entfernen von „binary/isolinux/prompt.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
E: config/binary_local-hooks/10-syslinux_customize failed (exit non-zero). You should check for errors.
P: Begin unmounting filesystems...

real    0m26.351s
user    0m16.925s
sys 0m1.744s
lb build failed (1).

real    0m28.401s
user    0m18.841s
sys 0m1.824s

I looked to see if I was setting a proxy in my environment:

env|grep -i proxy
TAILS_BUILD_OPTIONS=noproxy

I’ll dig around to try to figure out why this is happening.

#30 Updated by cypherpunks 2014-07-27 17:52:06

I had a bunch of warnings like this:

W: Failed to fetch http://deb.tails.boum.org/dists/feature-linux-3.14-2/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/testing/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/wheezy/Release.gpg  Could not connect to 127.0.0.1:8118 (127.0.0.1). - connect (111: Connection refused)

W: Failed to fetch http://deb.torproject.org/torproject.org/dists/sid/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy-backports/Release.gpg  Unable to connect to 127.0.0.1:8118:

W: Some index files failed to download. They have been ignored, or old ones used instead.

I decided to install this:

apt-get install apt-cacher-ng

I configured it by telling it to use Port 8118:

cat /etc/apt-cacher-ng/acng.conf|grep 8118
Port:8118

I restarted it:

/etc/init.d/apt-cacher-ng restart

I restarted the build process. I hope that this will ensure that all packages that are needed will be installed as expected.

The build still failed with the following (but the warnings are now gone!):

P: Begin executing local hooks...
P: customize buggy syslinux help
P: customize syslinux menu
rm: Entfernen von „binary/isolinux/exithelp.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
rm: Entfernen von „binary/isolinux/prompt.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
E: config/binary_local-hooks/10-syslinux_customize failed (exit non-zero). You should check for errors.
P: Begin unmounting filesystems...

real    1m6.259s
user    0m52.851s
sys 0m3.580s
lb build failed (1).

real    1m8.303s
user    0m54.743s
sys 0m3.684s

#31 Updated by intrigeri 2014-07-27 18:00:14

lb build without cleaning first is not supported. See https://tails.boum.org/contribute/build/#index5h2.

#32 Updated by cypherpunks 2014-07-27 18:00:31

I’ve restarted the full build with the apt-cacher-ng proxy running:

lb clean --all && lb config && time lb build

There are lots of little warnings like this:

Ign http://deb.torproject.org sid/main Translation-en
Ign http://deb.tails.boum.org devel/main Translation-en
Ign http://deb.tails.boum.org feature-linux-3.14-2/main Translation-en
Fetched 137 kB in 6s (22.0 kB/s)
Reading package lists...
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
Reading package lists...
Building dependency tree...
Reading state information...
P: Configuring file /etc/kernel-img.conf
P: Begin executing preseed...
P: Begin executing local preseeds...
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
P: Begin installing tasks...
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object '/usr/lib/libeatmydata/libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.

It does however continue to build. I’ll update this ticket again at the next failure or if it is successful. :)

#33 Updated by cypherpunks 2014-07-27 18:11:54

During the build we have some other warnings:

Setting up libxss1:i386 (1:1.2.2-1) ...
Setting up libxxf86dga1:i386 (2:1.1.3-2+deb7u1) ...
Setting up libyaml-0-2:i386 (0.1.4-2+deb7u4) ...
Setting up libuuid-perl (0.02-5) ...
Setting up linux-base (3.5) ...
Setting up libklibc (2.0.1-3.1) ...
Setting up klibc-utils (2.0.1-3.1) ...
Setting up initramfs-tools (0.115~bpo70+1) ...
update-initramfs: deferring update (trigger activated)
Setting up linux-image-3.14-2-486 (3.14.13-2) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-3.14-2-486
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: could not determine root device from /etc/fstab
libkmod: ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.2.0-4-686-pae/modules.dep.bin'
W: intel-microcode: cpuid kernel driver missing
W: intel-microcode: disabling IUCODE_TOOL_SCANCPUS option
live-boot: core filesystems devices utils udev wget blockdev.
Setting up linux-image-3.14-2-amd64 (3.14.13-2) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-3.14-2-amd64
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: could not determine root device from /etc/fstab
libkmod: ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.2.0-4-686-pae/modules.dep.bin'
W: intel-microcode: cpuid kernel driver missing
W: intel-microcode: disabling IUCODE_TOOL_SCANCPUS option
live-boot: core filesystems devices utils udev wget blockdev.

We also have some surprising network fetches:

invoke-rc.d: policy-rc.d denied execution of start.
Setting up p7zip-full (9.20.1~dfsg.1-4) ...
Setting up file-roller (3.4.2-1) ...
Setting up firmware-atheros (0.43) ...
Setting up firmware-b43-installer (1:018-2) ...
A chroot environment has been detected.
Remember this firmware needs kernel >= 2.6.25.
--2014-07-27 18:08:45--  http://www.lwfinger.com/b43-firmware/broadcom-wl-5.100.138.tar.bz2
Resolving www.lwfinger.com (www.lwfinger.com)... 173.254.28.119
Connecting to www.lwfinger.com (www.lwfinger.com)|173.254.28.119|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13514651 (13M) [application/x-bzip2]
Saving to: `broadcom-wl-5.100.138.tar.bz2'

100%[======================================================================>] 13,514,651  1.71M/s   in 9.1s    

2014-07-27 18:08:55 (1.41 MB/s) - `broadcom-wl-5.100.138.tar.bz2' saved [13514651/13514651]

Deleting old extracted firmware...
broadcom-wl-5.100.138/

Yowza! That seems like it needs a security review! I see a few other fetches during the build process and I was surprised by that network activity.

Other warnings/errors:

Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.14-2-amd64
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: could not determine root device from /etc/fstab
libkmod: ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.2.0-4-686-pae/modules.dep.bin'
W: intel-microcode: cpuid kernel driver missing
W: intel-microcode: disabling IUCODE_TOOL_SCANCPUS option
live-boot: core filesystems devices utils udev wget blockdev.
Setting up ntfsprogs (1:2012.1.15AR.5-2.1) ...
Setting up libreoffice-core (1:3.5.4+dfsg2-0+deb7u2) ...
Setting up libreoffice-style-galaxy (1:3.5.4+dfsg2-0+deb7u2) ...
Setting up openjdk-6-jre-lib (6b32-1.13.4-1~deb7u1) ...

And the build continues…

#34 Updated by cypherpunks 2014-07-27 18:15:46

More warnings:

Reading state information...
P: Begin executing hooks...
P: Begin executing hacks...
update-rc.d: warning:  stop runlevel arguments (6) do not match kexec-load Default-Stop values (0 6)
update-initramfs: Generating /boot/initrd.img-3.14-2-amd64
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: could not determine root device from /etc/fstab
live-boot: core filesystems devices utils udev wget blockdev.

#35 Updated by cypherpunks 2014-07-27 18:26:54

My last build had the following output:

path relative to the source directories
Cannot stat sortlist entry "usr/share/iceweasel/modules/sessionstore/SessionStorage.jsm"
This is probably because you're using the wrong file
path relative to the source directories
Cannot stat sortlist entry "usr/share/nautilus/ui/nautilus-navigation-window-ui.xml"
This is probably because you're using the wrong file
path relative to the source directories
Cannot stat sortlist entry "usr/share/pixmaps/nautilus/erase.png"
This is probably because you're using the wrong file
path relative to the source directories

And then the system hard locked. It appears that this was during:

32309 root      20   0  416m 302m  832 S 177,5  9,2  15:15.62 mksquashfs

I’m now in that process again and I hope it won’t melt the system again…

#36 Updated by cypherpunks 2014-07-27 18:27:39

Unsure if this is directly related but on Wheezy during boot/build I have the following output in dmesg:

[  132.375844] tracker-miner-f[3643]: segfault at 1aac ip b746ebda sp bfba6644 error 4 in libgobject-2.0.so.0.3200.4[b7444000+50000]

#37 Updated by cypherpunks 2014-07-27 18:44:39

In the end, the build failed again:

Reading state information...
P: Begin creating manifest...
P: Begin installing local packages lists...
P: Begin install linux-image...
P: Begin installing syslinux...
Reading package lists...
Building dependency tree...
Reading state information...
P: Begin installing disk information...
P: Begin copying binary local includes...
1 block
P: Begin executing local hooks...
P: customize buggy syslinux help
P: customize syslinux menu
P: adding CPU autodetection to the syslinux menu
Reading package lists...
Building dependency tree...
Reading state information...
P: fixing syslinux installation
P: including syslinux in the ISO filesystem
Hit http://ftp.us.debian.org experimental Release.gpg
Hit http://ftp.de.debian.org wheezy Release.gpg
Hit http://mozilla.debian.net wheezy-backports Release.gpg
Hit http://security.debian.org wheezy/updates Release.gpg
Hit http://ftp.us.debian.org unstable Release.gpg
Hit http://deb.torproject.org wheezy Release.gpg
Hit http://deb.tails.boum.org devel Release.gpg
Hit http://ftp.de.debian.org wheezy Release
Get:1 http://ftp.de.debian.org wheezy/main Sources [5957 kB]
Hit http://mozilla.debian.net wheezy-backports Release
Hit http://ftp.us.debian.org testing Release.gpg
Hit http://security.debian.org wheezy/updates Release
Get:2 http://security.debian.org wheezy/updates/main Sources [121 kB]
Get:3 http://mozilla.debian.net wheezy-backports/iceweasel-esr Sources [10.9 kB]
Hit http://deb.torproject.org sid Release.gpg
Hit http://deb.tails.boum.org feature-linux-3.14-2 Release.gpg
Hit http://ftp.us.debian.org wheezy-backports Release.gpg
Get:4 http://security.debian.org wheezy/updates/contrib Sources [14 B]
Get:5 http://security.debian.org wheezy/updates/non-free Sources [14 B]
Hit http://mozilla.debian.net wheezy-backports/iceweasel-esr i386 Packages
Hit http://ftp.us.debian.org experimental Release
Hit http://deb.torproject.org wheezy Release
Hit http://security.debian.org wheezy/updates/main i386 Packages
Get:6 http://ftp.de.debian.org wheezy/contrib Sources [47.7 kB]
Hit http://deb.tails.boum.org devel Release
Get:7 http://ftp.de.debian.org wheezy/non-free Sources [93.4 kB]
Hit http://ftp.us.debian.org unstable Release
Hit http://security.debian.org wheezy/updates/contrib i386 Packages
Hit http://deb.torproject.org sid Release
Hit http://ftp.de.debian.org wheezy/main i386 Packages
Hit http://ftp.us.debian.org testing Release
Get:8 http://deb.torproject.org wheezy/main Sources [2852 B]
Hit http://deb.tails.boum.org feature-linux-3.14-2 Release
Hit http://security.debian.org wheezy/updates/non-free i386 Packages
Hit http://ftp.de.debian.org wheezy/contrib i386 Packages
Hit http://ftp.us.debian.org wheezy-backports Release
Get:9 http://deb.tails.boum.org devel/main Sources [19.5 kB]
Hit http://security.debian.org wheezy/updates/contrib Translation-en
Hit http://deb.torproject.org wheezy/main i386 Packages
Hit http://ftp.de.debian.org wheezy/non-free i386 Packages
Hit http://security.debian.org wheezy/updates/main Translation-en
Hit http://deb.tails.boum.org devel/main i386 Packages
Hit http://ftp.de.debian.org wheezy/contrib Translation-en
Ign http://mozilla.debian.net wheezy-backports/iceweasel-esr Translation-en
Hit http://ftp.us.debian.org experimental/main i386 Packages/DiffIndex
Hit http://security.debian.org wheezy/updates/non-free Translation-en
Get:10 http://deb.torproject.org sid/main Sources [2843 B]
Hit http://ftp.de.debian.org wheezy/main Translation-en
Hit http://ftp.us.debian.org experimental/main Translation-en/DiffIndex
Hit http://ftp.de.debian.org wheezy/non-free Translation-en
Hit http://deb.torproject.org sid/main i386 Packages
Get:11 http://deb.tails.boum.org feature-linux-3.14-2/main Sources [14 B]
Hit http://deb.tails.boum.org feature-linux-3.14-2/main i386 Packages
Hit http://ftp.us.debian.org unstable/main i386 Packages/DiffIndex
Hit http://ftp.us.debian.org unstable/contrib i386 Packages/DiffIndex
Hit http://ftp.us.debian.org unstable/non-free i386 Packages/DiffIndex
Hit http://ftp.us.debian.org unstable/contrib Translation-en/DiffIndex
Hit http://ftp.us.debian.org unstable/main Translation-en/DiffIndex
Hit http://ftp.us.debian.org unstable/non-free Translation-en/DiffIndex
Hit http://ftp.us.debian.org testing/main i386 Packages/DiffIndex
Hit http://ftp.us.debian.org testing/contrib i386 Packages/DiffIndex
Hit http://ftp.us.debian.org testing/non-free i386 Packages/DiffIndex
Hit http://ftp.us.debian.org testing/contrib Translation-en/DiffIndex
Hit http://ftp.us.debian.org testing/main Translation-en/DiffIndex
Hit http://ftp.us.debian.org testing/non-free Translation-en/DiffIndex
Get:12 http://ftp.us.debian.org wheezy-backports/main Sources [342 kB]
Get:13 http://ftp.us.debian.org wheezy-backports/contrib Sources [6739 B]
Get:14 http://ftp.us.debian.org wheezy-backports/non-free Sources [8437 B]
Hit http://ftp.us.debian.org wheezy-backports/main i386 Packages/DiffIndex
Hit http://ftp.us.debian.org wheezy-backports/contrib i386 Packages/DiffIndex
Hit http://ftp.us.debian.org wheezy-backports/non-free i386 Packages/DiffIndex
Hit http://ftp.us.debian.org wheezy-backports/contrib Translation-en/DiffIndex
Hit http://ftp.us.debian.org wheezy-backports/main Translation-en/DiffIndex
Hit http://ftp.us.debian.org wheezy-backports/non-free Translation-en/DiffIndex
Get:15 http://ftp.us.debian.org experimental/main Sources [206 kB]
Get:16 http://ftp.us.debian.org unstable/main Sources [7375 kB]
Ign http://deb.torproject.org wheezy/main Translation-en
Get:17 http://ftp.us.debian.org unstable/contrib Sources [55.9 kB]
Get:18 http://ftp.us.debian.org unstable/non-free Sources [105 kB]
Ign http://deb.tails.boum.org devel/main Translation-en
Ign http://deb.torproject.org sid/main Translation-en
Get:19 http://ftp.us.debian.org testing/main Sources [6867 kB]
Ign http://deb.tails.boum.org feature-linux-3.14-2/main Translation-en
Get:20 http://ftp.us.debian.org testing/contrib Sources [47.1 kB]
Get:21 http://ftp.us.debian.org testing/non-free Sources [98.7 kB]
Fetched 21.4 MB in 15s (1398 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  make
Suggested packages:
  debian-keyring make-doc
Recommended packages:
  gcc c-compiler build-essential fakeroot libalgorithm-merge-perl
The following NEW packages will be installed:
  dpkg-dev make
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1754 kB of archives.
After this operation, 2785 kB of additional disk space will be used.
Err http://ftp.de.debian.org/debian/ wheezy/main make i386 3.81-8.2
  500  Connection timeout
Get:1 http://ftp.de.debian.org/debian/ wheezy/main dpkg-dev all 1.16.15 [1356 kB]
Failed to fetch http://ftp.de.debian.org/debian/pool/main/m/make-dfsg/make_3.81-8.2_i386.deb  500  Connection timeout
Fetched 1356 kB in 60s (22.3 kB/s)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
E: config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem failed (exit non-zero). You should check for errors.
P: Begin unmounting filesystems...

real    42m13.174s
user    50m30.469s
sys 2m43.842s
lb build failed (1).

real    47m33.128s
user    55m41.457s
sys 2m50.547s

#38 Updated by cypherpunks 2014-07-27 19:06:58

I tried the build again:

Hit http://ftp.us.debian.org wheezy-backports/non-free Translation-en/DiffIndex
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
P: Begin building root filesystem image...
W: skipping binary_rootfs
P: Begin creating manifest...
W: skipping binary_manifest
P: Begin installing local packages lists...
P: Begin install linux-image...
W: skipping binary_linux-image
P: Begin installing syslinux...
W: skipping binary_syslinux
P: Begin installing disk information...
W: skipping binary_disk
P: Begin copying binary local includes...
W: skipping binary_local-includes
P: Begin executing local hooks...
P: customize buggy syslinux help
P: customize syslinux menu
rm: Entfernen von „binary/isolinux/exithelp.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
rm: Entfernen von „binary/isolinux/prompt.cfg“ nicht möglich: Datei oder Verzeichnis nicht gefunden
E: config/binary_local-hooks/10-syslinux_customize failed (exit non-zero). You should check for errors.
P: Begin unmounting filesystems...

real    0m47.476s
user    0m28.498s
sys 0m2.180s
lb build failed (1).

real    0m49.842s
user    0m30.430s
sys 0m2.348s

I had to apply the following patch for the 10-syslinux_customize script to correctly function:

diff --git a/config/binary_local-hooks/10-syslinux_customize b/config/binary_local-hooks/10-syslinux_customize
index 214c487..eee6683 100755
--- a/config/binary_local-hooks/10-syslinux_customize
+++ b/config/binary_local-hooks/10-syslinux_customize
@@ -57,4 +57,4 @@ sed -i -e '/^include stdmenu\.cfg/a include tails.cfg' "${CFG_FILE}"
 sed -e 's,/isolinux/,,' -i "${SYSLINUX_PATH}/stdmenu.cfg"

 # remove useless files that break incremental upgrades on Wheezy
-rm "${SYSLINUX_PATH}"/{exithelp,prompt}.cfg
+# rm "${SYSLINUX_PATH}"/{exithelp,prompt}.cfg

#40 Updated by cypherpunks 2014-07-27 19:22:44

I’m testing the build by performing the following commands:

isohybrid tails-i386-feature_linux-3.14-2-1.1.1-20140727.iso.isohbrid --entry 4 --type 0x1c

This outputs the following message:

isohybrid: Warning: more than 1024 cylinders: 1052
isohybrid: Not all BIOSes will be able to boot this device

Then I dd’ed it to the usb disk for testing like so:

time dd if=tails-i386-feature_linux-3.14-2-1.1.1-20140727.iso.isohbrid of=/dev/sdb bs=16M

#41 Updated by intrigeri 2014-07-27 19:22:48

> There are lots of little warnings like this:

All this is fully expected.

#42 Updated by intrigeri 2014-07-27 19:24:01

> More warnings:

Expected too.

#43 Updated by intrigeri 2014-07-27 19:24:36

> My last build had the following output:
>

> path relative to the source directories
> Cannot stat sortlist entry "usr/share/iceweasel/modules/sessionstore/SessionStorage.jsm"
> This is probably because you're using the wrong file
> path relative to the source directories
> Cannot stat sortlist entry "usr/share/nautilus/ui/nautilus-navigation-window-ui.xml"
> This is probably because you're using the wrong file
> path relative to the source directories
> Cannot stat sortlist entry "usr/share/pixmaps/nautilus/erase.png"
> This is probably because you're using the wrong file
> path relative to the source directories

> 

Yeah, our squashfs sort list is outdated. There’s a ticket about it somewhere.
No big deal, and no real harm done.

#44 Updated by intrigeri 2014-07-27 19:25:02

> Unsure if this is directly related but on Wheezy during boot/build I have the following output in dmesg:
>

> [  132.375844] tracker-miner-f[3643]: segfault at 1aac ip b746ebda sp bfba6644 error 4 in libgobject-2.0.so.0.3200.4[b7444000+50000]
> 

Please file a dedicated bug report.

#45 Updated by intrigeri 2014-07-27 19:25:54

> I tried the build again:

All these “skipping” lines indicate that you did not lb clean --all.

#46 Updated by intrigeri 2014-07-27 19:32:07

> During the build we have some other warnings:

>

> [...]
> libkmod: ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.2.0-4-686-pae/modules.dep.bin'
> W: intel-microcode: cpuid kernel driver missing
> W: intel-microcode: disabling IUCODE_TOOL_SCANCPUS option
> live-boot: core filesystems devices utils udev wget blockdev.
> [...]
> libkmod: ERROR ../libkmod/libkmod.c:554 kmod_search_moddep: could not open moddep file '/lib/modules/3.2.0-4-686-pae/modules.dep.bin'
> W: intel-microcode: cpuid kernel driver missing
> W: intel-microcode: disabling IUCODE_TOOL_SCANCPUS option
> 

> We also have some surprising network fetches: […]

Please file a ticket for each individual problem, thanks.

#47 Updated by intrigeri 2014-07-27 19:33:26

> In the end, the build failed again:
>

> [...]
> Err http://ftp.de.debian.org/debian/ wheezy/main make i386 3.81-8.2
>   500  Connection timeout
> Get:1 http://ftp.de.debian.org/debian/ wheezy/main dpkg-dev all 1.16.15 [1356 kB]
> Failed to fetch http://ftp.de.debian.org/debian/pool/main/m/make-dfsg/make_3.81-8.2_i386.deb  500  Connection timeout
> Fetched 1356 kB in 60s (22.3 kB/s)
> [...]
> 

That’s a network connection problem on your side, or a problem with apt-cacher-ng.

#48 Updated by intrigeri 2014-07-27 19:34:53

> This outputs the following message:
>

> isohybrid: Warning: more than 1024 cylinders: 1052
> isohybrid: Not all BIOSes will be able to boot this device
> 

This is off-topic on this ticket, and I think we already have a ticket about it somewhere.

#49 Updated by ioerror 2014-07-27 19:38:25

intrigeri wrote:
> > There are lots of little warnings like this:
>
> All this is fully expected.

My feeling is that if warnings are expected - we should state something like “the manual build process will have warnings but no errors; feel free to file a bug for any warning you see but please note that those are low priority and may already be known.”

That sounds like a very straight forward documentation update.

#50 Updated by dawuud 2014-07-27 19:55:45

My build patch for the vargrant build method as mention in comment 18:
https://labs.riseup.net/code/issues/7661#note-18

I build a Tails iso with the new kernel… I dd’ed it to a USB disk and tested it out;
Everything worked fine. I got on the wifi, clocks synced, tor started… browser worked.

#51 Updated by ioerror 2014-07-27 23:27:28

intrigeri wrote:
> > […]
>
> * What branch are you building from?

I have been building from devel but eventually settled on feature/linux-3.14-2 with a few custom patches.

diff --git a/config/binary_local-hooks/10-syslinux_customize b/config/binary_local-hooks/10-syslinux_customize
index 214c487..eee6683 100755
--- a/config/binary_local-hooks/10-syslinux_customize
+++ b/config/binary_local-hooks/10-syslinux_customize
@@ -57,4 +57,4 @@ sed -i -e '/^include stdmenu\.cfg/a include tails.cfg' "${CFG_FILE}" 
 sed -e 's,/isolinux/,,' -i "${SYSLINUX_PATH}/stdmenu.cfg" 

 # remove useless files that break incremental upgrades on Wheezy
-rm "${SYSLINUX_PATH}"/{exithelp,prompt}.cfg
+# rm "${SYSLINUX_PATH}"/{exithelp,prompt}.cfg

> * May you please git clean -fdx and retry?

I did a fresh clone in a new directory. I did not apply the patch above. I then did the wiki build as you requested.

> * What does ./build-wiki in a clean Git tree do?

I did the following:

git clone git://git.tails.boum.org/tails
git checkout po
git checkout wiki
git checkout master
git checkout devel
git checkout feature/linux-3.14-2
./build-wiki

It builds the wiki as expected. I will now apply the above (10-syslinux_customize) patch and try to build the full live CD like so:

su -
lb clean --all && lb config && lb build

The lb clean --all && lb config commands worked perfectly and the lb build took a while but it eventually produced a functional ISO!

Restating the obvious for Manual builds to work on Wheezy:

• We need a documentation patch (add note about apt-cacher-ng and configuration)
• A merge of feature/linux-3.14-2 (which I hear is scheduled to happen tomorrow)
• A patch and merge of that single line 10-syslinux_customize patch listed above

This should ensure that manual builds work on Wheezy and that all the basics for such a build are documented.

It would be great to get a summay of why the vagrant builds aren’t working out of the box.

#55 Updated by kytv 2014-08-02 19:43:17

ioerror wrote:
> It would be great to get a summay of why the vagrant builds aren’t working out of the box.

FWIW, they work fine for me. I was getting an error with installing dpkg-dev, but bugfix/7691-fix-ftbfs-with-apt-get-source-syslinux took care of that. Now builds work manually as well as with Vagrant.

(Except for right now with the temporary outage of *.boum.org)