Tails

#1 Updated by intrigeri 2018-12-03 18:10:18

• blocks Feature #15506: Core work 2018Q4: Foundations Team added

#5 Updated by intrigeri 2018-12-10 09:41:33

• Target version changed from Tails_3.11 to Tails_3.12

#6 Updated by intrigeri 2018-12-30 12:32:50

• blocks Feature #15507: Core work 2019Q1: Foundations Team added

#7 Updated by intrigeri 2018-12-30 12:33:04

• blocked by deleted (Feature #15506: Core work 2018Q4: Foundations Team)

#8 Updated by hefee 2019-01-04 15:33:11

• Assignee set to hefee

#9 Updated by hefee 2019-01-11 16:07:40

• Assignee changed from hefee to bertagaz
• QA Check set to Info Needed

So far I can see there are only two packages, that get transmitted by Tails. Others are successfully sopped by ferm.

In the journal you can see entries like this, as we expected them to get dropped:

Nov 21 14:37:49 amnesia kernel: Dropped outbound packet: IN= OUT=eth0 SRC=fe80:0000:0000:0000:5254:00ff:fe42:37d2 DST=ff02:0000:0000:0000:0000:0000:0000:0002 LEN=56 TC=0 HOPLIMIT=255 FLOWLBL=0 PROTO=ICMPv6 TYPE=133 CODE=0·

Unfortunately the pcap file does only have timedelta as time, so I can’t link the pcap together with the ferm start. Can you give me the timestamp, when the pcap started? Than we should see, when the packages got transmitted and were not catched.

geb, nodens:
I think, we should be able to control the output on all interfaces and not just say, yeah well, this packages sent are not that bad. If Tails starts to support IPv6, we need to enable these packages. But currently Tails is not ready for IPv6, so we should disable everything and also try to catch for those two packages…

#10 Updated by intrigeri 2019-01-11 17:10:36

• Assignee changed from bertagaz to hefee

> Can you give me the timestamp, when the pcap started? Than we should see, when the packages got transmitted and were not catched.

AFAICT all the artifacts we have are attached to this ticket. I think this info is in the debug log. If it’s not explicit, let me know, I’ll check our test suite code to tell you which debug log line indicates the start of the capture.

(Also, you mean packets, not packages, right? :)

#11 Updated by hefee 2019-01-21 01:28:32

• Assignee changed from hefee to intrigeri

intrigeri wrote:

> AFAICT all the artifacts we have are attached to this ticket. I think this info is in the debug log. If it’s not explicit, let me know, I’ll check our test suite code to tell you which debug log line indicates the start of the capture.

Sorry my mistake, the pcap has a absolute time but normaly only secs form start are displayed. Than I can match the journal and pcap.

The interesting packets:

15:33:57 - Router Sollicitation
15:33:57 - Neighbor Sollicitation

I can match the pcap with the journal by the dchp-client packets:

15:37:44 - DHCP Request (pcap)
Nov 21 14:37:46 amnesia dhclient[1137]: DHCPREQUEST of 10.2.1.45 on eth0 to 255.255.255.255 port 67

-> We have around 1h 2secs diff between journal and pcap. But the saved journal starts at 14:34:28 UTC. We are interested what was happening at 14:33:55 UTC. So it is maybe a shutdown issue?

Unfortunately the debuglog uses completely different timestamps (sec from start). And tells me, that the test fails at 2:56:48. (It would help, if the timestapms would be UTC too). So if I speculate, that this is the end time of the journal. I see this scenario, that is triggering the packets:

02:52:09.157565101: execution complete
    And I can open the Additional Software configuration window from the notification 
# features/step_definitions/additional_software_packages.rb:136
02:52:09.158840925: spawning as root: poweroff
    And I shutdown Tails and wait for the computer to power off
# features/step_definitions/common_steps.rb:562
02:52:19.485028828: [log] CLICK on L(1023,384)@S(0)[0,0 1024x768]                                  
02:52:22.893751238: [log]  TYPE " autotest_never_use_this_option blacklist=psmouse #ENTER."
02:53:46.493521850: calling as root: echo 'hello?'

That would match to my thought, that the packets are triggered from the previous Tails run and are a shutdown issue.

Can it be, that the tcpdump starts along with the start of the scenario and that reboots do not test the tcpdump, so only the last journal is saved?

#13 Updated by intrigeri 2019-01-21 13:26:28

• Assignee changed from intrigeri to anonym

I won’t have time to look into this in time for 3.12, but anonym might have time and could be excited about it. If that does not happen, let’s organize the next steps at the FT meeting.

#14 Updated by anonym 2019-01-29 15:30:08

• Target version changed from Tails_3.12 to Tails_3.13

#15 Updated by intrigeri 2019-02-06 14:24:46

• Assignee deleted (anonym)

intrigeri wrote:
> I won’t have time to look into this in time for 3.12, but anonym might have time and could be excited about it. If that does not happen, let’s organize the next steps at the FT meeting.

Did not happen, let’s discuss this at the FT meeting.

#16 Updated by intrigeri 2019-02-16 06:03:42

• File 02_09_36_Recovering_in_offline_mode_after_Additional_Software_previously_failed_to_upgrade_and_then_succeed_to_upgrade_when_online.journal added
• File 02_09_36_Recovering_in_offline_mode_after_Additional_Software_previously_failed_to_upgrade_and_then_succeed_to_upgrade_when_online.pcap added
• File debug.log added

Here’s pcap, Journal and debug log for another instance of this bug (that also exposes Bug #16150).

#17 Updated by intrigeri 2019-02-16 06:06:30

• related to Bug #11521: The check_tor_leaks hook is fragile added

#19 Updated by intrigeri 2019-02-16 06:08:12

• Category set to Test suite
• QA Check deleted (Info Needed)

#20 Updated by intrigeri 2019-03-12 14:16:53

• Target version deleted (Tails_3.13)

#21 Updated by intrigeri 2019-03-12 16:10:50

• blocks Feature #16209: Core work: Foundations Team added

#22 Updated by intrigeri 2019-03-12 16:10:52

• blocked by deleted (Feature #15507: Core work 2019Q1: Foundations Team)

#23 Updated by intrigeri 2019-05-19 05:30:02

• Priority changed from Normal to Elevated

This has caused 3 out of the last 10 test suite runs on the stable branch to fail, which makes me spend an amount of time I can’t justify on analyzing such failures (and I would assume that some other folks on the RMs list do the same) => bumping priority. @anonym, looking for your next FT semi-procrastination task, for when you won’t be busy with Tor Browser 9 matters?

#24 Updated by anonym 2019-06-04 12:40:01

• Status changed from Confirmed to In Progress
• Assignee set to anonym
• Target version set to Tails_3.15
• % Done changed from 0 to 20
• Feature Branch set to feature/16150-unfiltered-pcaps feature/16150-disable-ipv6

intrigeri wrote:
> Anyone who works on this and wants to investigate the “leftover from previous scenario” hypothesis, see Bug #11521 where I’ve made the very same hypothesis a few years ago.

Looking at the pcap files we can rule this out: the offending packets are sent in the middle of the scenario, not the beginning which would be implied by your hypothesis.

In test_asp_75_firewallassertionerror.pcap the offending packets are:

MAC source         MAC destination    Info
50:54:00:ad:40:c6  33:33:00:00:00:02  Router Solicitation from 50:54:00:ad:40:c6
50:54:00:ad:40:c6  33:33:ff:ad:40:c6  Neighbor Solicitation for fc00::5254:ff:fead:40c6

and the source indeed is the Tails system (50:54:00:ad:40:c6). It’s unfortunate that we do filtering already when we collect the pcap file, because other packets sent before/after this could perhaps shed some light on what’s going on. I’ve pushed the branch feature/16150-unfiltered-pcaps which collects full pcap files to investigate this further.

Also, since we are not supporting IPv6 and try but fail to block it, why not just completely disable it? I’ve pushed feature/16150-disable-ipv6 for that, and my assumption is that this should completely fix this problem. Any objections to this? Things I haven’t thought about?

Actually, disabling IPv6 should just work around whatever issue we have, but I think that is fine. When/if we add IPv6 support we’ll have think deep about tons of stuff including the Router Solicitation topic, so let’s defer any that work to then.

Any way, my plan with these branches are:

• Verify that feature/16150-disable-ipv6 fixes these failures
• See if any run of feature/16150-unfiltered-pcaps gives more clues about the real issue and if so take notes for the future work on adding IPv6 support (but not spend a lot of time on it)
• Get both branches merged:
  • feature/16150-disable-ipv6 as the fix
  • feature/16150-unfiltered-pcaps for its improved pcap logging (and it’s a nice refactoring, imho)

#25 Updated by anonym 2019-06-04 14:06:36

• Feature Branch changed from feature/16150-unfiltered-pcaps feature/16150-disable-ipv6 to feature/16148-unfiltered-pcaps feature/16148-disable-ipv6

#27 Updated by anonym 2019-06-04 18:04:40

• Feature Branch changed from feature/16148-unfiltered-pcaps feature/16148-disable-ipv6 to feature/16148-unfiltered-pcaps feature/16148-disable-ipv6 feature/16148-disable-ipv6+force-all-tests

#28 Updated by anonym 2019-06-05 08:31:40

• Status changed from In Progress to Needs Validation
• Assignee deleted (anonym)
• % Done changed from 20 to 50

We have test results:

• feature/16148-unfiltered-pcaps: the scenario still fails, as expected. The full pcap shows that after Tails sends a Router Solicitation, then the router sends a Router Advertisement (then Tails sends the Neighbor Solicitation). So this confirms what nodens said in Bug #16148#note-2: “If there is no ipv6 default gateway, it’s expected for the host to send icmpv6 multicast packets for router/neighbor discovery”. So we are at least not responding to an advertisment initiated by the router, which is good.
  I’m still convinced that disabling IPv6 is the proper solution for us now. The alternative is to find a way to just disable sending Router Solicitations (net.ipv6.conf.all.accept_ra=0 perhaps does that?) or make an exception for it in our filter (like we do for DHCP) but I don’t see any reason to keep this attack surface.

• feature/16148-disable-ipv6 (and another run: all scenarios passed, including this one, so disabling IPv6 would indeed solve our problem.

• feature/16148-disable-ipv6+force-all-tests: two unrelated failures, yay!

IMHO these branches are ready to be merged!

#29 Updated by hefee 2019-06-05 09:37:14

• Assignee set to hefee

#30 Updated by hefee 2019-06-05 10:04:30

• Assignee changed from hefee to anonym

anonym wrote:
> We have test results:
>
> * feature/16148-unfiltered-pcaps: the scenario still fails, as expected. The full pcap shows that after Tails sends a Router Solicitation, then the router sends a Router Advertisement (then Tails sends the Neighbor Solicitation). So this confirms what nodens said in Bug #16148#note-2: “If there is no ipv6 default gateway, it’s expected for the host to send icmpv6 multicast packets for router/neighbor discovery”. So we are at least not responding to an advertisment initiated by the router, which is good.
> I’m still convinced that disabling IPv6 is the proper solution for us now. The alternative is to find a way to just disable sending Router Solicitations (net.ipv6.conf.all.accept_ra=0 perhaps does that?) or make an exception for it in our filter (like we do for DHCP) but I don’t see any reason to keep this attack surface.

disabling IPv6 completely is consequent and I like this step. As we than can enable Ipv6, when we are ready for it. What we loose when we disable IPv6 is that there may be setups that have IPv6 only in the local network and use Ip4 mapping (https://en.wikipedia.org/wiki/IPv6#IPv4-mapped%20IPv6%20addresses).

> * feature/16148-disable-ipv6 (and another run: all scenarios passed, including this one, so disabling IPv6 would indeed solve our problem.

seems fine.

> * feature/16148-disable-ipv6+force-all-tests: two unrelated failures, yay!

seems fine.

> IMHO these branches are ready to be merged!

review passed. As I don’t have permission to merge, so you have to merge yourself.

#33 Updated by anonym 2019-06-06 10:10:28

• Assignee deleted (anonym)

hefee wrote:
> anonym wrote:
> > Actually I just remember that nodens suggested to disable by adding disable_ipv6=1 to the kernel command-line instead. Our current approach (sysctl during boot) actually creates a window during early boot where IPv6 is enabled. Since we block all network modules until after logging in via Tails Greeter, IPv6 will only be enabled for the loopback interface, so this is perhaps a bit pedantic. We’ll see what I feel tomorrow! (Your thoughts are also welcome, hefee!)
>
> Good thughts, but IMO this is irrelevant. sysctl should be triggered much earlier than any network traffic is going over the wire. sysctl.conf is read by systemd-sysctl.service that is triggered within sysinit.target. And only after this the other targets like network.target are started. That’s why I think your approach is safe, that we do not leave any ipv6 packet.

Makes sense to me!

> What I read is that we may need ipv6 enabled for loopback, as some programs except loopback with ipv6 capabilities.
> […]

Our test test results indicate that nothing broke due to this, but I still think it makes sense (it could save front desk from some hard-to-debug issues from users installing such programs) so I’ll add it.

> But if we do not see issues, keep it simple and don’t add this line. But this may also been solved if we remove the line ::1 localhost is in the /etc/hosts.

Our hosts file is already IPv4 only.

#34 Updated by anonym 2019-06-06 11:38:29

• Status changed from Needs Validation to In Progress

Applied in changeset commit:tails|24b2dd805e63019d945b9d204e49ee5f3ea521c9.

#35 Updated by anonym 2019-06-06 11:38:30

• Status changed from In Progress to Fix committed
• % Done changed from 50 to 100

Applied in changeset commit:tails|744501f6d33001bff8de84e7aa42d1609ea687c6.

#37 Updated by intrigeri 2019-06-07 10:58:57

• related to Bug #16788: Tests fail with "Found something in the pcap file that either is non-IP, or cannot be parsed (RuntimeError)" added

#38 Updated by anonym 2019-06-11 11:51:07

• Status changed from Fix committed to In Progress

Applied in changeset commit:tails|86bd6d5db9284e6204192dae2f7cae1ddaf20df2.

#39 Updated by anonym 2019-06-11 13:06:06

• Status changed from In Progress to Fix committed

#41 Updated by intrigeri 2019-06-19 07:58:39

• related to Bug #16825: "Found something that is not an ethernet packet" intermittent test failure added

#42 Updated by intrigeri 2019-07-03 08:04:22

• Status changed from Fix committed to Resolved
• Target version changed from Tails_3.15 to Tails_3.14.1

Fixed in 3.14.1.