Bug #14250

Applications menu stops working sometimes due to time going backwards

Added by goupille 2017-08-20 13:02:43 . Updated 2018-02-24 11:46:35 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Target version:
Tails_3.5
Start date:
2017-08-20
Due date:
% Done:

100%

Feature Branch:
bugfix/14250-apps-menu-when-time-goes-backwards
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

A few users reported that during some sessions, the ‘Applications’ menu button stops working moments after Tails finishes to start. Apparently, opening ‘Places’ then moving to ‘Applications’ works.

Upstream bug: https://bugzilla.gnome.org/show_bug.cgi?id=763886

Subtasks

Related issues

Related to Tails - Bug #15168: Improve UX when hardware clock is set to localtime in a timezone too far from UTC Resolved 2018-01-15
Blocks Tails - Feature #13245: Core work 2018Q1: Foundations Team Resolved 2017-06-29

History

#1 Updated by intrigeri 2017-08-25 16:48:19

  • Assignee changed from intrigeri to anonym

(Sharing Foundations Team work.)

#2 Updated by intrigeri 2017-09-19 06:10:37

  • Target version set to Tails_3.3

Putting this more obviously on anonym’s radar.

#3 Updated by goupille 2017-09-21 16:40:09

  • Status changed from New to Confirmed

switching this one to confirmed since it was reported by several sources

#4 Updated by intrigeri 2017-11-11 12:40:09

  • Priority changed from Normal to Elevated

This was 2nd on the list of hot topics for help desk in September, and according to them it keeps being one of the top reported problems => bumping priority.

#5 Updated by intrigeri 2017-11-11 12:42:59

  • Assignee changed from anonym to goupille
  • QA Check set to Info Needed

goupille wrote:
> I could not reproduce myself but I have a user’s whisperback report containing gnome-session error and warnings

I have no email with this ticket ID in the subject in my inbox. What’s the report ID?

#6 Updated by goupille 2017-11-11 15:26:07

> I have no email with this ticket ID in the subject in my inbox

that’s probably because I forgot to send it :)

I just sent it to you with the ticket number in the subject

#7 Updated by goupille 2017-11-11 15:27:01

  • Assignee changed from goupille to intrigeri

#8 Updated by intrigeri 2017-11-11 16:40:16

  • QA Check deleted (Info Needed)

(Requested info was provided.)

#9 Updated by intrigeri 2017-11-11 16:42:02

#10 Updated by intrigeri 2017-11-11 16:52:45

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to goupille
  • Target version changed from Tails_3.3 to Tails_3.5
  • % Done changed from 0 to 10
  • QA Check set to Info Needed

In the logs I see nothing fishy. One notable thing though is that the system time was changed twice (once by tordate, once by htpdate) and both times towards the past:

Sep 23 14:34:15 amnesia time[17824]: A Tor consensus file now contains a valid time interval.
Sep 23 14:34:15 amnesia time[17827]: We do not have a Tor verified consensus, let's use the unverified one.
Sep 23 14:34:15 amnesia time[17828]: Waiting for the chosen Tor consensus file to contain a valid time interval...
Sep 23 14:34:15 amnesia time[17830]: The chosen Tor consensus now contains a valid time interval, let's use it.
Sep 23 14:34:15 amnesia time[17834]: Tor: valid-after=2017-09-23 11:00:00 | valid-until=2017-09-23 14:00:00
Sep 23 14:34:15 amnesia time[17837]: Current time is 2017-09-23 14:34:15
Sep 23 14:34:15 amnesia time[17842]: Current time is not in valid Tor range, setting to middle of this range: [2017-09-23 12:30:00]
Sep 23 12:30:00 amnesia systemd[6610]: Time has been changed
[…]
Sep 23 12:30:13 amnesia systemd[1]: Starting Setting time using HTP...
[…]
Sep 23 11:34:43 amnesia systemd[6610]: Time has been changed

I suspect that GNOME Shell (or something else down the stack) might not be super happy with going back in time. I wish there was a reliable way to tell wether the hardware clock is in UTC or in local time but sadly AFAIK that’s impossible :/ I wonder if we should kill GNOME Shell (to force it to restart) when we do go back in time like this.

goupille, do you have more such reports? If so please send them to me, I’d like to check if there’s a correlation between this problem and going back in time.

> Apparently, opening ‘Places’ then moving to ‘Applications’ works.

This feels weird though. Is this the case for strictly more than one affected users?

I doubt we’ll fix it in time for 3.3 so postponing.

#11 Updated by goupille 2017-11-11 17:19:19

  • Assignee changed from goupille to intrigeri
  • QA Check deleted (Info Needed)

I just sent you another report. it is from the user mentioning the ‘places’ workaround, and he or she said that the time was never set to GMT and that it needed to be set manually.

#12 Updated by intrigeri 2017-11-18 11:21:25

  • Assignee changed from intrigeri to goupille
  • QA Check set to Info Needed

goupille wrote:
> I just sent you another report. it is from the user mentioning the ‘places’ workaround, and he or she said that the time was never set to GMT and that it needed to be set manually.

I can’t find this. The only related report I have received is “[related to Bug #14250]Re: Bug report: 2eec54c8d51f21f68b806d55fdbd68d2”.

#13 Updated by goupille 2017-11-18 18:21:30

  • Assignee changed from goupille to intrigeri
  • QA Check deleted (Info Needed)

my bad, I just resend it now

#14 Updated by intrigeri 2017-11-18 19:07:56

  • Assignee changed from intrigeri to goupille
  • QA Check set to Info Needed

goupille wrote:
> my bad, I just resend it now

Right, but you didn’t attach the full debug log and your MUA trimmed it.

#15 Updated by goupille 2017-11-18 20:52:40

  • Assignee changed from goupille to intrigeri
  • QA Check deleted (Info Needed)

now you should have them :)

#16 Updated by intrigeri 2017-11-18 21:19:34

  • Assignee changed from intrigeri to goupille
  • QA Check set to Info Needed

In the logs (ec826c287aaee84f677f11f65d4dca4f) I see that the user changed the system clock manually (org.gnome.controlcenter.datetime.configure) first thing after login, before Tails had any chance to do so automatically. So this does not help me understand the “she said that the time was never set to GMT and that it needed to be set manually” part. It might be that Tails would manage to set the timezone to GMT if the user waited enough; it might be that Tails would fail, and then it would be interesting to have this user report a bug about this, without tweaking the timezone manually. If you can have them do that, please file a dedicated ticket.

Regardless, in this case as well we go back in time, so my “GNOME Shell (or something else down the stack) might not be super happy with going back in time” hypothesis still holds.

I would like to see more reports, as 2 reports is a bit short wrt. validating my hypothesis: this was 2nd on the list of hot topics for help desk in September, so I assume there were surely more than 2 bug reports, no? Feel free to reassign to whoever was on Help Desk duty back then (or whoever felt they had enough data to classify this ticket this way).

#17 Updated by intrigeri 2017-12-24 10:37:52

intrigeri wrote:
> I would like to see more reports, as 2 reports is a bit short wrt. validating my hypothesis: this was 2nd on the list of hot topics for help desk in September, so I assume there were surely more than 2 bug reports, no? Feel free to reassign to whoever was on Help Desk duty back then (or whoever felt they had enough data to classify this ticket this way).

A month later: ping → goupille & other Help Desk people.

#18 Updated by goupille 2017-12-24 13:26:49

  • Assignee changed from goupille to intrigeri

I didn’t saw any reports about that since then. iirc, it was reported by several users, but only two sent us a relevant whisperback report.

#19 Updated by intrigeri 2017-12-26 06:19:47

  • QA Check deleted (Info Needed)

#20 Updated by intrigeri 2018-01-01 16:40:23

  • blocked by deleted (Feature #13244: Core work 2017Q4: Foundations Team)

#21 Updated by intrigeri 2018-01-01 16:40:25

#22 Updated by intrigeri 2018-01-02 16:00:02

  • Assignee deleted (intrigeri)
  • Target version deleted (Tails_3.5)

goupille wrote:
> I didn’t saw any reports about that since then. iirc, it was reported by several users, but only two sent us a relevant whisperback report.

OK, so we have:

  • one relevant bug report
  • another bug report where the user manually interferred in a way that might actually have caused the problem; I’ve asked them to retry without interferring but goupille tells me they stopped answering a while ago, we’ll see
  • many other people reporting the problem, but who apparently don’t suffer from it enough to bother reporting a bug with WhisperBack

With basically no actionable info, even if I should in theory treat this bug as higher than normal priority (which I doubt, given affected users are apparently not that eager to see it fixed), I don’t know how I could actually do so => dropping this from the Foundations Team plate until we have enough info.

#23 Updated by intrigeri 2018-01-02 16:00:10

  • blocked by deleted (Feature #13245: Core work 2018Q1: Foundations Team)

#24 Updated by intrigeri 2018-01-12 13:05:00

Received another bug report (9dd010f6db4a6b61c3d7059f959509c3), woohoo! Here again, I see the system time going backwards: the hardware clock seems to be in local timezone (UTC+1) and thus htpdate sets the system clock back 1h in the past to UTC.

The system has been running for more than 50 minutes when the bug report is sent; I see nothing weird but the log is long; ideally I would like to ask the bug reporter when exactly the bug happened but they did not provide an email address.

I’ll need one more bug report to confirm (or not) my hypothesis. Hopefully this time I’ll get contact info so I can ask the bug reporter to try stuff.

#25 Updated by anonym 2018-01-12 14:29:06

I can reproduce it every time like this:

  1. Boot Tails 3.4
  2. Wait for the “Tor is ready” notification
  3. Verify that the Applications menu button responds to clicks
  4. Open GNOME Terminal and run: sudo date -s "now - 1 hour"
  5. Notice that the Application menu button no longer responds to clicks
  6. Run: sudo date -s "now + 1 hour" (i.e. undo 4)
  7. Notice that the Application menu button is fixed and now responds to clicks again (i.e. we’re back to 3)

Interesting indeed! :)

#26 Updated by anonym 2018-01-12 15:05:22

More observations:

  • Unloading the apps-menu GNOME Shell plugin does not fix the issue
  • In fact, GNOME’s default Activities menu button is equally affected
  • One way to fix the Activities/Applications button is to press the keyboard shortcut for the activities overview (Super) — after that the button works (but can be broken again by changing the time)

#27 Updated by emmapeel 2018-01-13 10:20:21

anonym wrote:
> I can reproduce it every time like this:
>
> # Boot Tails 3.4
> # Wait for the “Tor is ready” notification
> # Verify that the Applications menu button responds to clicks
> # Open GNOME Terminal and run: sudo date -s "now - 1 hour"
> # Notice that the Application menu button no longer responds to clicks
> # Run: sudo date -s "now + 1 hour" (i.e. undo 4)
> # Notice that the Application menu button is fixed and now responds to clicks again (i.e. we’re back to 3)
>
> Interesting indeed! :)

I tried to reproduce it but my Applications menu never stops working. I get the ‘Favorites’ bar on the side as well. But I have never faced this issue before (Bug #14250).

#28 Updated by anonym 2018-01-14 13:22:15

emmapeel wrote:
> I tried to reproduce it but my Applications menu never stops working. I get the ‘Favorites’ bar on the side as well. But I have never faced this issue before (Bug #14250).

Interesting! I was getting very confused, cause I couldn’t reproduce it either in the exact same KVM guest and host environment (well, except different time, and that the host system had been suspended). But then after three failed reproduction attempts, the fourth one worked, and since then I cannot get it to fail again. So possibly there’s some nondeterminism involved.

BTW, I can reproduce this on Debian Stretch (in a KVM guest) with the default “Activities” button (I have to run systemctl stop time-sync.target otherwise it would undo my time change). So this is not anything Tails-specific.

Also, I just tried reproducing this in Virtualbox, and it wasn’t straight-forward. Eventually I managed to reproduce it, while I was also fiddling with gsettings set org.gnome.desktop.interface enable-animations true/false, which might or might not be relevant. Not sure what this means, except that the issue is not specific to KVM.

#29 Updated by Anonymous 2018-01-15 09:39:04

  • Subject changed from Applications menu stops working some times to Applications menu stops working sometimes due to time going backwards

So shouldn’t we investigate why time could go backwards without users interfering in first place?
And if users manually set the time, shouldn’t we support this usecase?

#30 Updated by intrigeri 2018-01-15 10:04:26

> So shouldn’t we investigate why time could go backwards without users interfering in first place?

We do this on purpose as part of https://tails.boum.org/contribute/design/Time_syncing/

#31 Updated by intrigeri 2018-01-15 12:13:45

  • Assignee set to intrigeri
  • Target version set to Tails_3.5

(At least for triaging and reporting upstream.)

#32 Updated by intrigeri 2018-01-15 13:10:42

Reproduced on current Debian sid (gnome-shell 3.26.2-2) and Fedora-Workstation-Live-x86_64-27-1.6.iso.

#33 Updated by intrigeri 2018-01-15 13:46:02

Reproduced on sid by changing the time via GNOME control center.
Upstream bug report: https://bugzilla.gnome.org/show_bug.cgi?id=763886

#34 Updated by intrigeri 2018-01-15 13:50:01

  • Description updated

#35 Updated by intrigeri 2018-01-15 14:24:42

  • % Done changed from 10 to 20
  • Type of work changed from Research to End-user documentation

I’ve reported all the details and test results I could upstream. I’m now going to document the workaround (click Places) on our Known Issues page so at least our help desk can point affected users somewhere.

Zooming out a little, even if/once this bug in the GNOME Shell stack is fixed, let’s be honest: making the time go backwards is bound to trigger a variety of bugs. I’ve filed Bug #15168 to discuss how we should tackle this more general problem.

#36 Updated by intrigeri 2018-01-15 14:24:49

  • related to Bug #15168: Improve UX when hardware clock is set to localtime in a timezone too far from UTC added

#37 Updated by intrigeri 2018-01-15 15:05:01

  • Status changed from In Progress to Resolved
  • % Done changed from 20 to 100

Applied in changeset commit:e2a548ac9346901fd644caa01ac2fe6f0daf0429.

#38 Updated by intrigeri 2018-01-15 15:15:24

#39 Updated by intrigeri 2018-01-15 16:13:04

  • Status changed from Resolved to In Progress
  • % Done changed from 100 to 70
  • Feature Branch set to bugfix/14250-apps-menu-when-time-goes-backwards
  • Type of work changed from End-user documentation to Code

Good news! Upstream proposed a patch that works for me on sid. Built a patched gnome-shell package, uploaded, will test and if happy this will be a candidate for Tails 3.5 (and then I’ll try to get it into Stretch proposed-updates).

#40 Updated by intrigeri 2018-01-16 12:21:36

  • Assignee changed from intrigeri to anonym
  • QA Check set to Ready for QA

intrigeri wrote:
> Good news! Upstream proposed a patch that works for me on sid. Built a patched gnome-shell package, uploaded, will test and if happy this will be a candidate for Tails 3.5

I can’t reproduce the bug on this branch.

> (and then I’ll try to get it into Stretch proposed-updates).

I’ll do that once the fix has been applied to upstream Git. No need for a ticket IMO.

#41 Updated by anonym 2018-01-19 22:45:25

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 70 to 100
  • QA Check changed from Ready for QA to Pass

The fix is very convincing, and I tried to reproduce a couple of times without success => merged!

#42 Updated by anonym 2018-01-23 19:50:25

  • Status changed from Fix committed to Resolved

#43 Updated by intrigeri 2018-02-24 11:46:35

intrigeri wrote:
> intrigeri wrote:
> > Good news! Upstream proposed a patch that works for me on sid. Built a patched gnome-shell package, uploaded, will test and if happy this will be a candidate for Tails 3.5
>
> I can’t reproduce the bug on this branch.
>
> > (and then I’ll try to get it into Stretch proposed-updates).
>
> I’ll do that once the fix has been applied to upstream Git. No need for a ticket IMO.

The upstream proposed fix was turned into a merge request.