Tails

#1 Updated by segfault 2019-12-18 16:40:59

sajolida, intrigeri, what do you think?

#6 Updated by intrigeri 2019-12-28 17:28:17

> I looked into ways to do that, and the easiest would be to add a persistence preset for persisting the user’s dconf database stored in /home/amnesia/.config/dconf/user. This file seems to contain all the dconf settings that were changed by the user. This means that we can still change the default values we set in /etc/dconf/db/local.d.

Wow! Thanks for looking into this. I’ve always assumed this was a non-trivial problem technically speaking, because the user’s dconf database would contain more info than that. So indeed, if we can confirm that the user’s dconf database only has the settings they’ve changed, the problem looks much more tractable than we thought on Feature #5979 :)

Below I’ll mostly play the devil’s advocate and raise potential issues, but I think it’s a great discovery that you’ve made, and I would really like us to dive deeper into it: it would solve a bunch of problems our users have (see the pile of related tickets).

From a technical standpoint, off the top of my head:

• This proposal looks just perfect for settings that contain atomic values. I can think of no trouble at all there.
• There’s a problem for settings that contain lists, such as /org/gnome/shell/enabled-extensions and /org/gnome/shell/favorite-apps: when a user adds an element to, or removes an element from, such a list, they may believe their persistent custom delta is “enabling GNOME Shell extension X” or “removing favorite app Y”; however, with the proposed implementation, their persistent custom delta is really “fork the whole list forever”. This could cause trouble later on:
  • If we don’t proactively take action to mitigate such problems when we modify the default value of such lists, then the user may be left with a partly broken Tails. For example, they would not get the upcoming OpenPGP applet GNOME Shell extension, and complain that we broke OpenPGP Applet; or they would not get the new KeePassXC shortcuts.
  • In theory, we could proactively take action to mitigate such problems, e.g. by applying these changes we make in the default settings to the persistent per-user settings, at login time, but this sort of upstream vs. downstream delta change management is not trivial (e.g. we may need to distinguish “we did not apply migration step X” from “we did apply migration step X but the user later reverted it”; for example, if the user removed the KeePassXC shortcut, our migration code should probably not re-add it forcibly, so we need to take note of what migration step we’ve already gone through). Another way to look at it is a 3-way merge.

From a UX standpoint, I can imagine some difficulties about how to convey to user in the Persistence Setup GUI what exactly is made persistent by this preset:

• there’s of course the GNOME (the desktop environment) config;
• many GTK apps use it to store their own settings and various kinds of state info, e.g. last opened files and so on, which is maybe not what I would expect if I choose to persist “configuration”.

Zooming out a fair bit, at this point, my biggest worries are:

• This will significantly increase the diversity of Tails systems ⇒ harder for us to help users, to reproduce bugs, and to debug issues, because many of them could be caused by something specific to the user’s persistent configuration. We already have the hardware axis that’s hard to cope with; this adds another one. This makes me worry for our help desk and for the FT folks who triage newly reported problems. Especially because there could be all kinds of personal info in dconf, so we probably can’t include more in WhisperBack reports than the list of modified settings (without their values).
• It gives users a very powerful tool, without giving them the means to tell when they’re using it and what the impact is: looking at my own dconf database outside of Tails, I realize how many of my actions change its content, in a way that will affect, somehow, how my system behaves in the future (e.g. some apps store details like the size of their window in there). When using a non-live, non-amnesic system, realistically it’s part of my expectations. But I think that’s a different question when using Tails with this persistence setting enabled, due to its broad and vaguely defined scope. For example, I remember an “intercept” interview I did a couple years back, when the user (who was using Tails as her main OS, with persistence, and was pretty tech-savvy) told me she loved the safety feeling that Tails gave, by coming back in a pristine state every time she starts it, regardless of what she did last time. I think it’s a matter of feeling in control of Tails and understanding the consequences of one’s actions. I’m worried that persisting the whole dconf database could be a game changer in this respect, and not for the better.

I think all of this can be solved. I don’t think that solving All The Things™ before we do anything is the best approach. Instead, I would love to see us find an iterative approach that allows us to try things out in this area, offer some of the benefits to users soon, learn about what problems happen in practice, solve the most important ones, and then go further. Ideas?

#7 Updated by segfault 2020-01-05 23:29:09

intrigeri wrote:
> > I looked into ways to do that, and the easiest would be to add a persistence preset for persisting the user’s dconf database stored in /home/amnesia/.config/dconf/user. This file seems to contain all the dconf settings that were changed by the user. This means that we can still change the default values we set in /etc/dconf/db/local.d.
>
> Wow! Thanks for looking into this. I’ve always assumed this was a non-trivial problem technically speaking, because the user’s dconf database would contain more info than that. So indeed, if we can confirm that the user’s dconf database only has the settings they’ve changed, the problem looks much more tractable than we thought on Feature #5979 :)

:)

> Below I’ll mostly play the devil’s advocate and raise potential issues, but I think it’s a great discovery that you’ve made, and I would really like us to dive deeper into it: it would solve a bunch of problems our users have (see the pile of related tickets).
>
> From a technical standpoint, off the top of my head:
>
> * This proposal looks just perfect for settings that contain atomic values. I can think of no trouble at all there.
> * There’s a problem for settings that contain lists, such as /org/gnome/shell/enabled-extensions and /org/gnome/shell/favorite-apps: when a user adds an element to, or removes an element from, such a list, they may believe their persistent custom delta is “enabling GNOME Shell extension X” or “removing favorite app Y”; however, with the proposed implementation, their persistent custom delta is really “fork the whole list forever”. This could cause trouble later on:
> If we don’t proactively take action to mitigate such problems when we modify the default value of such lists, then the user may be left with a partly broken Tails. For example, they would not get the upcoming OpenPGP applet GNOME Shell extension, and complain that we broke OpenPGP Applet; or they would not get the new KeePassXC shortcuts.
> In theory, we could proactively take action to mitigate such problems, e.g. by applying these changes we make in the default settings to the persistent per-user settings, at login time, but this sort of upstream vs. downstream delta change management is not trivial (e.g. we may need to distinguish “we did not apply migration step X” from “we did apply migration step X but the user later reverted it”; for example, if the user removed the KeePassXC shortcut, our migration code should probably not re-add it forcibly, so we need to take note of what migration step we’ve already gone through). Another way to look at it is a 3-way merge.

Indeed, we will have to find a way to handle lists. We could store information about which migrations we already applied in the .config/dconf directory.

> From a UX standpoint, I can imagine some difficulties about how to convey to user in the Persistence Setup GUI what exactly is made persistent by this preset:
>
> * there’s of course the GNOME (the desktop environment) config;
> * many GTK apps use it to store their own settings and various kinds of state info, e.g. last opened files and so on, which is maybe not what I would expect if I choose to persist “configuration”.

ACK. Maybe something like “System and App Settings” then?

> Zooming out a fair bit, at this point, my biggest worries are:
>
> * This will significantly increase the diversity of Tails systems ⇒ harder for us to help users, to reproduce bugs, and to debug issues, because many of them could be caused by something specific to the user’s persistent configuration. We already have the hardware axis that’s hard to cope with; this adds another one. This makes me worry for our help desk and for the FT folks who triage newly reported problems. Especially because there could be all kinds of personal info in dconf, so we probably can’t include more in WhisperBack reports than the list of modified settings (without their values).
> * It gives users a very powerful tool, without giving them the means to tell when they’re using it and what the impact is: looking at my own dconf database outside of Tails, I realize how many of my actions change its content, in a way that will affect, somehow, how my system behaves in the future (e.g. some apps store details like the size of their window in there). When using a non-live, non-amnesic system, realistically it’s part of my expectations. But I think that’s a different question when using Tails with this persistence setting enabled, due to its broad and vaguely defined scope. For example, I remember an “intercept” interview I did a couple years back, when the user (who was using Tails as her main OS, with persistence, and was pretty tech-savvy) told me she loved the safety feeling that Tails gave, by coming back in a pristine state every time she starts it, regardless of what she did last time. I think it’s a matter of feeling in control of Tails and understanding the consequences of one’s actions. I’m worried that persisting the whole dconf database could be a game changer in this respect, and not for the better.

I think it would help with both of these issues if the user could easily disable or reset the custom config in tails-persistence-setup (ideally without having to reboot, which is related to Feature #11529). Then we could ask users to try to reproduce their issues without custom config. And users could easily reset their config if they want to go back to a pristine state.

> I think all of this can be solved. I don’t think that solving All The Things™ before we do anything is the best approach. Instead, I would love to see us find an iterative approach that allows us to try things out in this area, offer some of the benefits to users soon, learn about what problems happen in practice, solve the most important ones, and then go further. Ideas?

I don’t see a simple iterative approach for this. Persisting the whole database is easy, persisting only certain settings is hard :/

#8 Updated by nodens 2020-01-06 09:45:35

segfault wrote:
> intrigeri wrote:
> > Zooming out a fair bit, at this point, my biggest worries are:
> >
> > * This will significantly increase the diversity of Tails systems ⇒ harder for us to help users, to reproduce bugs, and to debug issues, because many of them could be caused by something specific to the user’s persistent configuration. We already have the hardware axis that’s hard to cope with; this adds another one. This makes me worry for our help desk and for the FT folks who triage newly reported problems. Especially because there could be all kinds of personal info in dconf, so we probably can’t include more in WhisperBack reports than the list of modified settings (without their values).
> > * It gives users a very powerful tool, without giving them the means to tell when they’re using it and what the impact is: looking at my own dconf database outside of Tails, I realize how many of my actions change its content, in a way that will affect, somehow, how my system behaves in the future (e.g. some apps store details like the size of their window in there). When using a non-live, non-amnesic system, realistically it’s part of my expectations. But I think that’s a different question when using Tails with this persistence setting enabled, due to its broad and vaguely defined scope. For example, I remember an “intercept” interview I did a couple years back, when the user (who was using Tails as her main OS, with persistence, and was pretty tech-savvy) told me she loved the safety feeling that Tails gave, by coming back in a pristine state every time she starts it, regardless of what she did last time. I think it’s a matter of feeling in control of Tails and understanding the consequences of one’s actions. I’m worried that persisting the whole dconf database could be a game changer in this respect, and not for the better.
>
> I think it would help with both of these issues if the user could easily disable or reset the custom config in tails-persistence-setup (ideally without having to reboot, which is related to Feature #11529). Then we could ask users to try to reproduce their issues without custom config. And users could easily reset their config if they want to go back to a pristine state.

I like this. I think it would be even greater if there was a way for the user (and thus the helpdesk) to know what has changed, and maybe rest only parts of the changes.

ISTR gconf knows what has changed from the defaults, because you can see it in dconf-editor, but I’m not sure it considers the “system” settings to be the default (instead of each app defaults).

>
> > I think all of this can be solved. I don’t think that solving All The Things™ before we do anything is the best approach. Instead, I would love to see us find an iterative approach that allows us to try things out in this area, offer some of the benefits to users soon, learn about what problems happen in practice, solve the most important ones, and then go further. Ideas?
>
> I don’t see a simple iterative approach for this. Persisting the whole database is easy, persisting only certain settings is hard :/

I agree it is hard, because it means more work for each new upstream release, but we could make a whitelist of some settings that could be saved, starting small ?

Thanks for working on this !

#9 Updated by segfault 2020-01-06 20:45:21

nodens wrote:
> segfault wrote:
> > I think it would help with both of these issues if the user could easily disable or reset the custom config in tails-persistence-setup (ideally without having to reboot, which is related to Feature #11529). Then we could ask users to try to reproduce their issues without custom config. And users could easily reset their config if they want to go back to a pristine state.
>
> I like this. I think it would be even greater if there was a way for the user (and thus the helpdesk) to know what has changed, and maybe rest only parts of the changes.

We could replace the user-db:user line with service-db:keyfile/user in /etc/dconf/profile/user, then /home/amnesia/.config/dconf/user.txt will contain the config in plain text instead of binary, so it can be more easily inspected.

>
> ISTR gconf knows what has changed from the defaults, because you can see it in dconf-editor, but I’m not sure it considers the “system” settings to be the default (instead of each app defaults).

Our settings in /etc/dconf/db/local.d are considered the default ones.

> > > I think all of this can be solved. I don’t think that solving All The Things™ before we do anything is the best approach. Instead, I would love to see us find an iterative approach that allows us to try things out in this area, offer some of the benefits to users soon, learn about what problems happen in practice, solve the most important ones, and then go further. Ideas?
> >
> > I don’t see a simple iterative approach for this. Persisting the whole database is easy, persisting only certain settings is hard :/
>
> I agree it is hard, because it means more work for each new upstream release, but we could make a whitelist of some settings that could be saved, starting small ?

Only saving some settings is what is hard. We would have to somehow keep an up-to-date list of these whitelisted settings which can persist and restore during boot. The dconf tool supports watching dconf keys or directory for changes, but only one key or directory. So we would either have to start one dconf watch command per setting we want to persist, or implement a tool ourselves which uses the dconf dbus service to watch the whitelisted settings.

#10 Updated by nodens 2020-01-06 22:36:22

segfault wrote:
>
> Only saving some settings is what is hard. We would have to somehow keep an up-to-date list of these whitelisted settings which can persist and restore during boot. The dconf tool supports watching dconf keys or directory for changes, but only one key or directory. So we would either have to start one dconf watch command per setting we want to persist, or implement a tool ourselves which uses the dconf dbus service to watch the whitelisted settings.

We could work around this by storing the db in persistence, but restoring all keys to the default value except the whitelisted one at session start: it would mean parsing the db, so still write a tool, but it sounds easier than watching dbus constantly.

#12 Updated by segfault 2020-01-15 17:10:32

nodens wrote:
> segfault wrote:
> >
> > Only saving some settings is what is hard. We would have to somehow keep an up-to-date list of these whitelisted settings which can persist and restore during boot. The dconf tool supports watching dconf keys or directory for changes, but only one key or directory. So we would either have to start one dconf watch command per setting we want to persist, or implement a tool ourselves which uses the dconf dbus service to watch the whitelisted settings.
>
> We could work around this by storing the db in persistence, but restoring all keys to the default value except the whitelisted one at session start: it would mean parsing the db, so still write a tool, but it sounds easier than watching dbus constantly.

I don’t want to have to write a tool that parses dconf databases. But we could restore the user database to a non-default profile, and then use dconf read and dconf write to copy the whitelisted settings to the default profile. FTR, something like this might do:

echo "user-db:tps" | sudo tee /etc/dconf/profile/tps
mv /home/amnesia/.config/dconf/user /home/amnesia/.config/dconf/tps
key="/org/gnome/shell/favorite-apps"
value=$(DCONF_PROFILE=tps dconf read "${key}")
dconf write "${key}" "${value}"

(If we decide to persist plain text keyfiles instead of binary databases, as I suggested in a previous comment, we would have to use service-db:keyfile/tps instead of user-db:tps)

(Also, we could consider removing /home/amnesia/.config/dconf/tps after restoring all whitelisted settings, so that it can’t be accessed by other code run as amnesia).

#13 Updated by sajolida 2020-02-26 19:54:45

It’s super cool that you are investigating all this and explore
interesting technical possibilities! We have so many tickets and
requests related to such configs.

My concerns in terms of UX are similar to intrigeri’s concerns in
Feature #17367#note-6: storing the whole database seems too vague and too
dangerous for me. Users won’t have a clue of what’s stored in there and
what’s not.

I like the whitelisting approach, which could give us a way of both
controlling better the possible damages and get the user to understand
and choose what is stored. Remember that our entire paradigm on the
Persistent Storage is to allow users to choose what they want to save.
So we need a level of granularity that makes sense to them.

For example, we could have a mechanism around this database to advertise
persistence of the “Mouse & mouse Touchpad Settings”, the “Desktop
Background” (I’m not kidding!) or any subset of it that makes sense to
our users. But I really don’t see us advertising a persistence of the
whole thing.

For the sake of the anecdote, I persist myself of few such settings by
running a ~/.xsession with a few gsettings line in it each time I start
my Tails.

#14 Updated by segfault 2020-03-07 19:19:09

sajolida wrote:
> It’s super cool that you are investigating all this and explore
> interesting technical possibilities! We have so many tickets and
> requests related to such configs.
>
> My concerns in terms of UX are similar to intrigeri’s concerns in
> Feature #17367#note-6: storing the whole database seems too vague and too
> dangerous for me. Users won’t have a clue of what’s stored in there and
> what’s not.
>
> I like the whitelisting approach, which could give us a way of both
> controlling better the possible damages and get the user to understand
> and choose what is stored. Remember that our entire paradigm on the
> Persistent Storage is to allow users to choose what they want to save.
> So we need a level of granularity that makes sense to them.
>
> For example, we could have a mechanism around this database to advertise
> persistence of the “Mouse & mouse Touchpad Settings”, the “Desktop
> Background” (I’m not kidding!) or any subset of it that makes sense to
> our users. But I really don’t see us advertising a persistence of the
> whole thing.

Ok, thanks for your input. How about we add a configuration button for the “GNOME Settings” preset, like we have for the Additional Software one, which opens a window that allows selecting specific GNOME settings that get persisted?

That should be fairly simple to implement, because I wouldn’t have to add new features to tails-persistence-setup (which is written in Modern Perl, which I still fail to do even simple changes in), but could write a small standalone app that just writes the selected settings to a file like /live/persistence/TailsData_unlocked/dconf.conf, which we will then use as our whitelist for persisted settings during boot.

I will implement a prototype.

#15 Updated by intrigeri 2020-03-08 09:53:00

> We could replace the user-db:user line with service-db:keyfile/user in /etc/dconf/profile/user, then /home/amnesia/.config/dconf/user.txt will contain the config in plain text instead of binary, so it can be more easily inspected.

FWIW, https://anarc.at/blog/2020-03-02-moving-dconf-entries-to-git/ may have useful info on this topic.

#16 Updated by intrigeri 2020-03-08 10:02:44

> Ok, thanks for your input. How about we add a configuration button for the “GNOME Settings” preset, like we have for the Additional Software one, which opens a window that allows selecting specific GNOME settings that get persisted?
>
> That should be fairly simple to implement, because I wouldn’t have to add new features to tails-persistence-setup (which is written in Modern Perl, which I still fail to do even simple changes in), but could write a small standalone app that just writes the selected settings to a file like /live/persistence/TailsData_unlocked/dconf.conf, which we will then use as our whitelist for persisted settings during boot.

I don’t know if that’s ideal from a UX/design PoV, but as long as our persistence config tool is 100% Perl, it indeed seems much more straightforward to proceed like you’re saying, and it avoids me being in the critical path for maintenance, bugfixing, and future evolution of this extension. We could reconsider if/once we rewrite this — at least the frontend — into Python.

#17 Updated by sajolida 2020-03-11 23:31:29

Totally! This would also prevent overcrowding the Persistent Storage
settings.

I think that it would be a very good start (if it’s cheap). At some
point, I think that we should rewrite the Persistent Storage UI totally
and get rid of the Perl so that more people can work on it.

But we’re not there yet and by the time we get there, I hope that we’ll
have the resources to rethink the whole thing a bit more in depth.
Working on more features like the one you are describing here will help
us in doing so. Building them as prototypes is also great: it’ll start
being useful quickly and give me things to test when I’ll do usability
tests on getting started with Tails :)

@intrigeri: What do you think?

#23 Updated by intrigeri 2020-03-15 17:43:14

> I think that it would be a very good start (if it’s cheap).

+1

> At some point, I think that we should rewrite the Persistent Storage UI totally and get rid of the Perl so that more people can work on it.

For avoidance of doubt: I totally agree that it would make a lot of sense to migrate our Perl code bases to a language that folks either already know, or are happy to learn.

(
In passing, FWIW off-topic food for thought:
In many cases, Python is a great answer to that desire to migrate away from Perl, mostly thanks to its popularity.
In some cases, it could be that we want to preserve stricter typing across the migration than what Python offers, or to take benefit of the migration to decrease resources requirements further, in which case IMO we should at least consider Golang and Rust; aside of the “migrate away from Perl” thing, having a decent amount of literacy in one of these languages on our team would allow us to port more shell scripts into better languages, in situations where the memory footprint of a larger interpreter than /bin/sh is not OK.
)