Tails

#9 Updated by Anonymous 2018-04-12 12:16:43

Uploaded version 1.3 to Debian unstable today: https://tracker.debian.org/pkg/onionshare Tested it only in Debian and it works great. We should test this in Tails. Next week I’ll prepare a backport.

#10 Updated by emmapeel 2018-04-18 08:19:27

Not sure if OnionSHare is supposed to work on it, but while testing the ISO for the Additional Software I tried to use Onionshare and I could not use it.

There was a configuration to do with Tor, I configured it to use SocksPort 9050, but when saving the config, Onionshare crashes.

The ISO I was using was https://nightly.tails.boum.org/build_Tails_ISO_feature-14594-asp-gui/lastSuccessful/archive/

#11 Updated by Anonymous 2018-04-18 08:39:28

I’ve not yet tested that myself. I plan to work on this next week. Note that the problem you report might be due to Onionshare’s apparmor profile not allowing to save a tor configuration file. It might even be that this is work to be done by the foundations team.

#13 Updated by Anonymous 2018-04-24 15:54:18

Testing on Debian first:

- user needs to be in debian-tor group to use system tor instead of bundled tor

- torrc needs to have control port open / or we could use a socket file
- Config is written here: /home/user/.config/onionshare/onionshare.json

Also see https://github.com/micahflee/onionshare/wiki/Connecting-to-Tor

#15 Updated by Anonymous 2018-04-24 20:10:15

I have some more infos:

- I opened an issue upstream to update the AppArmor profiles https://github.com/micahflee/onionshare/issues/686

- I saw that in the Debian packaging they are not automatically installed. I made a commit to change this but I want to wait until we have working profiles before I upload a new version of Onionshare. See https://salsa.debian.org/pkg-privacy-team/onionshare/commit/db02265185dba5f80dcd42071f2fdf89c44ebdde

- In Tails, on top of permission problems/AppArmor, there are some problems talking to the control port:

Apr 24 20:06:32 amnesia onion-grater[2558]: /usr/bin/onionshare-gui (pid: 31235, user: amnesia, port: 35246, filter: onionshare): command filtered: GETCONF hiddenservicesinglehopmode
Apr 24 20:06:32 amnesia onion-grater[2558]: /usr/bin/onionshare-gui (pid: 31235, user: amnesia, port: 35246, filter: onionshare): command filtered: ADD_ONION NEW:RSA1024 Port=80,17623
Apr 24 20:06:35 amnesia onion-grater[2558]: /usr/bin/onionshare-gui (pid: 31235, user: amnesia, port: 35246, filter: onionshare): command filtered: DEL_ONION

I don’t know how to fix those. To reproduce: try sharing a file and click “Start server”.

#17 Updated by intrigeri 2018-04-27 08:09:05

> - I opened an issue upstream to update the AppArmor profiles
> - I saw that in the Debian packaging they are not automatically installed. I made
> a commit to change this but I want to wait until we have working profiles before
> I upload a new version of Onionshare.

I’m not sure it’s worth installing these profiles on Debian:

• On Debian we need all OnionShare usage modes (system and bundled Tor) to work so the corresponding AppArmor profile would need to be more permissive than what we want on Tails.
• The only reason why we added AppArmor confinement was for Onion Grater, which is not in Debian.

OTOH having these profiles included in the Debian package would simplify maintenance on our side (currently we ship them via tails.git). So I think we should ship them in the Debian package, but disabled by default (really disabled, not in complain mode).

> - In Tails, on top of permission problems/AppArmor, there are some problems talking to the control port:
> I don’t know how to fix those. To reproduce: try sharing a file and click “Start server”.

Probably we’ll need to edit config/chroot_local-includes/etc/onion-grater.d/onionshare.yml. We (Foundations Team) will do that in time for Tails 3.9 once the other blockers are gone. But no emergency, if we don’t manage to port our stuff in time, we could actually stick to OnionShare 0.9.x in Tails 3.x and upgrade to 1.3+ only in Tails 4.x :)

#36 Updated by Anonymous 2019-04-16 14:19:35

Quick tests

- Apparmor:
- {HOME}/.config/onionshare folder cannot be created. This folder will hold onionshare.json config file. ( this seems to have been addressed in 85d3f210804b69d675125b7d9e74753662009065. intrigeri: was this change upstreamed?)
- A non existing folder $HOME/OnionShare is the default location to store received files. This folder needs to be created, which is problematic WRT Apparmor. → @sajolida: This is also an issue for UX: do we want to have another top level folder in the home directory, that some people might not use? Or is this actually good, so people know about onionshare?

- Default Tor used is “built-in”, and needs to be manually changed to “connect using control port” which works.
This can be mitigated by shipping a working onionshare.json config file. I think this is work for FT. (And maybe 6cd5d82da30d4b7ce752d476eb054321cd3f63f8 / Bug #16306 addresses this, but I could not test it as I would need a nightly build for this.)

- version of tor does not support ephemeral onion services yet.
- there is an issue with /etc/machine-id (This was addressed by Bug #16306 but I could not test this modification.)

I tested this only with Tails 3.11 (because this was the only version I have available and I’m currently on a mobile network, so no way to download a more recent image).

I’ll do more testing once the network is back for me.

#37 Updated by intrigeri 2019-04-17 06:35:08

> - @{HOME}/.config/onionshare folder cannot be created. This folder will hold onionshare.json config file.
> ( this seems to have been addressed in 85d3f210804b69d675125b7d9e74753662009065. intrigeri: was this change upstreamed?)

No, it was not upstreamed: these changes were tested with the version that’s in Buster (1.3.2-1) and I had no idea if they would work with 2.0.

#38 Updated by Anonymous 2019-04-22 10:31:06

intrigeri wrote:
> > - @{HOME}/.config/onionshare folder cannot be created. This folder will hold onionshare.json config file.
> > ( this seems to have been addressed in 85d3f210804b69d675125b7d9e74753662009065. intrigeri: was this change upstreamed?)
>
> No, it was not upstreamed: these changes were tested with the version that’s in Buster (1.3.2-1) and I had no idea if they would work with 2.0.

I’ll look into it this week and will upstream the corresponding changes if necessary.

#39 Updated by Anonymous 2019-04-26 15:06:31

I’m now retesting in feature/buster using Onionshare 2.0-3 from Debian.

> - Apparmor:
> - {HOME}/.config/onionshare folder cannot be created. This folder will hold onionshare.json config file. > ( this seems to have been addressed in 85d3f210804b69d675125b7d9e74753662009065. intrigeri: was this change upstreamed?)

Looks good in Tails feature/buster.

TODO: upstream this part of the code.

> - A non existing folder $HOME/OnionShare is the default location to store received files. This folder needs to be created, which is problematic WRT Apparmor. → @sajolida: This is also an issue for UX: do we want to have another top level folder in the home directory, that some people might not use? Or is this actually good, so people know about onionshare?

TODO: Enable reading and writing to this folder in Apparmor files for Onionshare.
TODO: Ship this folder by default in Tails? → input needed from intrigeri and sajolida

> - Default Tor used is “built-in”, and needs to be manually changed to “connect using control port” which works.
> This can be mitigated by shipping a working onionshare.json config file. I think this is work for FT. (And maybe 6cd5d82da30d4b7ce752d476eb054321cd3f63f8 / Bug #16306 addresses this, but I could not test it as I would need a nightly build for this.)

Works fine in the nightly build.

> - version of tor does not support ephemeral onion services yet.

feature/buster comes with Tor 3.5.8.

> - there is an issue with /etc/machine-id (This was addressed by Bug #16306 but I could not test this modification.)

This is fixed in feature/buster.

However, Onionshare 2.0-3 does not work in Tails Buster. Whenever I want to share a file or receive a file I get the following error : “”ADD_ONION response didn’t have an OK status: command filtered“. This error comes from stem (”$":https://stem.torproject.org/_modules/stem/response/add_onion.html). Any idea what this is about? I think it happens when an OnionService needs to be set up. (OnionCircuits also had an issue with stem recently: Bug #16626.)

So if we want to provide OnionShare > 2.0 in Tails we need to fix this.

#40 Updated by intrigeri 2019-05-01 09:30:53

Meta: I’m happy to provide occasional guidance when needed here, but given this is about new features, not a blocker for 4.0 (Buster will ship with OnionShare 1.3), and thus not core work for now, I’ll take it easy.

> TODO: Ship this folder by default in Tails? → input needed from intrigeri and sajolida

I see no technical reason against shipping it. UX might be a different matter. I guess it’s somewhat related to the Tor Browser downloads directory ongoing discussion.

> However, Onionshare 2.0-3 does not work in Tails Buster. Whenever I want to share a file or receive a file I get the following error : “”ADD_ONION response didn’t have an OK status: command filtered“. This error comes from stem (”$":https://stem.torproject.org/_modules/stem/response/add_onion.html). Any idea what this is about? I think it happens when an OnionService needs to be set up. (OnionCircuits also had an issue with stem recently: Bug #16626.)

Likely config/chroot_local-includes/etc/onion-grater.d/onionshare.yml needs an update: either to allow the newly needed control verb/arguments, or to send back a dummy answer, or something.

#41 Updated by sajolida 2019-05-01 18:28:14

> @sajolida: This is also an issue for UX: do we want to have another top level folder in the home directory, that some people might not use? Or is this actually good, so people know about onionshare?

Do you know if there’s any reason why OnionShare decided to use a custom
“OnionShare” folder instead of using the generic “Downloads” folder?

If you don’t know I’ll have a quick look at the OnionShare tracker and
code. If there’s no good reason, I would propose to use “Downloads” in
Tails (and propose this upstream as well).

You said it’s the default location, but do I understand correctly and
people are prompted anyway about where to save the file before it is saved?

#42 Updated by Anonymous 2019-05-02 06:47:42

@sajolida: I don’t know why this folder is used. Users are not prompted, no, but when they open the settings they can choose a folder, or we can ship a config file that specifies a different folder (which is what we should probably do - and allow this folder in the AppArmor rules).

#43 Updated by sajolida 2019-05-03 11:53:01

The change is not explained in e980bc1 nor in the changelog and before it was using “Downloads”. I propose to use “Downloads” by default. In Feature #15463, I’m also suggesting to use “Downloads” for Tor Browser.

#44 Updated by Anonymous 2019-05-03 11:58:27

@sajolida:
> The change is not explained in e980bc1 nor in the changelog and before it was using “Downloads”. I propose to use “Downloads” by default. In Feature #15463, I’m also suggesting to use “Downloads” for Tor Browser.

I fully agree with this for Tails, and we can / should add this to the OnionShare config file that we ship as well as to the AppArmor profile.

#49 Updated by mig5 2019-08-30 06:48:15

> Do you know if there’s any reason why OnionShare decided to use a custom
“OnionShare” folder instead of using the generic “Downloads” folder?

Just wanted to let you know the reason, which is that in OnionShare 2, there is a new mode called ‘Receive Mode’, which lets people upload files to your OnionShare (as opposed to the original and default ‘Share Mode’ whereby you are sharing those files with OnionShare to users who download the files with Tor Browser).

For this reason, we don’t put those files in ‘Downloads’, because they are actually uploads, from remote users.

Note also that the files that get uploaded in Receive Mode, go into sub-folders of that folder, which use timestamped names.

#50 Updated by maqp 2019-09-01 18:17:29

> > However, Onionshare 2.0-3 does not work in Tails Buster. Whenever I want to share a file or receive a file I get the following error : “”ADD_ONION response didn’t have an OK status: command filtered“. This error comes from stem (”$":https://stem.torproject.org/_modules/stem/response/add_onion.html). Any idea what this is about? I think it happens when an OnionService needs to be set up. (OnionCircuits also had an issue with stem recently: Bug #16626.)
>
> Likely config/chroot_local-includes/etc/onion-grater.d/onionshare.yml needs an update: either to allow the newly needed control verb/arguments, or to send back a dummy answer, or something.

Hey guys,

While figuring out how to get TFC (a program I’m working on) to work with Tails, I had similar issues to Ricochet 2.0 with ADD_ONION being filtered. Here’s what I did to get a v3 Onion Service running for TFC on Tails 4.0~beta1.

I bypassed the the onion-grater filters by setting `disable_filtering` to True. I modified the onion-grater to record debug messages to a file, and in there was an entry

/opt/tfc/venv_relay/bin/python3.7 (pid: 6879, user: amnesia, port: 46096, filter: None): -> ADD_ONION ED25519-V3:YMPlHTv68oEculIujt2mRjTHQZOs24kUgkyA8JLWGUj2FZocjol99nlsAX2bntcKm+SK8OGapzieu5YlKQygsQ== Port=80,5000

The 88-char long Base64 string is the Onion Service’s private key in expanded form. Based on Tor’s testing code I some time ago wrote a helper function for OnionShare and TFC to generate expanded keys from raw 32-byte private keys:

https://github.com/micahflee/onionshare/issues/461#issuecomment-360971386

Based on the output in the debug file I created following content to tfc.yml file in /etc/onion-grater.d.

<code class="yaml">
---
- apparmor-profiles:
    - '/opt/tfc/venv_relay/bin/python3.7'
  users:
    - 'amnesia'
  commands:
    GETINFO:
      - 'version'
      - 'onions/current'
    ADD_ONION:
      - 'ED25519-V3:.*Port=80,5000'
    DEL_ONION:
      - '.+'
    GETCONF:
      - 'hiddenservicesinglehopmode'
  confs:
    __owningcontrollerprocess:
  events:
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    HS_DESC:
    STATUS_SERVER:
</code>

I replaced the expanded key with a wildcard (.*). Perhaps some kind of regex for Base64 would be more safe? Running the helper function multiple times shows the expanded private key is always the same length.

Finally, I have to say I’m diving into the deep end of the pool here so please let me know if I’m doing something terribly stupid. I’m not sure what the port numbers have to be for OnionShare, but hopefully there’s something useful here for getting OnionShare 2 to work.

#51 Updated by intrigeri 2019-09-01 18:25:04

> While figuring out how to get TFC (a program I’m working on) to work with Tails, I had similar issues to Ricochet 2.0 with ADD_ONION being filtered. Here’s what I did to get a v3 Onion Service running for TFC on Tails 4.0~beta1.

> I bypassed the the onion-grater filters by setting `disable_filtering` to True. I modified the onion-grater to record debug messages to a file, and in there was an entry

> /opt/tfc/venv_relay/bin/python3.7 (pid: 6879, user: amnesia, port: 46096, filter: None): -> ADD_ONION ED25519-V3:YMPlHTv68oEculIujt2mRjTHQZOs24kUgkyA8JLWGUj2FZocjol99nlsAX2bntcKm+SK8OGapzieu5YlKQygsQ== Port=80,5000

Indeed, that’s quite different from what we have for ADD_ONION in config/chroot_local-includes/etc/onion-grater.d/onionshare.yml. Thanks a lot!
I expect these debugging instructions will help us when we work on this ticket :)

#54 Updated by maqp 2019-09-03 23:03:20

Hey again everyone!

I had a moment today to look into OnionShare2.1 and the onion-grater permissions.

On Tails 4.0~beta1 I first enabled administration password, and then ran

$ sudo apt update
$ sudo apt install -y python3-flask python3-stem python3-pyqt5 python3-crypto python3-socks python-nautilus obfs4proxy python3-pytest build-essential fakeroot python3-all python3-stdeb dh-python python3-pip
$ git clone https://github.com/micahflee/onionshare.git $HOME/onionshare/
$ cd $HOME/onionshare/install/
$ torsocks python3.7 -m pip install -r requirements.txt
$ cd $HOME/onionshare/dev_scripts/

Next I figured out the onion-grater rules.

Before launching OnionShare2.1 I edited /usr/local/lib/onion-grater by setting disable_filtering to True, and then added some code that stores debug messages to a file.

I then restarted the service with

$ sudo service onion-grater restart

I then observed logged messages while altering the three main settings of OnionShare:

-Use a persistent address (does not use NEW:)
-Use legacy addresses (uses RSA1024 instead of ED25519-V3)
-Use client authorization (apparently adds Flags=BasicAuth and ClientAuth=onionshare)

Based on that I determined following rule set (whitespace added for clarity):

      - 'NEW:BEST       Flags=BasicAuth Port=1,1 ClientAuth=onionshare'

      - 'NEW:RSA1024                    Port=80,176([0-4][0-9]|50)'
      - 'NEW:RSA1024                    Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'
      - 'NEW:RSA1024    Flags=BasicAuth Port=80,176([0-4][0-9]|50)*.'
      - 'NEW:RSA1024    Flags=BasicAuth Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'

      - 'NEW:ED25519-V3                 Port=80,176([0-4][0-9]|50)'
      - 'NEW:ED25519-V3                 Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'
      - 'NEW:ED25519-V3 Flags=BasicAuth Port=80,176([0-4][0-9]|50)*.'
      - 'NEW:ED25519-V3 Flags=BasicAuth Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'

      - 'RSA1024:.*                     Port=80,176([0-4][0-9]|50)'
      - 'RSA1024:.*                     Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'
      - 'RSA1024:.*     Flags=BasicAuth Port=80,176([0-4][0-9]|50)'
      - 'RSA1024:.*     Flags=BasicAuth Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'

      - 'ED25519-V3:.*                  Port=80,176([0-4][0-9]|50)'
      - 'ED25519-V3:.*                  Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'
      - 'ED25519-V3:.*  Flags=BasicAuth Port=80,176([0-4][0-9]|50)'
      - 'ED25519-V3:.*  Flags=BasicAuth Port=80,176([0-4][0-9]|50) ClientAuth=onionshare'

Now, this list is slightly forgiving in the sense OnionShare 2.1 does not support ClientAuth with Ed25519 keys.
The program already prevents the setting combination, so adding regex check for it adds IMO unnecessary complexity.

Combining these rules we get

(?:NEW\:RSA1024
  |NEW\:ED25519-V3
  |RSA1024\:
  |ED25519-V3\:)
(?:| Flags\=BasicAuth)
 Port\=80,176([0-4][0-9]|50)
(?:| ClientAuth\=onionshare)$

or as one-liner

(?:NEW\:RSA1024|NEW\:ED25519-V3|RSA1024\:|ED25519-V3\:)(?:| Flags\=BasicAuth) Port\=80,176([0-4][0-9]|50)(?:| ClientAuth\=onionshare)$

This doesn’t work however because the wildcards were left out. This is because we add Base64 detection (from https://stackoverflow.com/a/475217):

(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?

Plugging that in we get

(?:NEW\:RSA1024
|NEW\:ED25519-V3
|RSA1024\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?
|ED25519-V3\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)
(?:| Flags\=BasicAuth)
 Port\=80,176([0-4][0-9]|50)
(?:| ClientAuth\=onionshare)$

or as one-liner

(?:NEW\:RSA1024|NEW\:ED25519-V3|RSA1024\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?|ED25519-V3\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)(?:| Flags\=BasicAuth) Port\=80,176([0-4][0-9]|50)(?:| ClientAuth\=onionshare)$

Next we run

$ sudo gedit /etc/onion-grater.d/onionshare2.yml

and add following content:

<code class="yaml">

---
- apparmor-profiles:
    - '/usr/bin/python3.7'
  users:
    - 'amnesia'
  commands:
    GETINFO:
      - 'version'
      - 'onions/current'
    ADD_ONION:
      - 'NEW:BEST Flags=BasicAuth Port=1,1 ClientAuth=onionshare'
      - '(?:NEW\:RSA1024|NEW\:ED25519-V3|RSA1024\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?|ED25519-V3\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)(?:| Flags\=BasicAuth) Port\=80,176([0-4][0-9]|50)(?:| ClientAuth\=onionshare)$'
    DEL_ONION:
      - '.+'
    GETCONF:
      - 'hiddenservicesinglehopmode'
  confs:
    __owningcontrollerprocess:
  events:
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    HS_DESC:
    STATUS_SERVER:

    DEL_ONION:
      - '.+'
    GETCONF:
      - 'hiddenservicesinglehopmode'
  confs:
    __owningcontrollerprocess:
  events:
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    HS_DESC:
    STATUS_SERVER:
</code>

Finally, we can launch OnionShare2 with

$ ./onionshare-gui

I checked each possible permutation of the three OnionShare settings, each one is able to bring the Onion Service online.

(Note that the /usr/bin/python3.7 is correct for this git repository installation but it needs to be changed for the release!)

Again, I hope this will make it less troublesome for you to integrate OnionShare 2.1 into Tails 4 :)

-Markus

#55 Updated by maqp 2019-09-03 23:44:25

I started thinking it might be useful to split the ADD_ONIONs for v2 and v3 services in case v2s are deprecated at some point:

<code class="yaml">
---
- apparmor-profiles:
    - '/usr/bin/python3.7'
  users:
    - 'amnesia'
  commands:
    GETINFO:
      - 'version'
      - 'onions/current'
    ADD_ONION:
      - 'NEW:BEST Flags=BasicAuth Port=1,1 ClientAuth=onionshare'
      - (?:NEW\:RSA1024|RSA1024\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)(?:| Flags\=BasicAuth) Port\=80,176([0-4][0-9]|50)(?:| ClientAuth\=onionshare)$
      - (?:NEW\:ED25519-V3|ED25519-V3\:(?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)(?:| Flags\=BasicAuth) Port\=80,176([0-4][0-9]|50)(?:| ClientAuth\=onionshare)$
    DEL_ONION:
      - '.+'
    GETCONF:
      - 'hiddenservicesinglehopmode'
  confs:
    __owningcontrollerprocess:
  events:
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    HS_DESC:
    STATUS_SERVER:

    DEL_ONION:
      - '.+'
    GETCONF:
      - 'hiddenservicesinglehopmode'
  confs:
    __owningcontrollerprocess:
  events:
    SIGNAL:
      suppress: true
    CONF_CHANGED:
      suppress: true
    HS_DESC:
    STATUS_SERVER:


</code>

#56 Updated by maqp 2019-09-03 23:48:20

Aaaand I forgot to add the quotes for the rules. I wish one could edit their posts here.

#60 Updated by maqp 2019-10-15 18:43:29

intrigeri wrote:
> This will become FT work for 5.0, because Debian Bullseye will include OnionShare 2.x. If someone finds time to do it earlier, for example in order to fix Bug #16913, or for some other reason, great!

Wrt Bug #16913 (“Hide Tor settings in OnionShare”), OnionShare 2.2 has added this as a feature:

>Hide the Tor connection settings if the ONIONSHARE_HIDE_TOR_SETTINGS environment variable is set (Tails compatibility)

Source https://github.com/micahflee/onionshare/releases/tag/v2.2