Bug #9412
Add SOCKS support to WhisperBack
100%
Description
We patch WhisperBack’s .desktop
file to start it with torsocks, and all I see is a long stream of:
WARNING torsocks[3823]: [syscall] Unsupported syscall number 240. Denying the call (in tsocks_syscall() at syscall.c:165)
… on stderr, and the WhisperBack GUI never appears. WhisperBack starts fine without torsocks, though.
Files
Subtasks
History
#1 Updated by intrigeri 2015-05-16 10:13:34
- related to
Bug #7755: Migrate WhisperBack to Python 3 added
#2 Updated by intrigeri 2015-05-16 10:16:29
Perhaps the right way to fix that would be to give WhisperBack native SOCKS5 support?
#3 Updated by alant 2015-06-15 10:43:35
- Subject changed from WhisperBack doesn't start under torsocks on Jessie to Add socks support to WhisperBack
intrigeri wrote:
> Perhaps the right way to fix that would be to give WhisperBack native SOCKS5 support?
That looks quite easy to do, so let’s do this way!
#4 Updated by intrigeri 2015-06-16 00:54:07
> That looks quite easy to do, so let’s do this way!
Woohoo :)
#5 Updated by intrigeri 2015-07-08 16:40:34
- Subject changed from Add socks support to WhisperBack to Add SOCKS support to WhisperBack
#6 Updated by intrigeri 2015-08-02 08:51:30
- Due date set to 2015-07-15
#8 Updated by alant 2015-08-03 16:11:37
> What’s your ETA on this one?
I wasn’t aware of a deadline but I can very well do that before october 15th. Can you take care of the packaging?
#9 Updated by intrigeri 2015-08-04 02:35:44
> I can very well do that before october 15th.
Excellent. Even better if it’s done earlier, e.g. by the end of August or the end of September.
> Can you take care of the packaging?
Sure!
#10 Updated by romeopapa 2015-08-10 05:26:19
Hi alant,
I was looking for simple things to do on python with jessie and intregeri said I might give you a hand on this high priority task.
I’ve based my work on the feature/python3 branch.
I’ve attached a very basic patch that should implement native SOCKS support, but I’m unable to test it as apparently it seems to fail upstream (see https://labs.riseup.net/code/issues/7755#note-38 )
Take care
#11 Updated by romeopapa 2015-08-11 08:11:48
Here’s the updated patch, with which I’ve been able to successfully send a message earlier today.
I’ve also published the branch on gitlab, on a clone of whisperback’s repo. Maybe it’s more convenient for you: https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
Take care,
#12 Updated by romeopapa 2015-08-11 08:18:44
To note, i was also going to supply a patch for tails/feature/jessie but intrigeri’s commit 72647c7b2c674c7488a32d8942562d1d692b9d94 (Replace patches that wrapped apps with torsocks with dynamic patching with a hook.) has made that unnecessary.
#13 Updated by intrigeri 2015-08-11 09:00:40
- QA Check set to Dev Needed
> I’ve also published the branch on gitlab, on a clone of whisperback’s repo.
> Maybe it’s more convenient for you:
Sure it is!
Before we review it in depth, may you please make the SOCKS port configurable? E.g. in Tails we’re going to use something else than 9050. If you’re not interested in that task, just say so :)
#14 Updated by romeopapa 2015-08-11 15:31:42
Sure, I can make the SOCKS posrt configurable.
> E.g. in Tails we’re going to use something else than 9050
- Should it be a constant that could be overloaded (in order of priority) by: a command line argument and an environmnent variable and a config file somewhere?
- or were you thinking about a GUI element?
#15 Updated by romeopapa 2015-08-11 17:25:36
>
OK, going for environment variables WHISPERBACK_SOCKS_HOST and WHISPERBACK_SOCKS_PORT.
#16 Updated by romeopapa 2015-08-11 21:29:25
romeopapa wrote:
> >
>
Actually I’ve seen the way the config.py file works so I’ve also submitted a patch to tails (feature/jessie
) so that the default values for SOCKS will be in there. Here’s the commit on my gitlab fork:
https://gitlab.com/romeopapa/tails/commits/feature/jessie-9412-add-socks-support-to-whisperback
You will find attached 0002-The-SOCKS-host-and-port-can-be-configured-in-whisper.patch, or the whole series of commits to whisperback at my gitlab fork (ready to be merged into feature/python3
)
https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
Some notes:
- I started with the environment variables and a system that would warn you in comprehensive terms if you weren’t using the tail’s hardcoded defaults (because I saw a wrongly configured socks proxy as a potential anonymity leak - but then I realised that so would be a misconfigured smtp server, etc..).
- I’ve added some data validation. It tries to verify that it is a correct hostname (follows RFC 1123) or a correct IPv4 address (pysocks is not compatible with ipv6). I could finish and ‘validate’ the rest of the parameters loaded by config.py
#17 Updated by romeopapa 2015-08-11 21:36:54
Sorry it’s probably not clear from my previous post, but I did not use any environment variables in the end.
Attached, one last patch (pushed on gitlab) that removes an unneeded import to ‘os’ I had just added.
#18 Updated by intrigeri 2015-08-12 02:01:27
> Sure, I can make the SOCKS posrt configurable.
:)
>> E.g. in Tails we’re going to use something else than 9050
> * Should it be a constant that could be overloaded (in order of priority) by: a command line argument and an environmnent variable and a config file somewhere?
See the “CONFIGURATION” section in README
, in order to make this new setting consistent with how the rest of WhisperBack is configured :)
> * or were you thinking about a GUI element?
No.
#19 Updated by romeopapa 2015-08-12 12:48:06
> See the “CONFIGURATION” section in README, in order to make this new setting consistent with how the rest of WhisperBack is configured :)
Should be good, followed the implementation. I believe task is complete.
#20 Updated by romeopapa 2015-08-12 15:29:52
Here’s another additional patch, that repacks the validation code of hostname/ipaddress into functions, and now validates the SMTP host and port as well.
The rest of the inputs are not verified.
Patch 0004 attached, and submitted to gitlab at:
https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
#21 Updated by BitingBird 2015-08-13 02:50:32
- Status changed from Confirmed to In Progress
- % Done changed from 0 to 30
- QA Check changed from Dev Needed to Ready for QA
#22 Updated by alant 2015-08-17 05:53:50
Hi romeopapa
Thanks a lot for your patches! I tested them and they work perfectly. Find below a few comments before I merge them.
- commit 6d195d8:
- merge 660a2c4 (do not import
os
)
- merge 660a2c4 (do not import
- commit a8cfaea: please rewrite it to:
- not
import os
- add default values for
socks_host
andsocks_port
to the definition ofsend_message_tls
inmail.py
- do not fix whitespace errors in unrelated commit (
# Load the python configuration file "config.py" from different locations
) - do not move
# XXX: Add sanity checks
because it still applies to the old lines validateHostnameOrIPv4
: see next commit. It would be even more clean if you added all the sanity checks in commit 8257943- please add the parameters you add to the “SMTP” section of
doc/config.py.sample
- fixup for docstring
It doesn't take any argument.
should be in a separate commit in another branch (or be dropped).
- not
- commit 660a2c4
- meld into 6d195d8
- commit 8257943
param subject The topic of the feedback
;def execute_threaded(self, func, args, progress_callback=None,
and next changes;param progress_callback
: do not fix whitespace errors in unrelated commit- put
validateHostnameOrIPv4
andvalidatePortValue
inutils.py
, where other validation functions are, and rename themvalidate_hostname_or_ipv4
andvalidate_port
(according to project coding style ane PEP8).
Congrats again for your contribution!
#23 Updated by romeopapa 2015-08-18 02:46:04
alant, thanks for the review
Having fun rebasing with git, will post cleaned up patches when I’m done.
#24 Updated by intrigeri 2015-08-18 04:48:09
- QA Check changed from Ready for QA to Dev Needed
#25 Updated by romeopapa 2015-08-18 05:41:12
- File cleaned-0001-Monkeypatching-the-entire-connection-through-the-SOC.patch added
- File cleaned-0002-The-SOCKS-host-and-port-can-be-configured-in-whisper.patch added
- File cleaned-0003-Validates-hosts-and-ports-for-SOCKS-and-SMTP.patch added
There we go.
You might want to have a look at the validate_port and validate_hostname_or_ipv4 functions. They return True/False (like the others in utils.py) but I don’t use thos values when calling the functions.
#26 Updated by intrigeri 2015-08-18 05:50:41
- QA Check changed from Dev Needed to Ready for QA
#27 Updated by alant 2015-08-18 10:14:20
romeopapa wrote:
> There we go.
>
Great! I still have a few issues though.
The change in debian/control
in 0b4d804 should be in 229e894 which is the commit adding the dependancy (or in a specific commit if you prefern but definately not in “The SOCKS host and port can be configured in whisperback’s config.py”
> You might want to have a look at the validate_port and validate_hostname_or_ipv4 functions. They return True/False (like the others in utils.py) but I don’t use thos values when calling the functions.
Your current proposal doesn’t make sense, because the return False
statement cannot be reached.
Also, you call the validation functions with the wrong numbers of arguments:
<code class="python">
whisperBack.utils.validate_hostname_or_ipv4(self.smtp_host, "smtp_host")
</code>
<code class="python">
def validate_hostname_or_ipv4(self, hostname_or_ipv4, valuename):
</code>
whisperBack.utils
is a module, not an object, so there is nothing to reference as self
. You are writing pure functions here, not object methods (that’s actually why they are best in utils.py
).
Finally, I’m not sure of the interest of passing valuename
here (see below).
I think there is a choice that needs to be made here:
- either you use
validate_
functions that raise exceptions, as in your 1st patch, and don’t set a return value - or you use
is_valid_
functions that return a boolean, as the others inutil.py
do, and then raise awhisperBack.exceptions.MisconfigurationException
in__check_conf
, e.g.:
<code class="python">
if not self.socks_host:
raise whisperBack.exceptions.MisconfigurationException('socks_host')
elif not whisperBack.utils.is_valid_hostname_or_ipv4(self.socks_host):
raise ValueError("socks_host is not a correct hostname or IPv4 address")
</code>
I tend to perfer the 2nd proposal which seems me more coherent with the current code, at the price of having less clear description of the cause of the lack of correctness.
To end with, no need to create patches and attach them here, pushing to your git repository is enough (for others that might want to follow the discussion, you can add an URL to its web interface).
Thanks by advance for your patience.
#28 Updated by romeopapa 2015-08-18 17:40:47
Firstly, I might have rushed things a little. Maybe if I actually tried to run the code I wrote, there’d be less things to change. Sorry about that. Secondly, thanks for putting things so eloquently and taking the time to walk me through all of this, I’m learning some good things!
> there is a choice that needs to be made here:
> * either you use validate_ functions that raise exceptions, as in your 1st patch, and don’t set a return value
> * or you use is_valid_ functions that return a boolean, as the others in util.py do, and then raise a whisperBack.exceptions.MisconfigurationException in __check_conf
>
> I tend to perfer the 2nd proposal which seems me more coherent with the current code, at the price of having less clear description of the cause of the lack of correctness.
Going with #2.
OK, I’ve repushed to https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
All the clean and fixed commits are in there. I have actually tested the code this time ;)
#29 Updated by alant 2015-08-20 04:51:07
- QA Check changed from Ready for QA to Dev Needed
romeopapa wrote:
> All the clean and fixed commits are in there. I have actually tested the code this time ;)
Great! One comment on the last commit:
<code>
try:
int(candidate)
except:
return False
</code>
Don’t use a generic except but only catch the expected exception; in our case except ValueError:
(https://www.python.org/dev/peps/pep-0008/#programming-recommendations).
#30 Updated by romeopapa 2015-08-20 05:37:30
> Don’t use a generic except but only catch the expected exception; in our case except ValueError: (https://www.python.org/dev/peps/pep-0008/#programming-recommendations).
The excerpt in question: “A bare except:
clause will catch SystemExit
and KeyboardInterrupt
exceptions, making it harder to interrupt a program with Control-C
, and can disguise other problems.”
I’ve ammended the last commit and repushed to https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
#31 Updated by intrigeri 2015-08-22 05:27:01
- QA Check changed from Dev Needed to Ready for QA
> I’ve ammended the last commit and repushed to
> https://gitlab.com/romeopapa/whisperback/commits/feature/python3-9412-add-socks-support-to-whisperback
Cool! Please update the “QA Check” field accordingly next time: https://tails.boum.org/contribute/merge_policy/#submit
#32 Updated by romeopapa 2015-08-22 08:49:30
intrigeri wrote:
> Cool! Please update the “QA Check” field accordingly next time: https://tails.boum.org/contribute/merge_policy/#submit
Hi intrigeri.. In case this message was directed towards me: I don’t think my user romeopapa on the redmine has the right to do that.. I think I can only add notes (and attachments) to a ticket, not much more (or perhaps I’m not allowed because I haven’t created THIS ticket).
Cheers!
#33 Updated by sajolida 2015-08-23 01:40:35
romeopapa: I granted you the “Contributor” role now. You should be able to do more stuff.
#34 Updated by alant 2015-08-23 07:15:20
- Status changed from In Progress to Fix committed
- Assignee deleted (
alant) - % Done changed from 30 to 100
- QA Check changed from Ready for QA to Pass
- Feature Branch set to whisperback:feature/9412-add-socks-support-to-whisperback
I merged your branch to whisperback’s feature/jessie
branch. Thanks for you contribution to Tails!
#35 Updated by romeopapa 2015-08-24 08:38:27
- Status changed from Fix committed to In Progress
- Assignee set to alant
- % Done changed from 100 to 80
- QA Check changed from Pass to Ready for QA
sajolida wrote:
> romeopapa: I granted you the “Contributor” role now. You should be able to do more stuff.
Thanks sajolida.
I’ve reopened the bug, there’s one last commit that must be merged. It is for Tails repository:
Default values in WhisperBack’s config file.
Here’s the commit (ba8f3ffe) in question.
Thanks,
PS: Sorry if I messed up the ticket’s modifications.. Reassigning to alant because he was in charge before.
#36 Updated by intrigeri 2015-09-01 11:00:42
- Deliverable for set to 266
#38 Updated by alant 2015-09-11 14:24:58
- QA Check deleted (
Ready for QA)
It should be ready in feature/jessie, but I’ve got strange build errors with the APT repository, so I can’t test yet.
#39 Updated by intrigeri 2015-09-14 08:52:44
- QA Check set to Dev Needed
> It should be ready in feature/jessie,
I see that commit:11c68aa introduces socks_port = 9500
, and I don’t understand how it can work, nor why that number was chosen.
I think we should use the SOCKS port our design doc specifies for Tails-specific applications: https://tails.boum.org/contribute/design/stream_isolation/ (which will need an update as part of merging this branch, by the way, once we use the right port).
> but I’ve got strange build errors with the APT repository, so I can’t test yet.
Replied separately, but tl;dr: feature/jessie
builds fine for me here, and the problem seems to be in your local build environment.
#40 Updated by alant 2015-09-14 13:09:10
Applied in changeset commit:c85411bb7f6a749a5c7607cb3309600428542f75.
#41 Updated by alant 2015-09-14 14:03:28
Applied in changeset commit:a25da197d39225a44e3c5cbd1855a9c9789b7023.
#42 Updated by intrigeri 2015-09-16 23:59:40
Looks good, thanks! Sent a report with that WhisperBack and asked frontdesk to forward it to me.
Two small notes, though:
- I find it sad that the versioning scheme for Jessie-specific releases was suddenly changed with no obvious reason, but well — I’ll try to remember it and not to mess up by assuming it’s still like it used to be.
- Pushed a commit on top to remove trailing whitespace introduced by the branch you merged — please make sure your Git prevents you from merging such stuff in the future.
#43 Updated by intrigeri 2015-09-17 13:21:20
- Status changed from In Progress to Resolved
- % Done changed from 80 to 100
- QA Check changed from Dev Needed to Pass
Frontdesk got the report, yay!