Feature #6623

Have wget directly use the Tor SOCKS proxy

Added by sajolida 2014-01-27 06:02:08 . Updated 2015-02-24 22:51:06 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Target version:
Tails_1.3
Start date:
2014-01-27
Due date:
% Done:

80%

Feature Branch:
feature/6623-wget-socks
Type of work:
Code
Blueprint:

Starter:
1
Affected tool:
Deliverable for:

Description

We should add a wget wrapper, that runs it with torsocks instead of using the HTTP proxy. See comment 2 for implementation ideas.

Subtasks

Related issues

Related to Tails - Bug #8603: Unexpected redirection while sending Host header with wget Resolved 2015-01-07

History

#1 Updated by sajolida 2014-01-27 06:02:52

Issue has been updated by winterfairy.

At least in the past I had some problem with getting wget to use the socks proxy. Whatever I tried I only got it to work through polipo, which has a download limit (either bug or feature) of 50 MB.

Instead I use curl with resume support as follows (where $URL is the URL to download):

curl C -O —socks5-hostname 127.0.0.1:9050 $URL

If wget is still broken in this regard (I haven’t tried), maybe creating a “torget” that does the curl command above is a better solution?

#2 Updated by sajolida 2014-01-27 06:03:30

Issue #6600 has been updated by intrigeri.

> At least in the past I had some problem with getting wget to use the socks proxy.
> Whatever I tried I only got it to work through polipo, which has a download limit
> (either bug or feature) of 50 MB.

This makes wget connect to the Tor SOCKS proxy instead of Polipo:

  1. Comment out the http_proxy and https_proxy options in /etc/wgetrc (note that we add these options ourselves at build time)
  2. Run http_proxy= HTTP_PROXY= https_proxy= HTTPS_PROXY= torsocks wget URL

So I guess we could very well drop our wgetrc patching, and add a wget
wrapper that does the above.

This probably breaks downloading stuff from the LAN, but I’m not sure
if our current configuration allows it anyway.

#3 Updated by intrigeri 2014-02-17 17:23:41

Tails 0.22.1: wget can fetch files from a FTP server on the LAN, but not from a HTTP server on the LAN. In other words, the wget wrapper idea is doable.

#4 Updated by intrigeri 2014-02-17 17:26:09

  • Tracker changed from Bug to Feature
  • Subject changed from Document the possibility to use wget in Tails to Have wget use the SOCKS proxy
  • Status changed from New to Confirmed
  • Type of work changed from Documentation to Code

#5 Updated by intrigeri 2014-02-17 17:27:07

  • Priority changed from Low to Normal

#6 Updated by intrigeri 2014-06-21 13:49:47

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

#7 Updated by intrigeri 2014-06-21 13:51:00

  • Description updated
  • Starter changed from No to Yes

#8 Updated by intrigeri 2014-06-21 13:51:51

  • Target version set to Sustainability_M1
  • Parent task set to Feature #5379

Flagging for 2.0, as it’s a subtask of Feature #5379 that is itself a 2.0 goal.

#9 Updated by BitingBird 2014-06-21 14:48:23

  • Target version changed from Sustainability_M1 to Hardening_M1

Right, except it’s 3.0 :)

#10 Updated by intrigeri 2014-11-02 00:23:31

  • Subject changed from Have wget use the SOCKS proxy to Have wget directly use the Tor SOCKS proxy

#11 Updated by intrigeri 2014-11-02 15:55:23

  • Assignee set to intrigeri
  • Feature Branch set to feature/6623-wget-socks

#12 Updated by intrigeri 2014-11-02 16:25:30

  • Assignee deleted (intrigeri)
  • Target version changed from Hardening_M1 to Tails_1.3
  • % Done changed from 10 to 50
  • QA Check set to Ready for QA

#13 Updated by sajolida 2014-12-12 19:47:34

In Tails 1.2.1, to have wget work fine with torsocks you can do:

torsocks wget --no-proxy URL

#14 Updated by alant 2014-12-23 16:36:18

  • Assignee set to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

/usr/local/bin/wget includes

unset http_proxy
unset HTTP_PROXY
unset https_proxy
unset HTTPS_PROX

while these variables are not set anymore (see 32897af7). Why? Please provide a reason or remove these lines.

#15 Updated by intrigeri 2014-12-24 12:09:06

  • Assignee changed from intrigeri to alant
  • % Done changed from 50 to 80
  • QA Check changed from Dev Needed to Ready for QA

alant wrote:
> /usr/local/bin/wget includes
>
> […]
>
> while these variables are not set anymore (see 32897af7). Why? Please provide a reason or remove these lines.

The following command-line depends on the fact that these environment variables are not set. Keeping these unset makes this clear, and is a potentially useful safety net IMO, e.g. in case:

  • one manually sets one of these envvars for some reason;
  • our build system or test suite or something sets one of these envvars, and uses this wget wrapper.

#16 Updated by alant 2014-12-24 15:59:01

> The following command-line depends on the fact that these environment variables are not set. Keeping these unset makes this clear, and is a potentially useful safety net IMO, e.g. in case:
>
> * one manually sets one of these envvars for some reason;
> * our build system or test suite or something sets one of these envvars, and uses this wget wrapper.

OK.

#17 Updated by alant 2014-12-25 16:49:00

  • Status changed from In Progress to Fix committed
  • Assignee deleted (alant)
  • QA Check changed from Ready for QA to Pass

#18 Updated by sajolida 2015-01-31 11:02:41

  • related to Bug #8603: Unexpected redirection while sending Host header with wget added

#19 Updated by BitingBird 2015-02-24 22:51:06

  • Status changed from Fix committed to Resolved