Bug #9713

Upgrade Electrum to >= 2.5.x for compatibility with all Electrum servers

Added by s7r 2015-07-08 17:15:16 . Updated 2015-12-16 11:36:04 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Target version:
Start date:
2015-07-08
Due date:
% Done:

100%

Feature Branch:
bugfix/9713-electrum-2.5
Type of work:
Code
Blueprint:

Starter:
1
Affected tool:
Deliverable for:

Description

Currently Tails comes with Electrum wallet 1.9.8. It is a very nice app to have.

I recommend upgrading it to 2.3.x since there are significant improvements and bug fixes, including important security fixes. The policy in Debian stable is not to include new version of packages, I know, but regarding Electrum this should not apply. The current version used in Tails is old, there was a change in Bitcoin’s blockchain (switched to v3 blocks) which 2.3.x knows about. The electrum client <-> electrum server communication is also improved in 2.3.x


Files


Subtasks


Related issues

Related to Tails - Bug #10754: Install Electrum 0.2.5.x in Tails/Jessie Resolved 2015-12-15
Related to Tails - Bug #10758: Update electrum.feature for Electrum 2.5.x Resolved 2015-12-15
Has duplicate Tails - Bug #10073: update electrum package Duplicate 2015-08-22
Blocks Tails - Bug #10475: Scenario "Using a persistent Electrum configuration" is fragile In Progress 2015-11-04

History

#1 Updated by acaster 2015-07-08 18:32:11

Hi,

Also there is a CoinJoin plugin for Electrum, right? Maybe it could be included if not there already.

Thanks.

#2 Updated by BitingBird 2015-07-09 05:03:36

  • Status changed from New to Rejected
  • Priority changed from Elevated to Normal

If someone packages the 2.3 version for the backports, it will be updated in Tails. Otherwise not, and we don’t have time to dedicate to that.

Please reopen when 2.3 is packaged :)

#3 Updated by s7r 2015-08-22 16:58:37

Ok, bug reported in Debian and fixed:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792231

We should upgrade in the next releases (wheezy/jessie).

#4 Updated by sajolida 2015-08-23 01:48:13

  • Status changed from Rejected to Confirmed
  • Starter set to Yes

Cool, 2.4.2 is now in unstable according to https://tracker.debian.org/pkg/electrum.

Next step is to try installing electrum 2.4.2 from unstable in Tails Wheezy, any volunteer? If it doesn’t work we will most likely have to wait until Tails Jessie (2.0).

#5 Updated by sajolida 2015-08-23 01:49:09

  • has duplicate Bug #10073: update electrum package added

#6 Updated by sajolida 2015-08-23 01:49:50

  • Subject changed from Upgrade Electrum bitcoin wallet to 2.3.x for compatibility with all Electrum servers to Upgrade Electrum to >= 2.3.x for compatibility with all Electrum servers

#7 Updated by intrigeri 2015-08-25 00:19:38

  • Assignee set to anonym

anonym, next steps are thus:

  • try installing Electrum from sid on Tails/Wheezy; s7r, maybe?
  • if it works as-is: update our APT pinning
  • if it doesn’t work as-is:
    • wait for 2.3+ to reach Debian testing
    • ask the person who uploaded it to wheezy-backports if they want to upload it to jessie-backports, and then to wheezy-backports-sloppy

#8 Updated by intrigeri 2015-08-25 00:19:56

  • Type of work changed from Code to Test

#9 Updated by s7r 2015-08-25 08:02:46

Hello,
Tried to install it on Tails/Wheezy (1.4.1).

Edited /etc/apt/preferences file and pinned electrum package to unstable. Run an apt-get update and apt-get install electrum and it said:
electrum 2.4.4-1 depends on python:any
python-electrum
Package 1.9.8 is going to be installed (the one available in wheezy backports).

Pinned python-electrum package to unstable as well in /etc/apt/preferences. - Same result.
Permanently deleted python-electrum package entry from /etc/apt/preferences. - Same result.

Maybe I am doing something wrong here?

#10 Updated by anonym 2015-08-26 02:37:40

  • Type of work changed from Test to Wait

Just looking at the dependencies of Sid’s python-electrum package tells me it cannot be installed on Wheezy straight away. It has: python:any (<< 2.8), python:any (>= 2.7.5-5~), but only 2.7.3-4+deb7u1 is available on Wheezy. Any way, I forcefully installed it like this (I think):

apt-get remove electrum python-electrum
apt-get install python-dnspython python-protobuf python-qrcode libprotobuf7 python-pbkdf2/sid python-requests/wheezy-backports python-urllib3/wheezy-backports
apt-get download electrum/sid python-electrum/sid
dpkg -i --force-depends electrum_*.deb python-electrum_*.deb


It seems to work, so I guess we need wheezy-backports-sloppy backports of electrum and python-electrum that just lower the dependency of python to >= 2.7.3-4. And we need python-pbkdf2 backported to Wheezy or we just install it from Sid if we’re ok with that.

So, let’s first wait until electrum 2.3+ reaches Debian testing.

#11 Updated by s7r 2015-09-07 00:22:03

Hi,

Looks like it’s in testing now:
https://tracker.debian.org/news/709815

#12 Updated by anonym 2015-09-11 05:50:30

s7r wrote:
> Looks like it’s in testing now:
> https://tracker.debian.org/news/709815

Thanks for the notification!

So I backported the electrum 2.4.4-1 and python-pbkdf2 1.3+20110613.git2a0fb15~ds0 packages to wheezy-backports-sloppy, which turned out to be trivial rebuilds (patches attached any way). Then I did this in a running session:

# config format has changed and will segfault if we use the old one, yay!
rm -r /home/amnesia/.electrum

apt-get update
apt-get remove electrum python-electrum
apt-get install python-dnspython python-protobuf python-qrcode libprotobuf7 python-requests/wheezy-backports python-urllib3/wheezy-backports
dpkg -i electrum_2.4.4-1~d70.wheezy+1_all.deb python-electrum_2.4.4-1~d70.wheezy+1_all.deb python-pbkdf2_1.3+20110613.git2a0fb15~ds0-3~d70.wheezy+1_all.deb

Electrum sets up a wallet and starts in offline mode. However, when I try to open Tools -> Network to configure the proxy and go online, I get:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/electrum_gui/qt/main_window.py", line 2832, in run_network_dialog
    NetworkDialog(self.wallet.network, self.config, self).do_exec()
  File "/usr/lib/python2.7/dist-packages/electrum_gui/qt/network_dialog.py", line 47, in __init__
    self.servers = network.get_servers()
AttributeError: 'NoneType' object has no attribute 'get_servers'

I cannot remember this error from my previous trial with the packages from sid. However, I tried replacing my backports with the backages from Sid, just like before, but I still get the same error. I’m quite confused. I am almost 100% sure that I managed to go online in my previous attempt. I do not think there’s any point in my uploading these packages any where until this is resolved.

s7r, could you try to get electrum working with the Sid packages, possibly based on my instructions above (but obviously something more has to be done — but remember to remote /home/amnesia/.electrum)? Possibly you may also want to check if electrum actually is broken in Debian Stretch/Sid and that’s what we’re hitting.

#13 Updated by anonym 2015-09-11 05:58:44

Hm. It seems that the electrum daemon sort of works:

electrum daemon --proxy socks5:127.0.0.1:9050 start
tail -f /home/amnesia/.electrum/daemon.log


I can see it downloading blockchain chunks. And there’s a fair bit of connection/SSL errors too. And electrum daemon status always reports connected: false

I’m not sure this backporting effort will be so easy that it’s worth it. Well, if you can get it to work in a live Tails session, s7r, let us know (with exacpt steps!) and I promise to try to get it into Tails.

#14 Updated by s7r 2015-10-16 16:01:36

  • Type of work changed from Research to Wait

I was able to install following the same steps as you did.
I have experienced the same problem, it generates a wallet (warns about persistence) and starts in offline mode. However, I can’t open the networking pop-up in order to enter the socks5 proxy address and connect to a server.

This is a known issue in 2.4.4 and it’s fixed, the patch will be included in 2.5 which is to be release in a few days. After Electrum 2.5 will be in sid, we will test it again in a live Tails session. The good thing is that in Electrum 2.5 the gui and daemon are mutually exclusive. So, we’ll wait few days until the bug is patched upstream.

Before deleting /home/amnesia/.electrum/config I saw that the old version of electrum which is shipped by Tails (1.9.8) comes with the config already there which has a socks5 127.0.0.1:9050 setting - maybe that’s why Electrum 1.9.8 doesn’t start in offline mode and 2.4.4 does. Could we do the same with 2.5 when it’s included so that users don’t have to do this extra step?

Also, can we upgrade python (are there other dependencies required to update) so we can avoid —force-depends and do a clean install?

#15 Updated by anonym 2015-10-17 06:08:09

s7r wrote:
> I was able to install following the same steps as you did.
> I have experienced the same problem, it generates a wallet (warns about persistence) and starts in offline mode. However, I can’t open the networking pop-up in order to enter the socks5 proxy address and connect to a server.
>
> This is a known issue in 2.4.4 and it’s fixed, the patch will be included in 2.5 which is to be release in a few days. After Electrum 2.5 will be in sid, we will test it again in a live Tails session. The good thing is that in Electrum 2.5 the gui and daemon are mutually exclusive. So, we’ll wait few days until the bug is patched upstream.

Thanks for the info! Feel free to notify the Debian maintainer of electrum to bump the package as soon as it’s released. Any way, as soon as you see the package is available, please reassign this ticket to me, and set the QA Check field to Dev Needed.

> Before deleting /home/amnesia/.electrum/config I saw that the old version of electrum which is shipped by Tails (1.9.8) comes with the config already there which has a socks5 127.0.0.1:9050 setting - maybe that’s why Electrum 1.9.8 doesn’t start in offline mode and 2.4.4 does. Could we do the same with 2.5 when it’s included so that users don’t have to do this extra step?

Definitely. :)

> Also, can we upgrade python (are there other dependencies required to update) so we can avoid —force-depends and do a clean install?

Backporting a newer python is likely not trivial. But that’s beside the point: the idea is to backport electrum, like I did above, and then it will be backported to wheezy’s python, so no --force-depends crap will be needed — it will already bi installed in Tails.

#16 Updated by anonym 2015-10-27 10:13:16

A user reported this:

$USER: electrum on tails is ancient :(
$USER: i get this when testing it; "error: Your client produced a transaction that is not accepted by the Bitcoin network any more. Please upgrade to Electrum 2.5.1 or newer"


So it seems we may have to backport 2.5.1, which is in Debin Unstable, but not yet in Debian Testing.

#17 Updated by anonym 2015-10-27 11:27:40

So I tried with a backported electrum 2.5.1 (trivial wheezy backport) and I got the same stack trace when trying to open the network settings. However, after restarting electrum the network settings works, and I can configure it to use tor and all is good. I haven’t found any way to make those settings load from .electrum/config. Not even the -p and -s options helps; it always starts in offline mode, the network settings do not work, and a restart is required before anything will work.

#18 Updated by anonym 2015-11-06 10:26:04

  • Subject changed from Upgrade Electrum to >= 2.3.x for compatibility with all Electrum servers to Upgrade Electrum to >= 2.5.x for compatibility with all Electrum servers

#19 Updated by anonym 2015-11-06 10:32:08

  • blocks Bug #10475: Scenario "Using a persistent Electrum configuration" is fragile added

#20 Updated by anonym 2015-11-06 12:09:56

Reminder: Try to backport Electrum 2.5.1 to Jessie and see if that works better. Perhaps Wheezy’s Python is just too old or something.

#21 Updated by anonym 2015-11-06 12:10:18

  • Assignee changed from s7r to anonym
  • Target version set to Tails_1.8
  • QA Check changed from Info Needed to Dev Needed

#22 Updated by s7r 2015-11-07 11:52:34

Hello anonym,

I have identified the issue:
https://github.com/spesmilo/electrum/issues/1531

Fixed in this commit:
https://github.com/spesmilo/electrum/commit/4a7c7a66162c04fcaeeecb2c79942ee06a2b4da5

It will be included in Electrum 2.5.3 which will be released very soon - after that we will package electrum to unstable, do the tests, and if all good we’ll corp it into Tails as soon as it is migrated to sid.

I don’t think the python version has anything to do with it, nor upgrading to Jessie - I think it’s because of Tails armor which doesn’t allow any internet traffic except what goes through Tor socks5 proxy. Electrum’s installwizard gets confused and doesn’t do what it should. That’s why second time when you start electrum it always works and it also works if you have the blockchain_headers file in ~/.electrum.

Anyway, the above commit should fix it and should allow us to ship electrum with a pre-generated config in ~/.electrum/conf containing just the socks5 proxy address, and users will be able to enter seeds, generate wallets, etc.

#23 Updated by s7r 2015-11-12 15:58:45

  • File <del>missing: config</del> added
  • Type of work changed from Wait to Test

Hello anonym,

We are good to go. Electrum 2.5.4 is in unstable.
https://tracker.debian.org/pkg/electrum

We released 2.5.3 and 2.5.4 in the same day, 2.5.4 only has a small fix - the minimum relay fee increased because the majority of the server operators raised the dust threshold fee due to a spam attack in the bitcoin network. 2.5.4 should be the latest which I think it’ll work for reasonable time in the future (it produces valid transactions, compatible with new servers and uses new seeds, etc.). There is no sense backporting an older version than 2.5.4 due to the nature of the changes / fixes in each release.

I have tested it and it works good, it doesn’t break the networking pop-up and starts correctly both with config or without config at all.

I recommend shipping it with a conf. file in ~/.electrum which will just contain the proxy set to localhost:9050. Attached a sample file. If you think other settings should be shipped by default with this config file please add them.

#24 Updated by sajolida 2015-11-17 02:48:54

Note that according to https://mailman.boum.org/pipermail/tails-dev/2015-November/009750.html our documentation needs updating to. I won’t have time to work on investigating this myself but I’m happy to review and edit whatever needs to be changed in the current doc.

#25 Updated by sajolida 2015-11-17 02:49:08

Recommend users to manually select a trusted .onion server to protect
against DoS after note about SPV vulnerability.

Make a note that Electrum uses mBTC as the default base unit. 1 mBTC =
0.001 BTC. It can be changed in preferences.

#26 Updated by anonym 2015-11-18 09:15:52

  • Status changed from Confirmed to In Progress

Applied in changeset commit:98baf43e6f586acfec938a5eeb1586d6fb168d69.

#27 Updated by s7r 2015-11-18 09:17:19

  • Status changed from In Progress to Confirmed

Electrum 2.5.4 is in testing as of now. It should be ready for inclusion in next Tails release.

@sajolida can you tell me what needs to be updated and where? The linked email is not very explicit. I could take the time and write some necessary guidelines, need to know more about what’s required.

What DoS vulnerability do you refer to? As far as I know, the known vulnerabilities are covered in Electrum 2.5.4. Waiting for at least 1 confirmation is always required in SPV (light weight) wallets due to a limitation of the protocol itself - it’ll always be like this and it’s a known tradeoff, not a bug.

Also, there are .onion servers out there. Only I run 3. Electrum servers are peer to peer advertised via IRC, so the user has to connect to at least 1 server in order to get a fresh updated list of available servers.

As for the mBTC unit instead of BTC, I am also not a big fan of it - maybe we could change this setting by default if we decide to ship with a config file?

#28 Updated by s7r 2015-11-18 09:39:20

  • Status changed from Confirmed to In Progress

#29 Updated by anonym 2015-11-18 14:13:46

  • File electrum_2.5.4-1~d70.wheezy+1+tails1.patch added
  • Assignee changed from anonym to s7r
  • % Done changed from 0 to 40
  • QA Check changed from Dev Needed to Info Needed
  • Feature Branch set to bugfix/9713-electrum-2.5

s7r wrote:
> We are good to go. Electrum 2.5.4 is in unstable.

Thanks for letting me know! I’ve now built a backport (see attached file for the relatively simple backport) and prepared a Tails feature branch; nightly builds should be available here shortly: http://nightly.tails.boum.org/build_Tails_ISO_bugfix-9713-electrum-2.5/lastSuccessful/archive/build-artifacts/

So using your suggested ~/.electrum/config (i.e. only proxy set), following the wizard and then picking “Auto connect” doesn’t work very well. While electrum starts fine, most of the time it simply will not synchronize (even after waiting 10 minutes). I still have to open the network settings and manually pick a server for stuff to progress. But sometimes (~20% chance?) it does work, so I guess a bad server is picked (?). However, I can see electrum connecting to multiple servers in the pool (on their advertised port, e.g. 50002) immediately when I start the wizard which I find odd. Especially since it then doesn’t pick a working server; otherwise I could imagine it being a test of which servers in the pool work, but it’s done pretty prematurely since I still haven’t been asked if I want to “Auto connect” or manually configure a server. Is this what you talk about in this quite:

> Electrum servers are peer to peer advertised via IRC, so the user has to connect to at least 1 server in order to get a fresh updated list of available servers.

Doing that lookup prematurely could be pretty serious in non-Tails environments since it leaks to the clearnet that you are using electrum => bitcoins, which you may want to remain a secret. The network shouldn’t be touched until you have had the option to configure electrum to use Tor! Relatedly, if I completely purge the config (i.e. no proxy) I can see electrum immediately (i.e. on wizard start) trying to do some sort of DNS lookup of many servers (which will fail in Tails, due to limitations of Tor’ DNSPort). Perhaps I’m missing something, but this looks pretty serious.

Returning back to Tails, what’s to be expected with your suggested config, really? What I’d expect is that no connection would be made until the wizard knows the user’s answer to “How do you want to connect”, and if “Auto connect” is picked, then it would try servers until it find one that works (without taking too much time on failing servers). Is that not the intention? If not, any suggestions on how we can make this user friendly in Tails? Pre-configure some know good server?

So, what remains is:

  • Solve the “server/synchronizing issue” described above.
  • Get electrum 2.5.4 into Jessie Backports; I don’t see the sense in getting this (and python-pbkdf2) into Wheezy Backports for the few remaining Tails/Wheezy releases unless it’s very easy. We can use a custom package until Tails/Jessie.

Let’s try to get this fixed in time for Tails 1.8 (on 2015-12-15)!

#30 Updated by sajolida 2015-11-22 04:02:52

Dev note from https://mailman.boum.org/pipermail/tails-dev/2015-November/009785.html: would it make sense to add preconfigured onion servers to Electrum in Tails. Maybe we should keep the discussion on tails-dev but I want to make sure this was raised here as well as it might impact what we ship in the ISO (and not the documentation only).

#31 Updated by sajolida 2015-11-22 05:22:19

> @sajolida can you tell me what needs to be updated and where? The
> linked email is not very explicit.

I forwarded you another one today and I hope they provide a bit more
information. Tell me if they still don’t and maybe we should continue
discussing this over email not to duplicate discussions between email
and Redmine. Also, I personally won’t have much time to dedicate to this
apart from a bit of guidance and review.

> What DoS vulnerability do you refer to?

Michael was referring to http://arxiv.org/pdf/1410.6079.pdf. From my
very limited understanding of the problem, using onion servers is the
way to go for people using Electrum over Tor. See my previous note about
this as I’m not sure how this is integrated in your branch.

> As far as I know, the known
> vulnerabilities are covered in Electrum 2.5.4. Waiting for at least 1
> confirmation is always required in SPV (light weight) wallets due to
> a limitation of the protocol itself - it’ll always be like this and
> it’s a known tradeoff, not a bug.
>
> Also, there are .onion servers out there. Only I run 3. Electrum
> servers are peer to peer advertised via IRC, so the user has to
> connect to at least 1 server in order to get a fresh updated list of
> available servers.

I’m not sure to understand whether the user of Electrum in Tails would
have to perform some specific action to use the onion servers in
Electrum in Tails or not.

If they have to perform a specific action, then this should be
documented. If they don’t have to perform a specific action, then maybe
we need to adapt the current warning about “Do not blindly trust the
bitcoin balance”.

> As for the mBTC unit instead of BTC, I am also not a big fan of it -
> maybe we could change this setting by default if we decide to ship
> with a config file?

I don’t think we should change the default from upstream unless we need
something that is Tails specific.

#32 Updated by s7r 2015-11-23 16:47:13

sajolida wrote:
> I forwarded you another one today and I hope they provide a bit more
> information. Tell me if they still don’t and maybe we should continue
> discussing this over email not to duplicate discussions between email
> and Redmine. Also, I personally won’t have much time to dedicate to this
> apart from a bit of guidance and review.

Thanks, replied there and also including some info from that email here.

> Michael was referring to http://arxiv.org/pdf/1410.6079.pdf. From my
> very limited understanding of the problem, using onion servers is the
> way to go for people using Electrum over Tor. See my previous note about
> this as I’m not sure how this is integrated in your branch.
>

This is unrelated to electrum. It applies to Bitcoin Core (the
software implementing bitcoin protocol - full nodes). Bitcoin core has
a peer-ban-score, where the ban score of a peer (remote server that
you are connected to) increases if that peers feeds you trash data or
tries to DoS you.

The attack described in that paper takes advantage of the scarce exit
power in the Tor network (~1000 IP addresses) and speaks from the
point of view where an attacker runs few Tor exit nodes that allow
bitcoin exit traffic, and then uses the other exits to feed enormous
amount of trash data to the bitcoin network, hoping the majority of
bitcoin nodes will ban the other exits which are not under the control
of the attacker. Then, you can only connect to a bitcoin peer via Tor
using one of attacker’s exit nodes. Hope this explains - it’s just a
summary.

Electrum servers don’t have such banning mechanism, they only have
limits of requests per wallet and per address (100/10000).

So, I think it’s not worth it to confuse the users and make reference
to this anywhere.

> I’m not sure to understand whether the user of Electrum in Tails would
> have to perform some specific action to use the onion servers in
> Electrum in Tails or not.
>
> If they have to perform a specific action, then this should be
> documented. If they don’t have to perform a specific action, then maybe
> we need to adapt the current warning about “Do not blindly trust the
> bitcoin balance”.
>
They don’t have to. Just update the documentation. I included as much as I could in my email.

People should only trust the server list in Preferences ->
Network because those are servers which are advertised and their utxo
set hash is checked. If one of them has a broken database or database
not matching the other servers it’ll get banned.

> I don’t think we should change the default from upstream unless we need
> something that is Tails specific.

Correct. Added the warning for including it in the documentation.

If you think it’s necessary to include the additions for documentation sent by me in this ticket, please copy them.

#33 Updated by s7r 2015-11-23 17:04:19

anonym:

This is desired when we are in “Auto-mode”. Trying to connect before installwizard finishes so we can select Auto-Connect or Select server manually indeed should be looked into. My guess is that it tries to connect because it has a proxy set up in the config file and thinks the network settings have been previously modified by the user.

Can you test with no config file at all and confirm if it still tries to connect randomly even BEFORE installwizard finishes so you can select Auto connect or manual?

Anyway, regardless, this probably won’t be fixed in time for Tails 1.8 so we should patch 2.5.4 as-it-is since that we can agree that the possible issue is not a security risk to Tails users, due to the nature of the Tails OS. Having 1.9.8 shipped doesn’t allow our users to spend their coins (low-S ecdsa signatures enforcement) which is worse.

My server is: 3ffk7iumtx3cegbi.onion port 50001 (TCP)

We can include it by default, it’s always up but sometimes has a few seconds lag (it doesn’t have SSD and leveldb is furious on normal hard disk drives). So users could connect to this server initially, and then change if they want. Still I don’t think we should advertise or recommend a certain Electrum server for our users to use. The server list in the network menu is dynamically updated and servers with wrong utxo set hashes are kicked. It would rock if we could get SSD, I’ll try to make an effort and switch to one in as fast as possible, maybe even attract donations in my server’s btc donations address.

#34 Updated by anonym 2015-11-24 05:30:32

s7r wrote:
> anonym:
>
> This is desired when we are in “Auto-mode”.

Sure, but surely not before we’re ready to connect, i.e. have no wallet, right? Besides, I haven’t explicitly configured Auto-mode to be on, but I guess it’s the default.

> Trying to connect before installwizard finishes so we can select Auto-Connect or Select server manually indeed should be looked into. My guess is that it tries to connect because it has a proxy set up in the config file and thinks the network settings have been previously modified by the user.

I think this is incorrect because…

> Can you test with no config file at all and confirm if it still tries to connect randomly even BEFORE installwizard finishes so you can select Auto connect or manual?

… I already did try without a proxy set (see above: “Relatedly, if I completely purge the config …”) and connections (DNS queries, in this case) were made “early”.

> Anyway, regardless, this probably won’t be fixed in time for Tails 1.8 so we should patch 2.5.4 as-it-is since that we can agree that the possible issue is not a security risk to Tails users, due to the nature of the Tails OS. Having 1.9.8 shipped doesn’t allow our users to spend their coins (low-S ecdsa signatures enforcement) which is worse.

Agreed! In fact, according to our reports, it seems users can’t even get their balance.

> My server is: 3ffk7iumtx3cegbi.onion port 50001 (TCP)

First, let’s have a general discussion about what fixing on a single server means for us, so we have a clearer picture:

  • A single server is a single point of failure. DoS is easier, for instance.
  • Thanks to SPV, the server can spoof the wallet balance. Hence the server operator can scam Tails users, e.g. s/he can buy stuff from a Tails user, and then bump their balance with that exact amount so it looks like they’ve received payment.

More?

Now about fixing on your server specifically (as a temporary solution):

> We can include it by default, it’s always up but sometimes has a few seconds lag (it doesn’t have SSD and leveldb is furious on normal hard disk drives).

So it’s in fact very sensitive to DoS, then (see my point above).

> So users could connect to this server initially, and then change if they want.

I’d prefer if we wouldn’t have to document this since we’d have to explain a lot of stuff to make users able to make some sort of informed choice, I guess.

> Still I don’t think we should advertise or recommend a certain Electrum server for our users to use. The server list in the network menu is dynamically updated and servers with wrong utxo set hashes are kicked.

Sure. The best would would be if Auto-mode worked better, since that also alleviates both my points above (“single point of failure”, “balance spoofing”).

Not sure what conclusions to draw from this yet. Awaiting input!

#35 Updated by s7r 2015-11-24 11:46:34

anonym wrote:

> … I already did try without a proxy set (see above: “Relatedly, if I completely purge the config …”) and connections (DNS queries, in this case) were made “early”.
>

Ok, will take care of this upstream so when we upgrade electrum in Tails this won’t be a problem any more. Don’t know how much it’ll take but hopefully not too long.

> First, let’s have a general discussion about what fixing on a single server means for us, so we have a clearer picture:
>
> * A single server is a single point of failure. DoS is easier, for instance.
>
> * Thanks to SPV, the server can spoof the wallet balance. Hence the server operator can scam Tails users, e.g. s/he can buy stuff from a Tails user, and then bump their balance with that exact amount so it looks like they’ve received payment.
>
So? This is a limitation, not a bug. This is how SPV works everywhere, which is why electrum shows a separate balance for unconfirmed transactions. The security warnings even request in full nodes to wait for at least 1 confirmation. This is just the problem with unconfirmed transactions.

Wasn’t this already a problem in 1.9.8? Why it became a big problem now?

Also, it’s kind of hard to target a particular Tails user because of Tor and etc. But I agree, what you said it’s not impossible.

> I’d prefer if we wouldn’t have to document this since we’d have to explain a lot of stuff to make users able to make some sort of informed choice, I guess.

+1

#36 Updated by anonym 2015-11-25 13:38:49

s7r wrote:
> anonym wrote:
>
> > … I already did try without a proxy set (see above: “Relatedly, if I completely purge the config …”) and connections (DNS queries, in this case) were made “early”.
> >
>
> Ok, will take care of this upstream so when we upgrade electrum in Tails this won’t be a problem any more. Don’t know how much it’ll take but hopefully not too long.

Excellent! Do you think it could be ready by 2015-12-10? If so, this definitely could go into Tails 1.8. An Electrum release (uploaded to Debian Sid) wouldn’t be necessary until then, just patches that apply cleanly on the 2.5.4 sources would be fine.

Also, it’s not clear to me whether this will improve Auto-mode, so that it can be used instead of fixing on a single server for all Tails users.

> > First, let’s have a general discussion about what fixing on a single server means for us, so we have a clearer picture:
> >
> > * A single server is a single point of failure. DoS is easier, for instance.
> >
> > * Thanks to SPV, the server can spoof the wallet balance. Hence the server operator can scam Tails users, e.g. s/he can buy stuff from a Tails user, and then bump their balance with that exact amount so it looks like they’ve received payment.
> >
> So? This is a limitation, not a bug. This is how SPV works everywhere, which is why electrum shows a separate balance for unconfirmed transactions. The security warnings even request in full nodes to wait for at least 1 confirmation. This is just the problem with unconfirmed transactions.

I’m not bashing SPV, I just think SPV with a fixed server for all Tails users is a bit worrisome. The difference between “fixed server” and “Auto-mode” becomes “the operator can always target a known Tails user” vs “the operator can target a known Tails user with probability 1/$POOL_SIZE”.

> Wasn’t this already a problem in 1.9.8? Why it became a big problem now?

This is indeed not a bigger problem now than then. I would just prefer to do it right this time, if possible. :) To be clear: if we have to fix on a single server (yours, or whatever) for Tails 1.8, I won’t block on this.

#37 Updated by s7r 2015-11-25 16:51:22

  • Type of work changed from Test to Wait

My option: respect the principles of decentralization, do not hardcode a server (single point of failure) and do it right: Make the package upstream work how it should for Tails.

I will take care of it and try as hard as possible to have it ready before Tails 1.8, when I’ll reassign this ticket to anonym.

If anyone doesn’t agree, change the type “Wait” and proceed.

#38 Updated by s7r 2015-12-04 17:35:10

  • Assignee changed from s7r to anonym
  • Type of work changed from Wait to Test

Fixed and patched. We have Electrum 2.5.4-2 in unstable which includes the fix. With the current priority we will have it in testing on 09.12.2015, hope that is in time for the next Tails release.

anonym - now it always connects on auto-connect. Sometimes in 3 seconds, sometimes in 10 seconds, sometimes in few minutes. Depends on what servers are randomly selected, some of them are behind DoS protection systems which filter Tor, etc. etc. But it will connect. Please test it yourself and let me know.

The other issue which doesn’t affect Tails (DNS bootstrap before wizard finishes) will be fixed at some point, but not right now since we have to change many things for that.

#39 Updated by s7r 2015-12-04 18:41:09

Average is about 22 seconds, all successful connects on a set of 150 auto-connect requests. I think the several seconds delay comes from downloading the headers from 7 or 8 other servers at the same time, in order to check the hash and identify if the user is connected to a malicious server which could lie about something (SPV, you know). This affects the same non-Tails operating system as well.

#40 Updated by s7r 2015-12-05 03:52:59

  • File deleted (config)

#41 Updated by s7r 2015-12-05 04:06:28

  • File config added
  • QA Check changed from Info Needed to Dev Needed

Solution: ship electrum 2.5.4-2 in Tails with the following files in ~/.electrum/:

config (only includes the proxy set to socks5, localhost:9050)

blockchain_headers (includes the headers of the block chain to block height #386826). The file is from my own full node, but feel free to verify it against any node in the bitcoin network. Headers are the same everywhere. We don’t need to ensure we always ship it with very last block header, if we ship it at height #386826 it means there is few more KB for electrum to download. Its size is ~29 MB. This is not by any means sensitive, it’s a bitcoin network wide available file, it’s just there to make the sync faster. In case there’s a reorganization, the file will just be updated by electrum. It saves about 30 MB of initial download which via Tor it takes some time obviously.

I cannot upload blockchain_headers file here because it has 30 MB, but download it from here along with the config:
https://www.sky-ip.org/electrum-tails/

#42 Updated by anonym 2015-12-09 15:08:12

Thanks a lot for your help, s7r! I’ve uploaded a electrum 2.5.4-2 backport to the feature branch’s APT suite, built an image and successfully tested it. Others can soon test the nightly builds: http://nightly.tails.boum.org/build_Tails_ISO_bugfix-9713-electrum-2.5/lastSuccessful/archive/build-artifacts/

Indeed, with this electrum finally seems to work as it should in Tails! FWIW, I’ve attached the patch of my backport against the 2.5.4-2 sources.

Regarding shipping blockchain_headers in ~/.electrum I think we’d rather not. As expected, it compresses poorly, so it will increase the image size with almost the full 30 MiB. Users that enable the persistence preset will only have to download it once any way, so the situation is not so bad — after all, we strongly encourage using persistence, for good reasons, so I think few users would actually really benefit from this.

Next I’ll try running our automated test suite’s Electrum scenarios.

#43 Updated by s7r 2015-12-12 06:38:56

Glad to hear this anonym!

yes, it doesn’t compress so well. This is just a decision we have to take. I am equally fine with both options, you decide. This is only related to end-user interaction/experience.

a) is it worth to extend the Tails image size with 30MiB and have electrum sync with any server almost instantly.

b) save 30MiB in Tails image size and have electrum sync in 20-30 seconds on first start, until it downloads the headers from other servers.

I want to highlight again that there is no security tradeoff if we ship blockchain_headers file and will benefit both the users in amensic mode and the users in persistent mode. I don’t know exactly, but I think we have more users running Tails in amnesic mode (and using electrum by entering the seed every time) and would prefer faster sync (already having blockchain_headers there) as opposite to users that will be affected if the image sizes increases with 30 MiB.

Downloading the headers from multiple randomly selected servers and checking against the server we are connected to is a major help, since we are in SPV mode. This will ensure that if we are connected to a malicious server which tries to withhold data from us, we’ll switch to another one immediately. So, I think that the 20-30 second initial sync delay is totally worth it.

#44 Updated by anonym 2015-12-14 13:04:36

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100

Applied in changeset commit:e1198d4309f8153c9804c7f03e77c9f4e18d03b3.

#45 Updated by anonym 2015-12-14 13:26:45

  • Assignee deleted (anonym)
  • QA Check changed from Ready for QA to Pass

#46 Updated by anonym 2015-12-14 13:53:40

  • related to Bug #10754: Install Electrum 0.2.5.x in Tails/Jessie added

#47 Updated by bertagaz 2015-12-15 10:44:09

  • related to Bug #10758: Update electrum.feature for Electrum 2.5.x added

#48 Updated by anonym 2015-12-16 11:33:30

  • Status changed from Fix committed to Resolved

#49 Updated by anonym 2015-12-16 11:36:04

  • Type of work changed from Test to Code