Feature #15211

Reduce our Chutney network

Added by anonym 2018-01-22 12:01:41 . Updated 2018-07-06 07:02:42 .

Status:
In Progress
Priority:
Normal
Assignee:
anonym
Category:
Test suite
Target version:
Start date:
2018-01-22
Due date:
% Done:

0%

Feature Branch:
test/15211-minimal-tor-network+force-all-tests
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Rationale: the network size is proportional to the RAM/CPU usage, so we shouldn’t run a network larger than we need. (Note: the Tor 0.3.2 series that just went stable seems to increase relay memory usage quite drastically for us, so this what was prompted the interest in this ticket.)

Currently we have:

NODES = Authority.getN(4) + \
        BridgeAuthority.getN(1) + \
        NonExitRelay.getN(20) + \
        ExitRelay.getN(10) + \
        Bridge.getN(3) + \
        BridgeObfs4.getN(3) + \
        OnionService.getN(1) + \
        Client.getN(1)


but we should be able to get away with the smallest possible network:

NODES = Authority.getN(1) + \
        BridgeAuthority.getN(1) + \
        NonExitRelay.getN(2) + \
        ExitRelay.getN(1) + \
        Bridge.getN(1) + \
        BridgeObfs4.getN(1) + \
        OnionService.getN(1) + \
        Client.getN(1)


or similar. Some thoughts:

  • What is the lowest number of authorities (incl. bridge authorities) needed for a consensus?
  • 2 non-exits + 1 exit is enough, but maybe we want some diversity? I actually cannot come up with any technical argument for this.
  • I guess testing one bridge at a time is enough — at least in the current code (ours, Tor Launcher) there’s no difference between the 1 vs many cases.

Subtasks

Related issues

Blocked by Tails - Bug #16792: Upgrade our Chutney fork Resolved

History

#1 Updated by anonym 2018-01-22 12:02:07

#2 Updated by anonym 2018-01-30 21:16:44

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20
  • Feature Branch set to test/15211-minimal-tor-network+force-all-tests

I suspect this is the minimal network:

NODES = Authority.getN(1) + \
        NonExitRelay.getN(2) + \
        ExitRelay.getN(1) + \
        Bridge.getN(1) + \
        BridgeObfs4.getN(1) + \
        OnionService.getN(1) + \
        Client.getN(1)

bit I haven’t tried a full run yet. I’ve pushed a branch that will run it on Jenkins.

#3 Updated by anonym 2018-02-01 16:53:34

Full runs on Jenkins:

I only did a quick analysis, but it seems like all failures are unrelated (<= none occur in both runs).

It is still conceivable that this branch reduces the reliability of the Tor network, and that we haven’t seen enough runs to expose that. But my hope is that what we will see is a reduction of failures that we suspect are due to too resource starvation (CPU, memory) on the isotesters.

#4 Updated by anonym 2018-02-02 14:23:53

  • % Done changed from 20 to 0

anonym wrote:
> It is still conceivable that this branch reduces the reliability of the Tor network, and that we haven’t seen enough runs to expose that. But my hope is that what we will see is a reduction of failures that we suspect are due to too resource starvation (CPU, memory) on the isotesters.

I started four more full test suite runs. When done we’ll have six runs in total, that I’ll analyze in detail w.r.t. this.

#5 Updated by anonym 2018-02-23 11:04:46

Still looks bad. I’ll bump the number of authorities from 1 to 3 just to see what happens.

#6 Updated by anonym 2018-02-26 10:58:20

  • Target version changed from Tails_3.6 to Tails_3.7

anonym wrote:
> Still looks bad. I’ll bump the number of authorities from 1 to 3 just to see what happens.

Might have improved it a bit, but seemingly not satisfactory.

#7 Updated by intrigeri 2018-03-10 08:13:42

anonym wrote:
> Note: the Tor 0.3.2 series that just went stable seems to increase relay memory usage quite drastically for us

Are these problems tracked somewhere upstream? Perhaps they would be good candidates the tails-needs keyword.

#8 Updated by intrigeri 2018-03-28 09:30:34

  • Target version changed from Tails_3.7 to Tails_3.8

#9 Updated by intrigeri 2018-05-25 13:22:41

  • Target version deleted (Tails_3.8)

#10 Updated by intrigeri 2018-07-06 07:02:42

Note that our chutney submodule is currently 125 commits (=~ 3 years) behind upstream’s. I see some more preconfigured minimal network flavours were added. Maybe a first next step would be to update our submodule.

#11 Updated by intrigeri 2018-07-27 08:52:21

  • blocked by deleted (Feature #13241: Core work: Test suite maintenance)

#12 Updated by intrigeri 2019-08-10 17:29:22

  • blocked by Bug #16792: Upgrade our Chutney fork added