Factor out stuff into a Tails Python library
There’s some code duplication between at least Tails Greeter (feature/spoof-mac branch) and WhisperBack which should be eliminated by being refactored into a shared Tails Python library:
cb_request_starting()callback used for the help browser
- probably more of the help browser setup code (since it was pretty much copy-pasted from WhisperBack into TG)
__get_localised_doc_link()from WhisperBack (isn’t in TG yet, but will at some point)
|Related to Tails - Feature #6786: Be consistent when checking if persistence is enabled in tails-additional-software||In Progress||2014-02-27|
|Related to Tails - Feature #6145: Factor out stuff into a shell library||Confirmed||2013-11-14|
|Related to Tails - Feature #9372: Use 'set -e' and 'set -u' in every shell script where reasonably doable||In Progress||2015-05-11|
|Related to Tails - Feature #11198: Port complex shell scripts into Python||Confirmed||2016-08-27|
Related to Tails -
#8 Updated by anonym 2015-10-16 04:46:16
I have a few additional ideas for such a Python library (plus steps towards a grander Python vision for Tails in general, see below). For starters, parts of the Shell library should migrate into the Python library, so the shell functions are thin wrappers around the Python library. The priority should be:
- Anything that heavily uses lists! (Think: whitespace issues in the shell)
- Anything complex should move there, e.g.
hardware.sh:mod_rev_dep()(recursion in the shell? why not, but please no :)), and I think we can get
common.sh:wait_until()/try_for()functions implemented in python with real timeouts, instead of the “sort of”-timeouts we have now.
- Many things in
tor.shcan be implemented reliably and trivially via
- Python probably has proper localization stuff, instead of our
However, maybe we can convert everything in the Shell library, and even aim for replacing more or less all our Shell scripts in the future? There are several Python modules that could make this easier for us. We could use e.g. plumbum for this:
- It can import any command-line tool in your
$PATHand integrate it into python like a function. It’s super easy to extract the exit code, stdout and stderr. There’s a simple interface to use such imported commands through
Popentoo, when needed. There’s a simple interface to use such imported commands through
Popentoo, when needed. All this works so well that I’m afraid we might continue to use the Shell tools instead of the more appropriate Python equivalents. :)
- It provides simple ways to do piping, backgrounding and similar convenient shell concepts.
- We get the equivalent of
set -euby default, and can work with it more easily since the program won’t exit, but throw an exception, so exception handling will make things much more easier for us.
- It makes command nesting with
sshetc much simpler and clearer than in the shell.
Overall, plumbum is quite brilliant. :) There are also alternatives, like sh, scriptine and so on, but I do not know what is best for our needs (I suspect plumbum, though). Also, for more complex subprocess interaction we may want to look at e.g. sarge as a replacement for the standard (but IMHO awkward)
subprocess Python module.
I’m digressing, a bit. I guess we really need another ticket. :)
#10 Updated by anonym 2015-10-16 06:25:34
> > Overall, plumbum is quite brilliant. :) There are also alternatives, like sh,
> FYI we already use
sh in our Python lib.
After a brief look, it looks good, too (and indeed very similar; apparently
sh as an inspiration). A major difference is how piping works:
- Normal shell:
ls -1 | wc -l
(ls['-1'] | wc['-l'])()or
chain = ls['-1'] | wc['-l']; chain()
plumbum works more similarly to the shell, the
(...)() stuff is ugly. The
sh way, i.e. function composition, is actually how it’d be done in most programming languages, which might be a good thing: instead of making us thing that we are “writing shell scripts in python”, it may promote the feeling that we are “writing python scripts with an easy way to invoke the shell”. ;)
Any way, most of the time we use pipes in the shell it’s for text processing, and Python has much better facilities for that. On the top of my head, we do some piping into
nc for control port stuff, but we should use
stem any way. I’m sure there’s more but, again, I digress.
#11 Updated by alant 2015-10-18 10:17:36
> kurono volunteered to help. I’ll let you folks discuss it.
kurono, feel free to ask any question or start working on this. You can assign the ticket to you if you want. I’ll be happy to give any help you wish and to review your work. I’ll try not to let you wait for a whole week, but don’t expect me to answer daily.
#14 Updated by intrigeri 2016-02-18 16:23:05
This is one of the last two remaining tickets on the obsolete “Sustainability_M1” (that doesn’t mean much anymore, since we have our new roadmap). kurono, please consider moving this ticket to e.g. the 2016 or 2017 milestone, or just emptying the Target version field entirely.