Feature #11754

Port complex build shell scripts to Python

Added by intrigeri 2016-08-31 01:16:51 . Updated 2020-04-01 09:03:16 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Build system
Target version:
Start date:
2016-08-31
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Starter:
Affected tool:
Deliverable for:

Description

(Splitting Feature #11198 into smaller pieces.)

team: hefee, muri, intrigeri (reviewer); kurono?


Subtasks


Related issues

Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by intrigeri 2016-08-31 01:17:13

  • Blueprint set to https://tails.boum.org/blueprint/Port_shell_scripts_to_Python/

#2 Updated by sycamoreone 2016-09-02 01:38:06

  • Assignee changed from sycamoreone to intrigeri

The build shell scripts are the one in config/binary_local-hooks/ and config/chroot_local-hooks/?

#3 Updated by intrigeri 2016-09-02 02:17:23

  • Assignee changed from intrigeri to sycamoreone

> The build shell scripts are the one in config/binary_local-hooks/ and config/chroot_local-hooks/?

Yes :)

#4 Updated by sycamoreone 2016-09-07 03:21:55

More questions:

What about the scripts in tails/auto? At least auto/build and auto/config are quite complex shell scripts.

Some of the scripts source functions or configuration variables from other shell scripts. All the scripts in config/binary_local-hooks source

“${LB_BASE:-/usr/share/live/build}”/scripts/build.sh

and scripts in chroot_local-hooks source

/usr/share/amnesia/build/variables (created by ./auto/build)

or

/usr/local/lib/tails-shell-library/*

We certainly don’t want to maintain a bash and a Python version of config files and the library. Either we keep all scripts that use these in bash or we convert the library and all the scripts that use it to Python. But the tails-shell-library is also used in many other places, so it might not be possible to move them to Python.

Some scripts that are sourced by other scripts are

10-tbb:. /usr/local/lib/tails-shell-library/tor-browser.sh
11-localize_browser:. /usr/local/lib/tails-shell-library/tor-browser.sh
11-localize_browser:. /usr/local/lib/tails-shell-library/common.sh
11-localize_browser:. /usr/local/lib/tails-shell-library/localization.sh
11-localize_browser:. /etc/amnesia/environment
13-override-tbb-branding:. /usr/local/lib/tails-shell-library/tor-browser.sh
20-xul-ext_symlinks:. /usr/local/lib/tails-shell-library/tor-browser.sh
98-remove_unwanted_files:. /usr/share/amnesia/build/variables

#5 Updated by intrigeri 2016-09-10 04:46:25

> What about the scripts in tails/auto? At least auto/build and auto/config are quite complex shell scripts.

I’d love to see them ported to Python. Beware though: I suspect that live-build (itself written in shell) sources these scripts, so it might be that we need at least a thin shell wrapper if we’re going to port most of that code to Python.

> We certainly don’t want to maintain a bash and a Python version of config files and the library. Either we keep all scripts that use these in bash or we convert the library and all the scripts that use it to Python.

Agreed.

The same config files could be usable both in Python and shell though, if they only contain static data encoded in some standard format.

> But the tails-shell-library is also used in many other places, so it might not be possible to move them to Python.

> Some scripts that are sourced by other scripts are
>

> 10-tbb:. /usr/local/lib/tails-shell-library/tor-browser.sh
> 11-localize_browser:. /usr/local/lib/tails-shell-library/tor-browser.sh
> 11-localize_browser:. /usr/local/lib/tails-shell-library/common.sh
> 11-localize_browser:. /usr/local/lib/tails-shell-library/localization.sh
> 11-localize_browser:. /etc/amnesia/environment
> 13-override-tbb-branding:. /usr/local/lib/tails-shell-library/tor-browser.sh
> 20-xul-ext_symlinks:. /usr/local/lib/tails-shell-library/tor-browser.sh
> 98-remove_unwanted_files:. /usr/share/amnesia/build/variables
> 

I think that those should be ported to Python eventually: chroot local hooks prompted this effort, see for example our initial porting example (feature/11198-python-scripting) :)

#6 Updated by intrigeri 2017-09-28 12:01:07

  • Assignee changed from sycamoreone to kurono

(as per updated roadmap, see parent ticket for details)

#7 Updated by intrigeri 2017-09-28 12:01:21

  • Target version set to 2019

#8 Updated by intrigeri 2018-09-14 08:25:24

  • Description updated
  • Assignee changed from kurono to hefee

As per summit 2018. kurono, feel free to re-add yourself to the team :)

#9 Updated by intrigeri 2020-04-01 09:01:53

#10 Updated by intrigeri 2020-04-01 09:03:16

  • Assignee deleted (hefee)
  • Target version changed from 2019 to 2020

(As per the “Assignee” section of https://tails.boum.org/contribute/working_together/roles/foundations_team/#tasks-management)