Feature #11151
Speed up the build of the website
100%
Description
As suggested in Bug #11135#note-5 we could measure the time to build the wiki in the past (without the Installation Assistant for example) and now, and try to set a realistic goal.
Then we should investigate on concrete changes to speed things up, like Bug #11135.
Subtasks
Feature #10792: Try to group inc/steps inlines | Resolved | 0 |
|||
Bug #11135: Disable translations of /news/reports | Resolved | 100 |
|||
Feature #11156: Disable translations about very old releases | Resolved | 100 |
|||
Feature #11158: Disable backlinks on website | Rejected | 0 |
|||
Bug #11227: Website: Broken links at https://tails.boum.org/security/ | Resolved | sajolida | 100 |
||
Feature #11249: Do stats on which pages are the least translated | Resolved | 0 |
Related issues
Related to Tails - |
Resolved | 2015-11-27 |
History
#1 Updated by sajolida 2016-02-21 17:54:27
- Tracker changed from Bug to Feature
- Subject changed from Speed the build of the wiki to Speed the build of the website
#2 Updated by sajolida 2016-02-21 19:52:38
Reference time, on a X200 with a P8700, building from RAM on Tails takes at commit edf97d3:
real 48m53.368s
user 48m4.740s
sys 0m15.344s
#3 Updated by intrigeri 2016-02-21 20:27:08
On intri’s laptop:
current
git reset —hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7 && time ./build-website —rebuild
1036.49s user 6.94s system 100% cpu 17:17.74 total
old school (no Farsi, no IA)
git reset --hard caa53db1bf9f0157207ba92f56daf23f527ac6e3^ && time ./build-wiki --rebuild
334.42s user 3.51s system 99% cpu 5:38.40 total
current minus Farsi
git reset --hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7
patch -p1<<EOF
--- a/ikiwiki.setup
+++ b/ikiwiki.setup
@@ -199,7 +199,6 @@ po_master_language: en|English
# slave languages (translated via PO files) format: ll|Langname
po_slave_languages:
- de|Deutsch
- - fa|Farsi
- fr|Français
- pt|Português
# PageSpec controlling which pages are translatable
EOF
find wiki/src -name *.fa.po -delete
time ./build-website --rebuild
738.55s user 5.02s system 100% cpu 12:18.17 total
current minus IA
git reset --hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7
rm -rf wiki/src/{install,upgrade}
time ./build-website --rebuild
604.88s user 5.70s system 99% cpu 10:10.96 total
Conclusions
no IA, no Farsi = about 5.5 minutes, and then IA alone adds 7 minutes, Farsi alone adds 5 minutes, and adding both together adds 12 minutes => almost 17.5 minutes
#4 Updated by sajolida 2016-02-21 20:29:17
From the same commit after doing rm -r install upgrade
I get:
real 30m22.595s
user 29m6.884s
sys 0m11.628s
which is 39% less and matches more or less what intrigeri gets.
Let’s try to reduce the build time by 1/3 then.
#5 Updated by sajolida 2016-02-21 21:21:24
After merging Bug #11135 and Bug #9671 I get:
real 43m19.589s
user 41m47.072s
sys 0m15.312s
which is 11% less already.
#6 Updated by sajolida 2016-02-22 17:45:41
Remove the old installation instructions (/install and /upgrade) also speeds about 4%.
#7 Updated by sajolida 2016-02-22 17:46:06
- related to
Feature #9323: Adapt current documentation with the Installation Assistant added
#8 Updated by intrigeri 2016-02-22 20:35:23
At commit:9244ee23feb893aa1bf956b825bf0d0aa225e145 (after merging the branches for Bug #11135, Bug #9671 and Feature #11156) I see:
779.86s user 4.99s system 101% cpu 12:56.60 total
i.e. –25% :)
#9 Updated by intrigeri 2016-02-22 20:35:36
- Subject changed from Speed the build of the website to Speed up the build of the website
- Status changed from Confirmed to In Progress
#10 Updated by anonym 2016-02-23 12:13:41
On my laptop (Intel i7-3520M CPU with the core used by ikiwiki running at 3.2 GHz, and a pretty fast SSD):
current
real 20m23.051s
user 20m39.604s
sys 0m7.780s
old school (no Farsi, no IA)
real 6m18.782s
user 6m20.312s
sys 0m3.664s
current minus Farsi
real 13m2.951s
user 13m19.244s
sys 0m5.444s
current minus IA
real 10m6.676s
user 10m8.244s
sys 0m4.436s
So very comparable figures to what you’ve all seen.
So, in my case the new target (- 1/3) would sill be 20m23s * 2/3 ~= 14m which is still pretty long. :/
#11 Updated by sajolida 2016-03-24 14:00:41
- related to
Bug #11227: Website: Broken links at https://tails.boum.org/security/ added
#12 Updated by intrigeri 2016-03-25 19:47:41
- related to deleted (
)Bug #11227: Website: Broken links at https://tails.boum.org/security/
#13 Updated by emmapeel 2016-04-29 08:35:05
In my Thinkpad the current website takes
real 9m18.094s
user 8m58.084s
sys 0m9.280s
I have 7.6 GiB memory, and an Intel® Core™ i5 CPU M 520 @ 2.40GHz × 4 processor
#14 Updated by sajolida 2016-04-30 14:39:06
- Status changed from In Progress to Resolved
- Assignee deleted (
sajolida)
When building Feature #9323 I get 11% less in comparison with master today, for a total of 25m41.124s which is 47% less than what I had initially. So I can close this ticket :) \o/
#15 Updated by sajolida 2016-05-13 06:57:14
On an X201 with an i3 M350:
real 10m55.939s
user 10m57.772s
sys 0m5.136s
#16 Updated by sajolida 2016-05-13 08:17:59
Today I’ve got 36m20.637s
on my machine. That’s 26% less than my reference time.