Bug #16549

White screen on LimeSurvey administration panel

Added by sajolida 2019-03-08 16:14:08 . Updated 2019-03-11 12:43:29 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Infrastructure
Target version:
Start date:
2019-03-08
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Today I tried to update LimeSurvey from 3.5.3 (Mar 16, 2018) to 3.15.9 (January 14, 2019) and got a white screen on the administration panel starting from 3.10.0 (June 11, 2018):

What I did (with a few more trials and errors in between):

  • Try to update straight to 3.15.9: white screen!
  • Rolled back to 3.5.3: works!
  • Bisected until I identified 3.10.0 as the first version with the white screen.
  • Rolled back to 3.9.0 and updated the database.
  • Updated again to 3.10.0: white screen again!

I also tried to:

  • Have a look at the Apache access and error logs: nothing in there.
  • Restart Apache: both /bin/systemctl restart apache2.service and systemctl restart apache2.service failed, systemctl status apache2.service was not very informative either.

My questions are:

  • How can I further debug this white screen?
  • Could I have more such debugging permissions in the future?
  • Is there anything to fix so I can restart Apache in the future (though it’s probably not the fix for this time)?

Assigning this ticket to intrigeri as a start as he’s the one who set up LimeSurvey, though this sounds more like a task for the sysadmin on duty.

This is blocking me for the VeraCrypt success metrics (Feature #15977) that are not so urgent but the last remaining bit of Sponsor_W.


Subtasks


Related issues

Blocks Tails - Feature #15977: Quantitative survey on VeraCrypt usage after the release Resolved 2018-09-25

History

#1 Updated by sajolida 2019-03-08 16:14:17

  • blocks Feature #15977: Quantitative survey on VeraCrypt usage after the release added

#2 Updated by intrigeri 2019-03-08 17:16:05

  • Assignee changed from intrigeri to sajolida

@sajolida, I can look into this myself in the next few days if needed but I’ll answer your questions now so you can try stuff in the meantime:

> * How can I further debug this white screen?

I have no idea, never having maintained a LimeSurvey setup myself. So random ideas:

  • Maybe the new version has new dependencies, or depends on newer versions of stuff?
  • Restarting Apache could be worth trying.
  • If you still have the local copy of this service that you’ve used to develop it initially, maybe try to reproduce there? (I’m tempted to prepare a Docker thing that sets up stuff with our Puppet code, as drebs did last year for the translation platform, so you can run, develop & debug your service locally in an environment where you’re root. But that won’t happen overnight.)

> * Could I have more such debugging permissions in the future?

Probably… once we’ve identified what permissions that could be :) If I end up debugging this myself, I’ll try to take notes of what I do as root.

> * Is there anything to fix so I can restart Apache in the future (though it’s probably not the fix for this time)?

You already can, as per the set of permissions you’ve requested & were given (but you need to run a specific command that might not be your first choice): https://git.tails.boum.org/puppet-tails/tree/files/limesurvey/sudo/limesurvey-admin

> Assigning this ticket to intrigeri as a start as he’s the one who set up LimeSurvey,

Fine by me.

> though this sounds more like a task for the sysadmin on duty.

Not necessarily: we have a different budget line for “Help service administrators set up their service”. But we have no process in place to share this work. Next year bertagaz will do most of it so it’ll be clearer who’s your point of contact is. Until then, happy to keep handling the LimeSurvey bits myself.

#3 Updated by sajolida 2019-03-08 17:58:54

  • Assignee changed from sajolida to intrigeri

> You already can, as per the set of permissions you’ve requested & were given (but you need to run a specific command that might not be your first choice): https://git.tails.boum.org/puppet-tails/tree/files/limesurvey/sudo/limesurvey-admin

I tried:

sajolida@survey:~$ /bin/systemctl restart apache2.service
Failed to restart apache2.service: The name org.freedesktop.PolicyKit1
was not provided by any .service files
See system logs and 'systemctl status apache2.service' for details.

which is the only line with restart in
https://git.tails.boum.org/puppet-tails/tree/files/limesurvey/sudo/limesurvey-admin.

If “a specific command that might not be your first choice” is not this
one please be more explicit.

#4 Updated by sajolida 2019-03-08 17:59:10

  • QA Check set to Info Needed

#5 Updated by sajolida 2019-03-08 17:59:57

  • Assignee changed from intrigeri to sajolida
  • QA Check deleted (Info Needed)

Ah, sudo /bin/systemctl restart apache2.service works! Sorry for the noise…

#6 Updated by sajolida 2019-03-09 10:38:59

  • Target version set to Tails_3.13

#7 Updated by sajolida 2019-03-09 11:26:56

  • Assignee changed from sajolida to intrigeri

> * Maybe the new version has new dependencies, or depends on newer versions of stuff?

No changes on this side I think.

> * Restarting Apache could be worth trying.

Done.

> * If you still have the local copy of this service that you’ve used to develop it initially, maybe try to reproduce there? (I’m tempted to prepare a Docker thing that sets up stuff with our Puppet code, as drebs did last year for the translation platform, so you can run, develop & debug your service locally in an environment where you’re root. But that won’t happen overnight.)

I do.

> Probably… once we’ve identified what permissions that could be :) If I end up debugging this myself, I’ll try to take notes of what I do as root.

I thought about PHP errors display but today I managed to activate them and they were none.

Today I found more things to try out (and discareded more things that didn’t work). But I also deleted some files by mistake :(

Can I have some backups? The last version before my deletion today of:

  • application/config/config.php
  • upload

Maybe the password of the MySQL database would be enough if backups are complicated to get.

Sorry for the mess! This adventure will help me write better doc for next time :)

#8 Updated by intrigeri 2019-03-10 09:16:32

  • Priority changed from Normal to Elevated

#9 Updated by intrigeri 2019-03-10 10:05:14

  • Assignee changed from intrigeri to groente

> Can I have some backups? The last version before my deletion today of:

> * application/config/config.php
> * upload

Hi @groente, AFAICT we have no documentation about restoring backups yet. This could be an opportunity for you to draft it, by doing it in a real-world situation and taking some notes. Alternatively, if time is scarce, feel free to just give me a few hints and I’ll do it myself, just let me know :)

The aforementioned files, that sajolida wants restored, live in /var/www/limesurvey on survey.lizard.

> Maybe the password of the MySQL database would be enough if backups are complicated to get.

@sajolida, I’ve just sent it to you. I don’t know when groente can restore the backups so I suggest you either coordinate with him, or try without them.

#10 Updated by sajolida 2019-03-10 20:03:50

  • Assignee changed from groente to intrigeri

I managed to update to 3.10 without the backups so I think I’m fine with only the password. Thanks!

I improved the documentation in d16f763bbb..1694bcd213 to prevent such data loss in the future (and be closer to the upstream recommendations).

The issue was that sometimes you have to update the database on the command line to get the admin panel work again (I don’t know why…).

But I still couldn’t update to 3.13 and this time I think I genuinely need your help and not only a rubber duck :)

I need my php command line to be allowed to allocate more memory. See the output below:

www-data@survey:~/limesurvey$ php application/commands/console.php updatedb
Update mysql:host=localhost;port=3306;dbname=limesurvey; with prefix : from 351 to 352

mmap() failed: [12] Cannot allocate memory
PHP Fatal error:  Out of memory (allocated 134991872) (tried to allocate 4096 bytes) in /var/www/limesurvey/framework/db/CDbCommand.php on line 518

mmap() failed: [12] Cannot allocate memory
PHP Fatal error:  Out of memory (allocated 134991872) (tried to allocate 4096 bytes) in /var/www/limesurvey/framework/base/CComponent.php on line 548

#11 Updated by intrigeri 2019-03-11 07:04:46

  • Assignee changed from intrigeri to sajolida

> I need my php command line to be allowed to allocate more memory.

I’ve given the VM ~256MB more RAM. Please retry :)

#12 Updated by sajolida 2019-03-11 12:43:29

  • Status changed from Confirmed to Resolved
  • Assignee deleted (sajolida)

Yay, I’m in 3.15.9 now!