Bug #16142
Consider giving sajolida access to the web logs of our website
0%
Description
I used to use internal.git:stats/logs/retrieve-logs.rb to retrieve the logs of our web server.
The last logs I can fetch with this tool date from November 11 (migration of the website?).
I need logs to work on Feature #14922, Bug #16009, and Bug #16010.
- I can continue working on
Feature #14922with the logs until November 11. - I’ll need logs for several weeks prior to 3.12 for
Bug #16009. Ideally starting from 3.11.
Tell me if I can help fix that.
Subtasks
History
#1 Updated by intrigeri 2018-11-19 14:00:58
- related to
Feature #14588: Self-host our website added
#2 Updated by intrigeri 2018-11-19 14:01:13
- blocks Feature #13284: Core work: Sysadmin (Adapt our infrastructure) added
#3 Updated by intrigeri 2018-11-19 14:10:55
- Assignee changed from intrigeri to sajolida
- QA Check set to Info Needed
Oops! When migrating the website, I remembered that we need these logs at least to get our monthly email with boot stats, so I took care of making an exception to our usual web log policy (keep logs for 24h) and ensured we keep 64 days of logs, like the previous setup did. But indeed, I forgot that there are other needs for these logs. Thanks for reminding me :)
> * I’ll need logs for several weeks prior to 3.12 for Bug #16009. Ideally starting from 3.11.
Rest assured no data has been lost: combining the logs from the previous setup with the new ones, so I think everything you need is in there.
In order to organize my work on this ticket I have a few questions:
- Short-term:
- When do you need these logs, latest, for this work? Please set the target version and due date accordingly.
- Will you need them once or more?
- Long-term: how often do you need to do stuff with these logs? The answer will help us tell whether we can justify investing time into giving you direct access: if your problem could be solved by “dear sysadmins, can you please run this script on the logs and send me the output?” or “dear sysadmins, can you please share a tarball of the logs with me?” twice a year, and our sysadmins answer with a decent latency, it may take centuries before there’s a good return on investment on the more involved solution :)
#4 Updated by sajolida 2018-11-19 17:44:42
> When do you need these logs, latest, for this work? Please set the target version and due date accordingly.
For this work I’m fine with trying to give you some scripts to run.
- For the download metrics I anyway wanted to write a script so we can
compute them on a regular basis. - For the ‘back’ metrics, I can also give you a script to run.
But I would need an extract from the logs after we mergedBug #16010and
before 3.12 so I can test my script.
> Will you need them once or more?
For this work, only once to report to the funder, plus getting an
extract to work on the ‘back’ metrics before 3.12.
> * Long-term: how often do you need to do stuff with these logs?
I remember using them:
- For the donation campaign analysis. I could also maybe try to write
some scripts for that after playing with an extract. - To experiment with Matomo. But I think I’m done with this for some
time… - Some other UX needs (search terms, etc.) but this is rare and
unscheduled.
I’m fine with not stressing you out with building a plan to give me
direct access right now and see how it goes to send you scripts to run
from now and then.
So in the end I’m not sure how to set the metadata for this ticket :)
#5 Updated by sajolida 2018-11-19 17:44:56
- Assignee changed from sajolida to intrigeri
#6 Updated by intrigeri 2018-11-20 15:06:57
> * For the download metrics I anyway wanted to write a script so we can compute them on a regular basis.
Happy to make it run via cron or similar so it sends us the data regularly: for things you would do regularly, the initial investment could be worth it.
>> * Long-term: how often do you need to do stuff with these logs? […]
> I’m fine with not stressing you out with building a plan to give me direct access right now and see how it goes to send you scripts to run from now and then.
OK, let’s do this and we’ll see how it goes. Note that “you” can be plural i.e. the sysadmins team :)
I’ll send you a sample of our current logs so you can ensure your scripts support their format.
> So in the end I’m not sure how to set the metadata for this ticket :)
Let’s leave it as-is for now and whenever you get bored or pissed of having to go through sysadmins, bump it back onto my less-long-term radar.
#7 Updated by intrigeri 2018-11-20 15:07:41
- Subject changed from Cannot retrieve web logs after November 11 to Consider giving sajolida access to the web logs of our website
- Priority changed from Normal to Low
- QA Check deleted (
Info Needed)
#8 Updated by intrigeri 2018-11-20 15:07:46
- blocked by deleted (
Feature #13284: Core work: Sysadmin (Adapt our infrastructure))
#9 Updated by sajolida 2018-11-21 14:33:21
> I’ll send you a sample of our current logs so you can ensure your scripts support their format.
Just to be extra explicit:
- I already have old logs to work on the downloads stats (
Feature #14922). - To work on the ‘back’ stats (
Bug #16010) I have to wait until it’s merged.
So I’ll ping you after Bug #16010 is merged to ask you for a bit of the more
recent logs.
#10 Updated by intrigeri 2019-04-07 09:31:20
- Assignee changed from intrigeri to sajolida
- QA Check set to Info Needed
5 months later, how has it been not to have direct access to the web logs?
It would be sweet to identify the queries you have to run regularly on these logs, so we automate the process and give you continuous access to up-to-date results :)
#11 Updated by intrigeri 2019-06-02 15:17:02
- Status changed from Confirmed to Rejected
- QA Check deleted (
Info Needed)
Please reopen if practice teaches you that you really need this.