Tails

#3 Updated by intrigeri 2015-04-19 06:08:33

> Status changed from In Progress to Resolved

> I couldn’t get any satisfying answer…

Maybe we need to do polling on our side, then?

#4 Updated by sajolida 2015-04-22 11:03:21

But they don’t even have an updated RSS feed… so we could write a
script that parses their webpage a notifies us whenever they have an
update. Is that it?

I could give it a try in ruby-nokogiri.

#5 Updated by intrigeri 2015-04-23 01:39:04

> But they don’t even have an updated RSS feed… so we could write a script that parses their webpage a notifies us whenever they have an update. Is that it?

This would indeed be a solution to the problem at hand :)

> I could give it a try in ruby-nokogiri.

I think that writing and maintaining new code is not needed. Looking for a link that matches that regexp on the UUI web page, and comparing the captured substring with the known last version, should be enough:

"$":http://www\.pendrivelinux\.com/downloads/Universal-USB-Installer/Universal-USB-Installer-([0-9.]+)\.exe

Using uscan (part of the devscripts package) with a suitable --watchfile option, and perhaps a fake, dynamically generated Debian source tree with just the needed files, should be enough. There are probably other suitable already existing programs in Debian. If you have trouble using/configuring uscan, feel free to reassign to me: I know this tool a little bit and could treat this task as funny procrastination ;)

#8 Updated by intrigeri 2015-04-29 08:26:58

> Cool! I did that and it worked great.

:)

> See the script in attachment.

I had a quick look:

• Given what follows, I guess we’ll want to use a saner programming language for this script before we extend its functionality. So I won’t comment on the shell bits themselves.
• Creating temporary files with fixed names is bad practice and can have security implications depending on how the script is run. Better use mktemp to set up the working directory.

> Now I wonder how we could integrate that on our website…

> We could maybe have this script running as a cronjob somewhere on lizard. In case of updates it would download the new UUI, commit it to a Git repo, and send an email notification. Then we (starting with me) would pull this repo, download UUI from another source, compare their hashes, and if they match, push the new version in an underlay of the website.

Yep. We can as well have the script do the double download + comparison itself (e.g. one download with Tor, and the other without) before it adds the file to some Git repo we would pull from. I’d love to see uscan offer this functionality, but I’m not dreaming: I doubt any of us has time to add it there.

> But maybe that’s overkill :)

If we want to self-host UUI, the more we automate, the better: IMO the only manual part should be testing that the new UUI works fine for our use cases (until we have automated tests running on Windows!), merging and pushing to the live website. We’re already spending way too much time doing robots’ work, let’s not add to the pile.

#11 Updated by intrigeri 2015-04-30 08:45:36

> Disclaimer: this script was only a proof-of-concept […]

Sure, well understood :)

>> I guess we’ll want to use a saner programming language

> Which one are you proposing?

We’re usually doing Python, Ruby or Perl for this kind of things. Dependencies in Jessie + backports (and Wheezy + backports if you need to see this live soon, as our Jenkins slaves are still running Wheezy).

> couldn’t make uscan work from temporary directories only because it needs to store the state of the previously fetched URL to be able to know whether it has been updated or not.

Really? I thought uscan relied on the last version found in debian/changelog, and both my past experience with it and uscan’s manpage seem to confirm this, so I’m a little bit confused.

Assuming I’m correct, here’s the way I would do it. I would store in a web-accessible location the version number of the UUI we’re distributing (in a plain-text file, to avoid having to look at directory content and parsing filenames), and when creating the fake Debian source package, I would retrieve that information and write it to debian/changelog so that uscan effectively compares the last available upstream version with the version we’re distributing. And then, once downloaded and verified, it would add new .exe and update that version number in the underlay repo it can push to.

> * where would this be run

From the top of my mind, either a cronjob on misc.lizard, or a Jenkins job. Preferably the latter, so that we benefit from its tracking of success/failure, stats, etc. But this shouldn’t matter much from your perspective anyway => if it does, please clarify what system requirements you have, that would be affected by our answer to this question :)

> * which directory structure would you find acceptable for that

Ideally, none at all unless that’s strictly needed. If persistence is required, then we’ll give you a /var/lib/whatever directory and your stuff will have to manage it properly. We can easily automated deleting old files on our side, if it helps. But as said above, I don’t think that any persistent state is needed here.

> * to which Git repo it would commit

If you need a repo for the new underlay, we’ll create it :)

> * whether we need a Git repo for that script

Yes.