Software delivery and updates¶
How a portal gets its software and how a new release is installed: on a Standard installation's one node, and on an Enterprise installation's two.
The only path¶
flowchart LR
D[Development] -- "a release tag" --> G[(The release bundles<br/>on the licensing portal)]
G -- "each node downloads its bundle" --> N1[App VM, dc1]
G -- "each node downloads its bundle" --> N2[App VM, dc2]flowchart LR
D[Development] -- "a release tag" --> G[(The release bundles<br/>on the licensing portal)]
G -- "each node downloads its bundle" --> N1[App VM, dc1]
G -- "each node downloads its bundle" --> N2[App VM, dc2]
- A release carries a tag
vNN.NN.NN(for examplev01.00.05). - Each node downloads its release as a bundle from the licensing portal (my.claugine.com), the bundle of its edition, Standard or Enterprise, and verifies its checksum before unpacking it. Nothing is cloned from GitHub (see Licensing). Settings → Portal shows Releases: bundles from the control plane, the installed version and the latest release; there is no source repository to set. Platform users see the portal's version at the right of the bottom line of every page.
- Installing a release: with automatic updates on, the portal installs a new release by itself, the same way as
Settings → Portal → Update portal, which checks in and installs the newest release offered to this
installation: on two nodes one node at a time (below), on one node after a backup. The Standard edition installs in
its weekly window. The steps of every operation are in Administration → Logs. Break-glass only
(the portal's UI is down):
sudo claugine-portal-update --ref <release>on a node, as a bundle too. - Old releases go on hold by themselves: releases older than the oldest version an installation in use runs. A newer release stays offered even when nobody runs it, and with no installation in use nothing is held. Naming a release on hold is refused with "on hold or not offered to this installation"; an installation can still download the version it runs.
- The oldest releases are withdrawn in time, once no installation runs them: their builds are deleted. Naming a
withdrawn release (
--ref) stops with "vNN.NN.NN was withdrawn: its build is no longer available." Nothing is unpacked. - Both refusals end with "Nothing was changed. Install the release the control plane offers instead (the portal's update does)."
- The nodes never pass code to each other. When a node needs the version the other node runs, it fetches that version itself.
- No installing backwards: neither the portal nor a restore installs a release older than the one it runs. The backup an update takes first keeps the data from before the update (see Update portal).
Standard: one way, automatic¶
A Standard installation installs releases in one window a week: Sunday, 07:02 local time, one hour wide. The hour is not a grace period you can use — the portal checks every ten minutes, so an exact one-minute window would be missed most weeks.
You cannot change the day, the time, or the release stream: Standard is pinned to stable releases. Settings → Portal shows what is pinned, as "Update stream stable only · installed in one window a week, Sunday 07:02".
The portal checks in with the licensing portal every ten minutes, whatever day it is — that is how the licence state and the edition stay current. Installing a release is the part that waits for the window.
When a release is due, the node downloads its bundle, installs it over the running portal, applies any database changes and restarts the service. There is no second node to move traffic to, so the portal is briefly unavailable.
A super admin can ask for a check immediately. That check goes ahead outside the window: it refreshes the licence state and the edition, and installs a release if one is due.
A released installation keeps running but receives no further updates (see Licensing).
Enterprise: two nodes¶
An Enterprise installation chooses its release streams and the time, can hold a version, and installs a release on its standby before its active node.
The rules¶
- Both nodes run the same version, except in the middle of an update. A switch-over by hand is refused while tasks run or when the nodes run different versions.
- One node at a time: the standby installs first; it takes over; then the other node installs.
- The database changes of a release run once, on the node that takes over, before it serves anyone. A standby's database replicates.
- A backup first: an update begins with a full backup.
Maintenance¶
Updating the portal or its OS (on one node or two), the edition upgrade, changing the encryption key, Add second node and Replace node are maintenance. While one of these runs, the portal starts no new work: a provisioning, a network change, a deletion, a configuration check, an igw apply, an archival or a task asked for meanwhile is refused with "Maintenance is running: try again when it ends" (the API answers 503). The operation first stops new work, then checks that nothing is running (else it does not start: wait until the work ends), and lets work start again when it has finished, done or failed.
On a pair, Switch over and both halves of Update portal and Update OS stop new work the same way before they check that nothing runs, and keep it stopped through the change of roles; the node that takes over starts no new work either until the operation has ended. A platform's Make consistent (Infrastructure → the site's Platforms) counts as running work: an update waits for it (it does not start while a repair runs), and Make consistent is refused while maintenance runs. A repair cut off by a restart is shown as interrupted - run Make consistent again.
A portal or platform backup that is running counts as running work too: an update or a switch over of the pair waits until it ends, like a deployment. While maintenance runs (an update, a switch over), no scheduled backup and no Back up now starts: the backup is made once the maintenance has ended. The backup an update makes before it installs is part of that update and runs. See Backup and restore.
Update portal¶
Settings → Portal → Update portal on two nodes:
- a backup;
- the standby installs the release;
- the standby takes over: the database changes of the release run there, once, before it serves anyone;
- the former active node — the standby now — installs the release;
- the two switch back.
The portal is unavailable for about a minute at each switch. On one node, Update portal installs after a backup.
There is no "new version under test" any more. The switch back only returns the original node to the active role: both nodes then run the new release.
The portal never installs a release older than the one it runs - neither Update portal nor automatic updates: a release's database changes only go forward. There is no way back to an earlier release: neither the portal nor a restore installs one. The backup every update takes first (Update portal, Update OS and automatic updates) keeps the data as it was before the update: restored on a freshly installed VM (Backup and restore), it comes back on the current release, with that release's database changes applied. Changes made in the portal after that backup are not in it.
Update OS¶
Settings → Portal → Update OS, the same way on two nodes: the standby's OS updates (rebooted if they need it); the switch over; the other node's OS updates (rebooted if needed); the switch back. On one node, Update OS installs and reboots if needed, the portal unavailable meanwhile.
Automatic¶
Settings → Portal → Updates → Automatic updates (the stable or all-releases stream): a new release is installed by itself, the same way as Update portal. When a step fails, the update stops there (a switch that fails before the other node took over leaves the first node serving), and the super admins get an e-mail.
Safeguards of the pair¶
- One operation at a time on a node (an update, a switch over, an OS update, a key change); one cut off by a restart is shown as interrupted (with the step it reached) and no longer blocks the next one.
- While a project is being archived, portal updates and OS maintenance wait for it, as for other running work. If an archival is cut off anyway (the node restarted), the project is saved again, its page says so, and the daily round archives it.
- A switch over that stops after the active node went read only ends with one node serving: when the other node took over, this node follows it; when it did not, this node serves again once the other confirms it stepped back; when the other does not answer, this node stays in maintenance until it is known which node serves - never two active nodes - and recovers by itself as soon as the other node answers (see When a switch over stops).
- An update that stopped half way (the nodes on different releases) is finished by Update portal, and once by automatic updates: the other node gets the release and the two switch back. The second half is recorded as pending before it starts and cleared only once the two nodes switched back. If it is cut off - the node restarts or its agent stops - even after both nodes already run the new release, Settings → Portal shows the update as not finished and Update portal finishes it.
- The database copy for the other node is checked (complete, length and checksum) before the database it replaces is dropped.
- Each node knows the other's certificate: recorded when the pair is set up (or a node replaced), or at the first contact after this release; another certificate is refused before anything is sent. A node reinstalled with a new certificate is paired again with Replace node.