Moved from GitHub dgraph/4644
Posted by sleto-it:
Experience Report
This is a feature request to add an additional upgrade method, the in-place upgrade, which can coexist with our current logical upgrade method
When the in-place upgrade will be available, this is a FR to support rolling upgrades (one node at a time) of Dgraph clusters, so that the upgrade can be done while the cluster remains up and running, and ideally while it keeps serving reads and/or reads and writes
What you wanted to do
- Upgrade in an easy way a running Dgraph cluster, especially when upgrading to a new patch release (e.g. from 1.1.0 to 1.1.1)
- Minimize operational efforts
- Minimize downtime
What you actually did
The upgrade is currently possible, but it always requires an export and import (logical upgrade)
Why that wasn’t great, with examples
While it is understandable and acceptable that sometime a logical upgrade might be required (e.g. due to some incompatible changes, to re-shard the cluster, etc), it is important to improve the upgrade experience as much as possible e.g. by introducing a new upgrade method: the in-place upgrade, which would allow the installation of the new package/version and the reuse of the current data directory. The upgrade process can be responsible to run some in-place upgrade operations in the existing data directory, when needed
In-place upgrade will then hopefully also allow rolling upgrades, where we upgrade one node at a time, the cluster remaining up and operational in the meantime
The problem related to the current upgrade approach is that, in addition to the export and import, it requires to set up a new cluster, if you want to minimize downtime (blue-green deployment for upgrades) - which, from an operational point of view, is costly in terms of required time and efforts