Better breaking change documentation

Documentation
#1

I think that the past few releases have been unclear about breaking changes, and the steps required when upgrading

It would go a long way if the release notes contained more insightful (and complete) information about what has changed.

From my own experience:

When upgrading from v20.07 to v20.11:

I was unaware I needed to export and reimport data

When upgrading from v20.07 to v21.03:

I was unaware that some flags had changed:
On the alpha the whitelist, lru_mb and mutations flag had changed.
On the zero, the idx flag had changed

None of this came up during my research prior to upgrading, and if it had, I could easily have prevented a lot of unnecessary downtime.

I’m sure many others would really appreciate better release notes as well :slight_smile:

2 Likes
#2

All major releases we recommend to export and reimport. Cuz the potential of having a breaking change is big.

This idea started here How to deal with flags in Dgraph

Hey @docs team, maybe we could have a place to share breaking info. We already note the breaking changes in the PRs. What do you guys think?

PS. Also ping @hardik.

#3

Yeah, if we didn’t clarify the flag changes, that seems like a documentation oversight. @vvbalaji

#4

On the topic of flags (command line arguments) breaking change, this is what is there:

In the Changelog (Releases · dgraph-io/dgraph · GitHub), this is what was there for superflags:

I could not find any mention of Resilient Rocket on Dgraph Blog landing or on the blog with dgraph tag (https://dgraph.io/blog/tags/dgraph/. In the past, for Tenacious T’Challa, there was: Dgraph v20.11: Tenacious T’Challa Release - Dgraph Blog

In announcements (Release Notes v21.03.0 - Resilient Rocket), for new flag format, there’s this:

  1. Flags Consolidation

On the general topic about upgrades and the need to backup/export and restore/load, it was once mentioned in the changelog, for example https://github.com/dgraph-io/dgraph/releases/tag/v20.03.0:

Note: This release requires you to export and re-import data prior to upgrading or rolling back. The underlying data format has been changed.

When Google searching for the topic dgraph upgrading or dgraph upgrade, there will be two links to docs and other links dicuss posts and github issue. For the documentation links, there were:

The other links that appeared in the Google search were customers inquiring about upgrades or asking for a better experience regarding upgrades (or upgrade documentation):

#5

I second this. Just having a “Quick upgrade checklist” section together with release notes would be enough, I think. It could read something like this:

Quick upgrade checklist for Dgraph v20.11 to v21.03:

  • Data export and import required
  • Most Alpha and Zero flags changed
  • Ratel removed from docker image (use old img for ratel until further notice)
1 Like
#6

This is a quick brain storm on ways the experience can be improved for upgrades and breaking changes:

  • Release/Changelog: Especially for breaking changes, more details on these (e.g. more details in commit string). In discuss posts, there can be further details, with links to docs.
  • Blogs: Released at time of engineering release, as this could have details on features and breaking changes.
  • Docs: In the top of navigation, have a landing zone for breaking changes and upgrades. This can link to pages elsewhere in the docs, but bring it all together under one area. Google searches can link to this.
  • SEO (Google Search): A few searches link to the new landing zone in docs.
  • Roadmap: For huge datasets (700+ GB), the backup/restore process could take hours, so mutations would need to be disabled during the process. Could in-place upgrades help in this?

cc’ing @hardik @zhenni @dmai @vvbalaji @Vijay @docs

3 Likes
#7

These are good inputs. I will update current release notes to be more informative from “Breaking Changes” perspective. Meanwhile you can refer to following documents to understand the Flag changes and Process of Upgrades

Super Flags : https://dgraph.io/docs/deploy/cli-command-reference/#dgraph-cli-superflags-in-release-v2103

Upgrading to 21.03 : https://dgraph.io/docs/deploy/dgraph-administration/#upgrade-from-v20110-to-v21030-for-enterprise-customers

1 Like
#8

We are continuosly innovating and improving the storage, compaction, indexing and other aspects of the product. So it is fair to anticipate that the underlying file system storage is not compatible across newer releases. As indicated we will do a better job on highlighting this in release notes and docs to set the right expectatoins. It would be comptabile for a patch release.

Flags: Thanks for your feedback.