Feature request: Schema/data migration tool

Users Dgraph
1

A lot of frameworks have built in database-migration libraries:

https://hexdocs.pm/ecto_sql/Ecto.Migration.html

As my schema evolves I’m constantly running into an issue where my existing data becomes invalid. For example, if I add an external id / xid (@id) to a type, then all nodes of that type in my graph are now invalid and will result in non-nullable field errors:

type Thing
  id: ID!
  xid: String! @id
}

{
  "errors": [
    {
      "message": "Non-nullable field 'xid' (type String!) was not present in result from Dgraph.  GraphQL error propagation triggered.",

It’d be great if there was a tool that worked like Rails migrations, where I could generate a new timestamped migration script that runs relevant logic that updates (or rolls back) the schema, and optionally modifies existing data based on the schema change.

In my above example, a migration script would modify the Thing schema, then I would write some migration logic that iterates through all Things in the graph and adds a UUID value.

EDIT: The docs actually acknowledge this problem: Schema Migration - GraphQL

Then it might error out saying:

A list was returned, but GraphQL was expecting just one item. This indicates an internal error - 
probably a mismatch between the GraphQL and Dgraph/remote schemas. The value was resolved
as null (which may trigger GraphQL error propagation) and as much other data as possible returned.

So, while making such a schema change, you need to make sure that the underlying data really honors the uniqueness constraint on the username field. If not, you need to do a data migration to honor such constraints.

No suggestion on how to perform a data migration is given however.

Perhaps Dgraph could have a similar tool to Fauna’s migration tool?

EDIT: For reference: it seems like the official solution right now is to perform upserts via Ratel?

EDIT: Well, this was my solution… a temporary endpoint in my application. Feels really hacky. Hopefully there’ll be a better solution for this soon. :slightly_smiling_face:

import { dgraphClient } from '$lib/configs/dgraph-client.ts';
import { v4 as uuidv4 } from 'uuid';

export async function post(request) {
  const wordQuery = `{
    words(func: type(Word)) {
      uid
    }
  }`;

  const words = await dgraphClient.newTxn().query(wordQuery);
  const mutationJson = [];

  words.data.words.forEach(word => {
    mutationJson.push(
      {
        "uid": `${word.uid}`,
        "Word.xid": `${uuidv4()}`
      }
    )
  });

  const txn = dgraphClient.newTxn();
  const mutation = await txn.mutate({ setJson: mutationJson, commitNow: true });

  return {
    body: {
      done: true
    }
  }
}
1 Like