Add store_xids to Liveloader

Issues Dgraph
, , ,
1

Moved from GitHub dgraph/5106

Posted by MichelDiz:

Experience Report

This ticket is related to #4917

–store_xids:

A feature that exists only in Bulkloader. It takes the XID or blank node and saves it to the same node as a property of it with the predicate name as “xid”. e.g.:

<_:MichelDiz> <name> "Michel" .

It will be recorded as 

{
  "data": {
    "q": [
      {
        "name": "Michel",
        "xid": "_:MichelDiz"
      }
    ]
  }

This functionality at first does not seem useful. But I’m sure it’s related to the approach on external IDs https://docs.dgraph.io/mutations/#external-ids

e.g.:

<0x321> <xid> "https://www.themoviedb.org/person/32-robin-wright" .

It can be useful in this case and we can use Upsert Block. But it is not useful for those who need to ingest large amounts of data. Only small cases.

2

MichelDiz commented :

Observation: One thing I noticed in recent tests is that store_xids stores only real XIDs. I think It should store any kind of value except UIDs in fact. So my example in this issue is slightly wrong. Cuz I’m exemplifying with a blank node <_:MichelDiz> not a XID.

Take for example the GraphQL feature Adding/Updating nodes for XID inside XID doesn't work if parent XID already exists · Issue #5128 · dgraph-io/dgraph · GitHub
The dev team behind the GraphQL feature is using XIDs that are not actually XIDs.
In general, XIDs are basically “URIs” from RDF concept. But we can extend this for any value except UIDs.

a bit of context XID in other graph DBs

We should make sure that these syntaxes are properly handled by store_xids.

<_:BlankNode> 

<eg:a> <eg:b> <"123"^^http://www.w3.org/2001/XMLSchema#int>
<eg:a> <eg:b> <"123"^^http://www.w3.org/2001/XMLSchema#unsignedByte>
<urn:lot2> ... ... .

<http://my.com/John> <http://purl.org/dc/elements/1.1/publisher> <http://publisher/
Xyz> .

<employee> 

 <employee/name>