Invalid Mutation Error


(Ami Pandat) #1

Hello,
How do I know that my data is in correct JSON format?
Because I am trying to run my JSON formatted data which I have converted from csv,but it is showing an error that invalid mutation.


(Michel Conrado) #2

We have docs related to JSON usage https://docs.dgraph.io/mutations/#json-mutation-format


(Ami Pandat) #3

I have converted my csv formatted data into JSON which contains 98522 triples. So when I add 20-25 triples it works fine but as I increase the data size it is showing rpc error: Transaction Aborted.
Is there any problem with data size?


(Michel Conrado) #4

I don’t thing so. I have a small project with a big JSON(see image below) and I run it via dgraph-js - It is fast and has no issues. This tree is a set of cities and gps localizations. Has more than 5k cities and double of gps track.

52

I performed this in a single transaction.


(Ami Pandat) #5

I want to perform data scaling operation. Like, 1Gb dataset[25 lac triples] then 5Gb dataset. But in Mozilla this 200mb dataset[98522 triples] is not running so any alternative way if you can suggest.


(Michel Conrado) #6

You can use LiveLoad https://docs.dgraph.io/deploy/#live-loader
It accepts JSON load.


(Manish R Jain) #7

Aborts can happen if you’re writing to the same data from different transactions. When you run live loader, try to set the concurrency so there are very few aborts. Aborts would cause your load speed to go down significantly, so avoiding them is the best. If you set the concurrency to 1, there’d be no aborts. So, start with the default, then go down until you reach a sweet spot (hopefully greater than 1).


(Ami Pandat) #8

Live loader doesn’t accept JSON Data. It only handles RDF N-Quad.


(Michel Conrado) #9

Yeah, but it is on master. You can use the Docker tag “master” to use the binarie just for this liveloader instance.

#pick the flag
-f, --files string    Location of *.rdf(.gz) or *.json(.gz) file(s) to load
[node1] (local) root@192.168.0.23 ~
$ docker run -it -p 5080:5080 -p 6080:6080 dgraph/dgraph:master dgraph live -h
Unable to find image 'dgraph/dgraph:master' locally
master: Pulling from dgraph/dgraph
124c757242f8: Already exists
9d866f8bde2a: Already exists
fa3f2f277e67: Already exists
398d32b153e8: Already exists
afde35469481: Already exists
275fe35919d5: Already exists
43b451c9b8c2: Pull complete
cf89c21a4062: Pull complete
Digest: sha256:7aad04f3895a3fdca1eac9b5a06c6d63d40312aa9e95b3af7af0a1e488050f79
Status: Downloaded newer image for dgraph/dgraph:master
Run Dgraph live loader

Usage:
  dgraph live [flags]

Flags:
  -a, --auth_token string        The auth token passed to the server for Alter operation of the schema file
  -b, --batch int                Number of N-Quads to send as part of a mutation. (default 1000)
  -c, --conc int                 Number of concurrent requests to make to Dgraph (default 10)
  -d, --dgraph string            Dgraph alpha gRPC server address (default "127.0.0.1:9080")
  -f, --files string             Location of *.rdf(.gz) or *.json(.gz) file(s) to load
      --format string            Specify file format (rdf or json) instead of getting it from filename
  -h, --help                     help for live
  -i, --ignore_index_conflict    Ignores conflicts on index keys during transaction (default true)
      --new_uids                 Ignore UIDs in load files and assign new ones.
  -s, --schema string            Location of schema file
      --tls_cacert string        The CA Cert file used to verify server certificates.
      --tls_cert string          (optional) The Cert file provided by the client to the server.
      --tls_key string           (optional) The private key file provided by the client to the server.
      --tls_server_name string   Used to verify the server hostname.
      --tls_use_system_ca        Include System CA into CA Certs. (default true)
  -C, --use_compression          Enable compression on connection to alpha server
  -x, --xidmap string            Directory to store xid to uid mapping
  -z, --zero string              Dgraph zero gRPC server address (default "127.0.0.1:5080")

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --bindall                          Use 0.0.0.0 instead of localhost to bind to all addresses on local machine. (default true)
      --block_rate int                   Block profiling rate. Must be used along with block profile_mode
      --config string                    Configuration file. Takes precedence over default values, but is overridden to values set with environment variables and flags.
      --cwd string                       Change working directory to the path specified. The parent must exist.
      --expose_trace                     Allow trace endpoint to be accessible from remote
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --profile_mode string              Enable profiling mode, one of [cpu, mem, mutex, block]
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

Note: Don’t use master for other purposes. Just to use liveloader. You can also use Bulkloader that already has JSON support.


(Ami Pandat) #10

Okay…but without docker I can’t use …right?


(Michel Conrado) #11

Yes you can! (LOL) - Just build Dgraph from master branch. On Dgraph’s Readme there’s the steps to do it.


(Ami Pandat) #12

ohh, okay okay…!
Thanks a ton to answer all my questions(some silly). I will surely mention your team in my thesis acknowledgement.


(Ami Pandat) #13

Hey,
To build Dgraph from master’s branch I have followed README.md and it shows that you have already installed it. So does it mean that version v1.0.13 is from master’s branch?
Because Dgraph live does not have flags --files in my installed version.


(Michel Conrado) #14

When you build from master you gonna have a bin at ~/go/src/github.com/dgraph-io/dgraph/dgraph path. You need to move this bin to a desired place and run like so:

./dgraph something...

No, master is master. Master is like v1.1 not v1.0.xx

Cheers.