Recommended way of generating triples in the W3C standard RDF N-Quad format - with variables

Users
,
1

Hello everyone.

The polite context to my question is at the bottom of this post but I’d like to value peoples time and put my question first.

I’ve never worked with triples, or the RDF N-Quad format. I’ve been writing Go on and off for about 3 years and only just decided to go all in on it for my backend services and I’m now using dgo. This question is in that context.

Question:
What are the best practices for generating the string literals for both the JSON and triples for queries and mutations?

The thoughts I have:

1.) I’ve gone through the docs extensively and had no problem with mutations and queries with JSON and the default marshalling libraries until I came across doing conditional upserts as described in the dgo readme (example included for clarity):

query = `
  query {
      user as var(func: eq(email, "wrong_email@dgraph.io"))
  }`
mu := &api.Mutation{
  Cond: `@if(eq(len(user), 1))`, // Only mutate if "wrong_email@dgraph.io" belongs to single user.
  SetNquads: []byte(`uid(user) <email> "correct_email@dgraph.io" .`),
}
req := &api.Request{
  Query: query,
  Mutations: []*api.Mutation{mu},
  CommitNow:true,
}

// Update email only if exactly one matching uid is found.
if _, err := dg.NewTxn().Do(ctx, req); err != nil {
  log.Fatal(err)
}

This was fine as it is but my problems started with the fact there was only an example with setNquads and I wasnt sure how to include the uid(your_var) in JSON based on the result of the query portion of the upsert. Is there a way to do this and can someone please show me an example? In the meantime I decided to use n-Quads instead which lead me to my next question. (I actually prefer n-Quads I think. So I’d love to know the answer to the next one).

3.) How do I do variable subsitution for triples? Is there a good/bad way to do it like with SQL queries to avoid injections etc?

  • Queries have txn.QueryWithVars which works for simple stuff.
  • Mutations dont as far as I can tell.
  • Do I use normal fmt.Sprintf(`query text %s`, cast all my vars from a type here to strings) (what I’ve done for now) and it works - just seems quite… tiresome to do that for all my queries.

4.) Can i use fmt.Sprintf for the JSON queries too? I found it hard to do understand how to use the Vars: property of Txn.do at the same time as setting a var in my query at the top level. So i just did it myself -_-’

Combine:

q := `query all($a: string) {
    all(func: eq(name, $a)) {
      name
    }
  }`

and

query = `
  query {
      user as var(func: eq(email, "wrong_email@dgraph.io"))
  }`

I’ve included what I hacked together (working!) below to show where I got to.

Example Gist

Is there a better way? Are there any good marshalling tools for n-Quads? Should dGraph have some built in or have something like mongoose was for MongoDB but for n-Quads? Sorry, I’ve stored up all my questions 0_0’ around this area.

What I will say though is that if I had this knowledge I feel I can build anything on dgraph :rocket: - docs are solid for the most part (types could do with some work on removing the old documetation pertaining to the older versions dgraph (or with a discalimer)).

Background and Hello!
I assessed dgraph to use about a year and half ago for my businesses and am extremely excited to see how far the project has come since then. Awesome work from all the team!

I’m using dgraph as the main database for my new business (a management platform for high value collectable items) - we’re launching in the classic car market in about 6 months.

Since the platform heavily weighs on the interactions between all players in the market a graph database was a must, and I’m taking the plunge on using one (dGraph) as the main DB (as inspired by an old talk by @mrjn).

2 Likes
2

Hi,

Thank you for the question. I will see if one of our engineers can take a look at your question and we will try and get back to you promptly.

Thanks
Shekar

3

Thanks @Shekar - If I can get all this sorted I’m very happy to help the dgraph ecosystem in any way that I can going forward like I did extensively for FeathersJS. Although David (daffl) and Eric (ekrysk) did pretty much everything haha 0_0’

4

You can serialize your data into JSON (e.g., Go’s json.Marshal) to create the equivalent JSON string based on your data. N-Quad triples is a line-oriented format in the form of <subject> <predicate> <object> . or <subject> <predicate> "value" . per line. You can use string substitution or string builders to create these if needed

You can find an example of referencing uid variables in the JSON-format mutation on the docs for upsert block: Get started with Dgraph

{
  "query": "{ v as var(func: eq(email, \"user@company1.io\")) }",
  "set": {
    "uid": "uid(v)",
    "name": "first last",
    "email": "user@company1.io"
  }
}

You could also include the condition in JSON format with the "cond" field:

{
  "query": "{ v as var(func: eq(email, \"user@company1.io\")) }",
  "cond": "@if(lt(len(v), 1))",
  "set": {
    "uid": "uid(v)",
    "name": "first last",
    "email": "user@company1.io"
  }
}

All three points you mention are correct. If there’s an aspect of GraphQL Variables that you find is missing for queries, let us know and we can look into adding support for it.

If you want to build the JSON strings yourself, you can use fmt.Sprintf.

Typically you’d have a set of pre-existing queries you’ll run for your app that could be configured to accept input values as GraphQL variables. So you’d have the GraphQL
Variables defined in the query string, and you’ll pass the input values as Vars dynamically.

2 Likes
5

Thanks I’ll have a go with all of this tomorrow :slight_smile:

Knowing its normal to do a lot of this just using string substitution makes me feel a lot more comfortable and I can put anything which gets sent from a client through a type to avoid injections as was mentioned on the recent Go Time podcast :slight_smile:

Thank you very much for your time :slight_smile:

6

This feeling I have too :slight_smile: …and as just being a small fullstack webdeveloper it comes true for what I’m doing.
Writing that, I much appreciate your questions, as they bring some more practice to this forum on how we can get more, easier and faster out of this great database.

Queries are just strings, so any way to substitute variables is possible I think.
Most times I prefer just to concatenate strings like

uid := `0x1234`
q := `{
 query(func: uid( ` + uid + `)) { ... }
}`

Using fmt.Sprintf has some overhead that might not be needed, if I know what I want to substitute.
Other ways might be text/template or GitHub - valyala/fasttemplate: Simple and fast template engine for Go. Later provides in my opinion the possibility to use a clear way of named template vars while being high performant.
Neither of these would avoid injections. A function with regexp could do the job but depends of where the input is coming from, what kind of injection might be possible to break the query or allows to leak data that should be kept secret. Thinking about that, I doubt there is a uniform or recommended way to do that.

I think NQuads are great for fine tuned manipulations, eg. setting an edge, updating a counter or only one field of a bigger struct. In any other case I prefer to use JSON. Creating custom (Un-)MarshalJSON methods for schema structs helps me to handle data and schemas to keep functions small and easy to understand which query or manipulate the database. Yet these marshal methods are just hacks, I hope I find some time to compile that to a more general, reuseable approach.

1 Like
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.