# Bulk loader fails to create graphql schema properly

**URL:** <https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746>\
**Category:** Dgraph\
**Tags:** graphql\
**Created:** [October 3, 2020, 9:39am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746 "2020-10-03T09:39:55Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 3, 2020, 9:39am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/1 "2020-10-03T09:39:55Z")

</div>

## Report a Dgraph Bug

### What version of Dgraph are you using?

20.07.1

### Have you tried reproducing the issue with the latest release?

already the latest version

### What is the hardware spec (RAM, OS)?

running on docker

### Steps to reproduce the issue (command/config used to run Dgraph).

trying to bulk load a json file with with this command:

```auto
dgraph bulk -f users.json -s schema.dql -g schema.graphql --map_shards=9 --reduce_shards=3 --http localhost:8000 --zero=dgraph-1.lan:5080

```

`schema.dql` is empty file and schema.qraphql is like:

```auto
type User {
    id: String! @id
    username: String @search(by: [hash, regexp])
    followees: [User]
}

```

when i move the output `p` folders to the alpha servers and start them, the created schema is this:

 ![Screenshot from 2020-10-03 13-07-54](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/b/b4ceb8e6b75e9347b70837fbd3f3f1da845b29e8.png)  
as you can see, there is no `User.` prefix and predicates is not graphql compatible.

### Expected behaviour and actual result.

to see `User.id`, etc.

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [October 4, 2020, 11:30pm UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/2 "2020-10-04T23:30:05Z")

</div>

@pawan any ideas?

---

<div class="post-metadata">

**Author:** ![pawan](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pawan/32/1946_2.png) [@pawan](https://discuss.dgraph.io/u/pawan)\
**Post date:** [October 5, 2020, 5:53am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/3 "2020-10-05T05:53:55Z")

</div>

What data does your `users.json` file contain? The file has to contain data in the correct format like `User.id`, `User.followees`, `User.username` and so on for data to be written correctly. If it doesn’t have the prefix, then Dgraph will store whatever it has.

Giving the GraphQL schema while doing bulk loading via `-g schema.graphql` just stores the GraphQL schema inside the Dgraph instance so that when you bring up the instance you have an API that you can hit.

---

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 7, 2020, 9:32am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/4 "2020-10-07T09:32:58Z")

</div>

@pawan i fixed the `users.json` file as you mentioned, and now the fields has correct names.  
but it seems that the schema itself is not registered inside Dgraph instance, as there is no index on them:

 ![Screenshot from 2020-10-07 12-58-07](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/2/2f8ad2b2b461975dd37f85cd9c3d5340da5bbbeb.png)  
and also querying by User type returns nothing:

```auto
{
  getUser(func: eq(dgraph.type, User)) {
    count(uid)
  }
}

```

i think it should be considered as a bug

---

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 7, 2020, 9:44am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/5 "2020-10-07T09:44:37Z")

</div>

even after applying the schema via admin api, the User type filter query returns nothing!

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [October 7, 2020, 10:32pm UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/6 "2020-10-07T22:32:07Z")

</div>

Do you see `User` in Ratel’s Types tab?

---

<div class="post-metadata">

**Author:** ![pawan](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pawan/32/1946_2.png) [@pawan](https://discuss.dgraph.io/u/pawan)\
**Post date:** [October 8, 2020, 7:03am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/7 "2020-10-08T07:03:33Z")

</div>

Did your data add the `dgraph.type` edge for nodes of `User` type? Can you share a small reproducible example here so that we can help you faster?

---

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 10, 2020, 8:25am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/8 "2020-10-10T08:25:49Z")

</div>

here is the simplified versions of schema and data files: [schema\_simple.graphql](https://discuss.dgraph.io/uploads/short-url/b9VaphHkW4y0U5RfhQVZIUgAvSI.graphql) (106 Bytes) and [users\_simple.json](https://discuss.dgraph.io/uploads/short-url/mGUoA5bthAziQs68U5cXyFOEZLL.json) (233 Bytes)  
which can be used with this command to reproduce the bug:

```auto
dgraph bulk -f users_simple.json -s schema_empty.dql -g schema_simple.graphql --http localhost:8000 --zero=localhost:5080

```

---

<div class="post-metadata">

**Author:** ![rajas](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rajas/32/4393_2.png) [@rajas](https://discuss.dgraph.io/u/rajas)\
**Post date:** [October 12, 2020, 5:15am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/9 "2020-10-12T05:15:22Z")

</div>

Thanks @mbande for providing the data files. I was able to reproduce the issue.  
Inside users\_simple.json file, `dgraph.type` attribute has not been set which is resulting in this behaviour. You need to explicitly set the `dgraph.type` attribute so that DQL or GraphQL queries could get to know type of a node.  
I added `dgraph.type` attribute to the two entities as follows

```auto
[                                                                               
{"User.id": "1", "dgraph.type": "User", "User.username": "a", "followees": [{"User.username": "b", "User.id": "2"}, {"User.username": "c", "User.id": "3"}]},
{"User.id": "2", "dgraph.type": "User", "User.username": "b", "followees": [{"User.username": "c", "User.id": "3"}]}
] 

```

After adding `dgraph.type`, the query

```auto
{
  getUser(func: eq(dgraph.type, User)) {
    count(uid)
  }
}

```

returns

```auto
"data": {
    "getUser": [
      {
        "count": 2
      }
    ]
  }

```

The GraphQL query

```auto
query{
  queryUser{
    id
    username
  }
}

```

returns

```auto
"data": {
    "queryUser": [
      {
        "id": "1",
        "username": "a"
      },
      {
        "id": "2",
        "username": "b"
      }
    ]
  }

```

---

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 12, 2020, 8:53am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/10 "2020-10-12T08:53:29Z")

</div>

@rajas i fixed the data file as you mentioned, and this query works fine:

```auto
{
 getUser(func: eq(dgraph.type, User)) {
   count(uid)
 }
}

```

but something strange is happened, the schema type is not registered:

 ![Screenshot from 2020-10-12 12-18-08](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/1/195f03e0e14f0030efc8c2dea4bd21067b4f089c.png)  
and the required indexes is not created:  
 ![Screenshot from 2020-10-12 12-18-06](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/1/19c6a406c9e25937ad6f594dcc5fc2082cfe247c.png)

---

<div class="post-metadata">

**Author:** ![mbande](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mbande/32/5243_2.png) [@mbande](https://discuss.dgraph.io/u/mbande)\
**Post date:** [October 12, 2020, 10:44am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/11 "2020-10-12T10:44:29Z")

</div>

also, because there is no schema and hence no uniqueness constraint at the bulk load time, there are many duplicate nodes in database.  
sample file to reproduce: [users\_simple.json](https://discuss.dgraph.io/uploads/short-url/tV2uY0mwCwWRuM13xziLspMtQf7.json) (348 Bytes)

---

<div class="post-metadata">

**Author:** ![rajas](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rajas/32/4393_2.png) [@rajas](https://discuss.dgraph.io/u/rajas)\
**Post date:** [October 13, 2020, 10:55am UTC](https://discuss.dgraph.io/t/bulk-loader-fails-to-create-graphql-schema-properly/10746/12 "2020-10-13T10:55:20Z")

</div>

Hi @mbande,

Thanks for providing with the updated json file. I was able to reproduce this issue.

Bulk Loader is meant to be used for loading large data into a new Dgraph cluster. The data is provided in json/rdf format using the -f flag. By providing the graphql schema with -g flag, the related graphql mutations and queries are generated which could then be used to query or mutate data.

But, the provided graphql schema is not used to create corresponding DQL types. This is done so as to avoid conflicts with the existing DQL types. To do that, you may use the `/admin/schema` graphql endpoint after using bulk loader for the correct DQL schema to reflect on ratel.

There are differences between GraphQL and DQL schemas. One of the difference is the @id directive in GraphQL which does not directly translate to a DQL schema. The problem of no uniqueness constraint is caused due to this. A possible workaround for this is to start an empty Dgraph cluster with the given GraphQL schema and then insert data using GraphQL endpoint. Although, this will be slower than bulk loader.