# Need expand(\_all\_) like behaviour where schema isn't known beforehand

**URL:** <https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268>\
**Category:** Dgraph\
**Created:** [December 8, 2021, 12:59pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268 "2021-12-08T12:59:29Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![pranaypratyush](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pranaypratyush/32/4264_2.png) [@pranaypratyush](https://discuss.dgraph.io/u/pranaypratyush)\
**Post date:** [December 8, 2021, 12:59pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/1 "2021-12-08T12:59:29Z")

</div>

## What I want to do

I want to log API responses from multiple sources in a single type. I can’t really guess all possible predicates before hand. It’s also possible that a sometimes a single predicate has different type for different API response.  
Then I need to expand all fields on some of these responses without really knowing the type before hand. I know there would be timestamp in most of them and I would need filtering based on it.

## What I did

Tried to modify the type on every append but it doesn’t work since some predicates collide.

Some other thread mentioned using facets but that seems counter-intuitive. More explanation if that is the right approach?  
I know `expand(_all_)` need all predicates but couldn’t there be a `dgraph.dyna` type that can be assigned only to the nodes that need it. This way you wouldn’t have to keep track of all possible predicates in all nodes.

---

<div class="post-metadata">

**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\
**Post date:** [December 8, 2021, 4:21pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/2 "2021-12-08T16:21:33Z")

</div>

> [@pranaypratyush](#):
>
> I can’t really guess all possible predicates before hand.

Once the data is inserted in the DB, the predicate is allocated as a default type. So you can find by quering like this `schema { type }`.

> [@pranaypratyush](#):
>
> More explanation if that is the right approach?

Can you share where this was suggested? The context should help here.

> [@pranaypratyush](#):
>
> I know `expand(_all_)` need all predicates but couldn’t there be a `dgraph.dyna` type that can be assigned only to the nodes that need it. This way you wouldn’t have to keep track of all possible predicates in all nodes.

You can create a fake Type and use it to kind of “debug the schema” and create the official Schema. But you would have to upsert all possible nodes with the `"dgraph.type": "myFakeType"` and then you can query with ` expand(myFakeType)`

---

<div class="post-metadata">

**Author:** ![iluminae](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/iluminae/32/7367_2.png) [@iluminae](https://discuss.dgraph.io/u/iluminae)\
**Post date:** [December 8, 2021, 5:15pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/3 "2021-12-08T17:15:16Z")

</div>

> [@MichelDiz](#):
>
> But you would have to upsert all possible nodes with the `"dgraph.type": "myFakeType"` and then you can query with ` expand(myFakeType)`

you do not have to have `"dgraph.type": "myFakeType"` on a node for `expand(myFakeType)` to work. The only thing that consults dgraph.type is `expand(_all_)`

---

<div class="post-metadata">

**Author:** ![pranaypratyush](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pranaypratyush/32/4264_2.png) [@pranaypratyush](https://discuss.dgraph.io/u/pranaypratyush)\
**Post date:** [December 8, 2021, 6:18pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/4 "2021-12-08T18:18:57Z")

</div>

> [@MichelDiz](#):
>
> Can you share where this was suggested? The context should help here.

> [@Partially typed schema](http://discuss.hypermode.com/t/partially-typed-schema/14591):
>
> I know in advance what structure I expect to receive for some parts of my data but one certain field can contain arbitrary JSON. I want to be able to return the original JSON by following the node and run some aggregates over that JSON since I know names of certain fields. What would be the best approach to handle that and what are my options? Currently running mutation = txn.create\_mutation(set\_obj=partial\_data) request = txn.create\_request(mutations=[mutation], commit\_now=True) result = tx…

> [@MichelDiz](#):
>
> You can create a fake Type and use it to kind of “debug the schema” and create the official Schema. But you would have to upsert all possible nodes with the `"dgraph.type": "myFakeType"` and then you can query with ` expand(myFakeType`

Didn’t quite get it. I made a schema from sample response but then some other response later on might have extra fields that I wont be able to query. How can I see these extra fields?  
(Plus the schema is getting filled with predicates, it’s ok I guess but avoiding it would be nice)

---

<div class="post-metadata">

**Author:** ![iluminae](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/iluminae/32/7367_2.png) [@iluminae](https://discuss.dgraph.io/u/iluminae)\
**Post date:** [December 8, 2021, 10:14pm UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/5 "2021-12-08T22:14:15Z")

</div>

seems like your use case is more along the lines of elasticsearch - any new field that comes in just automatically index and be able to search on - this is not the default modus operandi of dgraph. Any new fields ingested will be assigned to a group _unindexed_ until it is told to index them. Indicies are required to run many of the filters necessary to begin a query.

To make it simple: no, there is no built in way to `expand(everyPossibleField)` without extra work.

Suggestion: assuming you have a structured input, keep the information split into predicates by field _and_ keep them together in one field together in eg: JSON.

eg:

```auto
_:newLog <request_time> "2021/01/02-01:02:00" .
_:newLog <service_name> "thanos" .
_:newLog <response_code> "200" .
_:newLog <full_log> "{\"request_time\":\"2021/01/02-01:02:00\",\"service_name\":\"thanos\",\"response_code\":200}" .

```

then your queries will look like this:

```auto
{
  q(func: eq(service_name,"thanos")) @filter(eq(response_code,"200")) {
    full_log
  }
}

```

that way you will always get ‘all’ of the fields because they will be in full log no matter what you filter by. You can index whatever fields you need to whenever you decide you need to build indicies for them.

edit: and if this works for you, lobby for the JSON type to be added as a feature to the database, as it will make this feel a bit less icky.

---

<div class="post-metadata">

**Author:** ![pranaypratyush](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pranaypratyush/32/4264_2.png) [@pranaypratyush](https://discuss.dgraph.io/u/pranaypratyush)\
**Post date:** [December 9, 2021, 7:40am UTC](https://discuss.dgraph.io/t/need-expand-all-like-behaviour-where-schema-isnt-known-beforehand/16268/6 "2021-12-09T07:40:57Z")

</div>

Thanks for the reply. I might just separate out the fields that need searching (if they exist) and dump the rest in the full log string. And yeah json field would be awesome.  
The rest of my data is hard to model as well with many relationships, that’s the reason I chose a graph db for this application but now it seems that arangodb might be the better choice for this application. I do like dgraph but probably not the right fit here.