# Not to use trigrams when filtering by regexp

**URL:** <https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218>\
**Category:** Dgraph\
**Created:** [September 12, 2018, 4:15pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218 "2018-09-12T16:15:11Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\
**Post date:** [September 12, 2018, 4:15pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/1 "2018-09-12T16:15:11Z")

</div>

it’s strange that for using regexp expression in **filter** section, still need to create trigram index.  
it’s just filtering on already fetched data, not a search by some condition. when using it in filter section, it makes sense just to apply regular expression to a string value and filter out values that don’t match.

as for now, trigram index is needed, so need additional space to store it, and, what’s more important, is not possible to use regular expressions like `"^f.*"` to keep values starting with “f” letter only. whereas all nodes are already found by outcoming edges from some other node.

i found 1 workaround to add 2 junk letters to every string value i need to filter with “start with” condition, so `"^f.*"` condition becomes `"^AAf.*"` . it works but looks real ugly

so, my question is - why not using regular “regexp” go function when processing regular expression in “filter” section, and use trigram index when use it in “func” one?

---

<div class="post-metadata">

**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\
**Post date:** [September 12, 2018, 4:19pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/2 "2018-09-12T16:19:05Z")

</div>

@gus what do you think about it?

---

<div class="post-metadata">

**Author:** ![gus](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gus/32/1259_2.png) [@gus](https://discuss.dgraph.io/u/gus)\
**Post date:** [September 14, 2018, 11:22pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/3 "2018-09-14T23:22:57Z")

</div>

the reason you can bypass the check with junk is a bug in the 3rd party package. we could add support to match all instead of exact matching, i don’t think we need to change regexp packages yet. i’ll check.

---

<div class="post-metadata">

**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\
**Post date:** [September 15, 2018, 5:21pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/4 "2018-09-15T17:21:00Z")

</div>

actually, my question was not about why junk check passes, but why we need trigram index and 3 chars limit for regexp in **filter** section, when we just filter out already fetched predicate values

---

<div class="post-metadata">

**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\
**Post date:** [September 22, 2018, 8:25pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/5 "2018-09-22T20:25:47Z")

</div>

mmmmmmmm… any updates here?

---

<div class="post-metadata">

**Author:** ![gus](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gus/32/1259_2.png) [@gus](https://discuss.dgraph.io/u/gus)\
**Post date:** [October 2, 2018, 11:43pm UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/6 "2018-10-02T23:43:33Z")

</div>

Sorry I couldnt get back to you sooner. I’m reopening the original issue and will investigate. I don’t know why regexp isn’t used for filters, but i think it’s worth checking. My gut feeling is that regexps are very slow, not to mention that they can use lots of memory.

If others could weigh in on this, it would be great.

Ref: [Not to use trigrams when filtering by regexp · Issue #2565 · dgraph-io/dgraph · GitHub](https://github.com/dgraph-io/dgraph/issues/2565)

---

<div class="post-metadata">

**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\
**Post date:** [October 3, 2018, 11:52am UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/7 "2018-10-03T11:52:21Z")

</div>

good news! since it’s filtering only, i don’t see any performance issues here - quite every database (elasticsearch, for example) has ability to post-filter returned data by regexp

---

<div class="post-metadata">

**Author:** ![gus](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gus/32/1259_2.png) [@gus](https://discuss.dgraph.io/u/gus)\
**Post date:** [January 18, 2019, 4:11am UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/8 "2019-01-18T04:11:39Z")

</div>

I’ve made a PR to change the regexp behavior to use an index if found, otherwise will run without index as discussed.

---

<div class="post-metadata">

**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\
**Post date:** [January 18, 2019, 7:33am UTC](https://discuss.dgraph.io/t/not-to-use-trigrams-when-filtering-by-regexp/3218/9 "2019-01-18T07:33:47Z")

</div>

thank you! this is very important feature for us
