# Invalid memory address or nil pointer dereference error

**URL:** <https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424>\
**Category:** Dgraph\
**Tags:** status:accepted, ticket:created\
**Created:** [September 17, 2020, 8:32am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424 "2020-09-17T08:32:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 17, 2020, 8:32am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/1 "2020-09-17T08:32:46Z")

</div>

```auto
invalid memory address or nil pointer dereference
SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14fb4be

```

```auto
upsert{
  query{
    v as var(func: eq(id,12)) # this one has 1 000 000 followers
  }
  mutation {
   delete{
    uid(v) <followers> * .
   }
  }
}

```

followers has count and reverse index.

code at this line

> <https://github.com/dgraph-io/dgraph/blob/master/posting/index.go#L280>

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [September 18, 2020, 10:45am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/2 "2020-09-18T10:45:05Z")

</div>

@ibrahim ideas?

@BlankRain - wondering if you could share the dataset?

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 18, 2020, 11:35am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/3 "2020-09-18T11:35:27Z")

</div>

@BlankRain can you please show the stack trace? and the version on which you saw this panic.

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 22, 2020, 7:52am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/4 "2020-09-22T07:52:21Z")

</div>

```auto
Dgraph version : v20.07.0-12-g681fe9116
Dgraph codename : shuri-mod
Go version : go1.14.1

```

@chewxy we use twitter data:

```auto
wget http://an.kaist.ac.kr/\~haewoon/release/twitter_social_graph/twitter_rv.tar.gz 

# split by line
split -l 54535107 twitter_rv.net

```

then use the go code to convert csv data to rdf

```auto
package main
import (
    "path/filepath"
    "os"
    "fmt"
	"flag"
	"bufio"
	"io"
	"strings"
)

func getFilelist(path string) []string {
	files := []string{}
    err := filepath.Walk(path, func(path string, f os.FileInfo, err error) error {
        if f == nil {
			return err
		}
        if f.IsDir() {
			return nil
		}
		files = append(files,path)
        return nil
    })
    if err != nil {
        fmt.Printf("filepath.Walk() returned %v\n", err)
	}
	return files
}

func rdf(f string, separator string, ch chan string) {
	rv, err := os.Open(f)
	
	if err != nil {
		fmt.Println("open file err=", err)
	    rv.Close()
		return
	}

	defer rv.Close()

	out, _ := os.OpenFile(f+".rdf", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
	
	reader := bufio.NewReader(rv)
	wirter := bufio.NewWriter(out)

	nodes := make(map[string]bool)
	//循环的读取文件的内容
	errcount := 0
	for {
		str, err := reader.ReadString('\n') // 读到一个换行就结束
		if err == io.EOF { // io.EOF表示文件的末尾
			break
		}
		//输出内容
		line := strings.Split(str, separator)
		if len(line) != 2 {
			errcount++
			fmt.Printf("from now on find %d errors in %s\n", errcount, f)
			continue
		}
		src := strings.Trim(line[0], "\n")
		dst := strings.Trim(line[1], "\n")
		tpl := "_:v%s <id> \"%s\" . \n_:v%s <dgraph.type> \"twitter_user\" .\n"
		if !nodes[src] {
			lout := fmt.Sprintf(tpl, src, src, src)
			wirter.Write([]byte(lout))
			nodes[src] = true
		}
		if !nodes[dst] {
			lout := fmt.Sprintf(tpl, dst, dst, dst)
			wirter.Write([]byte(lout))
			nodes[dst] = true
		}

		lout := fmt.Sprintf("_:v%s <followers> _:v%s . \n", src, dst)
		wirter.Write([]byte(lout))
	}
	wirter.Flush()
	out.Close()
	ch <- f + ".rdf is ok!"
}

func main(){
    flag.Parse()
	separatorKey := flag.Arg(0)
	root := flag.Arg(1)

	separatorMap := map[string]string{"s": " ", "t": "\t", "c": ","}
	separator := separatorMap[separatorKey]

	if 0 == len(separator) {
		fmt.Printf("Please specify the separator！\n t for Tab \n s for Space \n c for comma \n such as: ./gocsv2rdf s ./test \n")
		return
	}

	if 0 == len(root) {
		fmt.Printf("Please specify the CSV folder！\n such as: ./gocsv2rdf s ./test \n")
		return
	}

	files :=getFilelist(root)
	fmt.Printf("%v\n", files)

	
	count := len(files)
	if 0 == count {
		fmt.Printf("Please specify the correct CSV folder！\n")
		return
	}
	ch := make(chan string, count)
	for i:= 0;i<count;i++{
      	name := files[i]
		fmt.Printf("Runninng for %s\n", name)
		//rdf(name, separator, ch)
		go rdf(name, separator, ch) //内存足够大就用这行代码,并行处理
	}
	i := 0
	for x := range ch {
		fmt.Println(x)
		i++
		if i == int(count) {
			break
		}
	}
	fmt.Println("Over!")
}

```

then bulk load or live load into dgraph.

The schema is

```auto
id: int @index(int) .
followers: [uid] .

type twitter_user {
    id
    followers
}

```

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 22, 2020, 8:21am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/5 "2020-09-22T08:21:57Z")

</div>

Hey @BlankRain, can you show me the complete stack trace of the crash? I’m trying to figure out what sequence of function calls led to this crash.

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 22, 2020, 8:36am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/6 "2020-09-22T08:36:15Z")

</div>

Hi ,  
This is the stack trace .

 ![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/a/a5cea2b46645f709e7c06234c957148bfc15e43f.png)

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 23, 2020, 6:17am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/7 "2020-09-23T06:17:40Z")

</div>

@BlankRain It looks like you’re running a modified version of dgraph

```auto
Dgraph version : v20.07.0-12-g681fe9116
Dgraph codename : shuri-mod
Go version : go1.14.1

```

Can you try running on a released version of dgraph? I don’t have the code you’re running and I cannot debug it 🙂 .

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 23, 2020, 7:17am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/8 "2020-09-23T07:17:37Z")

</div>

Hi  
The code based on the release of v20.07.0 .

> **[Release Dgraph v20.07.0 (Savvy Shuri) · dgraph-io/dgraph](https://github.com/dgraph-io/dgraph/releases/tag/v20.07.0)**
>
> Changed
> 
> GraphQL
> 
> Make updateGQLSchema always return the new schema. (#5540)
> Allow user to define and pass arguments to fields. (#5562)
> Move alias to end of graphql pipeline. (#5369)
> 
> 
> Return error...

I modified the code to fix some bugs .  
for example:  
[Bulk loader crashes during reduce phase - #26 by BlankRain](http://discuss.hypermode.com/t/bulk-loader-crashes-during-reduce-phase/9116/26)  
The git log ,I based is this one.

```auto
commit 7431be0dce0ffc42e3ff3e31b39fcdd5505e2bb3
Author: Martin Martinez Rivera <martinmr@dgraph.io>
Date: Mon Aug 3 11:54:25 2020 -0700

    add cluster lables to the jaeger containers (#5951) (#6009)

```

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 23, 2020, 7:37am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/9 "2020-09-23T07:37:49Z")

</div>

Thanks @BlankRain. So the call to `posting()` is failing because `uidPosting` is nil on line 273.

> <https://github.com/dgraph-io/dgraph/blob/7431be0dce0ffc42e3ff3e31b39fcdd5505e2bb3/posting/list.go#L260-L275>

But it looks like we initialize the `it.uidPosting` in `init` which is called by `iterate`

> <https://github.com/dgraph-io/dgraph/blob/7431be0dce0ffc42e3ff3e31b39fcdd5505e2bb3/posting/list.go#L142>

@BlankRain would you be able to share your dataset? We can try to reproduce it but I doubt if we can easily reproduce it on our end. If you can share your dataset, we can look at it to figure out what’s causing this.

@animesh2049 Do you see any reason `pb.Posting` can be `nil` here? The cluster is running in ludicrous mode.

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 23, 2020, 8:00am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/10 "2020-09-23T08:00:18Z")

</div>

I can share my dataset. It may be too huge to copy.  
My dataset is based on the twitter data.  
I can show you how to produce the data.

```auto
wget http://an.kaist.ac.kr/\~haewoon/release/twitter_social_graph/twitter_rv.tar.gz 

tar xvf twitter_rv.tar.gz 

split -l 54535107 twitter_rv.net

mkdir test

mv x* test

rm test/*.rdf
./gocsv2rdf t test

# then copy test/*.rdf out for bulk load and start the dgraph cluster

```

the source code of gocsv2rdf is here

```auto
package main
import (
    "path/filepath"
    "os"
    "fmt"
	"flag"
	"bufio"
	"io"
	"strings"
)

func getFilelist(path string) []string {
	files := []string{}
    err := filepath.Walk(path, func(path string, f os.FileInfo, err error) error {
        if f == nil {
			return err
		}
        if f.IsDir() {
			return nil
		}
		files = append(files,path)
        return nil
    })
    if err != nil {
        fmt.Printf("filepath.Walk() returned %v\n", err)
	}
	return files
}

func rdf(f string, separator string, ch chan string) {
	rv, err := os.Open(f)
	
	if err != nil {
		fmt.Println("open file err=", err)
	    rv.Close()
		return
	}

	defer rv.Close()

	out, _ := os.OpenFile(f+".rdf", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
	
	reader := bufio.NewReader(rv)
	wirter := bufio.NewWriter(out)

	nodes := make(map[string]bool)
	//循环的读取文件的内容
	errcount := 0
	for {
		str, err := reader.ReadString('\n') // 读到一个换行就结束
		if err == io.EOF { // io.EOF表示文件的末尾
			break
		}
		//输出内容
		line := strings.Split(str, separator)
		if len(line) != 2 {
			errcount++
			fmt.Printf("from now on find %d errors in %s\n", errcount, f)
			continue
		}
		src := strings.Trim(line[0], "\n")
		dst := strings.Trim(line[1], "\n")
		tpl := "_:v%s <id> \"%s\" . \n_:v%s <dgraph.type> \"twitter_user\" .\n"
		if !nodes[src] {
			lout := fmt.Sprintf(tpl, src, src, src)
			wirter.Write([]byte(lout))
			nodes[src] = true
		}
		if !nodes[dst] {
			lout := fmt.Sprintf(tpl, dst, dst, dst)
			wirter.Write([]byte(lout))
			nodes[dst] = true
		}

		lout := fmt.Sprintf("_:v%s <followers> _:v%s . \n", src, dst)
		wirter.Write([]byte(lout))
	}
	wirter.Flush()
	out.Close()
	ch <- f + ".rdf is ok!"
}

func main(){
    flag.Parse()
	separatorKey := flag.Arg(0)
	root := flag.Arg(1)

	separatorMap := map[string]string{"s": " ", "t": "\t", "c": ","}
	separator := separatorMap[separatorKey]

	if 0 == len(separator) {
		fmt.Printf("Please specify the separator！\n t for Tab \n s for Space \n c for comma \n such as: ./gocsv2rdf s ./test \n")
		return
	}

	if 0 == len(root) {
		fmt.Printf("Please specify the CSV folder！\n such as: ./gocsv2rdf s ./test \n")
		return
	}

	files :=getFilelist(root)
	fmt.Printf("%v\n", files)

	
	count := len(files)
	if 0 == count {
		fmt.Printf("Please specify the correct CSV folder！\n")
		return
	}
	ch := make(chan string, count)
	for i:= 0;i<count;i++{
      	name := files[i]
		fmt.Printf("Runninng for %s\n", name)
		//rdf(name, separator, ch)
		go rdf(name, separator, ch) //内存足够大就用这行代码,并行处理
	}
	i := 0
	for x := range ch {
		fmt.Println(x)
		i++
		if i == int(count) {
			break
		}
	}
	fmt.Println("Over!")
}

```

go build it then run it.  
all this may need at least 30min or more.

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 23, 2020, 8:05am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/11 "2020-09-23T08:05:18Z")

</div>

```auto
upsert{
  query{
    v as var(func: eq(id,12)) # this one has 1 000 000 followers
  }
  mutation {
   delete{
    uid(v) <followers> * .
   }
  }
}

```

I do some delete in upsert block.  
the reason `pb.Posting` can be `nil` may around here.

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 23, 2020, 8:24am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/13 "2020-09-23T08:24:15Z")

</div>

Thanks @BlankRain. I’ve accepted this as a bug. @animesh2049 will run some tests locally on your dataset and get back to you.

---

<div class="post-metadata">

**Author:** ![animesh2049](https://avatars.discourse-cdn.com/v4/letter/a/41988e/32.png) [@animesh2049](https://discuss.dgraph.io/u/animesh2049)\
**Post date:** [September 24, 2020, 12:04pm UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/14 "2020-09-24T12:04:45Z")

</div>

Hey @BlankRain I have pushed a change, commit id `7873f04087e671b5e45d87aaa1875680324a0f7b`. Can you please apply this commit and see if you are still getting the error.

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 25, 2020, 1:46am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/15 "2020-09-25T01:46:57Z")

</div>

hi, which branch ?

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [September 25, 2020, 2:33am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/16 "2020-09-25T02:33:31Z")

</div>

you can just `git checkout 7873f040`

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 25, 2020, 3:20am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/17 "2020-09-25T03:20:38Z")

</div>

> [@chewxy](#):
>
> git checkout 7873f040

ok,thanks ,let me try

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 25, 2020, 6:11am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/18 "2020-09-25T06:11:58Z")

</div>

just post the related pr here: [Initialize posting list in moveToNextPart by animesh2049 · Pull Request #6560 · dgraph-io/dgraph · GitHub](https://github.com/dgraph-io/dgraph/pull/6560)

---

<div class="post-metadata">

**Author:** ![animesh2049](https://avatars.discourse-cdn.com/v4/letter/a/41988e/32.png) [@animesh2049](https://discuss.dgraph.io/u/animesh2049)\
**Post date:** [September 28, 2020, 7:58am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/19 "2020-09-28T07:58:12Z")

</div>

Hey @BlankRain are you still facing the issue ?

---

<div class="post-metadata">

**Author:** ![BlankRain](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/blankrain/32/1592_2.png) [@BlankRain](https://discuss.dgraph.io/u/BlankRain)\
**Post date:** [September 28, 2020, 8:02am UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/20 "2020-09-28T08:02:49Z")

</div>

Sorry , we didn’t finish the test yet. Our cluster is broken. I will let you know when we finish the test.

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [September 30, 2020, 9:57pm UTC](https://discuss.dgraph.io/t/invalid-memory-address-or-nil-pointer-dereference-error/10424/21 "2020-09-30T21:57:51Z")

</div>

2 posts were split to a new topic: [Tokenizer panics (type error)](https://discuss.dgraph.io/t/tokenizer-panics-type-error/10705)