Two interesting test cases?

jchiu · September 6, 2016, 3:12am

Assume that the current value is Cars. If you set it to something new and set it back to Cars, the change would not happen.

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

The above case is easy to fix. The following case feels much more strange to me.

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "random" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "random" .
  }
}' | python -m json.tool

I got the following panic:

panic: runtime error: makeslice: len out of range

goroutine 32 [running]:
panic(0x862ae0, 0xc420132e80)
	/usr/lib/go-1.7/src/runtime/panic.go:500 +0x1a1
github.com/dgraph-io/dgraph/posting.(*List).merge(0xc42013e4e0, 0xc42000c400, 0x0, 0x0)
	/home/jchiu/go/src/github.com/dgraph-io/dgraph/posting/list.go:708 +0x234
github.com/dgraph-io/dgraph/posting.(*List).MergeIfDirty(0xc42013e4e0, 0xd1cd20, 0xc42000c408, 0xd1b2a0, 0xc420122f28, 0x8ce0e0)
	/home/jchiu/go/src/github.com/dgraph-io/dgraph/posting/list.go:692 +0x101
github.com/dgraph-io/dgraph/posting.mergeAndUpdate(0xc42013e4e0, 0xc42015b6b0)
	/home/jchiu/go/src/github.com/dgraph-io/dgraph/posting/lists.go:299 +0x50
github.com/dgraph-io/dgraph/posting.gentlyMerge(0xc420136e20)
	/home/jchiu/go/src/github.com/dgraph-io/dgraph/posting/lists.go:173 +0x286
created by github.com/dgraph-io/dgraph/posting.checkMemoryUsage
	/home/jchiu/go/src/github.com/dgraph-io/dgraph/posting/lists.go:253 +0x1dc

Line numbers are off. The panic happens on the following line. I did a printf and got sz=-1.

github.com

dgraph-io/dgraph/blob/master/posting/list.go#L645


			if err := fn(p); err != nil {
				return err
			}
			advance()
		}
		if err := fn(obj); err != nil {
			return err
		}
		if obj.Uid == next {
			advance()
		}
		return nil
	}
	if err := l.iterate(readTs, afterUid, fi); err != nil {
		return err
	}

	codec.RemoveRange(bm, 0, maxUid)
	uitr = bm.Iterator()
	for uitr.HasNext() {
		p.Uid = uitr.Next()

Side comments: I restore m p u before running any of these using a trivial script:

rm -Rf m p u
cp -R ../dgraph2/u ./
cp -R ../dgraph2/p ./

mrjn · September 6, 2016, 3:57am

Great catches! I’ll create test cases for both of these, and fix them.

mrjn · September 6, 2016, 1:56pm

Unable to reproduce the second case panic. @pawan - You said you were able to reproduce this problem?

jchiu · September 7, 2016, 12:41am

One question I have is: When deleting a value, do we require the user to know the current value? In some other query language, this current value might not be required? Looking at your code and the following, it seems that the earlier deletes are now banned. (Also do make sure that the first query returns “Cars”. Sometimes I forget to restore.)

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "random" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "random" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

Now, what if we ensure that the deletes go through. Do we still have panics?

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}' | python -m json.tool

curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }
}' | python -m json.tool

This time, instead of getting panics, we see that the last query returns newcar3 instead of nothing. The panic got converted into a logic error? (as you were saying yesterday)

mrjn · September 7, 2016, 1:38am

If you specify the exact value to be deleted, the right logic is to only execute it, if the value stored is the same. Otherwise, it’s a NOOP. That’s the change I made yesterday.

If you want to delete any value attached to the subject, the right instruction would be to set obj = *. That needs to be implemented.

jchiu · September 7, 2016, 1:56am

Sounds good to me. How about the second part where we seem to fail to delete the value “newcar3”?

mrjn · September 7, 2016, 2:38am

It works with my PR.

+ curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }'
+ python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   221  100   146  100    75   161k  84745 --:--:-- --:--:-- --:--:--  142k
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68"
    },
    "server_latency": {
        "json": "26.214\u00b5s",
        "parsing": "33.284\u00b5s",
        "processing": "46.459\u00b5s",
        "total": "107.226\u00b5s"
    }
}
+ curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
  }
}'
+ python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   123  100    35  100    88   3699   9301 --:--:-- --:--:-- --:--:--  9777
{
    "code": "ErrorOk",
    "message": "Done"
}
+ curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
+ python -m json.tool
}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   218  100   145  100    73  30290  15249 --:--:-- --:--:-- --:--:-- 36250
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68"
    },
    "server_latency": {
        "json": "16.408\u00b5s",
        "parsing": "30.501\u00b5s",
        "processing": "56.99\u00b5s",
        "total": "105.202\u00b5s"
    }
}
+ curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
+ python -m json.tool
}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   123  100    35  100    88   3891   9785 --:--:-- --:--:-- --:--:-- 11000
{
    "code": "ErrorOk",
    "message": "Done"
}
+ curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
}'
+ python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   252  100   179  100    73  71657  29223 --:--:-- --:--:-- --:--:-- 89500
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68",
        "type.object.name.en": "newcar3"
    },
    "server_latency": {
        "json": "48.689\u00b5s",
        "parsing": "92.511\u00b5s",
        "processing": "102.261\u00b5s",
        "total": "246.699\u00b5s"
    }
}
+ curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
+ python -m json.tool
}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   126  100    35  100    91   3820   9932 --:--:-- --:--:-- --:--:-- 10111
{
    "code": "ErrorOk",
    "message": "Done"
}
+ curl localhost:8080/query -XPOST -d '{
  debug(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
+ python -m json.tool
}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   218  100   145  100    73  28656  14426 --:--:-- --:--:-- --:--:-- 29000
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68"
    },
    "server_latency": {
        "json": "52.47\u00b5s",
        "parsing": "55.531\u00b5s",
        "processing": "63.661\u00b5s",
        "total": "173.509\u00b5s"
    }
}

jchiu · September 7, 2016, 2:50am

The first output is missing “Cars” and your first delete didn’t happen. You need to “restore” the database to its original state…

jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d '{
>   debug(_uid_: 0xc1cee22c7ab32e68 ) {
>       type.object.name.en
>     }
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   253  100   176  100    77  90395  39548 --:--:-- --:--:-- --:--:--  171k
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68",
        "type.object.name.en": "Cars"
    },
    "server_latency": {
        "json": "37.827\u00b5s",
        "parsing": "37.729\u00b5s",
        "processing": "246.729\u00b5s",
        "total": "323.682\u00b5s"
    }
}
jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d 'mutation {
>   delete {
>      _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   123  100    35  100    88   5634  14166 --:--:-- --:--:-- --:--:-- 17600
{
    "code": "ErrorOk",
    "message": "Done"
}
jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d '{
>   debug(_uid_: 0xc1cee22c7ab32e68 ) {
>       type.object.name.en
>     }
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   222  100   145  100    77   121k  66037 --:--:-- --:--:-- --:--:--  141k
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68"
    },
    "server_latency": {
        "json": "18.477\u00b5s",
        "parsing": "48.25\u00b5s",
        "processing": "45.528\u00b5s",
        "total": "114.021\u00b5s"
    }
}
jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d 'mutation {
>   set {
>      _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   123  100    35  100    88   5791  14562 --:--:-- --:--:-- --:--:-- 17600
{
    "code": "ErrorOk",
    "message": "Done"
}
jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d '{
>   debug(_uid_: 0xc1cee22c7ab32e68 ) {
>       type.object.name.en
>     }
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   253  100   176  100    77   136k  61354 --:--:-- --:--:-- --:--:--  171k
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68",
        "type.object.name.en": "newcar3"
    },
    "server_latency": {
        "json": "23.212\u00b5s",
        "parsing": "44.5\u00b5s",
        "processing": "52.675\u00b5s",
        "total": "121.919\u00b5s"
    }
}
jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d 'mutation {
>   delete {
>      _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   126  100    35  100    91   5865  15250 --:--:-- --:--:-- --:--:-- 18200
{
    "code": "ErrorOk",
    "message": "Done"
}

jchiu@jchiudesktop:~/dgraph$ 
jchiu@jchiudesktop:~/dgraph$ curl localhost:8080/query -XPOST -d '{
>   debug(_uid_: 0xc1cee22c7ab32e68 ) {
>       type.object.name.en
>     }
>   }
> }' | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   257  100   180  100    77  39911  17073 --:--:-- --:--:-- --:--:--  175k
{
    "debug": {
        "_uid_": "0xc1cee22c7ab32e68",
        "type.object.name.en": "newcar3"
    },
    "server_latency": {
        "json": "68.198\u00b5s",
        "parsing": "105.212\u00b5s",
        "processing": "154.177\u00b5s",
        "total": "331.647\u00b5s"
    }
}

mrjn · September 7, 2016, 2:58am

Okay. This script can reproduce the bug.

set -x
curl localhost:8080/query -XPOST -d 'mutation {
set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
		   
}

}'

curl localhost:8080/query -XPOST -d '{
me(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
}
}
}'

echo "Sleeping for 15s"
sleep 15

curl localhost:8080/query -XPOST -d '{
  me(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
  }'

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "Cars" .
  }
}'

curl localhost:8080/query -XPOST -d '{
  me(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
}'

curl localhost:8080/query -XPOST -d 'mutation {
  set {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}'

curl localhost:8080/query -XPOST -d '{
  me(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
}'

curl localhost:8080/query -XPOST -d 'mutation {
  delete {
     _uid_:0xc1cee22c7ab32e68 <type.object.name.en> "newcar3" .
  }
}'

curl localhost:8080/query -XPOST -d '{
  me(_uid_: 0xc1cee22c7ab32e68 ) {
      type.object.name.en
    }
}'

system · November 28, 2017, 1:00am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
SetJson. Deletion of the first zero in the string Users mutation	4	521	March 16, 2019
"set" mutation escape character in JSON error Users mutation	2	805	November 15, 2019
JSON Mutation Format - Mutations Documentation	4	846	December 29, 2020
Update: Using JSON to mutate on Dgraph. (cURL, Raw HTTP) Dgraph	1	2734	July 7, 2018
Partially typed schema Dgraph schema	5	548	June 30, 2021

Two interesting test cases?

Related Topics