Orderasc/desc by object that is missing the field data


(James) #1

ive some old data that did not declare field created_at during my initial test.
later on, i’ve inserted those fields.
when i tried to sort by the missing field, the records only show those with this field declared. is this a norm?

i have not test with filter, what if i filter by this field with not equal?
will those old records without this field being enter also went missing?

as seen below, the result output will defer from count

query all($nothing: string){
  
users(func: eq(_type, "users") orderdesc: created_at)   {
		uid
		email
		created_at
		updated_at
}
  users_count(func: eq(_type, "users"))  {
	count(uid)
}
}

(James) #2

ive been testing delete,
and realize some how the data become inconsistant.
sometimes the list shows all records, but sometimes it shows only those records with created_at field specified


(Michel Conrado (Support Engineer)) #3

Hi James,

Not sure, but by logic, it seems like this would happen anyway. Because Dgraph can only Sort by what is indexed. You can create a task in your application to hunt those nodes that have no date and add a generic date.

Fill in an Issue about it.

Well I did some tests on:

Dgraph version   : v1.0.9
Commit SHA-1     : 22c13fce
Commit timestamp : 2018-10-02 16:45:53 -0700
Branch           : HEAD
{
  
users(func: eq(_type, "users"), orderdesc: created_at)   {
		uid
		email
		created_at
		updated_at
}
  users_count(func: eq(_type, "users"))  {
	count(uid)
}
}

Result:

{
  "data": {
    "users": [
      {
        "uid": "0x4",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x5",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x6",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x7",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      }
    ],
    "users_count": [
      {
        "count": 8
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "processing_ns": 3002300
    },
    "txn": {
      "start_ts": 22
    }
  }
}

Mutation was:


{
   "set":[
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      }
   ]
}

(James) #4

hi @MichelDiz

okay thank you.
i will do the manual update for addition of new fields…

are there any mutation in dgraph based on certain filter(s) that set all nodes to the value i wish?
for example are those without created_at and set it to 0001-01-01 ?

thank you


(Michel Conrado (Support Engineer)) #5

Nops, but you could try to do this as an upsert procedure https://docs.dgraph.io/howto/#upsert-procedure
may work good. Use “has(created_at)” instead of “eq”.

If the searched Node has “created_at” it does nothing. If it does not, it returns the UID and you generate a mutation with it.


(James) #6

Hi @MichelDiz

okay noted, so ive to do mutation in application layer for all existing records.
but how do i query for those field without created_at?


(Michel Conrado (Support Engineer)) #7

Maybe something like this:

{
  
users(func: eq(_type, "users")) @filter(NOT has(created_at))   {
		uid
		email
		created_at #let it here just to ensure
		updated_at #let it here just to ensure
}

}

And then do a query for “updated_at” just in case.


(James) #8

Awesome, thank you :slight_smile:
i thought its !has but didnt work :blush:


(James) #9

hi @MichelDiz

i have this finding:
with this same query, i queries this multiple times on ratel ui,
and 50% of the time, it return results with actual data, but another 50% only return records with uid field (other records without created_at). and i could not delete those uid only records

query all($nothing: string){
	users(func: eq(_type, "users"), first: 10, orderdesc: created_at)   {
		uid
		email
		username
		password
		first_name
		last_name
		mobile_no
		office_no
		gender
		country
		confirmed_at
		locked_at
		deleted_at
		reset_token
		status
		signup_type
		online
		online_at
		ip_address
		created_at
		updated_at
		roles : with_role {
			uid
			name 
		}
		work_for : work_for {
			uid
			code
			name
			status
			country
			created_at
			updated_at 
			 
		}
	}

	users_count(func: eq(_type, "users"))  @filter(has(created_at)) {
		count(uid)
	}
}

ok result:

{
  "data": {
    "users": [
      {
        "uid": "0xea63",
        "email": "dsfsdf@example.com",
        "username": "dsfsdf@example.com",
        "password": "",
        "first_name": "dsfsd",
        "last_name": "dsfdsf",
        "mobile_no": "4",
        "office_no": "",
        "gender": "M",
        "country": "AD",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 1,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-10-16T09:24:48.24963+08:00",
        "updated_at": "2018-10-16T09:25:31.91972Z"
      },
      {
        "uid": "0xea62",
        "email": "ddsfsdf@example.com",
        "username": "ddsfsdf@example.com",
        "password": "",
        "first_name": "sdfdss",
        "last_name": "dfdsd",
        "mobile_no": "2324324",
        "office_no": "",
        "gender": "M",
        "country": "",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 1,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-10-15T03:00:36.077012+08:00",
        "updated_at": "2018-10-16T07:26:56.430786Z",
        "roles": [
          {
            "uid": "0xc369",
            "name": "superadmin"
          },
          {
            "uid": "0xc36b",
            "name": "manager"
          },
          {
            "uid": "0xc36c",
            "name": "staff"
          }
        ]
      },
      {
        "uid": "0x2720",
        "email": "test5@test.com",
        "username": "test5@test.com",
        "password": "",
        "first_name": "sdfsdf tan",
        "last_name": "sdfdsf sdfdsf",
        "mobile_no": "2423432",
        "office_no": "",
        "gender": "M",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-09-14T09:03:48.808329+08:00",
        "updated_at": "2018-10-16T05:56:07.528755Z",
        "roles": [
          {
            "uid": "0xc36b",
            "name": "manager"
          }
        ]
      },
      {
        "uid": "0x271e",
        "email": "tester@tester.com",
        "username": "tester@tester.com",
        "password": "",
        "first_name": "sdfsdfsdfdsf",
        "last_name": "sdfdsfdsf",
        "mobile_no": "3432423432",
        "office_no": "",
        "gender": "F",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "0001-01-01T00:00:00+06:55",
        "updated_at": "2018-10-16T09:25:17.077452Z",
        "roles": [
          {
            "uid": "0xc36a",
            "name": "admin"
          }
        ]
      },
      {
        "uid": "0x271f",
        "email": "test@test.com",
        "username": "test@test.com",
        "password": "",
        "first_name": "cdscdscds",
        "last_name": "tan",
        "mobile_no": "4324324324324234534",
        "office_no": "",
        "gender": "M",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "0001-01-01T00:00:00+06:55",
        "updated_at": "2018-10-16T09:23:08.259463Z"
      }
    ],
    "users_count": [
      {
        "count": 5
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 75100,
      "processing_ns": 12097800,
      "encoding_ns": 1925700
    },
    "txn": {
      "start_ts": 90490,
      "lin_read": {
        "ids": {
          "1": 4989
        }
      }
    }
  }
}

uid only result:

{
  "data": {
    "users": [
      {
        "uid": "0x2716"
      },
      {
        "uid": "0x2715"
      },
      {
        "uid": "0x2713"
      },
      {
        "uid": "0x2712"
      },
      {
        "uid": "0x271d"
      },
      {
        "uid": "0x2717"
      },
      {
        "uid": "0x271c"
      },
      {
        "uid": "0x271b"
      },
      {
        "uid": "0x2711"
      },
      {
        "uid": "0x2719"
      }
    ],
    "users_count": [
      {
        "count": 5
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 303600,
      "processing_ns": 13727200,
      "encoding_ns": 1918800
    },
    "txn": {
      "start_ts": 90489,
      "lin_read": {
        "ids": {
          "1": 4988
        }
      }
    }
  }
}

(Calum Moore) #10

I’ve also been seeing non-deterministic results when sorted value is missing.


(Michel Conrado (Support Engineer)) #11

This seems a different case, can you fill up an issue with more details?


(James) #12

ive created an issue here:


(system) #13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.