How to add "Date Type Facet" by using Dgraph4j

BlueRhino · November 23, 2018, 11:23am

Hi all,
Does Dgraph4J supports add “Date Type Facet” to a Predicate?
My code is:

DgraphProto.Mutation.Builder builder = DgraphProto.Mutation.newBuilder();
DgraphProto.NQuad.Builder builder1 = DgraphProto.NQuad.newBuilder().setSubject("_:e").setPredicate(
"address").setObjectValue(DgraphProto.Value.newBuilder().setStrVal(Long.toString(System.currentTimeMillis()/1000)).build());
builder1.addFacets(DgraphProto.Facet.newBuilder().setKey("sendDate").setValType(DgraphProto.Facet.ValType.DATETIME).setValue(ByteString.copyFromUtf8("1136214246L")));
builder.addSet(builder1.build());
transaction.mutate(builder.build());
transaction.commit();

Then I got an Exception:

Exception in thread "main" io.grpc.StatusRuntimeException: UNKNOWN: Error while parsing facet: [key:"sendDate" value:"1136214246L" val_type:DATETIME ] error: Time.UnmarshalBinary: unsupported version
	at io.grpc.Status.asRuntimeException(Status.java:526)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:434)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	............

Then I change String “1136214246L” to “2007-05-01T15:43:26.3452-07:00”.But It doesn’t work.
SO,could anyone tell me How to add “Date Type Facet” using Dgraph4j ?
Thanks

MichelDiz · November 23, 2018, 2:37pm

Is it RFC3339?

Note

Dgraph supports date and time formats for dateTime scalar type only if they are RFC 3339 compatible which is different from ISO 8601(as defined in the RDF spec). You should convert your values to RFC 3339 format before sending them to Dgraph.

time.Time (RFC3339 format [Optional timezone] eg: 2006-01-02T15:04:05.999999999+10:00 or 2006-01-02T15:04:05.999999999)

Source: Get started with Dgraph

BUT, Since this is Facets, you can add whatever you want there. This error must be something else. Add it as string.

ps. I’m not a Java Dev.

BlueRhino · November 24, 2018, 11:17am

Thank you for your replay.I try “RFC 3339” and “Long” format.But the result is the same.
I know I can insert it as a string, but is there a performance problem ?
In other words, does Dgraph optimize for “Date Type”?

MichelDiz · November 24, 2018, 1:12pm

I don’t think so, just testing it.

If it appears in the Facet in the right format. Dgraph will deal with date as it intent to.

You can also try to do what you doing with a JSON object (JSON mutation). That’s a good way to be sure that you may doing something wrong with the procedures in JAVA. If you are able to insert the Date Type as you’re doing but in JSON, so you have a clue.

Cheers

BlueRhino · November 24, 2018, 1:16pm

Ok,Thanks.

I’m going to try JSON to find a clue.

ps. I prefer to use RDF:)

gitlw · November 27, 2018, 10:09pm

The error occurred because the server is expecting a binary format of time

github.com

dgraph-io/dgraph/blob/master/types/conversion.go#L77


					return to, errors.Errorf("Invalid data for float %v", data)
				}
				i := binary.LittleEndian.Uint64(data)
				*res = math.Float64frombits(i)
			case BoolID:
				if len(data) == 0 || data[0] == 0 {
					*res = false
					return to, nil
				} else if data[0] == 1 {
					*res = true
					return to, nil
				}
				return to, errors.Errorf("Invalid value for bool %v", data[0])
			case DateTimeID:
				var t time.Time
				if err := t.UnmarshalBinary(data); err != nil {
					return to, err
				}
				*res = t
			case GeoID:
				w, err := wkb.Unmarshal(data)

If the client is go, the binary format should be created using the Marshal method in the time package
https://golang.org/src/time/time.go

But since this is the java client and it seems the implementation above is specific to go, we’ll need to figure something out. Will get back to this shortly after discussing with others internally.

gitlw · November 27, 2018, 10:25pm

Also on the server side, we expect the source format of facet to be “BinaryID”

github.com

dgraph-io/dgraph/blob/master/types/facets/utils.go#L197


/*
 * Copyright 2017-2018 Dgraph Labs, Inc. and Contributors
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package facets

import (
	"math"
	"sort"
	"strconv"
	"unicode"

	"github.com/dgraph-io/dgo/v200/protos/api"
	"github.com/dgraph-io/dgraph/protos/pb"
	"github.com/dgraph-io/dgraph/tok"
	"github.com/dgraph-io/dgraph/types"
	"github.com/pkg/errors"
)

// SortAndValidate sorts And validates the facets.
func SortAndValidate(fs []*api.Facet) error {
	if len(fs) == 0 {
		return nil
	}
	sort.Slice(fs, func(i, j int) bool {
		return fs[i].Key < fs[j].Key
	})
	for i := 1; i < len(fs); i++ {
		if fs[i-1].Key == fs[i].Key {
			return errors.Errorf("Repeated keys are not allowed in facets. But got %s",
				fs[i].Key)
		}
	}
	return nil
}

// CopyFacets makes a copy of facets of the posting which are requested in param.Keys.
func CopyFacets(fcs []*api.Facet, param *pb.FacetParams) (fs []*api.Facet) {
	if param == nil || fcs == nil {
		return nil
	}
	// facets and param.keys are both sorted,
	// We also need all keys if param.AllKeys is true.
	numKeys := len(param.Param)
	numFacets := len(fcs)
	for kidx, fidx := 0, 0; (param.AllKeys || kidx < numKeys) && fidx < numFacets; {
		f := fcs[fidx]
		switch {
		case param.AllKeys || param.Param[kidx].Key == f.Key:
			fcopy := &api.Facet{
				Key:     f.Key,
				Value:   nil,
				ValType: f.ValType,
			}
			if !param.AllKeys {
				fcopy.Alias = param.Param[kidx].Alias
			}
			fcopy.Value = make([]byte, len(f.Value))
			copy(fcopy.Value, f.Value)
			fs = append(fs, fcopy)
			kidx++
			fidx++
		case f.Key > param.Param[kidx].Key:
			kidx++
		default:
			fidx++
		}
	}
	return fs
}

// valAndValType returns interface val and valtype for facet.
func valAndValType(val string) (interface{}, api.Facet_ValType, error) {
	if len(val) == 0 { // empty string case
		return "", api.Facet_STRING, nil
	}
	// strings should be in quotes.
	if len(val) >= 2 && val[0] == '"' && val[len(val)-1] == '"' {
		uq, err := strconv.Unquote(val)
		return uq, api.Facet_STRING, errors.Wrapf(err, "could not unquote %q:", val)
	}
	if intVal, err := strconv.ParseInt(val, 0, 64); err == nil {
		return int64(intVal), api.Facet_INT, nil
	} else if numErr := err.(*strconv.NumError); numErr.Err == strconv.ErrRange {
		// if we have only digits in val, then val is a big integer : return error
		// otherwise try to parse as float.
		allNumChars := true
		for _, v := range val {
			if !unicode.IsDigit(v) {
				allNumChars = false
				break
			}
		}
		if allNumChars {
			return nil, api.Facet_INT, err
		}
	}
	if floatVal, err := strconv.ParseFloat(val, 64); err == nil {
		// We can't store NaN as it is because it serializes into invalid JSON.
		if math.IsNaN(floatVal) {
			return nil, api.Facet_FLOAT, errors.Errorf("Got invalid value: NaN")
		}

		return floatVal, api.Facet_FLOAT, nil
	} else if numErr := err.(*strconv.NumError); numErr.Err == strconv.ErrRange {
		return nil, api.Facet_FLOAT, err
	}
	if val == "true" || val == "false" {
		return val == "true", api.Facet_BOOL, nil
	}
	if t, err := types.ParseTime(val); err == nil {
		return t, api.Facet_DATETIME, nil
	}
	return nil, api.Facet_STRING, errors.Errorf("Could not parse the facet value : [%s]", val)
}

// FacetFor returns Facet for given key and val.
func FacetFor(key, val string) (*api.Facet, error) {
	v, vt, err := valAndValType(val)
	if err != nil {
		return nil, err
	}

	facet, err := ToBinary(key, v, vt)
	if err != nil {
		return nil, err
	}

	if vt == api.Facet_STRING {
		// tokenize val.
		facet.Tokens, err = tok.GetTermTokens([]string{v.(string)})
		if err == nil {
			sort.Strings(facet.Tokens)
		}
	}
	return facet, err
}

// ToBinary converts the given value into a binary value.
func ToBinary(key string, value interface{}, sourceType api.Facet_ValType) (
	*api.Facet, error) {
	// convert facet val interface{} to binary
	sourceTid, err := TypeIDFor(&api.Facet{ValType: sourceType})
	if err != nil {
		return nil, err
	}

	targetVal := &types.Val{Tid: types.BinaryID}
	if err = types.Marshal(types.Val{Tid: sourceTid, Value: value}, targetVal); err != nil {
		return nil, err
	}

	return &api.Facet{Key: key, Value: targetVal.Value.([]byte), ValType: sourceType}, nil
}

// TypeIDFor gives TypeID for facet.
func TypeIDFor(f *api.Facet) (types.TypeID, error) {
	switch f.ValType {
	case api.Facet_INT:
		return types.IntID, nil
	case api.Facet_FLOAT:
		return types.FloatID, nil
	case api.Facet_BOOL:
		return types.BoolID, nil
	case api.Facet_DATETIME:
		return types.DateTimeID, nil
	case api.Facet_STRING:
		return types.StringID, nil
	default:
		return types.DefaultID, errors.Errorf("Unrecognized facet type: %v", f.ValType)
	}
}

// ValFor converts Facet into types.Val.
func ValFor(f *api.Facet) (types.Val, error) {
	val := types.Val{Tid: types.BinaryID, Value: f.Value}
	facetTid, err := TypeIDFor(f)
	if err != nil {
		return types.Val{}, err
	}

	return types.Convert(val, facetTid)
}

This file has been truncated. show original

gitlw · November 29, 2018, 11:59pm

I’ve put a PR to fix this issue Validate and converting facets to binary by gitlw · Pull Request #2797 · dgraph-io/dgraph · GitHub

system · December 29, 2018, 11:59pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Dgraph4j datetime problem Dgraph	3	942	December 2, 2019
Aggregation of Facet with type datetime GraphQL kind:question	6	563	December 23, 2022
Facets aren't working with setSetJson Dgraph Clients untagged , dgraph-js	3	726	July 11, 2020
How to add facets - mutate, graphql and dgo Users	4	925	January 8, 2020
How to correctly set the type dateTime to a predicate from Java Users	2	1180	April 12, 2018

How to add "Date Type Facet" by using Dgraph4j

Related topics