How to update lambda functions for Dgraph v23.0.1

What I want to do

I want to know how to update lambda functions when using Dgraph v23.0.1

What I did

Previously for Dgraph v21.12.0, I used helm install Dgraph chart, I have settings of alpha like this:

alpha:
  extraEnvs:
    - name: lambda
      value: url=http://dgraph-lambda-dgraph-lambda.demo.svc:8686/graphql-worker

And settings of Lambda like this:

image:
  tag: 1.2.4

service:
  type: ClusterIP
  port: 8686
  targetPort: 8686

env:
  - name: DGRAPH_URL
    value: http://dgraph-release-dgraph-alpha-headless.demo.svc:8080

Everything works perfectly. I can send a http post to endpoint: http://localhost:8080/admin using Postman with the body like this to update Lambda functions.

mutation{
	updateLambdaScript(input:{set:{script:
    """async function myFunction({graphql}) {
            const results = await graphql(`query {
                queryProductVersion {
                    version
                    categoriesAggregate  {
                    count
                    }

            }}`)
            return  results.data.queryProductVersion.map(parseResult)
        }


        const parseResult = (item) => {
        return { id: item.version, label: item.version, value: item.categoriesAggregate.count}
        }

        addGraphQLResolvers({
            "Query.myFunction": myFunction,
        })
    """
	
	}}){
		lambdaScript{script}
	}
}

But ever since I upgraded Dgraph to v23.0.1, meanwhile used the same version of lambda and the configuration, even though the changes I made to the settings of alpha works, I can No Longer update lambda functions using the prior method.

alpha:
  extraEnvs:
    - name: DGRAPH_ALPHA_GRAPHQL
      value: lambda-url=http://dgraph-lambda-dgraph-lambda.demo.svc:8686/graphql-worker

Sending a http post to endpoint: http://localhost:8080/admin using Postman with the body as above returns error like this:

    "errors": [
        {
            "message": "Cannot query field \"updateLambdaScript\" on type \"Mutation\".",
            "locations": [
                {
                    "line": 2,
                    "column": 2
                }
            ]
        }
    ],

Then I tried what’s suggested in the doc Lamda on my self-hosted Dgraph on aws with Postman. It returned error like:

Dgraph metadata

Only happens on v23.0.1

Hello…dear friends in Dgraph Community, anybody has time to take a look at this issue?

Not sure if it helps but we are linking our lambda source directly with

docker run -it --name lambda --rm -p 8686:8686 -v $(pwd)/docker/lambda/script.js:/app/script/script.js -e DGRAPH_URL=http://172.17.0.1:8080 dgraph/dgraph-lambda

for our local development. So $(pwd)/docker/lambda/ would be the relative path from wherever you call the docker command (for us it’s the root of the repo) to the directory where you’re scripts.js file sits.

We then have a build script for our lambda source and if script.js updates, the local lambda server restarts with the new source.

This is obviously only a solution for local development.

Not actually cause my team is using Dgraph Lambda that’s already deployed at AWS cloud. Lambda resolver is one of the features to be tested at AWS when we kick off Dgraph performance testing, and it shall be estimated with other stuff to help our company decide if we’re going to purchase enterprise features or not in the future for our project.

1 Like

Any solution for it installed by helm chart on k8s cluster? I’m not a fan of Dgraph cloud and I don’t think my project is gonna use it. But since the Lambda resolver works there, I bet there should be a solution for Prod right?

I suspect you are using a different config setup from v21.12 that’s not compatible with older or newer versions of Dgraph. 21.12 had bugs that necessitated us pulling back to v21.03 and more slowly rolling in the changes and features in 21.12 to release v22 and v23. The lambda changes in v21.12 are not yet incorporated int v23, and 21.12 is no longer a supported version. Regrettably, this includes upgrade testing and so on from 21.12 to v23.

My guess is you may want to rebuild the lambda from the ground up on v23.1.

Separately, we do now recommend people use the cloud service if at all possible. The cost is low relative to the underlying AWS costs, it includes support, 24/7 monitoring by our SRE team, upgrades, etc. The problems you are struggling with about how to set up k8s and deal with version changes may be best handled by getting onto the supported service.

That said, Dgraph does have a helm chart project. It is open-source vs supported code (e.g. the support team is not contractually obligated to handle tickets on it for paid customers) but it works well for most people.

Thanks for the clarification. Veritas Technologies LLC, which is the company that I’m working for, holds a very high level of security scrutiny as compulsory, which does not allow us to expose data on any other 3rd platform. This is the reason why I’m joining this community to search for help for the open-source project.

I’m glad to hear Veritas (which I know is a leader in security) feels confident using Dgraph internally.

For others who’s policies allow cloud DB usage, please know that Dgraph is highly secure, and we are now also SOC2 compliant. For most users the cloud offering is the best option.

For those who need to self-host, you may also consider a Dgraph annual support contract or consulting, query tuning etc. We absolutely respect and support all companies in self-hosting our FOSS software if desired. Enterprise features and support include binary, incremental backup, encryption, CDC, ABAC security rules, 24/7 support, etc. for those who choose to buy vs build those capabilities.

env:
  - name: DGRAPH_URL
    value: http://dgraph-release-dgraph-alpha-headless.demo.svc:8080

This is the only change that I made for Dgraph v23.0.1, nothing changed for Lambda chart. In fact, they’s no error installing and launching both as pod and service, problem seems to occur at the communication between lambda and the alpha.

Noted that the last commit for Lambda chart is on Apr 13, 2021, I wonder whether or not there’s gonna be any contributor continue to maintain and update it. Speaking about rebuild the lambda from the ground up on v23.1, do you mean rebuild the image? Could you elaborate on it? Thanks in advance.

Hi Ann.

I’m not really an expert in the lambda configs, but it may be that the configuration change described here for v21.12 (Zion (v21.12) Documentation and Setting Up a Lambda Server) is used in your old configuration, and the v23 configuration is the same as the v21.03, v22 and prior config options. I suggest looking at the dgraph lambda configuration docs here: Lambda Server - Deploy.

This is only a theory, and I have not compared the v23 and v21.12 configurations. I just noticed there is a change described in the above discuss link.

Self-hosted is certainly the approach we’d like to adopt to continue using Dgraph. Right now we’re still exploring it in our dev environment at AWS, focusing on data writing, query/mutate via its embedded GraphQL api. We haven’t finalize the start time of performance & load testing in our QA environment, but I assume as long as it starts, we’ll discover many issues which if can NOT be solved by the community’s help it shall rely on consulting. I can’t tell whether or not Veritas is going to purchase annual support contract at this moment, it depends on the testing report which helps us decide whether it’s worthy buying the enterprise features & sign annual support contract or not. But it’s a very good start. I hope to be notified once the lambda incorporation is completed, let’s keep in touch. Cheers!