# Dgraph does not work with mounted volumes

**URL:** <https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179>\
**Category:** Dgraph\
**Tags:** badger, status:accepted, kind:bug, good-first-issue, ticket:created\
**Created:** [September 10, 2020, 11:08am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179 "2020-09-10T11:08:22Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![tvvignesh](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tvvignesh/32/4140_2.png) [@tvvignesh](https://discuss.dgraph.io/u/tvvignesh)\
**Post date:** [September 10, 2020, 11:08am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/1 "2020-09-10T11:08:22Z")

</div>

Hi. I use Dgraph inside Docker within an Ubuntu VM from a Windows 10 Host (scripted using Vagrant) and I noticed few issues when using mounted volumes for storing Dgraph data.

## Case 1:

If the mounted volume is an [SMB share](https://www.vagrantup.com/docs/synced-folders/smb.html) , it works without any issues

## Case 2:

If the mounted volume is a [normal synced folder](https://www.vagrantup.com/docs/synced-folders/basic_usage), it fails with thes error which was reported here:

> <https://github.com/dgraph-io/dgraph/issues/2699>
>
> If you suspect this could be a bug, follow the template.
> 
> \- What version of Dg…raph are you using?
> Dgraph version : v1.0.9
> Commit SHA-1 : 22c13fce
> Commit timestamp : 2018-10-02 16:45:53 -0700
> Branch : HEAD
> 
> \- Have you tried reproducing the issue with latest release?
> No
> 
> \- What is the hardware spec (RAM, OS)?
> 12GB RAM, Windows 10 1809 64-bit
> Docker for Windows Edge Beta (Version 2.0.0.0-beta1-win75 (19925) Channel: edge 915f68b)
> Windows Subsystem for Linux (latest Windows 10 1809)
> Ubuntu 18.04 running in WSL with latest updates
> 
> \- Steps to reproduce the issue (command/config used to run Dgraph).
> 1. Install and setup Docker for Windows
> 2. Set your C drive on Windows to be shared with Docker for Windows
> 3. Make sure your C drive is mounted at \`/c/\` and not \`/mnt/c\` in WSL:
> \`\`\`
> sudo nano /etc/wsl.conf
> 
> \# Now make it look like this and save the file when you're done:
> \[automount\]
> root = /
> options = "metadata"
> \`\`\`
> 3. Create the directory \`/c/dgraph\` in WSL.
> 4. Create the dgraph container:
> \`\`\`
> docker run -it -d -p 5080:5080 -p 6080:6080 -p 8080:8080 -p 9080:9080 -p 8000:8000 -v /c/dgraph:/dgraph --name dgraph dgraph/dgraph dgraph zero
> \`\`\`
> 4. Dgraph fails to launch:
> \`\`\`
> Dgraph version : v1.0.9
> Commit SHA-1 : 22c13fce
> Commit timestamp : 2018-10-02 16:45:53 -0700
> Branch : HEAD
> 
> For Dgraph official documentation, visit https://docs.dgraph.io.
> For discussions about Dgraph , visit https://discuss.dgraph.io.
> To say hi to the community , visit https://dgraph.slack.com.
> 
> Licensed under Apache 2.0 + Commons Clause. Copyright 2015-2018 Dgraph Labs, Inc.
> 
> 
> Setting up grpc listener at: 0.0.0.0:5080
> Setting up http listener at: 0.0.0.0:6080
> 2018/10/29 04:17:47 Error while opening WAL store error: sync zw: invalid argument
> While syncing directory: zw.
> github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.syncDir
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:461
> github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.helpRewrite
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/manifest.go:275
> github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.helpOpenOrCreateManifestFile
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/manifest.go:134
> github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.openOrCreateManifestFile
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/manifest.go:116
> github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.Open
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:234
> github.com/dgraph-io/dgraph/dgraph/cmd/zero.run
> /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:181
> github.com/dgraph-io/dgraph/dgraph/cmd/zero.init.0.func1
> /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:59
> github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(\*Command).execute
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:702
> github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(\*Command).ExecuteC
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:783
> github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(\*Command).Execute
> /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:736
> github.com/dgraph-io/dgraph/dgraph/cmd.Execute
> /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:47
> main.main
> /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:24
> runtime.main
> /usr/local/go/src/runtime/proc.go:201
> runtime.goexit
> /usr/local/go/src/runtime/asm\_amd64.s:1333
> \`\`\`
> \- Expected behaviour and actual result.
> Dgraph should launch correctly.

> <https://github.com/dgraph-io/dgraph/issues/1325>
>
> Docker image runs fine on Windows when no volume is mounted.
> \`\`\`
> docker run -it …-p 8080:8080 -p 9090:9090 --name dgraph dgraph/dgraph dgraph --bindall=true
> \`\`\`
> 
> but it throws an error when we try to mount a volume
> \`\`\`
> docker run -it -p 8080:8080 -p 9090:9090 -v ~/dgraph:/dgraph --name dgraph dgraph/dgraph dgraph --bindall=true
> \`\`\`
> 
> Error:
> \`\`\`
> 2017/08/07 10:28:31 Error while creating badger KV WAL store error: sync w: invalid argument 
> Directory entry for compaction log 
> github.com/dgraph-io/badger.newLevelsController 
> /home/travis/gopath/src/github.com/dgraph-io/badger/levels.go:142 
> github.com/dgraph-io/badger.NewKV 
> /home/travis/gopath/src/github.com/dgraph-io/badger/kv.go:199 
> github.com/dgraph-io/dgraph/dgraph.(\*ServerState).initStorage 
> /home/travis/gopath/src/github.com/dgraph-io/dgraph/dgraph/server.go:69 
> github.com/dgraph-io/dgraph/dgraph.NewServerState 
> /home/travis/gopath/src/github.com/dgraph-io/dgraph/dgraph/server.go:54 
> main.main 
> /home/travis/gopath/src/github.com/dgraph-io/dgraph/cmd/dgraph/main.go:688 
> runtime.main 
> /home/travis/.gimme/versions/go1.8.linux.amd64/src/runtime/proc.go:185 
> runtime.goexit 
> /home/travis/.gimme/versions/go1.8.linux.amd64/src/runtime/asm\_amd64.s:2197
> \`\`\`
> 
> The error is while trying to do a \`syncDir\` in badger. From what I understand the \`syncDir\` should be performed in the Ubuntu Docker container and then Docker should handle syncing the directory with that on the host but it looks like its trying to fsync directly to Windows.

(I don’t use WSL but a normal Ubuntu VM)

## Case 3:

If I don’t use the shared folder (from host) within the VM, it works.

So, basically the problem is with the shared folder getting mounted from Windows host to Ubuntu VM.

---

<div class="post-metadata">

**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\
**Post date:** [September 11, 2020, 5:54am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/2 "2020-09-11T05:54:50Z")

</div>

@joaquin any ideas what’s going on?

---

<div class="post-metadata">

**Author:** ![tvvignesh](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tvvignesh/32/4140_2.png) [@tvvignesh](https://discuss.dgraph.io/u/tvvignesh)\
**Post date:** [September 11, 2020, 2:55pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/3 "2020-09-11T14:55:10Z")

</div>

While this may no longer be needed for me since I have moved away from Vagrant to hosting my dev env online, it might be relevant to others who face such issues.

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 11, 2020, 4:40pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/4 "2020-09-11T16:40:02Z")

</div>

This might be related to [[Windows] potential directory sync API misuse · Issue #699 · dgraph-io/badger · GitHub](https://github.com/dgraph-io/badger/issues/699) which has been fixed.

---

<div class="post-metadata">

**Author:** ![joaquin](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/joaquin/32/2619_2.png) [@joaquin](https://discuss.dgraph.io/u/joaquin)\
**Post date:** [September 14, 2020, 4:24pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/5 "2020-09-14T16:24:31Z")

</div>

@tvvignesh What is the provider used with Vagrant? Hyper-V? Virtualbox? Do you have a **`Vagrantfile`** you could share with us?

---

<div class="post-metadata">

**Author:** ![tvvignesh](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tvvignesh/32/4140_2.png) [@tvvignesh](https://discuss.dgraph.io/u/tvvignesh)\
**Post date:** [September 15, 2020, 4:29pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/6 "2020-09-15T16:29:48Z")

</div>

@joaquin Virtualbox without HyperV on Win 10 Home with SMB mount. Sure. You can find the vagrantfile below (changed a few things, but you may want to customize it to make it run on your side)

```auto
# -*- mode: ruby -*-
# vi: set ft=ruby :

## PLEASE CHANGE THESE ACCORDING TO YOUR OWN NEED

BOX_BASE = "bento/ubuntu-20.04"
# Custom Name for the Vagrant box
BOX_NAME = "tc-ubuntu-workspace"
# amount of RAM for Vagrant box
BOX_RAM_MB = "6144"
# number of CPUs for Vagrant box
BOX_CPU_COUNT = "4"
# Private IP of the BOX
BOX_IP_PRIVATE = "192.168.30.10"
# Hostname of the BOX
HOSTNAME = "host.my"
# Your Email
EMAIL = "xyz@example.com"
# Your Email
FULL_NAME = "My Name"
# SMB USERNAME
SMB_USERNAME = "xyz@example.com"

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|

  config.vm.hostname = HOSTNAME

  config.vm.box = BOX_BASE

  config.ssh.username = "vagrant"

  config.vm.network "forwarded_port", guest: 22, host: 2222, host_ip: "127.0.0.1", id: 'ssh'

  # Create a private network, which allows host-only access to the machine using a specific IP.
  config.vm.network "private_network", ip: BOX_IP_PRIVATE

  config.vm.box_check_update = true

   if Vagrant::Util::Platform.windows? then
     config.vm.synced_folder "../workspace", "/opt/workspace", type: "smb", smb_username: SMB_USERNAME, mount_options: ["mfsymlinks"], owner: "vagrant", group: "vagrant"
   else
    config.vm.synced_folder "../workspace", "/opt/workspace", owner: "vagrant", group: "vagrant"
   end

  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  #
  config.vm.provider "virtualbox" do |vb|
    vb.name = BOX_NAME
    # Display the VirtualBox GUI when booting the machine
    # vb.gui = true

    vb.customize ["modifyvm", :id, "--cableconnected1", "on"]

    vb.customize ["modifyvm", :id, "--cpus", BOX_CPU_COUNT]

    vb.memory = BOX_RAM_MB

    vb.customize ["modifyvm", :id, "--accelerate3d", "off"]
    # vb.customize ["modifyvm", :id, "--vram", "128"]

    # integration with host
    vb.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
    # vb.customize ["modifyvm", :id, "--draganddrop", "bidirectional"]

    # vb.customize ["modifyvm", :id, "--audio", "dsound"]
    # vb.customize ["modifyvm", :id, "--audioout", "on"]
    # vb.customize ["modifyvm", :id, "--audioin", "on"]
    # vb.customize ["modifyvm", :id, "--graphicscontroller", "vmsvga"]
    # Default host uses a USB mouse instead of PS2
    #vb.customize ["modifyvm", :id, "--mouse", "usb"]
    # Enable the use of hardware virtualization extensions (Intel VT-x or AMD-V) in the processor of your host system
    # vb.customize ["modifyvm", :id, "--hwvirtex", "on"]
    # it is also required if you want to use more than one virtual CPU in a VM.
    # vb.customize ["modifyvm", :id, "--ioapic", "on"]

    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/vagrant","1"]
    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/opt/workspace","1"]
  end

  # PROVISION SHELL SCRIPTS

  # IMPORT SSH KEYS

  config.vm.provision "file", source: "../my-ssh-keys/id_ed25519.pub", destination: "/home/vagrant/.ssh/id_ed25519.pub"
  config.vm.provision "file", source: "../my-ssh-keys/id_ed25519", destination: "/home/vagrant/.ssh/id_ed25519"

  config.vm.provision "shell", inline: "cat /home/vagrant/.ssh/id_ed25519.pub >> /home/vagrant/.ssh/authorized_keys"

  # UPDATE OS, SETUP GUI, INSTALL ALL THE SOFTWARE AND TOOLS REQUIRED

  # config.vm.provision "shell", path: "./scripts/initial-scripts.sh"
  # config.vm.provision "shell", path: "./scripts/software-setup.sh"
  # config.vm.provision "shell", path: "./scripts/docker.sh"
  # config.vm.provision "shell", path: "./scripts/velero.sh"
  # config.vm.provision "shell", path: "./scripts/terraform.sh"
  # config.vm.provision "shell", path: "./scripts/postinstall.sh", env: {"EMAIL" => EMAIL, "FULL_NAME" => FULL_NAME}
  # TRIGGERS

  config.trigger.after [:provision] do |t|
    t.name = "Reboot after provisioning"
    t.run = { :inline => "vagrant reload" }
  end
end

```

---

<div class="post-metadata">

**Author:** ![joaquin](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/joaquin/32/2619_2.png) [@joaquin](https://discuss.dgraph.io/u/joaquin)\
**Post date:** [September 18, 2020, 10:29pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/7 "2020-09-18T22:29:35Z")

</div>

As a compare point, I wanted to try this out Win10Pro w/ Hyper/V. I have another system with Win10Home + Virtualbox, which I will try next.

These will be the steps I use for this test.

```auto
vagrant up --no-provision
vagrant ssh

```

On the guest system I then run this:

```bash
sudo su
curl -OsL https://gist.githubusercontent.com/darkn3rd/ad9f736b67cc552412736cf09be79c88/raw/422ed657385f16a8c36e77261b1150af45923001/dgraph_dev_env.sh
bash dgraph_dev_env.sh

```

Then I log out and log in as vagrant, so that I pick up access to docker and docker-compose is in the path:

```bash
cd /opt/workarea
docker-compose up -d

```

I slightly modified the `Vagrantfile`:

```ruby
# -*- mode: ruby -*-
# vi: set ft=ruby :

BOX_BASE = "bento/ubuntu-20.04"
BOX_NAME = "tc-ubuntu-workspace"
BOX_RAM_MB = "6144"
BOX_CPU_COUNT = "4"
BOX_IP_PRIVATE = "192.168.30.10"
HOSTNAME = "host.my"
EMAIL = "xyz@example.com"
FULL_NAME = "My Name"
SMB_USERNAME = "xyz@example.com"

Vagrant.configure("2") do |config|
  config.vm.hostname = HOSTNAME
  config.vm.box = BOX_BASE
  config.ssh.username = "vagrant"
  config.vm.network "forwarded_port", guest: 22, host: 2222, host_ip: "127.0.0.1", id: 'ssh'
  config.vm.network "private_network", ip: BOX_IP_PRIVATE
  config.vm.box_check_update = true
  
# if Vagrant::Util::Platform.windows? then
# config.vm.synced_folder "./workspace", "/opt/workspace", type: "smb", smb_username: SMB_USERNAME, mount_options: ["mfsymlinks"], owner: "vagrant", group: "vagrant"
# else
    config.vm.synced_folder "./workspace", "/opt/workspace", owner: "vagrant", group: "vagrant"
# end

  config.vm.provider "virtualbox" do |vb|
    vb.name = BOX_NAME
    vb.customize ["modifyvm", :id, "--cableconnected1", "on"]
    vb.customize ["modifyvm", :id, "--cpus", BOX_CPU_COUNT]
    vb.memory = BOX_RAM_MB
    vb.customize ["modifyvm", :id, "--accelerate3d", "off"]
    vb.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/vagrant","1"]
    vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/opt/workspace","1"]
  end
end

```

---

<div class="post-metadata">

**Author:** ![joaquin](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/joaquin/32/2619_2.png) [@joaquin](https://discuss.dgraph.io/u/joaquin)\
**Post date:** [September 19, 2020, 12:08am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/8 "2020-09-19T00:08:59Z")

</div>

## Test Results

| Vagrant Provider | Share Type | Results |
| --- | --- | --- |
| Hyper/V | SMB | pass |
| Virtualbox | SMB | pass |
| Virtualbox | default sync | fail |

## Zero Logs

```auto
[Decoder]: Using assembly version of decoder
[Sentry] 2020/09/19 00:04:41 Integration installed: ContextifyFrames
[Sentry] 2020/09/19 00:04:41 Integration installed: Environment
[Sentry] 2020/09/19 00:04:41 Integration installed: Modules
[Sentry] 2020/09/19 00:04:41 Integration installed: IgnoreErrors
[Decoder]: Using assembly version of decoder
[Sentry] 2020/09/19 00:04:41 Integration installed: ContextifyFrames
[Sentry] 2020/09/19 00:04:41 Integration installed: Environment
[Sentry] 2020/09/19 00:04:41 Integration installed: Modules
[Sentry] 2020/09/19 00:04:41 Integration installed: IgnoreErrors
I0919 00:04:42.029401 15 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
I0919 00:04:42.204425 15 init.go:102]

Dgraph version : v20.07.1
Dgraph codename : shuri-1
Dgraph SHA-256 : 29d6804d57da4b9ddcc2173c5fe1631b97031d1b364b8b75e52cfa180ec36de6
Commit SHA-1 : d7123d93e
Commit timestamp : 2020-09-17 14:57:43 -0700
Branch : HEAD
Go version : go1.14.4

For Dgraph official documentation, visit https://dgraph.io/docs/.
For discussions about Dgraph , visit http://discuss.hypermode.com.
To say hi to the community , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.

I0919 00:04:42.204682 15 run.go:127] Setting up grpc listener at: 0.0.0.0:5080
I0919 00:04:42.205718 15 run.go:127] Setting up http listener at: 0.0.0.0:6080
I0919 00:04:42.206477 15 run.go:298] Opening zero BadgerDB with options: {Dir:zw ValueDir:zw SyncWrites:false TableLoadingMode:2 ValueLogLoadingMode:2 NumVersionsToKeep:1 ReadOnly:false Truncate:true Logger:0xc00006d3b0 Compression:2 InMemory:false MaxTableSize:67108864 LevelSizeMultiplier:10 MaxLevels:7 ValueThreshold:1048576 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 KeepL0InMemory:false BlockCacheSize:0 IndexCacheSize:0 LoadBloomsOnOpen:false NumLevelZeroTables:5 NumLevelZeroTablesStall:15 LevelOneSize:268435456 ValueLogFileSize:67108864 ValueLogMaxEntries:1000000 NumCompactors:2 CompactL0OnClose:true LogRotatesToFlush:2 ZSTDCompressionLevel:3 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s BypassLockGuard:false ChecksumVerificationMode:0 DetectConflicts:true managedTxns:false maxBatchCount:0 maxBatchSize:0}
[Sentry] 2020/09/19 00:04:42 Sending fatal event [a1e0d417f4cc441b84dce20eb5a94b18] to o318308.ingest.sentry.io project: 1805390
2020/09/19 00:04:42 sync zw: invalid argument
While syncing directory: zw.
github.com/dgraph-io/badger/v2.syncDir
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/dir_unix.go:115
github.com/dgraph-io/badger/v2.helpRewrite
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:294
github.com/dgraph-io/badger/v2.helpOpenOrCreateManifestFile
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:147
github.com/dgraph-io/badger/v2.openOrCreateManifestFile
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:127
github.com/dgraph-io/badger/v2.Open
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/db.go:286
github.com/dgraph-io/dgraph/dgraph/cmd/zero.run
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:300
github.com/dgraph-io/dgraph/dgraph/cmd/zero.init.0.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:79
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
Error while opening WAL store
github.com/dgraph-io/dgraph/x.Checkf
        /ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:51
github.com/dgraph-io/dgraph/dgraph/cmd/zero.run
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:301
github.com/dgraph-io/dgraph/dgraph/cmd/zero.init.0.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:79
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373

```

**Alpha Logs**

```auto
[Decoder]: Using assembly version of decoder
[Sentry] 2020/09/19 00:04:41 Integration installed: ContextifyFrames
[Sentry] 2020/09/19 00:04:41 Integration installed: Environment
[Sentry] 2020/09/19 00:04:41 Integration installed: Modules
[Sentry] 2020/09/19 00:04:41 Integration installed: IgnoreErrors
[Decoder]: Using assembly version of decoder
[Sentry] 2020/09/19 00:04:41 Integration installed: ContextifyFrames
[Sentry] 2020/09/19 00:04:41 Integration installed: Environment
[Sentry] 2020/09/19 00:04:41 Integration installed: Modules
[Sentry] 2020/09/19 00:04:41 Integration installed: IgnoreErrors
I0919 00:04:42.017222 14 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
I0919 00:04:42.018316 14 util_ee.go:126] KeyReader instantiated of type <nil>
I0919 00:04:42.204421 14 init.go:102]

Dgraph version : v20.07.1
Dgraph codename : shuri-1
Dgraph SHA-256 : 29d6804d57da4b9ddcc2173c5fe1631b97031d1b364b8b75e52cfa180ec36de6
Commit SHA-1 : d7123d93e
Commit timestamp : 2020-09-17 14:57:43 -0700
Branch : HEAD
Go version : go1.14.4

For Dgraph official documentation, visit https://dgraph.io/docs/.
For discussions about Dgraph , visit http://discuss.hypermode.com.
To say hi to the community , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.

I0919 00:04:42.204436 14 run.go:714] x.Config: {PortOffset:0 QueryEdgeLimit:1000000 NormalizeNodeLimit:10000 PollInterval:1s GraphqlExtension:true}
I0919 00:04:42.204453 14 run.go:715] x.WorkerConfig: {ExportPath:export NumPendingProposals:256 Tracing:0.01 MyAddr:alpha1:7080 ZeroAddr:[zero1:5080] RaftId:0 WhiteListedIPRanges:[{Lower:10.0.0.0 Upper:10.255.255.255} {Lower:172.16.0.0 Upper:172.31.255.255} {Lower:192.168.0.0 Upper:192.168.255.255} {Lower:172.20.0.0 Upper:172.31.255.255}] MaxRetries:-1 StrictMutations:false AclEnabled:false AbortOlderThan:5m0s SnapshotAfter:10000 ProposedGroupId:0 StartTime:2020-09-19 00:04:41.655328533 +0000 UTC m=+0.016049008 LudicrousMode:false EncryptionKey: **** LogRequest:0}
I0919 00:04:42.204486 14 run.go:716] worker.Config: {PostingDir:p BadgerTables:mmap BadgerVlog:mmap BadgerWalTables:mmap BadgerWalVlog:mmap BadgerCompressionLevel:3 WALDir:w MutationsMode:0 AuthToken: AllottedMemory:1024 PBlockCacheSize:0 PIndexCacheSize:0 WBlockCacheSize:0 WIndexCacheSize:0 HmacSecret: **** AccessJwtTtl:0s RefreshJwtTtl:0s AclRefreshInterval:0s}
I0919 00:04:42.205056 14 server_state.go:78] Setting Badger Compression Level: 3
I0919 00:04:42.205066 14 server_state.go:99] Setting Badger table load option: mmap
I0919 00:04:42.205069 14 server_state.go:111] Setting Badger value log load option: mmap
I0919 00:04:42.205072 14 server_state.go:150] Opening write-ahead log BadgerDB with options: {Dir:w ValueDir:w SyncWrites:false TableLoadingMode:2 ValueLogLoadingMode:2 NumVersionsToKeep:1 ReadOnly:false Truncate:true Logger:0x2be83e8 Compression:2 InMemory:false MaxTableSize:67108864 LevelSizeMultiplier:10 MaxLevels:7 ValueThreshold:1048576 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 KeepL0InMemory:false BlockCacheSize:0 IndexCacheSize:0 LoadBloomsOnOpen:false NumLevelZeroTables:5 NumLevelZeroTablesStall:15 LevelOneSize:268435456 ValueLogFileSize:1073741823 ValueLogMaxEntries:10000 NumCompactors:2 CompactL0OnClose:true LogRotatesToFlush:2 ZSTDCompressionLevel:3 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s BypassLockGuard:false ChecksumVerificationMode:0 DetectConflicts:false managedTxns:false maxBatchCount:0 maxBatchSize:0}
[Sentry] 2020/09/19 00:04:42 Sending fatal event [da497c698ab74ab085a92806bb381616] to o318308.ingest.sentry.io project: 1805390
2020/09/19 00:04:42 sync w: invalid argument
While syncing directory: w.
github.com/dgraph-io/badger/v2.syncDir
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/dir_unix.go:115
github.com/dgraph-io/badger/v2.helpRewrite
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:294
github.com/dgraph-io/badger/v2.helpOpenOrCreateManifestFile
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:147
github.com/dgraph-io/badger/v2.openOrCreateManifestFile
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/manifest.go:127
github.com/dgraph-io/badger/v2.Open
        /go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.2-0.20200827131741-d5a25b83fbf4/db.go:286
github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:153
github.com/dgraph-io/dgraph/worker.InitServerState
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:54
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:718
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:97
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
Error while creating badger KV WAL store
github.com/dgraph-io/dgraph/x.Checkf
        /ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:51
github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:154
github.com/dgraph-io/dgraph/worker.InitServerState
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:54
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:718
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:97
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373

```

@ibrahim Do you have any suggestions? Is there anything I can try out?

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 19, 2020, 9:29am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/10 "2020-09-19T09:29:39Z")

</div>

The `file.Sync` call here is failing because zw is a directory.

> <https://github.com/dgraph-io/badger/blob/536f6d74c52667bf485f53568c51e7fce1e9ed3a/dir_unix.go#L112>

We might be misusing the sync API. This should be easy to fix.

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [September 20, 2020, 5:57pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/14 "2020-09-20T17:57:09Z")

</div>

It looks like fsync and virtual box shared directories don’t work well together. Calling `fsync` on a shared volume returns a `invalid argument` error (this is fsync returning error, not badger or dgraph).

> <https://stackoverflow.com/questions/30232772/fsync-on-newly-created-folder/30263126#30263126>

> <https://github.com/ipfs/go-ds-flatfs/issues/28#issue-297979457>
>
> Basically, we try to sync a directory, windows fails with "invalid argument" ~(w…indows doesn't support syncing directories) and vbox forwards this error~.
> 
> ~We actually have a check to avoid syncing directories on windows. Unfortunately... in this case, we think we're on Linux. I believe the correct fix is to detect that the error is an "invalid argument" error and ignore it (we can't sync anyways, nothing really failed).~
> 
> 
> \---
> 
> Edit: It turns out that vboxsf just doesn't support fsync on directories at all. Really, we probably should just be ignoring fsync errors on directories entirely (possibly logging them?). There's not much we can do anyways.

The only thing we can do is ignore the `fsync` error but I don’t think people use dgraph or badger very often on shared virtual box volumes and so we don’t need to do anything.

---

<div class="post-metadata">

**Author:** ![joaquin](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/joaquin/32/2619_2.png) [@joaquin](https://discuss.dgraph.io/u/joaquin)\
**Post date:** [September 21, 2020, 1:19am UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/15 "2020-09-21T01:19:15Z")

</div>

I was digging around for vboxsf vs. fsync() and found these

[https://forums.virtualbox.org/viewtopic.php?f=1&t=87454](https://forums.virtualbox.org/viewtopic.php?f=1&t=87454)

and comments in the source code (not sure if this is what implements it):

[https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Additions/linux/sharedfolders/regops.c](https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Additions/linux/sharedfolders/regops.c)

I conclusion it seems the only route is to not sure vboxsf as it doesn’t support fsync, and use a network file share like SMB or NFS.

---

<div class="post-metadata">

**Author:** ![tvvignesh](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tvvignesh/32/4140_2.png) [@tvvignesh](https://discuss.dgraph.io/u/tvvignesh)\
**Post date:** [September 24, 2020, 5:34pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/16 "2020-09-24T17:34:53Z")

</div>

@ibrahim @joaquin Interesting. I had my own fair share of trouble with Virtualbox and Volumes. Even SMB has issues (not with Dgraph but with other tools where it takes 10x the time to run a script over a normal synced folder) and NFS is not officially supported in Windows Host (Sadly had to use windows as the host because loads of applications run only on windows 😟 ). That is why I shifted my entire dev env online to GCE and everything is great now.

Going plain docker was not advisable for me since I wanted the entire dev env setup, rather I run containers within the VM. This might occur to anyone who actually uses VM for development. You might want to add this somewhere in the website sometime later.

Great response as always guys. Cheers.

---

<div class="post-metadata">

**Author:** ![joaquin](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/joaquin/32/2619_2.png) [@joaquin](https://discuss.dgraph.io/u/joaquin)\
**Post date:** [September 24, 2020, 8:00pm UTC](https://discuss.dgraph.io/t/dgraph-does-not-work-with-mounted-volumes/10179/17 "2020-09-24T20:00:57Z")

</div>

@tvvignesh Nice to hear.

On Windows side of things: NFS is supported, but well, only on the servers (ref: [NFS Overview](https://docs.microsoft.com/en-us/windows-server/storage/nfs/nfs-overview)). With SMB, I would recommend sure to use SMB3.0 with vagrant, as the default SMB2.0 has problems:

```ruby
smb_sync_opts = { type: "smb", mount_options: %w[mfsymlinks vers=3.0] }
config.vm.synced_folder ".", "/vagrant", smb_sync_opts

```

On VMs vs Docker: Docker is insanely popular for local dev environments, but for myself personally, I find on some occasions, such as simulating full CI/CD scenario, I need VMs.

I added some docs under systemd for using vagrant to test dgraph with systemd configuration for Ubuntu 18.04 and CentOS 8 with Vagrant using either Libvirt (KVM) and Virtualbox providers:

- [dgraph/contrib/systemd/ha\_cluster/tests at master · dgraph-io/dgraph · GitHub](https://github.com/dgraph-io/dgraph/tree/master/contrib/systemd/ha_cluster/tests)

Updated with above link with support for SMB3.0 as default for Virtualbox (on Windows) and Hyper/V providers.