Hello, and congratulation on the great project.
I am evaluating the possible use of Dgraph for a solution within my company.
I started from a JSON Wikidata dump, simplified the structure a bit by removing references, using English data only, and keeping all the connecting properties while removing many scalar ones, as they are not interesting to us. The final result has around 100M vertices and more than 1B edges; I imported using bulk mode.
I deployed a single physical (on-premise) machine cluster with three zero and six alpha nodes, split in two groups, using Docker; the machine has 48 cores and 64GB RAM. Now, running a term query on a property called label (schema entry: string @index(term) ) I get unacceptable slowness (see timing below, 1’ 41"). I am also noting that, while running the query, the system seems to be mostly idle, check the CPU metric after the command.
I am adding some details below; if this is interesting for you, I can provide the entire database (bulk loaded), as well as my docker-compose.yml, just let me know.
Thanks so much, have a great day
giuseppe
Details
root@d055ae8f9bc8:/data# dgraph version
Dgraph version : v21.03.1
Dgraph codename : rocket-1
Dgraph SHA-256 : a00b73d583a720aa787171e43b4cb4dbbf75b38e522f66c9943ab2f0263007fe
Commit SHA-1 : ea1cb5f35
Commit timestamp : 2021-06-17 20:38:11 +0530
Branch : HEAD
Go version : go1.16.2
jemalloc enabled : true
time curl -X POST http://localhost:28080/query -H ‘Content-Type: application/dql’ -d ‘{ var( func: allofterms(label, “General
Motors Company”) ) { A as QID } var( func: allofterms(aliases, “General Motors Company”) ) { B as QID } WD( func: uid(A,
B) ) { name label aliases description P31 { QID } P279 { QID } } }’
{“data”:{“WD”:[{“name”:“Q29570”,“label”:“Chevrolet”,“aliases”:[“Chevy”,“Chevrolet Division of General Motors Company y Nahuelito Gimenez”],“description”:“American automobile division of GM”,“P31”:[{“QID”:“Q10429667”},{“QID”:“Q334453”},{“QID”:“Q786820”}]},{“name”:“Q81965”,“label”:“General Motors”,“aliases”:[“GM”,“General Motors Corporation”,“General Motors Company, LLC”,“GMC”],“description”:“automotive manufacturing corporation based in Detroit, Michigan, USA”,“P31”:[{“QID”:“Q891723”},{“QID”:“Q21980538”},{“QID”:“Q786820”}]},{“name”:“Q67604297”,“label”:“Pilot Life Insurance Company, Petitioner v. Everate W. Dedeaux [Case 85-1043] / Metropolitan Life Insurance Company, Petitioner v. Arthur Taylor [Case 85-686] / General Motors Corporation, Petitioner v. Arthur Taylor. [Case 85-688 (NAID 118974763)”,“description”:“item in the National Archives and Records Administration’s holdings”,“P31”:[{“QID”:“Q18593264”}]},{“name”:“Q19035456”,“label”:“Pick Manufacturing Company v. General Motors Corporation”,“P31”:[{“QID”:“Q2334719”}]},{“name”:“Q59356336”,“label”:“224 West 57th Street”,“aliases”:[“Demarest and Peerless Company Building”,“Demarest \u0026 Peerless Building”,“General Motors Building”,“A. T. Demarest \u0026 Company Building”,“Argonaut Building”,“Peerless Motor Car Company Building”],“description”:“Commercial building in Manhattan, New York”,“P31”:[{“QID”:“Q1021645”}]},{“name”:“Q26349054”,“label”:“HHE determination report no. HHE-73-73-143, Inland Manufacturing Company, General Motors Corporation, Dayton, Ohio”,“description”:“field study”,“P31”:[{“QID”:“Q1402850”}]},{“name”:“Q67604509”,“label”:“Eastman Kodak Company, Petitioner v. Image Technical Services, Inc., et al. [Case 90-1029] / General motors Corporation, et al., Petitioners v. Evert Romein, et al. [Case 90-1390] / United States, Petitioner v. R. L. C. [Case 90-1 (NAID 118974953)”,“description”:“item in the National Archives and Records Administration’s holdings”,“P31”:[{“QID”:“Q18593264”}]}]},“extensions”:{“server_latency”:{“parsing_ns”:928457,“processing_ns”:101801347836,“encoding_ns”:145195,“assign_timestamp_ns”:724950,“total_ns”:101803227071},“txn”:{“start_ts”:1400},“metrics”:{“num_uids”:{"":0,“P279”:7,“P31”:7,“QID”:16,"_total":58,“aliases”:7,“description”:7,“label”:7,“name”:7}}}}
0.01s user 0.02s system 0% cpu 1:41.86 total
