# A lot of sstables, each is 67Mb - is it ok? **URL:** \ **Category:** Badger\ **Created:** [March 29, 2019, 7:02pm UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255 "2019-03-29T19:02:31Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [March 29, 2019, 7:02pm UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/1 "2019-03-29T19:02:32Z")
seems like they are never compacted. only 16Gb is loaded but query performance is way slow. should these files be compacted? how to debug why it’s not happening? thanks ```auto root@ip-172-16-25-70:/dgraph/p# ls -lh total 16G -rw-r--r-- 1 root root 159M Mar 29 16:14 000477.vlog -rw-r--r-- 1 root root 70M Mar 29 09:19 014865.sst -rw-r--r-- 1 root root 70M Mar 29 09:19 014888.sst -rw-r--r-- 1 root root 70M Mar 29 09:19 014889.sst -rw-r--r-- 1 root root 70M Mar 29 09:20 014890.sst -rw-r--r-- 1 root root 70M Mar 29 09:21 014928.sst -rw-r--r-- 1 root root 70M Mar 29 09:21 014931.sst -rw-r--r-- 1 root root 70M Mar 29 09:21 014951.sst -rw-r--r-- 1 root root 70M Mar 29 09:21 014952.sst -rw-r--r-- 1 root root 70M Mar 29 09:21 014954.sst -rw-r--r-- 1 root root 70M Mar 29 09:22 014977.sst -rw-r--r-- 1 root root 70M Mar 29 09:22 014978.sst -rw-r--r-- 1 root root 70M Mar 29 09:22 014997.sst -rw-r--r-- 1 root root 67M Mar 29 09:22 014999.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015023.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015024.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015025.sst -rw-r--r-- 1 root root 8.2K Mar 29 09:23 015026.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015027.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015029.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015053.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015054.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015059.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015062.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015063.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015087.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015088.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015089.sst -rw-r--r-- 1 root root 67M Mar 29 09:23 015091.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015092.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015093.sst -rw-r--r-- 1 root root 70M Mar 29 09:23 015094.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015124.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015125.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015126.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015127.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015150.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015151.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015152.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015153.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015175.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015176.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015177.sst -rw-r--r-- 1 root root 67M Mar 29 09:24 015178.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015201.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015202.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015203.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015232.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015233.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015234.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015235.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015268.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015269.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015270.sst -rw-r--r-- 1 root root 67M Mar 29 09:25 015271.sst -rw-r--r-- 1 root root 67M Mar 29 09:26 015289.sst -rw-r--r-- 1 root root 66M Mar 29 09:26 015290.sst -rw-r--r-- 1 root root 66M Mar 29 09:26 015291.sst -rw-r--r-- 1 root root 66M Mar 29 09:26 015328.sst -rw-r--r-- 1 root root 66M Mar 29 09:27 015339.sst -rw-r--r-- 1 root root 66M Mar 29 09:27 015340.sst -rw-r--r-- 1 root root 66M Mar 29 09:27 015343.sst -rw-r--r-- 1 root root 67M Mar 29 09:28 015359.sst -rw-r--r-- 1 root root 66M Mar 29 09:28 015383.sst -rw-r--r-- 1 root root 66M Mar 29 09:30 015426.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015473.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015474.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015475.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015476.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015491.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015495.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015496.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015497.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015498.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015499.sst -rw-r--r-- 1 root root 67M Mar 29 09:31 015500.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015533.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015534.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015535.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015536.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015537.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015555.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015557.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015560.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015562.sst -rw-r--r-- 1 root root 67M Mar 29 09:32 015563.sst -rw-r--r-- 1 root root 66M Mar 29 09:32 015564.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015583.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015586.sst -rw-r--r-- 1 root root 66M Mar 29 09:33 015588.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015611.sst -rw-r--r-- 1 root root 66M Mar 29 09:33 015612.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015613.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015614.sst -rw-r--r-- 1 root root 66M Mar 29 09:33 015615.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015616.sst -rw-r--r-- 1 root root 66M Mar 29 09:33 015636.sst -rw-r--r-- 1 root root 67M Mar 29 09:33 015637.sst -rw-r--r-- 1 root root 66M Mar 29 09:33 015639.sst -rw-r--r-- 1 root root 66M Mar 29 09:34 015664.sst -rw-r--r-- 1 root root 66M Mar 29 09:34 015665.sst -rw-r--r-- 1 root root 66M Mar 29 09:34 015668.sst -rw-r--r-- 1 root root 67M Mar 29 09:34 015689.sst -rw-r--r-- 1 root root 66M Mar 29 09:35 015720.sst -rw-r--r-- 1 root root 66M Mar 29 09:36 015762.sst -rw-r--r-- 1 root root 67M Mar 29 09:36 015781.sst -rw-r--r-- 1 root root 67M Mar 29 09:36 015783.sst -rw-r--r-- 1 root root 67M Mar 29 09:37 015807.sst -rw-r--r-- 1 root root 66M Mar 29 09:37 015809.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015832.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015833.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015835.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015837.sst -rw-r--r-- 1 root root 3.5K Mar 29 09:38 015838.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015858.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015859.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015860.sst -rw-r--r-- 1 root root 67M Mar 29 09:38 015861.sst -rw-r--r-- 1 root root 4.3K Mar 29 09:38 015863.sst -rw-r--r-- 1 root root 67M Mar 29 09:39 015874.sst -rw-r--r-- 1 root root 1.7K Mar 29 09:39 015878.sst -rw-r--r-- 1 root root 67M Mar 29 09:39 015906.sst -rw-r--r-- 1 root root 794 Mar 29 09:39 015907.sst -rw-r--r-- 1 root root 67M Mar 29 09:40 015933.sst -rw-r--r-- 1 root root 70M Mar 29 09:40 015952.sst -rw-r--r-- 1 root root 67M Mar 29 09:41 015962.sst -rw-r--r-- 1 root root 67M Mar 29 09:41 015963.sst -rw-r--r-- 1 root root 732 Mar 29 09:41 015964.sst -rw-r--r-- 1 root root 67M Mar 29 09:41 015980.sst -rw-r--r-- 1 root root 67M Mar 29 09:41 015982.sst -rw-r--r-- 1 root root 66M Mar 29 09:41 015983.sst -rw-r--r-- 1 root root 66M Mar 29 09:41 016013.sst -rw-r--r-- 1 root root 66M Mar 29 09:41 016014.sst -rw-r--r-- 1 root root 70M Mar 29 09:41 016015.sst -rw-r--r-- 1 root root 66M Mar 29 09:41 016016.sst -rw-r--r-- 1 root root 70M Mar 29 09:41 016019.sst -rw-r--r-- 1 root root 70M Mar 29 09:41 016021.sst -rw-r--r-- 1 root root 70M Mar 29 09:41 016023.sst -rw-r--r-- 1 root root 70M Mar 29 09:41 016024.sst -rw-r--r-- 1 root root 67M Mar 29 09:42 016048.sst -rw-r--r-- 1 root root 70M Mar 29 09:42 016051.sst -rw-r--r-- 1 root root 70M Mar 29 09:42 016054.sst -rw-r--r-- 1 root root 66M Mar 29 09:42 016089.sst -rw-r--r-- 1 root root 67M Mar 29 09:42 016092.sst -rw-r--r-- 1 root root 66M Mar 29 09:42 016093.sst -rw-r--r-- 1 root root 66M Mar 29 09:42 016113.sst -rw-r--r-- 1 root root 444 Mar 29 09:42 016117.sst -rw-r--r-- 1 root root 66M Mar 29 09:42 016118.sst -rw-r--r-- 1 root root 66M Mar 29 09:42 016119.sst -rw-r--r-- 1 root root 66M Mar 29 09:43 016142.sst -rw-r--r-- 1 root root 66M Mar 29 09:43 016145.sst -rw-r--r-- 1 root root 66M Mar 29 09:43 016172.sst -rw-r--r-- 1 root root 70M Mar 29 09:43 016173.sst -rw-r--r-- 1 root root 70M Mar 29 09:43 016174.sst -rw-r--r-- 1 root root 70M Mar 29 09:43 016175.sst -rw-r--r-- 1 root root 67M Mar 29 09:43 016176.sst -rw-r--r-- 1 root root 66M Mar 29 09:43 016178.sst -rw-r--r-- 1 root root 66M Mar 29 09:43 016179.sst -rw-r--r-- 1 root root 66M Mar 29 09:44 016205.sst -rw-r--r-- 1 root root 66M Mar 29 09:45 016251.sst -rw-r--r-- 1 root root 66M Mar 29 09:45 016257.sst -rw-r--r-- 1 root root 66M Mar 29 09:45 016282.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016345.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016346.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016349.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016371.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016373.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016379.sst -rw-r--r-- 1 root root 67M Mar 29 09:47 016380.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016418.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016419.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016423.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016424.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016425.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016426.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016427.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016428.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016430.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016433.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016434.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016435.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016436.sst -rw-r--r-- 1 root root 67M Mar 29 09:48 016437.sst -rw-r--r-- 1 root root 3.0K Mar 29 09:48 016439.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016440.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016441.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016442.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016443.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016444.sst -rw-r--r-- 1 root root 28K Mar 29 09:48 016445.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016446.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016447.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016448.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016449.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016450.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016451.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016452.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016454.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016455.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016457.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016458.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016459.sst -rw-r--r-- 1 root root 2.8K Mar 29 09:48 016460.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016461.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016462.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016463.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016464.sst -rw-r--r-- 1 root root 7.0K Mar 29 09:48 016466.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016467.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016468.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016470.sst -rw-r--r-- 1 root root 70M Mar 29 09:48 016471.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016472.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016473.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016474.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016475.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016476.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016477.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016479.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016480.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016481.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016482.sst -rw-r--r-- 1 root root 1004 Mar 29 09:48 016483.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016484.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016487.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016488.sst -rw-r--r-- 1 root root 304 Mar 29 09:48 016489.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016490.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016492.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016493.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016494.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016495.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016496.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016497.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016498.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016499.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016501.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016502.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016503.sst -rw-r--r-- 1 root root 66M Mar 29 09:48 016504.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016505.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016506.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016507.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016508.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016509.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016510.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016511.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016512.sst -rw-r--r-- 1 root root 1.6M Mar 29 09:49 016513.sst -rw-r--r-- 1 root root 13K Mar 29 09:49 016514.sst -rw-r--r-- 1 root root 67M Mar 29 09:49 016515.sst -rw-r--r-- 1 root root 11K Mar 29 09:49 016516.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016517.sst -rw-r--r-- 1 root root 4.7K Mar 29 09:49 016518.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016519.sst -rw-r--r-- 1 root root 2.4K Mar 29 09:49 016520.sst -rw-r--r-- 1 root root 2.2K Mar 29 09:49 016521.sst -rw-r--r-- 1 root root 66M Mar 29 09:49 016522.sst -rw-r--r-- 1 root root 2.3K Mar 29 09:49 016523.sst -rw-r--r-- 1 root root 1.8K Mar 29 09:49 016524.sst -rw-r--r-- 1 root root 5.4M Mar 29 16:01 016526.sst -rw-r--r-- 1 root root 2 Mar 29 16:02 LOCK -rw-r--r-- 1 root root 110K Mar 29 16:02 MANIFEST ``` ---
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [March 29, 2019, 7:04pm UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/2 "2019-03-29T19:04:32Z")
on previous load we already had 800 sstables - so node took 6 minutes to start ---
**Author:** ![mrjn](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mrjn/32/775_2.png) [@mrjn](https://discuss.dgraph.io/u/mrjn)\ **Post date:** [April 4, 2019, 12:57am UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/3 "2019-04-04T00:57:08Z")
Can you share your `badger info` output? And yes, it is ok to have a lot of SSTables. Badger does not look through each and every one of them when doing a key lookup. ---
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [April 6, 2019, 8:33am UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/4 "2019-04-06T08:33:44Z")
sorry, but i don’t think that is’s ok. now it’s 1466, and it is increasing more and more. insertion dramatically slows down, queries slow down as well i don’t know any other lsm-based systems with such a lot of sstables being not merged. and if you insert data in a random order (which is true for all realtime data) you have to look into _all_ of them. for now, we are loading ~1 billion of edges for more than 4 days and insertion speed now is way low, so i am not sure that we will complete this insertion ![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/1/1b354bd434dad5f8af036cf1e08f94c755f31bdf.png) will share `badger info` a bit later ---
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [April 6, 2019, 10:47am UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/5 "2019-04-06T10:47:53Z")
so, yep, now alpha is crashing all the time with OOM error on 64Gb machine. so, no way to continue insertion. whereas there is only 44Gb data on one node and 115G on other (replicas is set to 3, so full copy on each node). i suppose, that just because of a lot of not merged sstables it eats so much space and memory, and running so slow. ```auto [ec2-user@ip-172-18-xxx-xxx ~]$ for ip in $ips; do echo $ip; ssh $ip 'sudo du /vol/dgraph -sh'; done 172.18.xxx.xxx 115G /vol/dgraph 172.18.xxx.xxx 107G /vol/dgraph 172.18.xxx.xxx 44G /vol/dgraph ``` ---
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [April 6, 2019, 11:51am UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/6 "2019-04-06T11:51:33Z")
`badger info` on one node (which is the biggest one) full listing is here: [badger-info.txt - Google Drive](https://drive.google.com/file/d/15OgZtP32H7ApiNL2Js_Vwl7kseYwjaak/view?usp=sharing) ```auto bash-4.2# /home/ec2-user/go/bin/badger info --dir . Listening for /debug HTTP requests at port: 8080 [2019-04-06T10:39:14Z] MANIFEST 694 kB MA [12 hours earlier] 121374.sst 43 MB L0 [12 hours earlier] 121089.sst 68 MB L1 [12 hours earlier] 121091.sst 70 MB L1 [12 hours earlier] 121093.sst 69 MB L1 [12 hours earlier] 121094.sst 29 MB L1 [2 days earlier] 091226.sst 379 B L2 [19 hours earlier] 114155.sst 1.3 MB L2 [17 hours earlier] 116204.sst 2.7 MB L2 [15 hours earlier] 118977.sst 68 MB L2 [13 hours earlier] 120159.sst 12 kB L2 [12 hours earlier] 121095.sst 68 MB L2 [12 hours earlier] 121097.sst 68 MB L2 [12 hours earlier] 121098.sst 68 MB L2 [12 hours earlier] 121100.sst 68 MB L2 [12 hours earlier] 121101.sst 68 MB L2 [12 hours earlier] 121102.sst 68 MB L2 [12 hours earlier] 121103.sst 68 MB L2 [12 hours earlier] 121104.sst 68 MB L2 [12 hours earlier] 121105.sst 68 MB L2 [12 hours earlier] 121106.sst 68 MB L2 [12 hours earlier] 121107.sst 68 MB L2 [12 hours earlier] 121108.sst 68 MB L2 [12 hours earlier] 121109.sst 68 MB L2 [12 hours earlier] 121110.sst 68 MB L2 [12 hours earlier] 121111.sst 68 MB L2 [12 hours earlier] 121112.sst 68 MB L2 [12 hours earlier] 121113.sst 68 MB L2 [12 hours earlier] 121114.sst 68 MB L2 [12 hours earlier] 121115.sst 68 MB L2 [12 hours earlier] 121116.sst 68 MB L2 [12 hours earlier] 121117.sst 68 MB L2 [12 hours earlier] 121119.sst 68 MB L2 [12 hours earlier] 121120.sst 68 MB L2 [12 hours earlier] 121122.sst 68 MB L2 [12 hours earlier] 121123.sst 68 MB L2 [12 hours earlier] 121125.sst 68 MB L2 [12 hours earlier] 121127.sst 68 MB L2 [12 hours earlier] 121129.sst 68 MB L2 [12 hours earlier] 121130.sst 68 MB L2 [12 hours earlier] 121132.sst 68 MB L2 [12 hours earlier] 121133.sst 68 MB L2 [12 hours earlier] 121135.sst 68 MB L2 [12 hours earlier] 121137.sst 68 MB L2 [12 hours earlier] 121138.sst 68 MB L2 [12 hours earlier] 121140.sst 68 MB L2 [12 hours earlier] 121141.sst 68 MB L2 [12 hours earlier] 121143.sst 68 MB L2 [12 hours earlier] 121146.sst 9.8 MB L2 [2 days earlier] 091236.sst 70 MB L3 [2 days earlier] 091238.sst 70 MB L3 [2 days earlier] 091239.sst 70 MB L3 [2 days earlier] 091240.sst 70 MB L3 [2 days earlier] 091241.sst 70 MB L3 [2 days earlier] 091473.sst 70 MB L3 [2 days earlier] 091475.sst 70 MB L3 [2 days earlier] 091477.sst 70 MB L3 [2 days earlier] 091479.sst 70 MB L3 [2 days earlier] 091481.sst 70 MB L3 [2 days earlier] 091482.sst 70 MB L3 [2 days earlier] 091484.sst 70 MB L3 [2 days earlier] 092631.sst 70 MB L3 [2 days earlier] 092962.sst 70 MB L3 [2 days earlier] 093476.sst 70 MB L3 [2 days earlier] 094137.sst 70 MB L3 [2 days earlier] 094570.sst 70 MB L3 [2 days earlier] 095165.sst 70 MB L3 [2 days earlier] 095746.sst 70 MB L3 [2 days earlier] 096285.sst 70 MB L3 [2 days earlier] 096837.sst 70 MB L3 [2 days earlier] 097701.sst 70 MB L3 [2 days earlier] 098531.sst 70 MB L3 [2 days earlier] 099350.sst 70 MB L3 [1 day earlier] 100197.sst 70 MB L3 [1 day earlier] 100805.sst 70 MB L3 [1 day earlier] 101693.sst 70 MB L3 [1 day earlier] 102528.sst 70 MB L3 [1 day earlier] 103605.sst 70 MB L3 [1 day earlier] 105281.sst 70 MB L3 [1 day earlier] 105569.sst 70 MB L3 [1 day earlier] 106595.sst 70 MB L3 [1 day earlier] 106934.sst 70 MB L3 [1 day earlier] 107886.sst 70 MB L3 [1 day earlier] 109047.sst 70 MB L3 [1 day earlier] 110167.sst 70 MB L3 [1 day earlier] 110575.sst 70 MB L3 [22 hours earlier] 111483.sst 70 MB L3 [17 hours earlier] 115548.sst 70 MB L3 [17 hours earlier] 115779.sst 70 MB L3 [17 hours earlier] 116215.sst 70 MB L3 [17 hours earlier] 116217.sst 70 MB L3 [17 hours earlier] 116219.sst 22 MB L3 [17 hours earlier] 116333.sst 29 MB L3 [17 hours earlier] 116347.sst 68 MB L3 [17 hours earlier] 116350.sst 68 MB L3 [17 hours earlier] 116351.sst 68 MB L3 [17 hours earlier] 116354.sst 68 MB L3 [17 hours earlier] 116601.sst 70 MB L3 [16 hours earlier] 117589.sst 70 MB L3 [16 hours earlier] 118028.sst 68 MB L3 [16 hours earlier] 118029.sst 68 MB L3 [16 hours earlier] 118031.sst 68 MB L3 [16 hours earlier] 118032.sst 68 MB L3 [16 hours earlier] 118034.sst 68 MB L3 [16 hours earlier] 118035.sst 17 MB L3 [15 hours earlier] 118771.sst 12 MB L3 [14 hours earlier] 119143.sst 68 MB L3 [14 hours earlier] 119279.sst 70 MB L3 [13 hours earlier] 120353.sst 68 MB L3 [13 hours earlier] 120356.sst 70 MB L3 [13 hours earlier] 120359.sst 70 MB L3 [13 hours earlier] 120362.sst 70 MB L3 [13 hours earlier] 120489.sst 68 MB L3 [13 hours earlier] 120490.sst 68 MB L3 [13 hours earlier] 120491.sst 53 MB L3 [13 hours earlier] 120529.sst 70 MB L3 [13 hours earlier] 120531.sst 70 MB L3 [13 hours earlier] 120533.sst 70 MB L3 [13 hours earlier] 120535.sst 70 MB L3 [13 hours earlier] 120537.sst 70 MB L3 [13 hours earlier] 120539.sst 70 MB L3 [13 hours earlier] 120542.sst 70 MB L3 [13 hours earlier] 120544.sst 70 MB L3 [13 hours earlier] 120545.sst 70 MB L3 [13 hours earlier] 120547.sst 70 MB L3 [13 hours earlier] 120549.sst 70 MB L3 [13 hours earlier] 120552.sst 70 MB L3 [13 hours earlier] 120554.sst 70 MB L3 [13 hours earlier] 120556.sst 70 MB L3 [12 hours earlier] 120875.sst 70 MB L3 [12 hours earlier] 120877.sst 70 MB L3 [12 hours earlier] 120878.sst 70 MB L3 [12 hours earlier] 120880.sst 70 MB L3 [12 hours earlier] 120881.sst 70 MB L3 [12 hours earlier] 120883.sst 70 MB L3 [12 hours earlier] 120885.sst 70 MB L3 [12 hours earlier] 120887.sst 70 MB L3 [12 hours earlier] 120889.sst 70 MB L3 [12 hours earlier] 120891.sst 70 MB L3 [12 hours earlier] 120893.sst 70 MB L3 [12 hours earlier] 120895.sst 70 MB L3 [12 hours earlier] 120897.sst 70 MB L3 [12 hours earlier] 120899.sst 70 MB L3 [12 hours earlier] 120901.sst 73 MB L3 [12 hours earlier] 120903.sst 73 MB L3 [12 hours earlier] 120905.sst 73 MB L3 [12 hours earlier] 120907.sst 73 MB L3 [12 hours earlier] 120909.sst 73 MB L3 [12 hours earlier] 120911.sst 73 MB L3 [12 hours earlier] 120913.sst 73 MB L3 [12 hours earlier] 120915.sst 73 MB L3 [12 hours earlier] 120917.sst 73 MB L3 [12 hours earlier] 120920.sst 70 MB L3 [12 hours earlier] 120923.sst 70 MB L3 [12 hours earlier] 120925.sst 73 MB L3 [12 hours earlier] 120927.sst 73 MB L3 [12 hours earlier] 120929.sst 73 MB L3 [12 hours earlier] 120931.sst 73 MB L3 [12 hours earlier] 120933.sst 73 MB L3 [12 hours earlier] 120937.sst 73 MB L3 [12 hours earlier] 120939.sst 73 MB L3 [12 hours earlier] 120943.sst 73 MB L3 [12 hours earlier] 120945.sst 73 MB L3 [12 hours earlier] 120947.sst 73 MB L3 [12 hours earlier] 120950.sst 70 MB L3 [12 hours earlier] 120952.sst 68 MB L3 [12 hours earlier] 120954.sst 68 MB L3 [12 hours earlier] 120956.sst 68 MB L3 [12 hours earlier] 120958.sst 68 MB L3 [12 hours earlier] 120960.sst 68 MB L3 [12 hours earlier] 120962.sst 68 MB L3 [12 hours earlier] 120964.sst 68 MB L3 [12 hours earlier] 120966.sst 68 MB L3 [12 hours earlier] 120968.sst 68 MB L3 [12 hours earlier] 120970.sst 68 MB L3 [12 hours earlier] 120972.sst 68 MB L3 [12 hours earlier] 120974.sst 68 MB L3 [12 hours earlier] 120976.sst 68 MB L3 [12 hours earlier] 120978.sst 68 MB L3 [12 hours earlier] 120980.sst 68 MB L3 [12 hours earlier] 120982.sst 68 MB L3 [12 hours earlier] 120984.sst 68 MB L3 [12 hours earlier] 120986.sst 68 MB L3 [12 hours earlier] 120988.sst 68 MB L3 [12 hours earlier] 120990.sst 68 MB L3 [12 hours earlier] 120992.sst 68 MB L3 [12 hours earlier] 120994.sst 68 MB L3 [12 hours earlier] 120996.sst 68 MB L3 [12 hours earlier] 120998.sst 68 MB L3 [12 hours earlier] 121000.sst 68 MB L3 [12 hours earlier] 121002.sst 68 MB L3 [12 hours earlier] 121004.sst 68 MB L3 [12 hours earlier] 121006.sst 68 MB L3 [12 hours earlier] 121008.sst 68 MB L3 [12 hours earlier] 121010.sst 68 MB L3 [12 hours earlier] 121012.sst 68 MB L3 [12 hours earlier] 121014.sst 68 MB L3 [12 hours earlier] 121016.sst 68 MB L3 [12 hours earlier] 121018.sst 68 MB L3 [12 hours earlier] 121020.sst 68 MB L3 [12 hours earlier] 121022.sst 68 MB L3 [12 hours earlier] 121024.sst 68 MB L3 [12 hours earlier] 121026.sst 68 MB L3 [12 hours earlier] 121028.sst 68 MB L3 [12 hours earlier] 121030.sst 67 MB L3 [12 hours earlier] 121118.sst 70 MB L3 [12 hours earlier] 121121.sst 70 MB L3 [12 hours earlier] 121124.sst 70 MB L3 [12 hours earlier] 121126.sst 70 MB L3 [12 hours earlier] 121128.sst 70 MB L3 [12 hours earlier] 121131.sst 70 MB L3 [12 hours earlier] 121134.sst 70 MB L3 [12 hours earlier] 121136.sst 70 MB L3 [12 hours earlier] 121139.sst 70 MB L3 [12 hours earlier] 121142.sst 70 MB L3 [12 hours earlier] 121145.sst 70 MB L3 [12 hours earlier] 121147.sst 68 MB L3 [12 hours earlier] 121148.sst 70 MB L3 [12 hours earlier] 121149.sst 68 MB L3 [12 hours earlier] 121150.sst 68 MB L3 [12 hours earlier] 121151.sst 70 MB L3 [12 hours earlier] 121152.sst 68 MB L3 [12 hours earlier] 121153.sst 70 MB L3 [12 hours earlier] 121154.sst 68 MB L3 [12 hours earlier] 121155.sst 68 MB L3 [12 hours earlier] 121156.sst 70 MB L3 [12 hours earlier] 121157.sst 68 MB L3 [12 hours earlier] 121158.sst 70 MB L3 [12 hours earlier] 121159.sst 68 MB L3 [12 hours earlier] 121160.sst 70 MB L3 [12 hours earlier] 121161.sst 70 MB L3 [12 hours earlier] 121162.sst 70 MB L3 [12 hours earlier] 121163.sst 70 MB L3 [12 hours earlier] 121164.sst 70 MB L3 [12 hours earlier] 121165.sst 70 MB L3 [12 hours earlier] 121166.sst 70 MB L3 [12 hours earlier] 121167.sst 70 MB L3 [12 hours earlier] 121168.sst 70 MB L3 [12 hours earlier] 121169.sst 70 MB L3 [12 hours earlier] 121170.sst 70 MB L3 [12 hours earlier] 121171.sst 70 MB L3 [12 hours earlier] 121172.sst 70 MB L3 [12 hours earlier] 121173.sst 70 MB L3 [12 hours earlier] 121174.sst 70 MB L3 [12 hours earlier] 121175.sst 70 MB L3 [12 hours earlier] 121176.sst 70 MB L3 [12 hours earlier] 121177.sst 70 MB L3 [12 hours earlier] 121178.sst 70 MB L3 [12 hours earlier] 121179.sst 70 MB L3 [12 hours earlier] 121180.sst 70 MB L3 [12 hours earlier] 121181.sst 70 MB L3 [12 hours earlier] 121182.sst 70 MB L3 [12 hours earlier] 121183.sst 70 MB L3 [12 hours earlier] 121184.sst 70 MB L3 [12 hours earlier] 121185.sst 70 MB L3 [12 hours earlier] 121186.sst 70 MB L3 [12 hours earlier] 121187.sst 70 MB L3 [12 hours earlier] 121188.sst 70 MB L3 [12 hours earlier] 121189.sst 70 MB L3 [12 hours earlier] 121190.sst 70 MB L3 [12 hours earlier] 121191.sst 70 MB L3 [12 hours earlier] 121192.sst 70 MB L3 [12 hours earlier] 121193.sst 70 MB L3 [12 hours earlier] 121194.sst 70 MB L3 [12 hours earlier] 121195.sst 70 MB L3 [12 hours earlier] 121196.sst 68 MB L3 [12 hours earlier] 121197.sst 70 MB L3 [12 hours earlier] 121198.sst 70 MB L3 [12 hours earlier] 121199.sst 38 MB L3 [12 hours earlier] 121200.sst 70 MB L3 [12 hours earlier] 121201.sst 70 MB L3 [12 hours earlier] 121202.sst 70 MB L3 [12 hours earlier] 121203.sst 70 MB L3 [12 hours earlier] 121204.sst 70 MB L3 [12 hours earlier] 121205.sst 70 MB L3 [12 hours earlier] 121206.sst 70 MB L3 [12 hours earlier] 121207.sst 70 MB L3 [12 hours earlier] 121208.sst 70 MB L3 [12 hours earlier] 121209.sst 70 MB L3 [12 hours earlier] 121210.sst 70 MB L3 [12 hours earlier] 121211.sst 70 MB L3 [12 hours earlier] 121212.sst 70 MB L3 [12 hours earlier] 121213.sst 70 MB L3 [12 hours earlier] 121214.sst 68 MB L3 [12 hours earlier] 121215.sst 68 MB L3 [12 hours earlier] 121216.sst 70 MB L3 [12 hours earlier] 121217.sst 68 MB L3 [12 hours earlier] 121218.sst 68 MB L3 [12 hours earlier] 121219.sst 70 MB L3 [12 hours earlier] 121220.sst 71 MB L3 [12 hours earlier] 121221.sst 73 MB L3 [12 hours earlier] 121222.sst 68 MB L3 [12 hours earlier] 121223.sst 70 MB L3 [12 hours earlier] 121224.sst 73 MB L3 [12 hours earlier] 121225.sst 70 MB L3 [12 hours earlier] 121226.sst 73 MB L3 [12 hours earlier] 121227.sst 70 MB L3 [12 hours earlier] 121228.sst 70 MB L3 [12 hours earlier] 121229.sst 73 MB L3 [12 hours earlier] 121230.sst 70 MB L3 [12 hours earlier] 121231.sst 73 MB L3 [12 hours earlier] 121232.sst 64 MB L3 [12 hours earlier] 121233.sst 73 MB L3 [12 hours earlier] 121235.sst 73 MB L3 [12 hours earlier] 121237.sst 73 MB L3 [12 hours earlier] 121238.sst 73 MB L3 [12 hours earlier] 121239.sst 73 MB L3 [12 hours earlier] 121240.sst 70 MB L3 [12 hours earlier] 121241.sst 70 MB L3 [12 hours earlier] 121242.sst 73 MB L3 [12 hours earlier] 121243.sst 73 MB L3 [12 hours earlier] 121244.sst 73 MB L3 [12 hours earlier] 121245.sst 73 MB L3 [12 hours earlier] 121246.sst 73 MB L3 [12 hours earlier] 121247.sst 73 MB L3 [12 hours earlier] 121248.sst 73 MB L3 [12 hours earlier] 121250.sst 73 MB L3 [12 hours earlier] 121251.sst 73 MB L3 [12 hours earlier] 121252.sst 73 MB L3 [12 hours earlier] 121253.sst 70 MB L3 [12 hours earlier] 121254.sst 70 MB L3 [12 hours earlier] 121255.sst 70 MB L3 [12 hours earlier] 121256.sst 70 MB L3 [12 hours earlier] 121257.sst 70 MB L3 [12 hours earlier] 121258.sst 70 MB L3 [12 hours earlier] 121259.sst 70 MB L3 [12 hours earlier] 121260.sst 70 MB L3 [12 hours earlier] 121261.sst 73 MB L3 [12 hours earlier] 121262.sst 73 MB L3 [12 hours earlier] 121263.sst 73 MB L3 [12 hours earlier] 121264.sst 73 MB L3 [12 hours earlier] 121265.sst 73 MB L3 [12 hours earlier] 121266.sst 73 MB L3 [12 hours earlier] 121267.sst 73 MB L3 [12 hours earlier] 121268.sst 73 MB L3 [12 hours earlier] 121269.sst 73 MB L3 [12 hours earlier] 121270.sst 73 MB L3 [12 hours earlier] 121271.sst 70 MB L3 (a lot of lines here) [17 hours earlier] 003579.vlog 87 MB VL [17 hours earlier] 003586.vlog 71 MB VL [17 hours earlier] 003587.vlog 72 MB VL [17 hours earlier] 003588.vlog 70 MB VL [17 hours earlier] 003589.vlog 67 MB VL [17 hours earlier] 003591.vlog 69 MB VL [17 hours earlier] 003593.vlog 64 MB VL [17 hours earlier] 003594.vlog 67 MB VL [17 hours earlier] 003595.vlog 61 MB VL [17 hours earlier] 003597.vlog 62 MB VL [17 hours earlier] 003598.vlog 57 MB VL [17 hours earlier] 003599.vlog 57 MB VL [17 hours earlier] 003600.vlog 81 MB VL [17 hours earlier] 003601.vlog 81 MB VL [17 hours earlier] 003602.vlog 81 MB VL [17 hours earlier] 003603.vlog 81 MB VL [17 hours earlier] 003604.vlog 77 MB VL [17 hours earlier] 003605.vlog 93 MB VL [17 hours earlier] 003606.vlog 85 MB VL [17 hours earlier] 003607.vlog 88 MB VL [17 hours earlier] 003608.vlog 81 MB VL [17 hours earlier] 003609.vlog 88 MB VL [17 hours earlier] 003610.vlog 78 MB VL [17 hours earlier] 003611.vlog 85 MB VL [17 hours earlier] 003612.vlog 83 MB VL [17 hours earlier] 003624.vlog 90 MB VL [17 hours earlier] 003625.vlog 207 MB VL [17 hours earlier] 003626.vlog 94 MB VL [17 hours earlier] 003627.vlog 76 MB VL [17 hours earlier] 003628.vlog 70 MB VL [17 hours earlier] 003629.vlog 70 MB VL [17 hours earlier] 003630.vlog 68 MB VL [17 hours earlier] 003631.vlog 68 MB VL [17 hours earlier] 003632.vlog 68 MB VL [17 hours earlier] 003633.vlog 97 MB VL [17 hours earlier] 003635.vlog 70 MB VL [16 hours earlier] 003637.vlog 69 MB VL [16 hours earlier] 003638.vlog 67 MB VL [16 hours earlier] 003639.vlog 69 MB VL [16 hours earlier] 003640.vlog 69 MB VL [16 hours earlier] 003641.vlog 68 MB VL [16 hours earlier] 003642.vlog 67 MB VL [16 hours earlier] 003643.vlog 67 MB VL [16 hours earlier] 003644.vlog 69 MB VL [16 hours earlier] 003645.vlog 68 MB VL [16 hours earlier] 003646.vlog 67 MB VL [16 hours earlier] 003647.vlog 68 MB VL [16 hours earlier] 003648.vlog 68 MB VL [16 hours earlier] 003652.vlog 70 MB VL [16 hours earlier] 003653.vlog 68 MB VL [16 hours earlier] 003654.vlog 68 MB VL [16 hours earlier] 003655.vlog 67 MB VL [16 hours earlier] 003656.vlog 68 MB VL [16 hours earlier] 003657.vlog 70 MB VL [16 hours earlier] 003658.vlog 68 MB VL [16 hours earlier] 003659.vlog 68 MB VL [16 hours earlier] 003660.vlog 69 MB VL [16 hours earlier] 003661.vlog 71 MB VL [16 hours earlier] 003662.vlog 71 MB VL [16 hours earlier] 003663.vlog 70 MB VL [16 hours earlier] 003665.vlog 68 MB VL [16 hours earlier] 003667.vlog 67 MB VL [16 hours earlier] 003669.vlog 69 MB VL [16 hours earlier] 003670.vlog 68 MB VL [16 hours earlier] 003671.vlog 68 MB VL [16 hours earlier] 003672.vlog 68 MB VL [16 hours earlier] 003673.vlog 109 MB VL [16 hours earlier] 003674.vlog 68 MB VL [16 hours earlier] 003675.vlog 70 MB VL [16 hours earlier] 003676.vlog 68 MB VL [16 hours earlier] 003677.vlog 68 MB VL [16 hours earlier] 003678.vlog 67 MB VL [16 hours earlier] 003679.vlog 67 MB VL [15 hours earlier] 003680.vlog 69 MB VL [15 hours earlier] 003681.vlog 69 MB VL [15 hours earlier] 003682.vlog 69 MB VL [15 hours earlier] 003683.vlog 70 MB VL [15 hours earlier] 003684.vlog 68 MB VL [15 hours earlier] 003685.vlog 68 MB VL [15 hours earlier] 003686.vlog 69 MB VL [15 hours earlier] 003687.vlog 69 MB VL [15 hours earlier] 003688.vlog 70 MB VL [15 hours earlier] 003689.vlog 68 MB VL [15 hours earlier] 003690.vlog 68 MB VL [15 hours earlier] 003691.vlog 68 MB VL [15 hours earlier] 003692.vlog 69 MB VL [15 hours earlier] 003693.vlog 73 MB VL [15 hours earlier] 003694.vlog 69 MB VL [15 hours earlier] 003695.vlog 68 MB VL [15 hours earlier] 003696.vlog 68 MB VL [15 hours earlier] 003697.vlog 68 MB VL [15 hours earlier] 003698.vlog 68 MB VL [15 hours earlier] 003699.vlog 69 MB VL [15 hours earlier] 003700.vlog 69 MB VL [15 hours earlier] 003701.vlog 69 MB VL [15 hours earlier] 003702.vlog 72 MB VL [15 hours earlier] 003703.vlog 69 MB VL [15 hours earlier] 003704.vlog 67 MB VL [15 hours earlier] 003705.vlog 70 MB VL [15 hours earlier] 003706.vlog 68 MB VL [14 hours earlier] 003707.vlog 69 MB VL [14 hours earlier] 003708.vlog 68 MB VL [14 hours earlier] 003709.vlog 68 MB VL [14 hours earlier] 003710.vlog 69 MB VL [14 hours earlier] 003711.vlog 70 MB VL [14 hours earlier] 003712.vlog 70 MB VL [14 hours earlier] 003713.vlog 68 MB VL [14 hours earlier] 003714.vlog 68 MB VL [14 hours earlier] 003715.vlog 69 MB VL [14 hours earlier] 003716.vlog 67 MB VL [14 hours earlier] 003717.vlog 68 MB VL [14 hours earlier] 003718.vlog 71 MB VL [14 hours earlier] 003719.vlog 68 MB VL [14 hours earlier] 003720.vlog 67 MB VL [14 hours earlier] 003721.vlog 68 MB VL [14 hours earlier] 003722.vlog 68 MB VL [14 hours earlier] 003723.vlog 68 MB VL [14 hours earlier] 003724.vlog 68 MB VL [14 hours earlier] 003725.vlog 70 MB VL [14 hours earlier] 003726.vlog 68 MB VL [14 hours earlier] 003727.vlog 69 MB VL [14 hours earlier] 003728.vlog 67 MB VL [14 hours earlier] 003729.vlog 67 MB VL [13 hours earlier] 003730.vlog 68 MB VL [13 hours earlier] 003731.vlog 69 MB VL [13 hours earlier] 003732.vlog 95 MB VL [13 hours earlier] 003733.vlog 68 MB VL [13 hours earlier] 003734.vlog 68 MB VL [13 hours earlier] 003735.vlog 87 MB VL [13 hours earlier] 003736.vlog 68 MB VL [13 hours earlier] 003737.vlog 70 MB VL [13 hours earlier] 003738.vlog 80 MB VL [13 hours earlier] 003739.vlog 447 MB VL [13 hours earlier] 003740.vlog 70 MB VL [13 hours earlier] 003741.vlog 68 MB VL [13 hours earlier] 003742.vlog 69 MB VL [13 hours earlier] 003743.vlog 69 MB VL [13 hours earlier] 003744.vlog 69 MB VL [13 hours earlier] 003745.vlog 69 MB VL [13 hours earlier] 003746.vlog 68 MB VL [13 hours earlier] 003747.vlog 121 MB VL [12 hours earlier] 003748.vlog 559 MB VL [12 hours earlier] 003749.vlog 69 MB VL [12 hours earlier] 003750.vlog 69 MB VL [12 hours earlier] 003751.vlog 69 MB VL [12 hours earlier] 003752.vlog 69 MB VL [12 hours earlier] 003753.vlog 102 MB VL [12 hours earlier] 003754.vlog 67 MB VL [12 hours earlier] 003755.vlog 69 MB VL [12 hours earlier] 003756.vlog 33 MB VL [EXTRA] [2019-04-06T10:39:14Z] LOCK 2 B [Summary] Level 0 size: 43 MB Level 1 size: 236 MB Level 2 size: 2.6 GB Level 3 size: 27 GB Level 4 size: 64 GB Total index size: 94 GB Value log size: 18 GB Abnormalities: 1 extra file. 0 missing files. 0 empty files. 0 truncated manifests. ``` ---
**Author:** ![makitka](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/makitka/32/1307_2.png) [@makitka](https://discuss.dgraph.io/u/makitka)\ **Post date:** [April 6, 2019, 9:23pm UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/7 "2019-04-06T21:23:45Z")
ok, sorry guys, seems like i was not familiar with level compaction strategy, only with size tiered one from cassandra. i’ve read some articles how rocksdb and hence badger works. so, number of sstables is not the case. anyway, now 2 nodes are crashing with OOM in a regular basis and loading is not continuing… ```auto fatal error: runtime: out of memory runtime stack: runtime.throw(0x148dd8a, 0x16) /usr/local/go/src/runtime/panic.go:608 +0x72 runtime.sysMap(0xcdd0000000, 0x4000000, 0x1fd43b8) /usr/local/go/src/runtime/mem_linux.go:156 +0xc7 runtime.(*mheap).sysAlloc(0x1fba720, 0x4000000, 0x1fba738, 0x7f9e93260e58) /usr/local/go/src/runtime/malloc.go:619 +0x1c7 runtime.(*mheap).grow(0x1fba720, 0x1, 0x0) /usr/local/go/src/runtime/mheap.go:920 +0x42 runtime.(*mheap).allocSpanLocked(0x1fba720, 0x1, 0x1fd43c8, 0x400) /usr/local/go/src/runtime/mheap.go:848 +0x337 runtime.(*mheap).alloc_m(0x1fba720, 0x1, 0x16, 0x7f9e9359efff) /usr/local/go/src/runtime/mheap.go:692 +0x119 runtime.(*mheap).alloc.func1() /usr/local/go/src/runtime/mheap.go:759 +0x4c runtime.(*mheap).alloc(0x1fba720, 0x1, 0x7f9e93010016, 0x7f9e93260e58) /usr/local/go/src/runtime/mheap.go:758 +0x8a runtime.(*mcentral).grow(0x1fbbfd8, 0x0) /usr/local/go/src/runtime/mcentral.go:232 +0x94 runtime.(*mcentral).cacheSpan(0x1fbbfd8, 0x7f9e93260e58) /usr/local/go/src/runtime/mcentral.go:106 +0x2f8 runtime.(*mcache).refill(0x7fbdafe456c0, 0xc1bbacad16) /usr/local/go/src/runtime/mcache.go:122 +0x95 runtime.(*mcache).nextFree.func1() /usr/local/go/src/runtime/malloc.go:749 +0x32 runtime.systemstack(0x0) /usr/local/go/src/runtime/asm_amd64.s:351 +0x66 runtime.mstart() /usr/local/go/src/runtime/proc.go:1229 goroutine 1864 [running]: runtime.systemstack_switch() /usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc1bbacad20 sp=0xc1bbacad18 pc=0x8e1d60 runtime.(*mcache).nextFree(0x7fbdafe456c0, 0xe0e416, 0xc63ec97480, 0x20, 0x40) /usr/local/go/src/runtime/malloc.go:748 +0xb6 fp=0xc1bbacad78 sp=0xc1bbacad20 pc=0x8923d6 runtime.mallocgc(0xa0, 0x14401e0, 0xc586e8eb01, 0x203373) /usr/local/go/src/runtime/malloc.go:903 +0x793 fp=0xc1bbacae18 sp=0xc1bbacad78 pc=0x892d23 runtime.newobject(0x14401e0, 0x1) /usr/local/go/src/runtime/malloc.go:1032 +0x38 fp=0xc1bbacae48 sp=0xc1bbacae18 pc=0x893108 github.com/dgraph-io/dgraph/protos/pb.(*PostingList).Unmarshal(0xcdcfffb4f0, 0xc6dbd349a0, 0x36, 0x6c, 0x8b1e42, 0x8) /ext-go/1/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:13680 +0x44e fp=0xc1bbacaf48 sp=0xc1bbacae48 pc=0xe98a0e github.com/dgraph-io/dgraph/posting.ReadPostingList.func1(0xc6dbd349a0, 0x36, 0x6c, 0x6c, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/mvcc.go:208 +0x74 fp=0xc1bbacafa0 sp=0xc1bbacaf48 pc=0xf99844 github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*Item).Value(0xcdcf673d90, 0xc1bbacb070, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/iterator.go:109 +0x17a fp=0xc1bbacb010 sp=0xc1bbacafa0 pc=0xe2ac3a github.com/dgraph-io/dgraph/posting.ReadPostingList(0xc7212a13c0, 0x18, 0x18, 0xcdcffd8fa0, 0x0, 0x64, 0x100) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/mvcc.go:206 +0x305 fp=0xc1bbacb0b8 sp=0xc1bbacb010 pc=0xf95415 github.com/dgraph-io/dgraph/posting.getNew(0xc7212a13c0, 0x18, 0x18, 0xc0001b0300, 0x0, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/mvcc.go:245 +0x1d3 fp=0xc1bbacb170 sp=0xc1bbacb0b8 pc=0xf956f3 github.com/dgraph-io/dgraph/posting.(*LocalCache).Get(0xcd89ea1ae0, 0xc7212a13c0, 0x18, 0x18, 0x18, 0x4, 0xc7212a13c0) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/lists.go:212 +0xc2 fp=0xc1bbacb1c8 sp=0xc1bbacb170 pc=0xf941a2 github.com/dgraph-io/dgraph/posting.(*Txn).Get(0xcd89eef360, 0xc7212a13c0, 0x18, 0x18, 0xc7212a13c0, 0x18, 0x18) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/oracle.go:76 +0x4d fp=0xc1bbacb210 sp=0xc1bbacb1c8 pc=0xf959bd github.com/dgraph-io/dgraph/posting.(*Txn).addIndexMutation(0xcd89eef360, 0x15b9f00, 0xc000038018, 0xcdcfd02dc0, 0xc77ee8ce4c, 0x4, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/index.go:120 +0x7c fp=0xc1bbacb260 sp=0xc1bbacb210 pc=0xf8a4ec github.com/dgraph-io/dgraph/posting.(*Txn).addIndexMutations(0xcd89eef360, 0x15b9f00, 0xc000038018, 0xc1bbacb440, 0x1, 0x1) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/index.go:109 +0x166 fp=0xc1bbacb2f0 sp=0xc1bbacb260 pc=0xf8a366 github.com/dgraph-io/dgraph/posting.(*List).AddMutationWithIndex(0xcdcf3a0930, 0x15b9f00, 0xc000038018, 0xc1cc1a7e00, 0xcd89eef360, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/posting/index.go:390 +0x479 fp=0xc1bbacb4d0 sp=0xc1bbacb2f0 pc=0xf8c2e9 github.com/dgraph-io/dgraph/worker.runMutation(0x15b9f00, 0xc000038018, 0xc1cc1a7e00, 0xcd89eef360, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/worker/mutation.go:88 +0x2e6 fp=0xc1bbacb688 sp=0xc1bbacb4d0 pc=0x10d4fc6 github.com/dgraph-io/dgraph/worker.(*node).applyMutations(0xc192b591a0, 0x15b9f00, 0xc000038018, 0xc1c0645580, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:246 +0x6e4 fp=0xc1bbacb920 sp=0xc1bbacb688 pc=0x10c7604 github.com/dgraph-io/dgraph/worker.(*node).applyCommitted(0xc192b591a0, 0xc1c0645580, 0xc1cc16d660, 0x15) /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:271 +0xbc2 fp=0xc1bbacbc60 sp=0xc1bbacb920 pc=0x10c8692 github.com/dgraph-io/dgraph/worker.(*node).processApplyCh.func1(0xc1c0645a00, 0xb, 0x10) /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:385 +0x1a5 fp=0xc1bbacbd68 sp=0xc1bbacbc60 pc=0x10f4375 github.com/dgraph-io/dgraph/worker.(*node).processApplyCh(0xc192b591a0) /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:412 +0x241 fp=0xc1bbacbfd8 sp=0xc1bbacbd68 pc=0x10c8d71 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc1bbacbfe0 sp=0xc1bbacbfd8 pc=0x8e3e41 created by github.com/dgraph-io/dgraph/worker.(*node).InitAndStartNode /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:1161 +0x4e0 ``` ---
**Author:** ![dmai](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/dmai/32/1254_2.png) [@dmai](https://discuss.dgraph.io/u/dmai)\ **Post date:** [April 8, 2019, 4:31pm UTC](https://discuss.dgraph.io/t/a-lot-of-sstables-each-is-67mb-is-it-ok/4255/8 "2019-04-08T16:31:36Z")
If you can share a heap profile of the Alpha before it runs out of memory we can take a closer look. [https://docs.dgraph.io/howto/#profiling-information](https://docs.dgraph.io/howto/#profiling-information)