To get accurate memory usage inforation, we need the current RSS(Resident Set Size) but linux doesn’t export it directly. It only has maxRSS in the rusage struct, which is the maximum memory that was used at any point of time by the process. So, the other option would be to read the /proc/self/stat to get the current RSS. What are your thoughts on this? @core-devs
Link:http://nadeausoftware.com/articles/2012/07/c_c_tip_how_get_process_resident_set_size_physical_memory_use http://linux.die.net/man/2/getrusage https://golang.org/pkg/runtime/#MemStats
anacrolix
August 24, 2016, 7:51am
2
It’s difficult to determine if ru_maxrss is the maximum RSS were the process to page in everything at the present time (which is what you want), or the max RSS over the lifetime of the process. Especially with the wording: “For RUSAGE_CHILDREN, this is the resident set size of the largest child, not the maximum resident set size of the process tree”. It also makes no sense for the kernel to track a maximum observed value and not provide the current value. I’d assume for now that the meaning of ru_maxrss is the value you want until you observe it to be otherwise.
Yeah, agreed. But a source says “The ru_maxrss field of struct rusage is the “high watermark” for the RSS. It indicates the peek RAM use for this process”. So, it wasn’t clear on if it was the maximum over time or otherwise.
kernel - getrusage system call: what is "maximum resident set size" - Unix & Linux Stack Exchange .
ashwin95r
August 24, 2016, 10:37am
4
Okay I tested it out. Tough the number matches up the one shown by tools like htop, it does give out the maximum usage till now.
Total memory: 791
INFO[0030] Counters ignored=0 len_cnq=0 package=loader parsed=891584 pending=134 processed=891450 read=901451
INFO[0031] Counters ignored=0 len_cnq=0 package=loader parsed=928031 pending=34 processed=927997 read=937999
INFO[0032] Counters ignored=0 len_cnq=0 package=loader parsed=968299 pending=658 processed=967641 read=977692
INFO[0034] Counters ignored=0 len_cnq=0 package=loader parsed=993580 pending=2649 processed=990931 read=998430
Total memory: 951
INFO[0035] Counters ignored=0 len_cnq=0 package=loader parsed=1022641 pending=1667 processed=1020974 read=1032643
INFO[0036] Counters ignored=0 len_cnq=0 package=loader parsed=1071371 pending=2856 processed=1068515 read=1076291
INFO[0037] Counters ignored=0 len_cnq=0 package=loader parsed=1104553 pending=21 processed=1104532 read=1114534
INFO[0038] Counters ignored=0 len_cnq=0 package=loader parsed=1165148 pending=670 processed=1164478 read=1165147
INFO[0039] Counters ignored=0 len_cnq=0 package=loader parsed=1196842 pending=2426 processed=1194416 read=1206843
Total memory: 1081
2016/08/24 10:36:52 Memory usage over threshold. STW. Allocated MB: 1081
2016/08/24 10:36:52 Calling merge on all lists.
INFO[0040] Counters ignored=0 len_cnq=867 package=loader parsed=1244394 pending=3867 processed=1240527 read=1254396
2016/08/24 10:36:56 Merged lists. Calling GC.
2016/08/24 10:36:58 Trying to free OS memory
2016/08/24 10:37:00 Memory usage after calling GC. Allocated MB: 1144
Total memory: 1144
2016/08/24 10:37:00 Memory usage over threshold. STW. Allocated MB: 1144
2016/08/24 10:37:00 Calling merge on all lists.
INFO[0049] Counters ignored=0 len_cnq=10000 package=loader parsed=1255037 pending=13000 processed=1242037 read=1265039
2016/08/24 10:37:01 Merged lists. Calling GC.
2016/08/24 10:37:02 Trying to free OS memory
2016/08/24 10:37:03 Memory usage after calling GC. Allocated MB: 1144
Total memory: 1144
2016/08/24 10:37:03 Memory usage over threshold. STW. Allocated MB: 1144
2016/08/24 10:37:03 Calling merge on all lists.
2016/08/24 10:37:03 Merged lists. Calling GC.
2016/08/24 10:37:04 Trying to free OS memory
INFO[0052] Counters ignored=0 len_cnq=10000 package=loader parsed=1256545 pending=13000 processed=1243545 read=1266547
2016/08/24 10:37:05 Memory usage after calling GC. Allocated MB: 1144
INFO[0053] Counters ignored=0 len_cnq=9714 package=loader parsed=1256691 pending=12712 processed=1243979 read=1266547
INFO[0054] Counters ignored=0 len_cnq=0 package=loader parsed=1273636 pending=750 processed=1272886 read=1273635
Total memory: 1144
2016/08/24 10:37:07 Memory usage over threshold. STW. Allocated MB: 1144
2016/08/24 10:37:07 Calling merge on all lists.
INFO[0055] Counters ignored=0 len_cnq=0 package=loader parsed=1291401 pending=2064 processed=1289337 read=1292331
2016/08/24 10:37:07 Merged lists. Calling GC.
2016/08/24 10:37:08 Trying to free OS memory
INFO[0056] Counters ignored=0 len_cnq=10000 package=loader parsed=1302338 pending=13000 processed=1289338 read=1312340
2016/08/24 10:37:09 Memory usage after calling GC. Allocated MB: 1144
INFO[0058] Counters ignored=0 len_cnq=0 package=loader parsed=1308558 pending=2259 processed=1306299 read=1312340
INFO[0059] Counters ignored=0 len_cnq=0 package=loader parsed=1339505 pending=971 processed=1338534 read=1348569
Total memory: 1144
INFO[0060] Counters ignored=0 len_cnq=0 package=loader parsed=1379708 pending=2932 processed=1376776 read=1385650
2016/08/24 10:37:12 Memory usage over threshold. STW. Allocated MB: 1144
2016/08/24 10:37:12 Calling merge on all lists.
2016/08/24 10:37:12 Merged lists. Calling GC.
2016/08/24 10:37:13 Trying to free OS memory
As can be seen, even after calling the GC, the usage info given by Maxrss remains the same indicating it is the maximum till that point.
mrjn
August 24, 2016, 12:32pm
5
Linux is going to be our main play field. So, we could use the proc stat feature, to get the exact memory usage by the process. I think Mac might also support it.
ashwin95r
November 28, 2017, 1:00am
6
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.