psbrandt•15mo ago

Tracking Down Lost Memory

Hi, I have a long running service that creates many JsRuntimes over time and I have found that I am slowly leaking memory. While trying to figure out what's going on, I found that the following simple piece of code seems to leak memory:

fn main() {
  deno_core::JsRuntime::new(deno_core::RuntimeOptions::default());
}

fn main() {
  deno_core::JsRuntime::new(deno_core::RuntimeOptions::default());
}

I am using valgrind to check for memory leaks and this is what I'm getting with DHAT*:

$ valgrind --tool=dhat target/debug/rusty-v8-memtest
==1505== Total:     2,766,643 bytes in 3,129 blocks
==1505== At t-gmax: 664,059 bytes in 1,134 blocks
==1505== At t-end:  36,733 bytes in 659 blocks

$ valgrind --tool=dhat target/debug/rusty-v8-memtest
==1505== Total:     2,766,643 bytes in 3,129 blocks
==1505== At t-gmax: 664,059 bytes in 1,134 blocks
==1505== At t-end:  36,733 bytes in 659 blocks

This is what I'm getting with memcheck*:

$ valgrind --tool=memcheck target/debug/rusty-v8-memtest
==1499== HEAP SUMMARY:
==1499==     in use at exit: 36,733 bytes in 659 blocks
==1499==   total heap usage: 3,129 allocs, 2,470 frees, 2,766,643 bytes allocated
==1499==
==1499== LEAK SUMMARY:
==1499==    definitely lost: 1,953 bytes in 14 blocks
==1499==    indirectly lost: 1,443 bytes in 23 blocks
==1499==      possibly lost: 1,440 bytes in 5 blocks
==1499==    still reachable: 31,897 bytes in 617 blocks
==1499==         suppressed: 0 bytes in 0 blocks
==1499== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

$ valgrind --tool=memcheck target/debug/rusty-v8-memtest
==1499== HEAP SUMMARY:
==1499==     in use at exit: 36,733 bytes in 659 blocks
==1499==   total heap usage: 3,129 allocs, 2,470 frees, 2,766,643 bytes allocated
==1499==
==1499== LEAK SUMMARY:
==1499==    definitely lost: 1,953 bytes in 14 blocks
==1499==    indirectly lost: 1,443 bytes in 23 blocks
==1499==      possibly lost: 1,440 bytes in 5 blocks
==1499==    still reachable: 31,897 bytes in 617 blocks
==1499==         suppressed: 0 bytes in 0 blocks
==1499== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

My memory leak could still absolutely be in my own code, but are the leaks above expected? The DHAT viewer output can be seen here: https://github.com/psbrandt/rusty-v8-memtest/blob/master/dhat-output/master/dhat-master.pdf. I also compiled a custom V8 without WASM (which seems to be one source of the leaks) and I the leaked memory is reduced, but not to zero. Am I doing something wrong or is there something I can do to clean up after a JsRuntime (besides dispose and/or dispose_platform)? Any ideas on how I could debug further? *Some output deleted to meet post length limit restriction.

3 Replies

Deno AI Helper•15mo ago

Based on the documentation, there are several ways you can debug and track down memory leaks in Deno. 1. Deno.metrics: This API provides information about the count of started and completed synchronous and asynchronous operations, and the amount of data that has been sent over the operations interface. In Deno 1.8, a new field was added to Deno.metrics called ops. This field contains per operation information about how often the API was invoked and how much data has been transmitted over it. This allows for far more granular instrumentation of the runtime. 2. Deno.memoryUsage: This function returns an object describing the memory usage of the Deno process and the V8 subsystem measured in bytes. It provides information about the resident set size, the total size of the heap for V8, the amount of the heap used for V8, and memory associated with JavaScript objects outside of the JavaScript isolate. 3. Debugging with Chrome DevTools: Deno has built-in debugger capabilities that make it possible to use remote debuggers like Chrome Devtools. These tools are extremely useful when trying to pin-point bugs or profile your application. 4. Profiling with V8 flags: The documentation provides a list of useful V8 flags that can be used during profiling, such as --prof, --log-internal-timer-events, --log-timer-events, --track-gc, --log-source-code, and --track-gc-object-stats. 5. Debugging with LLDB: To debug the Deno binary, you can use rust-lldb. It should come with rustc and is a wrapper around LLDB. Remember to ensure that all resources are properly closed after use. In Deno 1.19, better errors for ops and resource sanitizers in Deno.test were introduced. If a test case is leaking resources, it will fail with a concise and useful

Deno AI Helper•15mo ago

message, including a stack trace of where the async operation was started. If you've tried all these methods and are still experiencing memory leaks, it might be a good idea to open an issue on the Deno GitHub repository to get help from the Deno community. 🦕

Helpful Links: Deno 1.19 Release Notes Deno 1.19 Release Notes Deno 1.20 Release Notes Deno 1.20 Release Notes

psbrandtOP•15mo ago

I guess I should have mentioned I already opened this issue: https://github.com/denoland/rusty_v8/issues/1348