D
Deno

help

Investigating memory leak after adding new app dependency

Ddantheman3/24/2023
Hi, I added an OpenTelemetry metrics reporting loop to a Deno program and found that Deno began leaking memory (300MB+, until it hits OOM and gets killed). This program was previously stable at 40MB RAM usage. To begin, I graphed Deno.memoryUsage() and it seems like the leaked memory is all external. How else can I investigate the issue? Anything else I can add to my typescript to observe this?
Bbartlomieju3/24/2023
@danopia any chance you could say what the script does? There was another "memory leak" issue opened just yesterday and we're investigating
Ddantheman3/24/2023
I added an OpenTelemetry-JS metric reporting loop which ultimately does a fetch() every 20 seconds. But there's a bunch of code inside all the OpenTelemetry libraries that I haven't seen, so I don't know what else it does before the fetch() call 😦 I also checked Deno.resources() which looks flat, and I have a list of dispatched ops in a 30-minute window via Deno.metrics(). I can try putting together a sharable reproduction later today but getting it minimal might be tricky. I was hoping for some way I can self-diagnose what all the external memory is
Ddantheman3/24/2023
ok, I'll keep investigating later in the day and see if I can isolate which layer of my code is triggering it 🙂 @.bartlomieju Oh, do you mean #18369 memory leak for >1.31.0? I already verified my behavior is the same on 1.30.0 so what I'm looking at is not a regression I'm trying to get a minimal repro of my issue put together now and it seems like some irresponsible Response cloning here: https://github.com/open-telemetry/opentelemetry-js/blob/0c37daa0b1fe59a04810514ab4f995070ed84825/experimental/packages/opentelemetry-instrumentation-fetch/src/fetch.ts#L342-L344 I don't see anything cancelling the body of resClone4Hook Yea, that opentelemetry-js code looks like it was my memory leak ^^ The fetch response bodies must be sitting in external
Bbartlomieju3/24/2023
Yes, that what I meant Alright! Glad that you found it. If you have any suggestions how we could improve visibility to things like that I'm very open to feedback. On related note, we thought about adding more first class support for visibility/observability - Node.js has "diagnostics channel", but it doesn't seem like a silver bullet. So again - any feedback and requests here are much appreciated

Looking for more? Join the community!

Recommended Posts
VScode deno lint blew up again after update.It seems every time there's an extension update VScode lint blows up and will stop recognizing top limport cache not workingI can't install cache of my http imports:Why isn't this error caught?Basic code example: ``` try { Deno.remove("fakefile"); } catch (err) { console.log("File doesn'tUsing dynamic import with npm compatibility gives intermittent error "__DENO_NODE_GLOBAL_THIS_167935This is pretty hard to google because the number at the end of the error changes everytime it happenGraphql codegendoes anyone have a solution to generate the sdk using graphq-request with Deno?Deno Third Party Modules Web HooksIs there a way to see the webhook payload URL a package you have published is listening for?Does Deno Deploy support location based analytics?The free version does not seem to support it. Does the paid version support it?IPC solution for a process manager👋 I'm at day 7 of working on Pup (https://github.com/hexagon/pup) - a universal process manager rSpecifying config remotely or jsx on installI'm struggling quite a lot here. I originally recommended using this command: ```bash deno install -Is it possible to know what version of a dynamically-imported module was just imported?Right now, you can run code like `import('npm:lodash')`, which will get the latest version of lodashWhy am I getting an "Uncaught DOM Exception" in the Deno CLI?To replicate: 1. `deno` in the command line to initialize a Deno REPL ``` // Get an array of random Better way to allow downloading of files besides serving entire fileI'm trying to allow users to download files that are in a private folder which cannot be accessed puShare state between islandsHi. How could I share state between island. E.g. shopping cart. I click on add product and in some oIslands not working with Cloudflare DNS proxyI have deployed Deno on a remote server with nginx-ingress-controller that offers self-signed certifUpdate sub-directory for module already added to deno.land/xThe title pretty much says it all, but the module in question is `hot_mod`, and the current subdir iShould we, or how should we, use Deno as a sandbox for untrusted code?I've been using Deno in this context because of some of the nice security model features. However, ideno_bindgen type error: deno-ts(2345)Anyone else getting this with `deno_bindgen`? > Argument of type 'bigint' is not assignable to paramUsing Rust FFI in a public Deno moduleI'm writing a Deno module that I intend to publish on deno.land. It will be using a Rust library (puPublishing to deno.land/x: GitHub webhook says it's delivered?I followed these directions (https://deno.land/add_module) to publish a 3rd party module to deno.lanDeno tsWhenever i run my deno file using a ts config like this documentation describes: https://deno.land/