Robbie Posts

Robbie

•Created by Robbie on 12/18/2024 in #help

Issues with importing from npm - should I switch to esm.sh?

Is importing from esm.sh recommended over npm? Recently I've been having a lot of issues with imports from npm inside Jupyter. For example:

import * as crypto from "node:crypto";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.714.0";

import * as crypto from "node:crypto";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.714.0";

results in an error:

Stack trace:
Error: Cannot find module '@aws-sdk/credential-provider-node'
Require stack:
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/runtimeConfig.js
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/index.js
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/index.js
    at Function.Module._resolveFilename (node:module:619:15)
    at Function.Module._load (node:module:497:27)
    at Module.require (node:module:681:19)
    at require (node:module:812:16)
    at Object.<anonymous> (file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/runtimeConfig.js:58:4)
    at Module._compile (node:module:745:34)
    at loadMaybeCjs (node:module:770:10)
    at Object.Module._extensions..js (node:module:755:12)
    at Module.load (node:module:662:32)

Stack trace:
Error: Cannot find module '@aws-sdk/credential-provider-node'
Require stack:
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/runtimeConfig.js
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/index.js
- /Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/index.js
    at Function.Module._resolveFilename (node:module:619:15)
    at Function.Module._load (node:module:497:27)
    at Module.require (node:module:681:19)
    at require (node:module:812:16)
    at Object.<anonymous> (file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_1/dist-cjs/runtimeConfig.js:58:4)
    at Module._compile (node:module:745:34)
    at loadMaybeCjs (node:module:770:10)
    at Object.Module._extensions..js (node:module:755:12)
    at Module.load (node:module:662:32)

and

import * as crypto from "node:crypto";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.714.0";
import { DynamoDBDocument } from "npm:@aws-sdk/lib-dynamodb@3.431.0";

import * as crypto from "node:crypto";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.714.0";
import { DynamoDBDocument } from "npm:@aws-sdk/lib-dynamodb@3.431.0";

results in a different error:

Stack trace:
TypeError: [ERR_MODULE_NOT_FOUND] Cannot find module 'file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_2/index.js' imported from 'file:///Users/robbie/Developer/iyield/$deno$repl.mts'
    at async <anonymous>:2:22

Stack trace:
TypeError: [ERR_MODULE_NOT_FOUND] Cannot find module 'file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/client-dynamodb/3.714.0_2/index.js' imported from 'file:///Users/robbie/Developer/iyield/$deno$repl.mts'
    at async <anonymous>:2:22

If I switch to:

import * as crypto from "node:crypto";
import { DynamoDB } from "https://esm.sh/@aws-sdk/client-dynamodb@3.714.0";
import { DynamoDBDocument } from "https://esm.sh/@aws-sdk/lib-dynamodb@3.714.0";

import * as crypto from "node:crypto";
import { DynamoDB } from "https://esm.sh/@aws-sdk/client-dynamodb@3.714.0";
import { DynamoDBDocument } from "https://esm.sh/@aws-sdk/lib-dynamodb@3.714.0";

The errors go away.

2 replies

DDeno

•Created by Robbie on 11/28/2024 in #help

Import errors when importing into Jupyter

I'm seeing some weird behaviour when I try to import a module into Jupyter. A simple reproduction is:

import { unmarshall } from "npm:@aws-sdk/util-dynamodb@3.699.0";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.431.0";
console.log(unmarshall);

import { unmarshall } from "npm:@aws-sdk/util-dynamodb@3.699.0";
import { DynamoDB } from "npm:@aws-sdk/client-dynamodb@3.431.0";
console.log(unmarshall);

If I run that code in Jupyter it works fine but when I move it to a file foo.ts and import that module in Jupyter...

import * as foo from "./foo.ts";

import * as foo from "./foo.ts";

I get an error:

Stack trace:
TypeError: [ERR_MODULE_NOT_FOUND] Cannot find module 'file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/util-dynamodb/3.699.0_1/index.js' imported from 'file:///private/tmp/foo.ts'
    at async <anonymous>:4:13

Stack trace:
TypeError: [ERR_MODULE_NOT_FOUND] Cannot find module 'file:///Users/robbie/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/util-dynamodb/3.699.0_1/index.js' imported from 'file:///private/tmp/foo.ts'
    at async <anonymous>:4:13

What's strange is if I change the order of the two import lines it works... but then the error will come back if I add a 3rd import from a different @aws-sdk package. Does anyone have any idea what's going on and whether there is any workaround?

1 replies

DDeno

•Created by Robbie on 11/28/2024 in #help

Replacing `fs.createWriteStream()` with Deno equivalent

Hi, I am trying to download files from S3 to disk and it works if I use fs.createWriteStream() as follows:

import * as fs from "node:fs";
import { S3 } from "npm:@aws-sdk/client-s3@3.701.0";

const s3 = new S3();
const response = await s3.getObject({ Bucket: "mybucket", Key: "mykey" });
const result = await response.Body.pipe(fs.createWriteStream("out.dat"));

import * as fs from "node:fs";
import { S3 } from "npm:@aws-sdk/client-s3@3.701.0";

const s3 = new S3();
const response = await s3.getObject({ Bucket: "mybucket", Key: "mykey" });
const result = await response.Body.pipe(fs.createWriteStream("out.dat"));

How would I achieve the same thing with Deno's standard library instead of node:fs? I have tried the following:

const s3 = new S3();
const response = await s3.getObject({ Bucket: "mybucket", Key: "mykey" });
const output = await Deno.open("out.dat", { write: true, create: true });
const outputWriter = output.writable.getWriter();
await outputWriter.ready;
const result = await response.Body.pipe(outputWriter);

const s3 = new S3();
const response = await s3.getObject({ Bucket: "mybucket", Key: "mykey" });
const output = await Deno.open("out.dat", { write: true, create: true });
const outputWriter = output.writable.getWriter();
await outputWriter.ready;
const result = await response.Body.pipe(outputWriter);

But I get an error:

Stack trace:
TypeError: dest.on is not a function
    at IncomingMessageForClient.Readable.pipe (ext:deno_node/_stream.mjs:2688:12)
    at <anonymous>:11:36
    at eventLoopTick (ext:core/01_core.js:175:7)

Stack trace:
TypeError: dest.on is not a function
    at IncomingMessageForClient.Readable.pipe (ext:deno_node/_stream.mjs:2688:12)
    at <anonymous>:11:36
    at eventLoopTick (ext:core/01_core.js:175:7)

Any ideas?

9 replies