abi
abiā€¢17mo ago

Read one byte at a time from ReadableStream

What would be the most obvious way to read a single byte from a ReadableStream at a time?
14 Replies
crowlKats
crowlKatsā€¢17mo ago
i assume its a readablestream with the type set to "bytes"?
abi
abiā€¢17mo ago
Well, Iā€™m not sure about the bytes part, but assuming that, what would be the best way? Only thing I can think of is a custom transform stream which enqueues the source chunks as 1-byte chunks.
crowlKats
crowlKatsā€¢17mo ago
well, if it is a bytes stream, then you can do:
const reader = stream.getReader({ mode: "byob" });
const buf = new Uint8Array(1);
await reader.read(buf);
// buf now contains the single byte
const reader = stream.getReader({ mode: "byob" });
const buf = new Uint8Array(1);
await reader.read(buf);
// buf now contains the single byte
abi
abiā€¢17mo ago
Ah there I go, thanks! alright, so in my case, the source happens to be Deno.Conn and it seams that its readable is not a readable byte stream... and i can't figure out how to convert it into one šŸ˜¦ i guess this is basically what i'm looking for: https://streams.spec.whatwg.org/#example-rbs-push or rather this one: https://streams.spec.whatwg.org/#example-rs-push-backpressure
crowlKats
crowlKatsā€¢17mo ago
it should be; let me check yea it is. what is not working?
abi
abiā€¢17mo ago
const listener = Deno.listen({ port: 8888 })
for await (const conn of listener) {
const source = conn.readable
source.pipeThrough(
new TransformStream({
transform: (chunk, controller) => {
controller.enqueue(chunk)
},
}),
).getReader({ mode: "byob" })
}
const listener = Deno.listen({ port: 8888 })
for await (const conn of listener) {
const source = conn.readable
source.pipeThrough(
new TransformStream({
transform: (chunk, controller) => {
controller.enqueue(chunk)
},
}),
).getReader({ mode: "byob" })
}
then i just make a request: $ curl -v 127.0.0.1:8888
error: Uncaught TypeError: Cannot use a BYOB reader with a non-byte stream
).getReader({ mode: "byob" })
^
at setUpReadableStreamBYOBReader (internal:ext/web/06_streams.js:3344:13)
at acquireReadableStreamBYOBReader (internal:ext/web/06_streams.js:307:5)
at ReadableStream.getReader (internal:ext/web/06_streams.js:4784:16)
at file:///Users/abi/Code/ozaman/think.ts:10:5
error: Uncaught TypeError: Cannot use a BYOB reader with a non-byte stream
).getReader({ mode: "byob" })
^
at setUpReadableStreamBYOBReader (internal:ext/web/06_streams.js:3344:13)
at acquireReadableStreamBYOBReader (internal:ext/web/06_streams.js:307:5)
at ReadableStream.getReader (internal:ext/web/06_streams.js:4784:16)
at file:///Users/abi/Code/ozaman/think.ts:10:5
is it the .pipeThrough that's causing issues?
crowlKats
crowlKatsā€¢17mo ago
oh yea, that is indeed the issue sadly TransformStreams currently dont properly support byte streams :(
abi
abiā€¢17mo ago
ah well :/ then i'm going to have to take a different route i guess
crowlKats
crowlKatsā€¢17mo ago
GitHub
TransformStream byte streams Ā· Issue #616 Ā· whatwg/streams
@isonmad submitted a pull request #601 to support asymmetric default->byte TransformStream objects. Although only default->default and default->byte transforms are possible at ...
abi
abiā€¢17mo ago
thanks i have a tcp server, and clients will be sending both "string messages" and binary content at different times. i would like to have two readable streams, sourced from the same tcp client. one for text and one for binary. i want them to never read the same data twice. i need to be able to switch the "active mode" at any given moment. how would you go about this personally in deno? for example a client would send something like this
HELLO.
NICE WEATHER.
BLOB.
<...binary content...>
<binary sequence [13, 10, 13, 10] found>
THANKS.
BYE.
HELLO.
NICE WEATHER.
BLOB.
<...binary content...>
<binary sequence [13, 10, 13, 10] found>
THANKS.
BYE.
crowlKats
crowlKatsā€¢17mo ago
do these binary messages possibly contain the bytes for newlines? and same question for strings
abi
abiā€¢17mo ago
string messages are delimited by CRLF. binary content will contain CRLF, but that's not necessarily their delimiter. CRLF, period, CRLF, is. in other words, [13, 10, 46, 13, 10] would be the delimiter byte sequence for binary content and [13, 10] for text content.
crowlKats
crowlKatsā€¢17mo ago
hm it would be a lot easier if they both had the same delimiter, and then you could use https://deno.land/std@0.177.0/streams/mod.ts?s=DelimiterStream, and have the first byte of each message tell you the type of message, and use a transformstream to check for this indentifying byte and if its a string, decode the chunk i guess what you could do is the delimiter would be [13, 10] & instead of first byte, check if the chunk ends with [13, 10 46] to tell if its a buf or not
abi
abiā€¢17mo ago
unfortunately i didn't define the protocol, but i actually made a PR for adding some functionality to DelimiterStream, hoping it will make it past review. not sure if it would help here but... you're making a lot of sense right now, i think i'm getting a bit ahead of myself. i'll give it another go without tee'ing and transforming unnecessarily.