abi•2y ago

Read one byte at a time from ReadableStream

What would be the most obvious way to read a single byte from a ReadableStream at a time?

14 Replies

i assume its a readablestream with the type set to "bytes"?

abi•2y ago

Well, I’m not sure about the bytes part, but assuming that, what would be the best way? Only thing I can think of is a custom transform stream which enqueues the source chunks as 1-byte chunks.

crowlKats•2y ago

well, if it is a bytes stream, then you can do:

const reader = stream.getReader({ mode: "byob" });
const buf = new Uint8Array(1);
await reader.read(buf);
// buf now contains the single byte

const reader = stream.getReader({ mode: "byob" });
const buf = new Uint8Array(1);
await reader.read(buf);
// buf now contains the single byte

abi•2y ago

Ah there I go, thanks! alright, so in my case, the source happens to be Deno.Conn and it seams that its readable is not a readable byte stream... and i can't figure out how to convert it into one 😦 i guess this is basically what i'm looking for: https://streams.spec.whatwg.org/#example-rbs-push or rather this one: https://streams.spec.whatwg.org/#example-rs-push-backpressure

crowlKats•2y ago

it should be; let me check yea it is. what is not working?

abi•2y ago

const listener = Deno.listen({ port: 8888 })
for await (const conn of listener) {
  const source = conn.readable
  source.pipeThrough(
    new TransformStream({
      transform: (chunk, controller) => {
        controller.enqueue(chunk)
      },
    }),
  ).getReader({ mode: "byob" })
}

const listener = Deno.listen({ port: 8888 })
for await (const conn of listener) {
  const source = conn.readable
  source.pipeThrough(
    new TransformStream({
      transform: (chunk, controller) => {
        controller.enqueue(chunk)
      },
    }),
  ).getReader({ mode: "byob" })
}

then i just make a request: $ curl -v 127.0.0.1:8888

error: Uncaught TypeError: Cannot use a BYOB reader with a non-byte stream
  ).getReader({ mode: "byob" })
    ^
    at setUpReadableStreamBYOBReader (internal:ext/web/06_streams.js:3344:13)
    at acquireReadableStreamBYOBReader (internal:ext/web/06_streams.js:307:5)
    at ReadableStream.getReader (internal:ext/web/06_streams.js:4784:16)
    at file:///Users/abi/Code/ozaman/think.ts:10:5

error: Uncaught TypeError: Cannot use a BYOB reader with a non-byte stream
  ).getReader({ mode: "byob" })
    ^
    at setUpReadableStreamBYOBReader (internal:ext/web/06_streams.js:3344:13)
    at acquireReadableStreamBYOBReader (internal:ext/web/06_streams.js:307:5)
    at ReadableStream.getReader (internal:ext/web/06_streams.js:4784:16)
    at file:///Users/abi/Code/ozaman/think.ts:10:5

is it the .pipeThrough that's causing issues?

crowlKats•2y ago

oh yea, that is indeed the issue sadly TransformStreams currently dont properly support byte streams :(

abi•2y ago

ah well :/ then i'm going to have to take a different route i guess

crowlKats•2y ago

https://github.com/whatwg/streams/issues/616 for reference

GitHub

TransformStream byte streams · Issue #616 · whatwg/streams

@isonmad submitted a pull request #601 to support asymmetric default->byte TransformStream objects. Although only default->default and default->byte transforms are possible at ...

abi•2y ago

thanks i have a tcp server, and clients will be sending both "string messages" and binary content at different times. i would like to have two readable streams, sourced from the same tcp client. one for text and one for binary. i want them to never read the same data twice. i need to be able to switch the "active mode" at any given moment. how would you go about this personally in deno? for example a client would send something like this

HELLO.
NICE WEATHER.
BLOB.
<...binary content...>
<binary sequence [13, 10, 13, 10] found>
THANKS.
BYE.

HELLO.
NICE WEATHER.
BLOB.
<...binary content...>
<binary sequence [13, 10, 13, 10] found>
THANKS.
BYE.

crowlKats•2y ago

do these binary messages possibly contain the bytes for newlines? and same question for strings

abi•2y ago

string messages are delimited by CRLF. binary content will contain CRLF, but that's not necessarily their delimiter. CRLF, period, CRLF, is. in other words, [13, 10, 46, 13, 10] would be the delimiter byte sequence for binary content and [13, 10] for text content.

crowlKats•2y ago

hm it would be a lot easier if they both had the same delimiter, and then you could use https://deno.land/std@0.177.0/streams/mod.ts?s=DelimiterStream, and have the first byte of each message tell you the type of message, and use a transformstream to check for this indentifying byte and if its a string, decode the chunk i guess what you could do is the delimiter would be [13, 10] & instead of first byte, check if the chunk ends with [13, 10 46] to tell if its a buf or not

abi•2y ago

unfortunately i didn't define the protocol, but i actually made a PR for adding some functionality to DelimiterStream, hoping it will make it past review. not sure if it would help here but... you're making a lot of sense right now, i think i'm getting a bit ahead of myself. i'll give it another go without tee'ing and transforming unnecessarily.