batching tasks

I'm calling an api and get an array of 2000+ items. I want to chunk these into batches of 100 and then process them all concurrently.

I've tried BatchQueueBatchQueue but it looks like it stops after processing only the first batch.

const batchQueue = new BatchQueue(100)
const data = await response.json() as ApiResponse[];

let processedCount = 0;

this.batchQueue.queue(
  ...data.map((spin) => async () => {
    try {
      await this.processSpin(spin);
      processedCount++;
    } catch (error) {
        console.error(error);
    }
  }),
);

await this.batchQueue.run();

const batchQueue = new BatchQueue(100)
const data = await response.json() as ApiResponse[];

let processedCount = 0;

this.batchQueue.queue(
  ...data.map((spin) => async () => {
    try {
      await this.processSpin(spin);
      processedCount++;
    } catch (error) {
        console.error(error);
    }
  }),
);

await this.batchQueue.run();

What's the best way to do this preferably without third party modules?

Deno•15mo ago•

11 replies

beerman

batching tasks

const batchQueue = new BatchQueue(100)
const data = await response.json() as ApiResponse[];

let processedCount = 0;

this.batchQueue.queue(
  ...data.map((spin) => async () => {
    try {
      await this.processSpin(spin);
      processedCount++;
    } catch (error) {
        console.error(error);
    }
  }),
);

await this.batchQueue.run();

const batchQueue = new BatchQueue(100)
const data = await response.json() as ApiResponse[];

let processedCount = 0;

this.batchQueue.queue(
  ...data.map((spin) => async () => {
    try {
      await this.processSpin(spin);
      processedCount++;
    } catch (error) {
        console.error(error);
    }
  }),
);

await this.batchQueue.run();

What's the best way to do this preferably without third party modules?

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

Deno

Chat about Deno, a modern runtime for JavaScript and TypeScript.

20,934 Members

Join

batching tasks

batching tasks

Continue the conversation

Deno

Continue the conversation

Deno

Similar Threads