pMapIterable
currently yields mapper
results in the order they were obtained from the input iterable
. In other words, pMapIterable
always waits for, and returns, the first promise in the queue. In my use case, the order of my input iterable
is insignificant: it is essentially an unordered stream. In this use case, I want to async
ly map, with bounded concurrency, each element of the iterable and treat the results as another unordered stream: the output order does not need to match the input order. Forcing the output order to match the input order introduces head-of-line blocking: mapper
results later in the promises
queue that are currently available are blocked from further processing by pending results earlier in the queue. Additionally, new promises cannot be started even though old promises have fulfilled, underutilizing the allowed concurrency
.
I'd like to propose adding a new pMapIterable
option that treats the promises
buffer like a pool rather than a queue. In other words, the returned async iterable would yield results from the promises
buffer as soon as they are available, rather than in FIFO order. Perhaps it could be called preserveOrder
and default to true
.
I'd be happy to work on a PR if you're interested in accepting such a feature: I think it should be relatively straightforward to devise an implementation based around the conditional use of await Promise.race(promises)
instead of await promises[0]
.
P.S.: Thanks for all these nifty p-
utils!
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too