Skip to content

voiage.parallel.distributed.distributed_chunk_map

distributed_chunk_map([positional or keyword] chunks: Sequence[Sequence[Item]] | Iterable[Sequence[Item]] = None, [positional or keyword] worker_func: Callable[[Sequence[Item]], ChunkResult] = None, [keyword-only] config: ClusterExecutionConfig | None = None, [keyword-only] n_workers: int | None = None, [keyword-only] use_processes: bool | None = None, [keyword-only] executor_factory: Callable[[int, bool], Executor] | None = None) -> list[ChunkResult]

Execute chunk-oriented work in input order across local or distributed CPU workers.

Parameters:

  • chunks Sequence[Sequence[Item]] | Iterable[Sequence[Item]]
  • worker_func Callable[[Sequence[Item]], ChunkResult]
  • config ClusterExecutionConfig | None (default: None)
  • n_workers int | None (default: None)
  • use_processes bool | None (default: None)
  • executor_factory Callable[[int, bool], Executor] | None (default: None)

Returns: list[ChunkResult]