Iterator utils¶
- dpipe.itertools.pam(functions: Iterable[Callable], *args, **kwargs)[source]¶
Inverse of
map
. Apply a sequence of callables to fixed arguments.Examples
>>> list(pam([np.sqrt, np.square, np.cbrt], 64)) [8, 4096, 4]
- dpipe.itertools.zip_equal(*args: Union[Sized, Iterable]) Iterable[Tuple] [source]¶
zip over the given iterables, but enforce that all of them exhaust simultaneously.
Examples
>>> zip_equal([1, 2, 3], [4, 5, 6]) # ok >>> zip_equal([1, 2, 3], [4, 5, 6, 7]) # raises ValueError # ValueError is raised even if the lengths are not known >>> zip_equal([1, 2, 3], map(np.sqrt, [4, 5, 6])) # ok >>> zip_equal([1, 2, 3], map(np.sqrt, [4, 5, 6, 7])) # raises ValueError
- dpipe.itertools.head_tail(iterable: Iterable) Tuple[Any, Iterable] [source]¶
Split the
iterable
into the first and the rest of the elements.Examples
>>> head, tail = head_tail(map(np.square, [1, 2, 3])) >>> head, list(tail) 1, [4, 9]
- dpipe.itertools.peek(iterable: Iterable) Tuple[Any, Iterable] [source]¶
Return the first element from
iterable
and the whole iterable.Notes
The incoming
iterable
might be mutated, use the returned iterable instead.Examples
>>> original_iterable = map(np.square, [1, 2, 3]) >>> head, iterable = peek(original_iterable) >>> head, list(iterable) 1, [1, 4, 9] # list(original_iterable) would return [4, 9]
- dpipe.itertools.lmap(func: Callable, *iterables: Iterable) list [source]¶
Composition of list and map.
- dpipe.itertools.pmap(func: Callable, iterable: Iterable, *args, **kwargs) Iterable [source]¶
Partial map. Maps
func
overiterable
usingargs
andkwargs
as additional arguments.
- dpipe.itertools.dmap(func: Callable, dictionary: dict, *args, **kwargs)[source]¶
Transform the
dictionary
by mappingfunc
over its values.args
andkwargs
are passed as additional arguments.Examples
>>> dmap(np.square, {'a': 1, 'b': 2}) {'a': 1, 'b': 4}
- dpipe.itertools.zdict(keys: Iterable, values: Iterable) dict [source]¶
Create a dictionary from
keys
andvalues
.
- dpipe.itertools.flatten(iterable: Iterable, iterable_types: Optional[Union[tuple, type]] = None) list [source]¶
Recursively flattens an
iterable
as long as it is an instance ofiterable_types
.Examples
>>> flatten([1, [2, 3], [[4]]]) [1, 2, 3, 4] >>> flatten([1, (2, 3), [[4]]]) [1, (2, 3), 4] >>> flatten([1, (2, 3), [[4]]], iterable_types=(list, tuple)) [1, 2, 3, 4]
- dpipe.itertools.filter_mask(iterable: Iterable, mask: Iterable[bool]) Iterable [source]¶
Filter values from
iterable
according tomask
.
- dpipe.itertools.extract(sequence: Sequence, indices: Iterable)[source]¶
Extract
indices
fromsequence
.
- dpipe.itertools.negate_indices(indices: Iterable, length: int)[source]¶
Return valid indices for a sequence of len
length
that are not present inindices
.
- dpipe.itertools.make_chunks(iterable: Iterable, chunk_size: int, incomplete: bool = True)[source]¶
Group
iterable
into chunks of sizechunk_size
.- Parameters
iterable –
chunk_size –
incomplete – whether to yield the last chunk in case it has a smaller size.
- dpipe.itertools.collect(func: Callable)[source]¶
Make a function that returns a list from a function that returns an iterator.
Examples
>>> @collect >>> def squares(n): >>> for i in range(n): >>> yield i ** 2 >>> >>> squares(3) [1, 4, 9]
- dpipe.itertools.stack(axis: int = 0, dtype: Optional[dtype] = None)[source]¶
Stack the values yielded by a generator function along a given
axis
.dtype
(if any) determines the data type of the resulting array.Examples
>>> @stack(1) >>> def consecutive(n): >>> for i in range(n): >>> yield i, i+1 >>> >>> consecutive(3) array([[0, 1, 2], [1, 2, 3]])