Iterator utils

dpipe.itertools.pam(functions: Iterable[Callable], *args, **kwargs)

Inverse of map. Apply a sequence of callables to fixed arguments.

Examples

>>> list(pam([np.sqrt, np.square, np.cbrt], 64))
[8, 4096, 4]

dpipe.itertools.zip_equal(*args: Union[Sized, Iterable]) → Iterable[Tuple]

zip over the given iterables, but enforce that all of them exhaust simultaneously.

Examples

>>> zip_equal([1, 2, 3], [4, 5, 6]) # ok
>>> zip_equal([1, 2, 3], [4, 5, 6, 7]) # raises ValueError
# ValueError is raised even if the lengths are not known
>>> zip_equal([1, 2, 3], map(np.sqrt, [4, 5, 6])) # ok
>>> zip_equal([1, 2, 3], map(np.sqrt, [4, 5, 6, 7])) # raises ValueError

dpipe.itertools.head_tail(iterable: Iterable) → Tuple[Any, Iterable]

Split the iterable into the first and the rest of the elements.

Examples

>>> head, tail = head_tail(map(np.square, [1, 2, 3]))
>>> head, list(tail)
1, [4, 9]

dpipe.itertools.peek(iterable: Iterable) → Tuple[Any, Iterable]

Return the first element from iterable and the whole iterable.

Notes

The incoming iterable might be mutated, use the returned iterable instead.

Examples

>>> original_iterable = map(np.square, [1, 2, 3])
>>> head, iterable = peek(original_iterable)
>>> head, list(iterable)
1, [1, 4, 9]
# list(original_iterable) would return [4, 9]

dpipe.itertools.lmap(func: Callable, *iterables: Iterable) → list

Composition of list and map.

dpipe.itertools.pmap(func: Callable, iterable: Iterable, *args, **kwargs) → Iterable

Partial map. Maps func over iterable using args and kwargs as additional arguments.

dpipe.itertools.dmap(func: Callable, dictionary: dict, *args, **kwargs)

Transform the dictionary by mapping func over its values. args and kwargs are passed as additional arguments.

Examples

>>> dmap(np.square, {'a': 1, 'b': 2})
{'a': 1, 'b': 4}

dpipe.itertools.zdict(keys: Iterable, values: Iterable) → dict

Create a dictionary from keys and values.

dpipe.itertools.squeeze_first(inputs)

Remove the first dimension in case it is singleton.

dpipe.itertools.flatten(iterable: Iterable, iterable_types: Optional[Union[tuple, type]] = None) → list

Recursively flattens an iterable as long as it is an instance of iterable_types.

Examples

>>> flatten([1, [2, 3], [[4]]])
[1, 2, 3, 4]
>>> flatten([1, (2, 3), [[4]]])
[1, (2, 3), 4]
>>> flatten([1, (2, 3), [[4]]], iterable_types=(list, tuple))
[1, 2, 3, 4]

dpipe.itertools.filter_mask(iterable: Iterable, mask: Iterable[bool]) → Iterable

Filter values from iterable according to mask.

dpipe.itertools.extract(sequence: Sequence, indices: Iterable)

Extract indices from sequence.

dpipe.itertools.negate_indices(indices: Iterable, length: int)

Return valid indices for a sequence of len length that are not present in indices.

dpipe.itertools.make_chunks(iterable: Iterable, chunk_size: int, incomplete: bool = True)

Group iterable into chunks of size chunk_size.

Parameters

• iterable –

• chunk_size –

• incomplete – whether to yield the last chunk in case it has a smaller size.

dpipe.itertools.collect(func: Callable)

Make a function that returns a list from a function that returns an iterator.

Examples

>>> @collect
>>> def squares(n):
>>>     for i in range(n):
>>>         yield i ** 2
>>>
>>> squares(3)
[1, 4, 9]

dpipe.itertools.stack(axis: int = 0, dtype: Optional[dtype] = None)

Stack the values yielded by a generator function along a given axis. dtype (if any) determines the data type of the resulting array.

Examples

>>> @stack(1)
>>> def consecutive(n):
>>>     for i in range(n):
>>>         yield i, i+1
>>>
>>> consecutive(3)
array([[0, 1, 2],
       [1, 2, 3]])

dpipe.itertools.recursive_conditional_map(xr, f, condition)

Walks recursively through iterable data structure xr. Applies f on objects that satisfy condition.