A Python Development Utility You Should Know

A library to perform different tasks on iterables and do them neatly!

Photo by Yasamine June on Unsplash

Iterables in Python consist of lists, sets, dictionaries, and tuples. They are used across any program we write for any number of times, and often we perform a range of operations on them with varying complexity. 

These operations can be from simple traversals to complex maps and filter operations using the two map and filterdesignated Python functions. 

The Pipe library

Pipe is a library that enables us to use an infix syntax in Python when dealing with these iterables to perform any number of tasks. 

We can separate different filters we want to use on our iterable with the help of pipes (the | symbol) which you might also know as the OR symbol that we typically use in conditional statements in Python. 

 Let’s install it and I’ll show you some of the best use cases I’ve found from it! 

pip install pipe

Traversing a nested list the easy way

A concise way to traverse (or print) a complex nested list or list of lists of lists is easily done with pipe’s traversecommand. 

list([[1, 2], [3, 4], [5], [6, [7, 8, 9]]] | traverse)

Out:
[1, 2, 3, 4, 5, 6, 7, 8, 9]

Remove duplicates 

Removing duplicates from a list for example and returning with or without absolute values is easily done with dedupfunction:

print("With signs: ", list([-1, 0, 0, 0, 1, 2, 3] | dedup))
print("Ignoring signs: ", list([-1, 0, 0, 0, 1, 2, 3] | dedup(key=abs)))

# Out
With signs:  [-1, 0, 1, 2, 3] 
Ignoring signs:  [-1, 0, 2, 3]

Remove duplicates but only the consecutive ones:

If you only want to remove the duplicate values that appear consecutively in a given iterable, you can do so with the unique function:

list([1, 1, 2, 2, 3, 3, 1, 2, 3] | uniq)

Out:
[1, 2, 3, 1, 2, 3]

This function also has an key optional parameter to ignore signs of the values:

list([1, -1, 1, 2, -2, 2, 3, 3, 1, 2, 3] | uniq(key=abs))

Out:
[1, 2, 3, 1, 2, 3]

Permutations

Get all permutations of a given set of characters in an iterable can also be done: 

list(['X', 'Y', 'Z'] | permutations(2))

Out:
[('X', 'Y'), ('X', 'Z'), ('Y', 'X'), ('Y', 'Z'), ('Z', 'X'), ('Z', 'Y')]

You can also do it for any given string, provided you first make sure the input to the pipe function is an iterable. It can done via a for loop: 

print("With two letters:")
for item in 'THEN' | permutations(2):
    print(item)

Out: 
With two letters: 
('T', 'H') ('T', 'E') ('T', 'N') ('H', 'T') ('H', 'E') ('H', 'N') ('E', 'T') ('E', 'H') ('E', 'N') ('N', 'T') ('N', 'H') ('N', 'E')

Transpose a matrix

Swap the row and column values or perform a transpose of a given Python matrix (comprised of a list of list or something similar) can be done by the transpose function:

list([[1, 2, 3], [4, 5, 6], [7, 8, 9]] | transpose)

Out:
[(1, 4, 7), (2, 5, 8), (3, 6, 9)]

Apply “filter” functions to iterables

The select function in pipe is similar to the map and filter functions that come built-in with Python. They do in fact perform similar functions with the iterables:

mylist = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# find the even numbers
list(mylist | select(lambda num: num % 2 == 0))

# Out:
[False, True, False, True, False, True, False, True, False, True]

However, it does come with a nice advantage of being able to chain the select function with a where function; something like this: 

list(mylist | where (lambda num: num > 5)
            | select(lambda num: num % 2 == 0)
)

Out:
[True, False, True, False, True]

You can also combine dictionaries with lists to perform specific operations on either key or value objects by chaining multiple methods together, similar to the above example. 

Miscellaneous functions

Apart from these, the library also has some common functions like groupby and chain that do similar functions as the built-in module functions from itertools. 

list([[1, 2], [3, 4], [5]] | chain)

Out:
[1, 2, 3, 4, 5]

Skip elements from an iterable with theskip function: 

list((1, 2, 3, 4, 5) | skip(2))

Out:
[3, 4, 5]

You can also use the itertools.islice as a Pipe function:

list((1, 2, 3, 4, 5, 6, 7, 8, 9) | islice(2, 8, 2))

Out:
[3, 5, 7]

And so on, you can read about them in detail in the documentation. 

A few parting words

So that was a rather brief yet probing discussion on this useful library I came across one afternoon. I hope you find some value from it, seeing that it does allow you to perform a ton of functions that are a little too complicated to write via list comprehensions or other similar means. 

But beware of doing this on very large iterables though. There may a runtime caveat associated with using it that I haven’t explored here, and if you’re dealing with one, do make sure to check for runtime differences when using some of the pipe functions with larger data! 

But, do experiment on your own if you can with custom pipe functions. The documentation on the GitHub page does have useful info on that too. 

Lastly, if you found this tutorial helpful, share this publication! :)