Coroutines in Python for Data Engineering (1)

Coroutine — greenlet

from greenlet import greenlet

def test1():
    print("Test1 begins")
    gr2.switch()
    print("Test1 ends")

def test2():
    print("Test2 begins")
    gr1.switch()
    print("Test2 ends")

gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch()

Greenlet provides a more straightforward way switch() to switch between functions, instead of yield, next and send.

In the code above, we created two greenlet object, gr1 and gr2 . gr1 is actually calling test and gr2 is calling test2 . From the print of the code we can tell that test2 is executed beforetest1 returns, but they still runs in a single thread, so the above code cannot take advantage of a multi core CPU.

Here is the signiture of the constructor of greenlet object ,

greenlet(run=None, parent=None)

, where the run parameter is which function the greenlet will run, like test1and test2in our example, and the parent parameter is to specify what is the parent coroutine of this coroutine. When a child coroutine exits, it will return to its parent. There is a main coroutine which will be create implictly by the program. The main coroutine is the default parent for all coroutines is we won’t specify the parent paramenter.

Coroutine — eventlet

Eventlet is build on greenlet by implementing its own greenThread and scheduler called hub. Hub is singleton and…