[python]yield使用的最佳实践[1]

7月 16, 2017 2条评论 12,797次阅读 1人点赞

http://www.dabeaz.com/coroutines/Coroutines.pdf

准备

python: 2.7.12
os: mac os

结论 [使用场景]

迭代生成数据(生产者)
接收数据(消费者)
中断(协作式的任务)

任何脱离以上三个场景的使用yield，请使用其它方法处理，yield是很好用，但不至于到处都可以使用的地步

协程和生成器-Coroutines and Generators

在python2.5 开始，生成器就已经加入一些新的特性，比如协程(PEP-342)(啥，不了解PEP,自己查一下)
最为知名的，一个新的叫做send()的方法
大部分的文档，或者书对于这些特性，都很少提及实际的应用场景

使用协程和生成器

生成器

先来看一段最简单的代码

In [1]: def cutdown(n):
   ...:     while n > 0:
   ...:         yield n
   ...:         n = n -1

for i in cutdown(5):
    print i 

output: 5 4 3 2 1

In [1]: def cutdown(n):

...: while n > 0:

...: yield n

...: n = n -1

for i in cutdown(5):

print i

output: 5 4 3 2 1

和直接返回值不一样的是，每次都返回其中的某一个
所以从技术层面来讲，这个在某些场景可以有for-loop的功效
虽然使用起来都是def 为开始的关键字，不过生成器的类型可等同于func ，我们对比一下就知道了

In [1]: def my_range():
   ...:     while i < 5:
   ...:         yield i
   ...:         i += 1
   ...:

In [2]: x = my_range()

In [3]: x
Out[3]: <generator object my_range at 0x10560f190>

In [1]: def my_range():

...: while i < 5:

...: yield i

...: i += 1

...:

In [2]: x = my_range()

In [3]: x

Out[3]: <generator object my_range at 0x10560f190>

一个实际的例子

python版本的 unix ‘head -f ‘

import time


def follw(thefile):
    # thefile.
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(1)
            continue
        yield line


with open('test.txt', 'r') as logfile:
    for line in follw(logfile):
        print line

import time

def follw(thefile):

# thefile.

while True:

line = thefile.readline()

if not line:

time.sleep(1)

continue

yield line

with open('test.txt', 'r') as logfile:

for line in follw(logfile):

print line

将生成器当成pipelines 使用

import time


def follw(thefile):
    # thefile.
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(1)
            continue
        yield line


def grep(pattern, lines):
    for line in lines:
        if pattern in line:
            yield line

logfile = open("test.txt")
loglines = follw(logfile)
pylines = grep("python", logfile)


for line in pylines:
    print line

import time

def follw(thefile):

# thefile.

while True:

line = thefile.readline()

if not line:

time.sleep(1)

continue

yield line

def grep(pattern, lines):

for line in lines:

if pattern in line:

yield line

logfile = open("test.txt")

loglines = follw(logfile)

pylines = grep("python", logfile)

for line in pylines:

print line

在这里，while True就失去了原来一直循环的作用，这里只对有的值做处理

将yield做为表达式使用

直接上代码，讲一下当作消费者使用时的实际场景

def coroutine(func):
    def wrapper(*args, **kwargs):
        # start the func
        cr = func(*args, **kwargs)
        cr.next()
        return cr
    return wrapper


@coroutine
def grep(pattern):
    print 'Now im looking for the :{}'.format(pattern)
    try:
        while True:
            line = yield
            if pattern in line:
                print line
    except GeneratorExit:
        print 'now the exit the coroutine'


g = grep('python')
g.close()
g.send('nice')