Python多进程机制实例详解
本文实例讲述了Python多进程机制。分享给大家供大家参考。具体如下：
在以前只是接触过PYTHON的多线程机制，今天搜了一下多进程，相关文章好像不是特别多。看了几篇，小试了一把。程序如下，主要内容就是通过PRODUCER读一个本地文件，一行一行的放到队列中去。然后会有相应的WORKER从队列中取出这些行。
importmultiprocessing importos importsys importQueue importtime defwriteQ(q,obj): q.put(obj,True,None) print"putsize:",q.qsize() defreadQ(q): ret=q.get(True,1) print"getsize:",q.qsize() returnret defproducer(q): time.sleep(5)#让进行休息几秒方便ps命令看到相关内容 pid=os.getpid() handle_file='/home/dwapp/joe.wangh/test/multiprocess/datafile' withopen(handle_file,'r')asf:#with...as...这个用法今天也是第一次看到的 forlineinf: print"producer<",pid,">isdoing:",line writeQ(q,line.strip()) q.close() defworker(q): time.sleep(5)#让进行休息几秒方便ps命令看到相关内容 pid=os.getpid() empty_count=0 whileTrue: try: task=readQ(q) print"worker<",pid,">isdoing:",task ''' 如果这里不休眠的话一般情况下所有行都会被同一个子进程读取到为了使实验效果更加清楚在这里让每个进程读取完 一行内容时候休眠5s这样就可以让其他的进程到队列中进行读取 ''' time.sleep(5) exceptQueue.Empty: empty_count+=1 ifempty_count==3: print"queueisempty,quit" q.close() sys.exit(0) defmain(): concurrence=3 q=multiprocessing.Queue(10) funcs=[producer,worker] foriinrange(concurrence-1): funcs.append(worker) foriteminfuncs: printstr(item) nfuncs=range(len(funcs)) processes=[] foriinnfuncs: p=multiprocessing.Process(target=funcs[i],args=(q,)) processes.append(p) print"concurrenceworkeris:",concurrence,"workingstart" foriinnfuncs: processes[i].start() foriinnfuncs: processes[i].join() print"allDONE" if__name__=='__main__': main()
实验结果如下：
dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>python1.py <functionproducerat0xb7b9141c> <functionworkerat0xb7b91454> <functionworkerat0xb7b91454> <functionworkerat0xb7b91454> concurrenceworkeris:3workingstart producer<28320>isdoing:line1 putsize:1 producer<28320>isdoing:line2 putsize:2 producer<28320>isdoing:line3 putsize:3 producer<28320>isdoing:line4 putsize:3 producer<28320>isdoing:line5 getsize:3 putsize:4 worker<28321>isdoing:line1 getsize:3 worker<28322>isdoing:line2 getsize:2 worker<28323>isdoing:line3 getsize:1 worker<28321>isdoing:line4 getsize:0 worker<28322>isdoing:line5 queueisempty,quit queueisempty,quit queueisempty,quit allDONE
程序运行期间在另外一个窗口进行ps命令可以观测到一些进程的信息
dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef|greppython dwapp13735118300Nov20pts/1200:00:05python dwapp2831927481814:04pts/000:00:00python1.py dwapp2832028319014:04pts/000:00:00python1.py dwapp2832128319014:04pts/000:00:00python1.py dwapp2832228319014:04pts/000:00:00python1.py dwapp2832328319014:04pts/000:00:00python1.py dwapp2832527849014:04pts/1300:00:00greppython dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef|greppython dwapp13735118300Nov20pts/1200:00:05python#此时28320进程也就是PRODUCER进程已经结束 dwapp2831927481114:04pts/000:00:00python1.py dwapp2832128319014:04pts/000:00:00python1.py dwapp2832228319014:04pts/000:00:00python1.py dwapp2832328319014:04pts/000:00:00python1.py dwapp2832827849014:04pts/1300:00:00greppython dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef|greppython dwapp13735118300Nov20pts/1200:00:05python dwapp2831927481014:04pts/000:00:00python1.py dwapp2832128319014:04pts/000:00:00python1.py dwapp2832228319014:04pts/000:00:00python1.py dwapp2832328319014:04pts/000:00:00[python]<defunct>#这里应该是代表28323进程（WORKER）已经运行结束了 dwapp2833127849014:04pts/1300:00:00greppython dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef|greppython dwapp13735118300Nov20pts/1200:00:05python dwapp2833727849014:05pts/1300:00:00greppython
希望本文所述对大家的Python程序设计有所帮助。