python如何控制进程或者线程的个数
背景
日常开发中,难免遇到并发场景,而并发场景难免需要做流量控制,即需要对并发的进程或者线程的总量进行控制。今天简单总结两种常用的控制线程个数的方法。
方法一:进程池/线程池
如下例demo所示,创建了一个大小是4的进程池,然后创建5个进程,并启动
frommultiprocessingimportPool importos,time,random deflong_time_task(name): print('Runtask%s(%s)...'%(name,os.getpid())) start=time.time() time.sleep(random.random()*3) end=time.time() print('Task%sruns%0.2fseconds.'%(name,(end-start))) if__name__=='__main__': print('Parentprocess%s.'%os.getpid()) p=Pool(4) foriinrange(5): p.apply_async(long_time_task,args=(i,)) print('Waitingforallsubprocessesdone...') p.close() p.join() print('Allsubprocessesdone.')
运行结果如下,可以看到第5个进程会等池子里的进程完成一个后才会被启动
Runtask0(32952)... Runtask1(32951)... Runtask2(32953)... Runtask3(32954)... Task2runs0.68seconds. Runtask4(32953)... Task1runs1.41seconds. Task0runs1.44seconds. Task4runs2.15seconds. Task3runs2.98seconds. Allsubprocessesdone.
方法二:queue
queue模块即队列,特别适合处理信息在多个线程间安全交换的多线程程序中。下面的demo展示了如何通过queue来限制线程的并发个数
importthreading importqueue importtime importrandom importos maxThreads=4 classStore(threading.Thread): def__init__(self,q): threading.Thread.__init__(self) self.queue=q #self.store=store defrun(self): try: print('Runtask(%s)...'%(os.getpid())) start=time.time() time.sleep(random.random()*3) end=time.time() t=threading.currentThread() #线程ID print('Threadid:%d'%t.ident) print('Threadname:%s'%t.getName()) print('Taskruns%0.2fseconds.'%(end-start)) exceptExceptionase: print(e) finally: self.queue.get() self.queue.task_done() defmain(): q=queue.Queue(maxThreads) forsinrange(6): q.put(s) t=Store(q) t.start() q.join() print('over') if__name__=='__main__': main()
运行结果如下:
Runtask(33259)... Runtask(33259)... Runtask(33259)... Runtask(33259)... Threadid:123145444999168 Threadname:Thread-13 Taskruns0.04seconds. Runtask(33259)... Threadid:123145394630656 Threadname:Thread-10 Taskruns1.02seconds. Runtask(33259)... Threadid:123145428209664 Threadname:Thread-12 Taskruns1.20seconds. Threadid:123145394630656 Threadname:Thread-17 Taskruns0.68seconds. Threadid:123145444999168 Threadname:Thread-14 Taskruns1.79seconds. Threadid:123145411420160 Threadname:Thread-11 Taskruns2.96seconds. over
以上就是python如何控制进程或者线程的个数的详细内容,更多关于python控制进程或线程的资料请关注毛票票其它相关文章!