python多線程示例- 給多任務(wù)處理增加監(jiān)視保護(hù)線程

繼上篇文章python多線程示例 -- 有限調(diào)度器處理隊(duì)列數(shù)據(jù)做一個(gè)拓展功能，給多任務(wù)處理增加一個(gè)時(shí)間限制保護(hù)

  • 新增一個(gè)計(jì)時(shí)線程，這里示例為計(jì)時(shí)12s，增加了一個(gè)信號(hào)量timeoutSignal
  • 當(dāng)計(jì)時(shí)線程休眠18s后，釋放timeoutSignal，主程序處理中獲取到時(shí)，會(huì)退出while循環(huán)，利用線程join函數(shù)來阻塞主線程，等到所有線程結(jié)束后，主進(jìn)程會(huì)結(jié)束
from queue import Queue, Empty
from threading import Lock, Thread, BoundedSemaphore, current_thread, activeCount
import logging
import time


class AppLogger:
    def __init__(self, moduleName, logfile):
        self._logger = logging.getLogger(moduleName)
        handler = logging.FileHandler(logfile)
        fmt = "%(asctime)-15s %(levelname)s %(filename)s %(lineno)d %(message)s"
        formatter = logging.Formatter(fmt)
        handler.setFormatter(formatter)
        self._logger.addHandler(handler)
        self._logger.setLevel(logging.INFO)

        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        self._logger.addHandler(console)

        self.warnning = self._logger.warning
        self.error = self._logger.error
        self.info = self._logger.info
        self.debug = self._logger.debug


dataQueueMaxLen = 100
schedulerMaxCount = 4


lock = Lock()

scheduleSources = BoundedSemaphore(schedulerMaxCount)

dataQueue = Queue(dataQueueMaxLen)

myLogger = AppLogger("myapp","test.log")

class Scheduler:
    def __init__(self, func, data):
        self._thread = Thread(target=func, args=data)
        self.record = "Scheduler(%s, %s)" % (func.__name__, *data)

    def __str__(self):
        return self.record

    def start(self):
        # myLogger.info(("START ", self.record))
        self._thread.start()

    def join(self):
        self._thread.join()

dataList = range(dataQueueMaxLen)

for data in dataList:
    dataQueue.put(data)

def testFunc(data):
    thd = current_thread()
    myLogger.info("Thread %s start testFunc %s sleep 5 seconds" % (thd , data))

    time.sleep(5)
    scheduleSources.release()
    myLogger.info("Thread %s exit testFunc %s" % (thd, data))

runningProcList = []

def startScheduler(func, data):
    if scheduleSources.acquire():
        proc = Scheduler(func, [data])
        proc.start()
        runningProcList.append(proc)


timeoutSignal = BoundedSemaphore(1)

def monitor():
    timeoutSignal.acquire()
    time.sleep(12)
    timeoutSignal.release()

if __name__ == "__main__":
    monitorThd = Thread(target=monitor)
    monitorThd.start()

    while not dataQueue.empty():
        if timeoutSignal.acquire(timeout=0):
            myLogger.warnning("timeout wait other thread end and Exit main process")
            break
        data = dataQueue.get()
        startScheduler(testFunc, data)
        myLogger.info("current running thread count %d " % activeCount())

    for proc in runningProcList:
        proc.join()

可以從最終日志輸出結(jié)果中可以看到 程序啟動(dòng)12后，主線程收到了超時(shí)信號(hào)2020-04-06 09:22:05,005 WARNING multiProcTest.py 95 timeout wait other thread end and Exit main process，退出while循環(huán)，主線程等待還未結(jié)束的線程退出后再結(jié)束

C:\Python36\python.exe D:/MyWorkSpace/CCMtask/multiProcTest.py
2020-04-06 09:21:48,721 INFO multiProcTest.py 67 Thread <Thread(Thread-2, started 76540)> start testFunc 0 sleep 5 seconds
2020-04-06 09:21:48,721 INFO multiProcTest.py 99 current running thread count 3 
2020-04-06 09:21:49,955 INFO multiProcTest.py 67 Thread <Thread(Thread-3, started 87780)> start testFunc 1 sleep 5 seconds
2020-04-06 09:21:49,955 INFO multiProcTest.py 99 current running thread count 4 
2020-04-06 09:21:49,971 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:49,955 INFO multiProcTest.py 67 Thread <Thread(Thread-4, started 88128)> start testFunc 2 sleep 5 seconds
2020-04-06 09:21:49,971 INFO multiProcTest.py 67 Thread <Thread(Thread-5, started 2312)> start testFunc 3 sleep 5 seconds
2020-04-06 09:21:49,971 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-2, started 76540)> exit testFunc 0
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-3, started 87780)> exit testFunc 1
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-4, started 88128)> exit testFunc 2
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-6, started 82564)> start testFunc 4 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-7, started 86944)> start testFunc 5 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-8, started 87232)> start testFunc 6 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,991 INFO multiProcTest.py 71 Thread <Thread(Thread-5, started 2312)> exit testFunc 3
2020-04-06 09:21:54,991 INFO multiProcTest.py 67 Thread <Thread(Thread-9, started 72020)> start testFunc 7 sleep 5 seconds
2020-04-06 09:21:54,991 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-6, started 82564)> exit testFunc 4
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-7, started 86944)> exit testFunc 5
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-8, started 87232)> exit testFunc 6
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-10, started 86476)> start testFunc 8 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-11, started 86644)> start testFunc 9 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-12, started 86228)> start testFunc 10 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:22:00,006 INFO multiProcTest.py 71 Thread <Thread(Thread-9, started 72020)> exit testFunc 7
2020-04-06 09:22:00,006 INFO multiProcTest.py 67 Thread <Thread(Thread-13, started 89180)> start testFunc 11 sleep 5 seconds
2020-04-06 09:22:00,006 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-10, started 86476)> exit testFunc 8
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-12, started 86228)> exit testFunc 10
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-11, started 86644)> exit testFunc 9
2020-04-06 09:22:05,005 INFO multiProcTest.py 67 Thread <Thread(Thread-14, started 86376)> start testFunc 12 sleep 5 seconds
2020-04-06 09:22:05,005 INFO multiProcTest.py 99 current running thread count 3 
2020-04-06 09:22:05,005 WARNING multiProcTest.py 95 timeout wait other thread end and Exit main process
2020-04-06 09:22:05,020 INFO multiProcTest.py 71 Thread <Thread(Thread-13, started 89180)> exit testFunc 11
2020-04-06 09:22:10,020 INFO multiProcTest.py 71 Thread <Thread(Thread-14, started 86376)> exit testFunc 12
2020-04-06 09:22:10,020 INFO multiProcTest.py 104 main proc exit

Process finished with exit code 0
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容