進(jìn)程和線程

什么是進(jìn)程(process)？

An executing instance of a program is called a process. （進(jìn)程就是資源的集合）
程序并不能單獨(dú)運(yùn)行，只有將程序裝載到內(nèi)存中，系統(tǒng)為它分配資源才能運(yùn)行，而這種執(zhí)行的程序就稱之為進(jìn)程。程序和進(jìn)程的區(qū)別就在于：程序是指令的集合，它是進(jìn)程運(yùn)行的靜態(tài)描述文本；進(jìn)程是程序的一次執(zhí)行活動(dòng)，屬于動(dòng)態(tài)概念。

在多道編程中，我們?cè)试S多個(gè)程序同時(shí)加載到內(nèi)存中，在操作系統(tǒng)的調(diào)度下，可以實(shí)現(xiàn)并發(fā)地執(zhí)行。這是這樣的設(shè)計(jì)，大大提高了CPU的利用率。進(jìn)程的出現(xiàn)讓每個(gè)用戶感覺到自己獨(dú)享CPU，因此，進(jìn)程就是為了在CPU上實(shí)現(xiàn)多道編程而提出的。

什么是線程(thread)？

線程是操作系統(tǒng)能夠進(jìn)行運(yùn)算調(diào)度的最小單位。它被包含在進(jìn)程之中，是進(jìn)程中的實(shí)際運(yùn)作單位。一條線程指的是進(jìn)程中一個(gè)單一順序的控制流，一個(gè)進(jìn)程中可以并發(fā)多個(gè)線程，每條線程并行執(zhí)行不同的任務(wù)

A thread is an execution context, which is all the information a CPU needs to execute a stream of instructions.

Suppose you're reading a book, and you want to take a break right now, but you want to be able to come back and resume reading from the exact point where you stopped. One way to achieve that is by jotting down the page number, line number, and word number. So your execution context for reading a book is these 3 numbers.

If you have a roommate, and she's using the same technique, she can take the book while you're not using it, and resume reading from where she stopped. Then you can take it back, and resume it from where you were.

Threads work in the same way. A CPU is giving you the illusion that it's doing multiple computations at the same time. It does that by spending a bit of time on each computation. It can do that because it has an execution context for each computation. Just like you can share a book with your friend, many tasks can share a CPU.

On a more technical level, an execution context (therefore a thread) consists of the values of the CPU's registers.

Last: threads are different from processes. A thread is a context of execution, while a process is a bunch of resources associated with a computation. A process can have one or many threads.

Clarification: the resources associated with a process include memory pages (all the threads in a process have the same view of the memory), file descriptors (e.g., open sockets), and security credentials (e.g., the ID of the user who started the process).

進(jìn)程與線程的區(qū)別？

1. Threads share the address space of the process that created it; processes have their own address space.
   線程共享內(nèi)存空間，進(jìn)程的內(nèi)存是獨(dú)立的
2. Threads have direct access to the data segment of its process; processes have their own copy of the data segment of the parent process.
3. Threads can directly communicate with other threads of its process; processes must use interprocess communication to communicate with sibling processes.
   同一進(jìn)程中的線程之間可以直接通信，進(jìn)程之間通信必須使用中間代理
4. New threads are easily created; new processes require duplication of the parent process.
   新線程易于創(chuàng)建，創(chuàng)建新進(jìn)程需要克隆其父進(jìn)程
5. Threads can exercise considerable control over threads of the same process; processes can only exercise control over child processes.
   線程可以控制與操作同一進(jìn)程中的其他線程，進(jìn)程只能操作其子進(jìn)程
6. Changes to the main thread (cancellation, priority change, etc.) may affect the behavior of the other threads of the process; changes to the parent process does not affect child processes.
   對(duì)主線程的修改可能會(huì)影響到同一進(jìn)程中的其他線程，對(duì)于父進(jìn)程的修改不會(huì)影響其子進(jìn)程

import threading
import time

def show(arg):
    time.sleep(2)
    print('thread+str(%s)' %arg)

for i in range(10):
    t = threading.Thread(target=show, args= (i,))
    t.start()

print('main thread stop')

#繼承式調(diào)用
import threading
import time

class MyThread(threading.Thread):
    def __init__(self, n):
        super(MyThread, self).__init__()
        self.n = n

    def run(self):
        time.sleep(2)
        print('thread+str(%s)' %self.n)

if __name__ == '__main__':
    for i in range(10):
        t = MyThread(i)
        t.start()
    print('main thread stop',threading.current_thread(), threading.active_count())

Join & Daemon

import threading
import time

def show(arg):
    time.sleep(2)
    print('thread+str(%s)' %arg)

start_time = time.time()
t_objs = []
for i in range(10):
    t = threading.Thread(target=show, args= (i,))
    t.start()
    t_objs.append(t)  #將實(shí)例加進(jìn)來

for t in t_objs:    #循環(huán)線程實(shí)例列表，等待所以線程執(zhí)行完畢
    t.join()

print('main thread stop')
print("cost:", time.time() - start_time)

守護(hù)線程：主線程結(jié)束時(shí)，不等待守護(hù)線程結(jié)束，直接結(jié)束

def show(arg):
    print(arg)
    time.sleep(2)
    print('thread+str(%s)' %arg)

start_time = time.time()

for i in range(10):
    t = threading.Thread(target=show, args= (i,))
    t.setDaemon(True)     #把當(dāng)前線程設(shè)置為守護(hù)線程
    t.start()
print('main thread stop')

Python GIL(Global Interpreter Lock)

In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly because CPython’s memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.)
上面的核心意思就是，無論啟多少個(gè)線程, Python在執(zhí)行的時(shí)候在同一時(shí)刻只允許一個(gè)線程運(yùn)行。當(dāng)然，不同的線程可能是在不同的CPU上運(yùn)行的。

線程鎖（互斥鎖Mutex）

由于線程之間是進(jìn)行隨機(jī)調(diào)度，并且每個(gè)線程可能只執(zhí)行n條執(zhí)行之后，當(dāng)多個(gè)線程同時(shí)修改同一條數(shù)據(jù)時(shí)可能會(huì)出現(xiàn)臟數(shù)據(jù)，所以，出現(xiàn)了線程鎖--保證在這個(gè)線程執(zhí)行完后別的線程才能調(diào)度（串行執(zhí)行）

import time
import threading


def addNum():
    global num  # 在每個(gè)線程中都獲取這個(gè)全局變量
    print('--get num:', num)
    # lock.acquire()  #加鎖后，程序就變串行了。所以，鎖的范圍盡量小
    time.sleep(3)
    lock.acquire()
    num -= 1  #(加鎖保證數(shù)據(jù)修改的一致型，防止此時(shí)對(duì)數(shù)據(jù)的修改還沒完成，而原數(shù)據(jù)又被其他線程取走了)
    lock.release()

num = 100  # 設(shè)定一個(gè)共享變量
thread_list = []
lock = threading.Lock()
for i in range(100):
    t = threading.Thread(target=addNum)
    t.start()
    thread_list.append(t)

for t in thread_list:  # 等待所有線程執(zhí)行完畢
    t.join()

print('final num:', num)

遞歸鎖（Rlock）

說白了就是在一個(gè)大鎖中還再包含子鎖

import threading, time

def run1():
    print("grab the first part data")
    lock.acquire()
    global num
    num += 1
    lock.release()
    return num

def run2():
    print("grab the second part data")
    lock.acquire()
    global num2
    num2 += 1
    lock.release()
    return num2

def run3():
    lock.acquire()
    res = run1()
    print('--------between run1 and run2-----')
    res2 = run2()
    lock.release()
    print(res, res2)

if __name__ == '__main__':
    num, num2 = 0, 0
    lock = threading.RLock()
    for i in range(10):
        t = threading.Thread(target=run3)
        t.start()

while threading.active_count() != 1:
    print(threading.active_count())
else:
    print('----all threads done---')
    print(num, num2)

信號(hào)量

信號(hào)量：允許同一時(shí)間幾個(gè)線程訪問公共數(shù)據(jù)

import threading, time

def run(n):
    semaphore.acquire()
    time.sleep(1)
    print("run the thread: %s\n" %n)
    semaphore.release()

if __name__ == '__main__':
     semaphore = threading.BoundedSemaphore(3)   #最多允許5個(gè)線程同時(shí)執(zhí)行
     for i in range(12):
         t = threading.Thread(target=run, args=(i,))
         t.start()

while threading.active_count() != 1:
     pass  #print(threading.active_count())
else:
    print('----all threads done----')

事件(event):通過Event來實(shí)現(xiàn)兩個(gè)或多個(gè)線程間的交互。

python線程的事件用于主線程控制其他線程的執(zhí)行，事件主要提供了三個(gè)方法 set、wait、clear。

事件處理的機(jī)制：全局定義了一個(gè)“Flag”，如果“Flag”值為 False，那么當(dāng)程序執(zhí)行 event.wait 方法時(shí)就會(huì)阻塞，如果“Flag”值為True，那么event.wait 方法時(shí)便不再阻塞。
? clear：將“Flag”設(shè)置為False
? set：將“Flag”設(shè)置為True

import threading, time

event = threading.Event()

def light():
    count = 0
    event.set()
    while True:
        if count < 5:
            print("\033[1;42mGreen light is on..\033[0m")
        elif count >= 5 and count < 10:
            event.clear()
            print("\033[1;41mRed light is on.\033[0m")
        else:
            count = 0
            event.set()
            print("\033[1;42mGreen light is on..\033[0m")
        time.sleep(1)
        count += 1

def car(name):
    while True:
        if event.isSet():
            print("[%s] is running" %name)
            time.sleep(1)
        else:
            print("[%s] sees red light, it's waiting" % name)
            event.wait()
            # print("\033[1;34m[%s] sees green light is on, it's keep going." % name)

lights = threading.Thread(target=light)
lights.start()

car1 = threading.Thread(target=car, args=("Jeep",))
car1.start()

隊(duì)列queue和生產(chǎn)者消費(fèi)者模型

隊(duì)列

作用：解耦，提高效率

queue 實(shí)例化方法

class queue.Queue(maxsize=0) # 先入先出
class queue.LifoQueue(maxsize=0) # last in first out
class queue.PriorityQueue(maxsize=0) # 存儲(chǔ)數(shù)據(jù)時(shí)可設(shè)置優(yōu)先級(jí)的隊(duì)列

常用方法(q =queue.queue()):
q.qsize() 返回隊(duì)列的大小
q.empty() 如果隊(duì)列為空，返回True,反之False
q.full() 如果隊(duì)列滿了，返回True,反之False
q.full 與 maxsize 大小對(duì)應(yīng)
q.get([block[, timeout]]) 獲取隊(duì)列，timeout等待時(shí)間
q.get_nowait() 相當(dāng)q.get(False)
非阻塞 q.put(item) 寫入隊(duì)列，timeout等待時(shí)間
q.put_nowait(item) 相當(dāng)q.put(item, False)
q.task_done() 在完成一項(xiàng)工作之后，q.task_done() 函數(shù)向任務(wù)已經(jīng)完成的隊(duì)列發(fā)送一個(gè)信號(hào)
q.join() 實(shí)際上意味著等到隊(duì)列為空，再執(zhí)行別的操作

生產(chǎn)者消費(fèi)者模型

什么是生產(chǎn)者消費(fèi)者模式

生產(chǎn)者消費(fèi)者模式是通過一個(gè)容器來解決生產(chǎn)者和消費(fèi)者的強(qiáng)耦合問題。生產(chǎn)者和消費(fèi)者彼此之間不直接通訊，而通過阻塞隊(duì)列來進(jìn)行通訊，所以生產(chǎn)者生產(chǎn)完數(shù)據(jù)之后不用等待消費(fèi)者處理，直接扔給阻塞隊(duì)列，消費(fèi)者不找生產(chǎn)者要數(shù)據(jù)，而是直接從阻塞隊(duì)列里取，阻塞隊(duì)列就相當(dāng)于一個(gè)緩沖區(qū)，平衡了生產(chǎn)者和消費(fèi)者的處理能力。
理解生產(chǎn)者消費(fèi)者模型及在Python編程中的運(yùn)用實(shí)例

import threading, queue
import time

q = queue.Queue()

def producer(name):
    count = 1
    while True:
        print('[%s]生產(chǎn)了骨頭%s' % (name, count))
        q.put('骨頭%s' %count)
        # print('[%s]生產(chǎn)了骨頭%s' %(name, count))
        count += 1
        time.sleep(0.5)

def cusumer(name):
    while True:
        print('[%s] 取到[%s] 并且吃了它...' %(name, q.get()))
        time.sleep(3)

p = threading.Thread(target=producer, args=('Alex',))
c1 = threading.Thread(target=cusumer, args=('dog1',))
c2 = threading.Thread(target=cusumer, args=('dog2',))
p.start()
c1.start()
c2.start()