關(guān)于golang 的readall清空reader

問(wèn)題

   body, err := ioutil.ReadAll(c.Request().Body) 
   代碼執(zhí)行完成后,c.Request().Body 內(nèi)容變成了空,這是為什么?

代碼分析

step1 調(diào)用內(nèi)部函數(shù),每次至少讀512字節(jié)(怎么定的呢?)

可以注意到底層調(diào)用了buffer的ReadFrom函數(shù)

// readAll reads from r until an error or EOF and returns the data it read
// from the internal buffer allocated with a specified capacity.
func readAll(r io.Reader, capacity int64) (b []byte, err error) {
    buf := bytes.NewBuffer(make([]byte, 0, capacity))
    // If the buffer overflows, we will get bytes.ErrTooLarge.
    // Return that as an error. Any other panic remains.
    defer func() {
        e := recover()
        if e == nil {
            return
        }
        if panicErr, ok := e.(error); ok && panicErr == bytes.ErrTooLarge {
            err = panicErr
        } else {
            panic(e)
        }
    }()
    _, err = buf.ReadFrom(r)
    return buf.Bytes(), err
}

// ReadAll reads from r until an error or EOF and returns the data it read.
// A successful call returns err == nil, not err == EOF. Because ReadAll is
// defined to read from src until EOF, it does not treat an EOF from Read
// as an error to be reported.
func ReadAll(r io.Reader) ([]byte, error) {
    return readAll(r, bytes.MinRead)
}

step2 buffer readFrom實(shí)現(xiàn)分析

Buffer定義,buf:存放的數(shù)據(jù),buf中off到len(buf)之間的數(shù)據(jù)為有效數(shù)據(jù),其余數(shù)據(jù)為0,讀的時(shí)候從buf[off]開(kāi)始,寫是在buf[len(buf)]。

// A Buffer is a variable-sized buffer of bytes with Read and Write methods.
// The zero value for Buffer is an empty buffer ready to use.
type Buffer struct {
    buf       []byte   // contents are the bytes buf[off : len(buf)]
    off       int      // read at &buf[off], write at &buf[len(buf)]
    bootstrap [64]byte // memory to hold first slice; helps small buffers avoid allocation.
    lastRead  readOp   // last read operation, so that Unread* can work correctly.
}
// ReadFrom reads data from r until EOF and appends it to the buffer, growing
// the buffer as needed. The return value n is the number of bytes read. Any
// error except io.EOF encountered during the read is also returned. If the
// buffer becomes too large, ReadFrom will panic with ErrTooLarge.
func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
    b.lastRead = opInvalid
    // If buffer is empty, reset to recover space.
    if b.off >= len(b.buf) {
        b.Truncate(0)
    }
    for {
        //buff 末尾使用的空間不足時(shí),擴(kuò)容
        if free := cap(b.buf) - len(b.buf); free < MinRead {
            // not enough space at end,buffer正常情況下是直接尾部寫入數(shù)據(jù),當(dāng)發(fā)現(xiàn)尾部空間不足時(shí),就需要調(diào)整buffer存儲(chǔ)形式,將數(shù)據(jù)區(qū)buf[off]到buf[len(buf)]對(duì)齊到頭部,攢出尾部空間來(lái)繼續(xù)存儲(chǔ)。

            newBuf := b.buf 
            if b.off+free < MinRead {  //發(fā)現(xiàn)可用的空間不夠用時(shí),就需要擴(kuò)容,擴(kuò)容辦法是原buf cap的兩倍+最小讀取數(shù)據(jù)大小
                // not enough space using beginning of buffer;
                // double buffer capacity
                newBuf = makeSlice(2*cap(b.buf) + MinRead)
            }
            copy(newBuf, b.buf[b.off:])//將buf數(shù)據(jù)區(qū)copy到臨時(shí)buf
            b.buf = newBuf[:len(b.buf)-b.off]//再?gòu)呐R時(shí)buf把數(shù)據(jù)copy回去,完成對(duì)齊頭部
            b.off = 0
        }
        m, e := r.Read(b.buf[len(b.buf):cap(b.buf)])//傳入buf 的長(zhǎng)度與cap之間的空間給reader
        b.buf = b.buf[0 : len(b.buf)+m]//buf取到真實(shí)讀取的字節(jié)位置
        n += int64(m)//讀取的總字節(jié)數(shù)增加
        if e == io.EOF {//讀到eof標(biāo)記時(shí)停止
            break
        }
        if e != nil {
            return n, e
        }
    }
    return n, nil // err is EOF, so return nil explicitly
}

step3 io.Reader 的Read實(shí)現(xiàn);以c.Request().Body為例

其中body是這么定義的,是封裝了Reader和Closer的interface

Body io.ReadCloser

那么body的reader到底是什么呢?body是在Request結(jié)構(gòu)體中定義的,內(nèi)容是請(qǐng)求的body體,因此連接建立起來(lái)后,body會(huì)被賦值,沿著這個(gè)思路,找到準(zhǔn)備body的reader代碼的地方,詳細(xì)查找路基如下:
【http啟動(dòng)服務(wù)】func (srv *Server) Serve(l net.Listener) =>【建立連接】go c.serve(ctx)=>【讀取request】c.readRequest(ctx)=>【讀取request詳細(xì)信息】readRequest(b *bufio.Reader, deleteHostHeader bool)=>【解析http協(xié)議,讀取信息】readTransfer(msg interface{}, r *bufio.Reader)

// Prepare body reader. ContentLength < 0 means chunked encoding
    // or close connection when finished, since multipart is not supported yet
    switch {
    case chunked(t.TransferEncoding):
        if noResponseBodyExpected(t.RequestMethod) {
            t.Body = NoBody
        } else {
            t.Body = &body{src: internal.NewChunkedReader(r), hdr: msg, r: r, closing: t.Close}
        }
    case realLength == 0:
        t.Body = NoBody
    case realLength > 0:
        t.Body = &body{src: io.LimitReader(r, realLength), closing: t.Close}
    default:
        // realLength < 0, i.e. "Content-Length" not mentioned in header
        if t.Close {
            // Close semantics (i.e. HTTP/1.0)
            t.Body = &body{src: r, closing: t.Close}
        } else {
            // Persistent connection (i.e. HTTP/1.1)
            t.Body = NoBody
        }
    }

至此,我們發(fā)現(xiàn)body是一個(gè)bufio.Reader,其定義如下

type Reader struct {
    buf          []byte //緩沖區(qū)
    rd           io.Reader // reader provided by the client
    r, w         int       // buf read and write positions
    err          error
    lastByte     int
    lastRuneSize int
}

在server.go中,我們注意到如下初始化,也就是說(shuō)body是一個(gè)帶緩沖區(qū)的io reader,并且connReader作為數(shù)據(jù)源的reader

c.r = &connReader{conn: c}
c.bufr = newBufioReader(c.r)

關(guān)鍵!從緩沖區(qū)讀取數(shù)據(jù)區(qū),并將buf的讀off加n,也就是將buf中已讀的數(shù)據(jù)清空

// Read reads data into p.
// It returns the number of bytes read into p.
// The bytes are taken from at most one Read on the underlying Reader,
// hence n may be less than len(p).
// At EOF, the count will be zero and err will be io.EOF.
func (b *Reader) Read(p []byte) (n int, err error) {
    n = len(p)
    if n == 0 {
        return 0, b.readErr()
    }
    if b.r == b.w {
        if b.err != nil {
            return 0, b.readErr()
        }
        if len(p) >= len(b.buf) {
            // Large read, empty buffer.
            // Read directly into p to avoid copy.
            n, b.err = b.rd.Read(p)
            if n < 0 {
                panic(errNegativeRead)
            }
            if n > 0 {
                b.lastByte = int(p[n-1])
                b.lastRuneSize = -1
            }
            return n, b.readErr()
        }
        // One read.
        // Do not use b.fill, which will loop.
        b.r = 0
        b.w = 0
        n, b.err = b.rd.Read(b.buf)////buf為空時(shí),直接從conn的reader中讀取,不涉及r、w的變化,buf還是空
        if n < 0 {
            panic(errNegativeRead)
        }
        if n == 0 {
            return 0, b.readErr()
        }
        b.w += n
    }

    // copy as much as we can
    n = copy(p, b.buf[b.r:b.w])//從緩沖區(qū)讀取數(shù)據(jù)區(qū)
    b.r += n//將buf的讀off加n,也就是將buf中已讀的數(shù)據(jù)清空
    b.lastByte = int(b.buf[b.r-1])
    b.lastRuneSize = -1
    return n, nil
}

結(jié)論

ioutil.ReadAll(c.Request().Body) 最終會(huì)調(diào)用bufio.Reader 的read方法來(lái)讀取數(shù)據(jù),buf被讀取后,數(shù)據(jù)會(huì)被清空,所以 ioutil.ReadAll(c.Request().Body) 再次調(diào)用時(shí),也讀不到數(shù)據(jù)啦~

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容