這是一個(gè)現(xiàn)網(wǎng)問題
問題現(xiàn)象
客戶頁面記錄展示不全，只展示了1000條，最新的不展示。
項(xiàng)目組一兄弟定位為，數(shù)據(jù)庫(kù)查詢到的記錄被get請(qǐng)求返回時(shí)給截掉了（由于最大size限制）
走給我審核時(shí)，第一直覺告訴我定位的原因肯定不對(duì)，因?yàn)槿绻怯捎趃et請(qǐng)求大小限制，不可能剛好截?cái)啾Ａ粽?000條記錄，而且如果是按字節(jié)大小截?cái)啵瑀esponse應(yīng)該會(huì)報(bào)錯(cuò)吧？
驗(yàn)證猜想
抱著懷疑的態(tài)度，在網(wǎng)上搜了一下GET返回時(shí)有什么限制，見如下鏈接
Limit on the length of the data that a webserver can return in response to a GET request
There are no limits on the amount of data returned on a HTTP response from Jetty.
You could stream data back to the client until shortly before the heat death of the universe.
Technically speaking, you can have a HTTP Response with no Content-Length
specified, which can be returned using either the Chunked Transfer-Encoding, or just a raw stream of bytes with a Connection: close
indicating when the data is complete (done being sent) by a close of the underlying connection. Both of which are essentially limit-less.
If you use a HTTP Response with Content-Length, be aware that Content-Length is, in practice, a 32-bit number, but more modern browsers support the 64-bit versions.
大體意思就是GET在response返回時(shí)沒有大小限制，說明定位原因不對(duì)。
根因推導(dǎo)
- 不是GET返回截?cái)嗟膯栴}，且返回不全，肯定是后臺(tái)截?cái)嗔耍笈_(tái)能從哪里截?cái)?？初步判斷為?shù)據(jù)庫(kù)查詢只返回了1000條記錄
- 進(jìn)一步搜索日志，發(fā)現(xiàn)果然是數(shù)據(jù)庫(kù)返回了1000條記錄
- 接著查看代碼，發(fā)現(xiàn)代碼在查數(shù)據(jù)庫(kù)時(shí)并未使用limit限制，進(jìn)一步猜測(cè)調(diào)用的beego orm的
All(container interface{}, cols ...string) (int64, error)有默認(rèn)過濾規(guī)則 - 網(wǎng)上搜索關(guān)于beego orm All方法的資料并查看源碼，發(fā)現(xiàn)All方法果然有默認(rèn)1000的限制返回?cái)?shù)
All / Values / ValuesList / ValuesFlat 受到 Limit 的限制，默認(rèn)最大行數(shù)為 1000
源碼：
func (o *querySet) All(container interface{}, cols ...string) (int64, error) {
return o.orm.alias.DbBaser.ReadBatch(o.orm.db, o, o.mi, o.cond, container, o.orm.alias.TZ, cols)
}
func (d *dbBase) ReadBatch(q dbQuerier, qs *querySet, mi *modelInfo, cond *Condition, container interface{}, tz *time.Location, cols []string) (int64, error) {
...
limit := tables.getLimitSQL(mi, offset, rlimit)
...
query := fmt.Sprintf("%s %s FROM %s%s%s T0 %s%s%s%s%s", sqlSelect, sels, Q, mi.table, Q, join, where, groupBy, orderBy, limit)
...
}
func (t *dbTables) getLimitSQL(mi *modelInfo, offset int64, limit int64) (limits string) {
if limit == 0 {
limit = int64(DefaultRowsLimit)
}
}
DefaultRowsLimit = 1000
Reference：
https://stackoverflow.com/questions/16172524/limit-on-the-length-of-the-data-that-a-webserver-can-return-in-response-to-a-get
http://zoomq.qiniudn.com/ZQScrapBook/ZqFLOSS/data/20131230002213/index.html
