buf_flush_page_cleaner_coordinator協(xié)調(diào)線程的主循環(huán)主線程以最多1s的間隔或者收到buf_flush_event事件就會觸發(fā)進(jìn)行一輪的刷臟。
批量刷臟主要有3個場景。
- 同步刷臟
如果 buf_flush_sync_lsn > 0, 則因為redo log free space 不夠了, 那么我們需要進(jìn)入同步刷臟階段了。同步刷臟場景下，所有需要寫臟數(shù)據(jù)庫的用戶線程都會堵塞，這是很嚴(yán)重的情況。 - 正常刷臟
最常見邏輯 srv_check_activity(last_activity), 也就是系統(tǒng)有正?；顒樱蠨ML/DDL, 這個時候會通過 page_cleaner_flush_pages_recommendation() 函數(shù)去合理的判斷應(yīng)該刷多少個page, 既不抖動, 也能夠滿足刷臟需求 - 空閑刷臟
如果系統(tǒng)沒有DML\DDL活動，且ret_sleep == OS_SYNC_TIME_EXCEEDED，說明比較空閑?？臻e的情況下因為服務(wù)器IO比較空閑，所以Innodb使用buf_flush_page_cleaner_coordinator線程本身進(jìn)行刷新，刷新的塊數(shù)計算比較簡單就是innodb_io_capacity設(shè)置的值。
/******************************************************************//**
page_cleaner thread tasked with flushing dirty pages from the buffer
pools. As of now we'll have only one coordinator.
@return a dummy parameter */
extern "C"
os_thread_ret_t
DECLARE_THREAD(buf_flush_page_cleaner_coordinator)(
/*===============================================*/
void* arg MY_ATTRIBUTE((unused)))
/*!< in: a dummy parameter required by
os_thread_create */
{
/* 忽略一些邏輯 */
while (srv_shutdown_state == SRV_SHUTDOWN_NONE) {
if (ret_sleep != OS_SYNC_TIME_EXCEEDED
&& srv_flush_sync
&& buf_flush_sync_lsn > 0) {
/* 場景1 */
} else if (srv_check_activity(last_activity)) {
ulint n_to_flush;
lsn_t lsn_limit = 0;
/* Estimate pages from flush_list to be flushed */
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_activity = srv_get_activity_count();
n_to_flush =
page_cleaner_flush_pages_recommendation(
&lsn_limit, last_pages);
} else {
n_to_flush = 0;
}
/* 場景2 */
} else if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
/* 場景3 */
/* no activity, slept enough */
}
} else {
/* no activity, but woken up by event */
n_flushed = 0;
}
}
」
三種場景下的具體工作
同步刷臟
pc_request(ULINT_MAX, lsn_limit),會把lsn小于lsn_limit的都flush到硬盤，同時coordinator線程本身也會參與刷臟。
/* woke up for flush_sync */
mutex_enter(&page_cleaner->mutex);
lsn_t lsn_limit = buf_flush_sync_lsn;
buf_flush_sync_lsn = 0;
mutex_exit(&page_cleaner->mutex);
/* Request flushing for threads */
pc_request(ULINT_MAX, lsn_limit);
ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();
/* Coordinator also treats requests */
while (pc_flush_slot() > 0) {}
/* only coordinator is using these counters,
so no need to protect by lock. */
page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
page_cleaner->flush_pass++;
/* Wait for all slots to be finished */
ulint n_flushed_lru = 0;
ulint n_flushed_list = 0;
pc_wait_finished(&n_flushed_lru, &n_flushed_list);
if (n_flushed_list > 0 || n_flushed_lru > 0) {
buf_flush_stats(n_flushed_list, n_flushed_lru);
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_SYNC_TOTAL_PAGE,
MONITOR_FLUSH_SYNC_COUNT,
MONITOR_FLUSH_SYNC_PAGES,
n_flushed_lru + n_flushed_list);
}
n_flushed = n_flushed_lru + n_flushed_list;
正常刷臟
通過page_cleaner_flush_pages_recommendation計算需要刷新的頁。
ulint n_to_flush;
lsn_t lsn_limit = 0;
/* Estimate pages from flush_list to be flushed */
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_activity = srv_get_activity_count();
n_to_flush =
page_cleaner_flush_pages_recommendation(
&lsn_limit, last_pages);
} else {
n_to_flush = 0;
}
/* Request flushing for threads */
pc_request(n_to_flush, lsn_limit);
ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();
/* Coordinator also treats requests */
while (pc_flush_slot() > 0) {
/* No op */
}
/* only coordinator is using these counters,
so no need to protect by lock. */
page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
page_cleaner->flush_pass++ ;
/* Wait for all slots to be finished */
ulint n_flushed_lru = 0;
ulint n_flushed_list = 0;
pc_wait_finished(&n_flushed_lru, &n_flushed_list);
if (n_flushed_list > 0 || n_flushed_lru > 0) {
buf_flush_stats(n_flushed_list, n_flushed_lru);
}
if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
last_pages = n_flushed_list;
}
n_evicted += n_flushed_lru;
n_flushed_last += n_flushed_list;
n_flushed = n_flushed_lru + n_flushed_list;
if (n_flushed_lru) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_LRU_BATCH_FLUSH_TOTAL_PAGE,
MONITOR_LRU_BATCH_FLUSH_COUNT,
MONITOR_LRU_BATCH_FLUSH_PAGES,
n_flushed_lru);
}
if (n_flushed_list) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_ADAPTIVE_TOTAL_PAGE,
MONITOR_FLUSH_ADAPTIVE_COUNT,
MONITOR_FLUSH_ADAPTIVE_PAGES,
n_flushed_list);
}
空閑刷臟
空閑刷臟是coordinator自己進(jìn)行，直接按照PCT_IO(100)來生成刷新數(shù)量。
#define PCT_IO(p) ((ulong) (srv_io_capacity * ((double) (p) / 100.0)))
buf_flush_lists(PCT_IO(100), LSN_MAX, &n_flushed);
n_flushed_last += n_flushed;
if (n_flushed) {
MONITOR_INC_VALUE_CUMULATIVE(
MONITOR_FLUSH_BACKGROUND_TOTAL_PAGE,
MONITOR_FLUSH_BACKGROUND_COUNT,
MONITOR_FLUSH_BACKGROUND_PAGES,
n_flushed);
}
http://mysql.taobao.org/monthly/2018/09/02/
http://www.itdecent.cn/p/6991304a8e26
https://mp.weixin.qq.com/s/o2OlvRiybIsqi7WU_Kvhiw
