1 Motion Estimation

圖片.png

monocular vision 單眼視覺(jué)

圖片.png

2 cases:

• optical flow is zero although there is motion in the scene
• optical flow is non-zero but there is no motion in the scene

圖片.png

In removing noise from a sequence, we are interested in using both the spatial neighbourhood of a pixel in a frame, as well as the temporal neighbourhood. In this case, we want to filter along the motion trajectory

圖片.png

motion based segmentation of the objects

圖片.png

直接重復(fù)前一幀(zero-order hold)的object會(huì)產(chǎn)生 jagged motion
duplicate frame t1 into frame t
this is straightforward way to generate missing frame, however, it can generate jagged motion, because during going from t1 to t2, the person has considerably moved

圖片.png

更平滑的approach是用motion compensation, find the motion between this block at frame t1 and t2

圖片.png

temporal filtering denoise就是利用加性噪聲時(shí)間平均減弱的原理, 加上motion estimation, 對(duì)于object block進(jìn)行平均減弱加性噪聲
the general idea is that i want to be able to find the motion and then perform a type of filtering, spacial temporal filtering along the motion trajectories

2 Block Matching

圖片.png

block matching的四個(gè)前提假設(shè)
① no change int the ambient light, 環(huán)境光沒(méi)有明顯改變
② objects are rigid, 剛性運(yùn)動(dòng)
③ objects are translated in the 3D world on a plane parallel to the image place, 物體在畫面上運(yùn)動(dòng)沒(méi)有明顯的大小變化, 運(yùn)動(dòng)是在一個(gè)平面(和成像平面平行)完成的
④ no objects appear or left the scene, 沒(méi)有物體出畫入畫

圖片.png

圖片.png

??exausting search, 就是窮盡所有可能

圖片.png

??將search region限定在一定區(qū)域內(nèi)

圖片.png

??進(jìn)一步減少比較的次數(shù), 就是空間二分

圖片.png

圖片.png

??用下采樣的方式減少計(jì)算量, 每四個(gè)pixel只用其中的一個(gè)pixel計(jì)算error

圖片.png

??另外一種減少計(jì)算量的方法, 不計(jì)算單個(gè)block的error, 而是計(jì)算16個(gè)方向上(x方向8個(gè), y方向8個(gè))的error, 在這個(gè)slide中, 原本要計(jì)算64個(gè)error, 現(xiàn)在只用計(jì)算16個(gè), 具體的算法和應(yīng)用場(chǎng)景具體用到的時(shí)候再去看一下

圖片.png

??大尺度的motion estimation結(jié)果怎么與小尺度的motion estimation結(jié)果聯(lián)合起來(lái)用, 就是motion estimation x2之后要怎么和當(dāng)前這個(gè)尺度計(jì)算出來(lái)的motion estimation結(jié)合起來(lái)得到一個(gè)motion estimation的輸出. serves as initial condition for the estimation of the highest levle??? 這個(gè)是什么意思, initial condition具體怎么用?
??這種尺度金字塔的方式經(jīng)常被使用, 能夠明顯 improve ME

圖片.png

??half-pel interpolation, 這是讓motion vector方向更精細(xì)的意思嗎(half-pel accurate motion vector)?

圖片.png

??不同的searching方法效果的比較, 這里motion vector是只有大小沒(méi)有方向嗎?

圖片.png

??BM的demo軟件