Depthwise separable convolution
深度級(jí)可分離卷積其實(shí)是一種可分解卷積操作(factorized convolutions).將卷積分解成兩個(gè)更小的操作:
- depthwise convolution
- pointwise convolution
在標(biāo)準(zhǔn)卷積中,其卷積核作用于所有輸入通道上(input channel).如輸出圖片為:(6,6,3),原來(lái)卷積操作是:(4,4,3,5).4X4的卷積大小,3是卷積的通道數(shù),5是卷積核的數(shù)量.其輸出尺寸為,其輸出特征為(3,3,5),如圖所示(同一個(gè)卷積核作用在所有通道上):
不同通道結(jié)果相加得到該卷積核的感受野的值.
深度分離卷積(depthwise convolution),針對(duì)每個(gè)輸入通道采用不同的卷積核，就是說(shuō)一個(gè)卷積核對(duì)應(yīng)一個(gè)輸入通道.如圖所示:
輸入有3個(gè)通道，對(duì)應(yīng)著有3個(gè)大小為(4,4,1) (4,4,1)(4,4,1)的深度卷積核，卷積結(jié)果共有3個(gè)大小為(3,3,1) (3,3,1)(3,3,1)
最后,采用pointwise convolution.pointwise convolution其實(shí)就是普通的卷積，只不過(guò)其采用1x1的卷積核。1 X 1的卷積核作用 用于將不同通道的值相加.
depthwise convolution先將通道(深度)分別進(jìn)行卷積操作,再用pointwise convolution(1X1的卷積)進(jìn)行通道間的卷積感受野值相加 最終結(jié)果與標(biāo)準(zhǔn)卷積類似.
官方示意圖如下所示:
計(jì)算量估計(jì)
假設(shè)輸入特征圖大小為:,而輸出特征圖大小為:
假設(shè)特征圖的width與height,與輸出圖大小一樣,兩者差別在于通道.
標(biāo)準(zhǔn)卷積
對(duì)于標(biāo)準(zhǔn)卷積():
每個(gè)卷積核的乘積計(jì)算量:
單個(gè)通道輸出特征圖邊長(zhǎng)所需的乘積計(jì)算量:
單個(gè)卷積核跨通道值相乘:
N個(gè)卷積:
depthwise convolution
對(duì)于每一個(gè)通道,都有一個(gè)卷積,所以其計(jì)算量為:
pointwise convolution
1*1卷積,其計(jì)算量為:
最后比較depthwise separable convolution和標(biāo)準(zhǔn)卷積如下：
一般情況下 N 比較大，那么如果采用3x3卷積核的話，depthwise separable convolution相較標(biāo)準(zhǔn)卷積可以降低大約9倍的計(jì)算量。其實(shí)，后面會(huì)有對(duì)比，參數(shù)量也會(huì)減少很多。
mobilenet v1 結(jié)構(gòu)
基礎(chǔ)結(jié)構(gòu)如圖:
原始的mobilenet v1 用于imagenet分類任務(wù)中,整個(gè)網(wǎng)絡(luò)有28層.
mobilenet 超參數(shù)
- width multiplier
- resolution multiplier
width multiplier主要按照比例減少通道數(shù):
其記為,取值范圍從(0,1]
總計(jì)算量變?yōu)?
DK · DK · αM · DF · DF + αM · αN · DF · DF
分辨率因子用來(lái)控制輸入的分辨率,記為:
DK · DK · αM · ρDF · ρDF + αM · αN · ρDF · ρDF
pytorch實(shí)現(xiàn):
在torch.nn.Conv2d中,group參數(shù)用來(lái)控制是否對(duì)輸入的每個(gè)通道單獨(dú)設(shè)置卷積:
- At groups=1, all inputs are convolved to all outputs.(groups=1的時(shí)候,為標(biāo)準(zhǔn)卷積)
- At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.(=2時(shí),用兩個(gè)卷積核,一個(gè)卷積核看一半,最后concat)
- At groups= in_channels, each input channel is convolved with its own set of filters.(每個(gè)通道放一個(gè)卷積核,這就是我們要的depthwise convolution)
所以,論文中的mobilenet基礎(chǔ)結(jié)構(gòu)為:
def conv_dw(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, inp, 3, stride, 1, groups=inp, bias=False),
nn.BatchNorm2d(inp),
nn.ReLU(inplace=True),
nn.Conv2d(inp, oup, 1, 1, 0, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True),
)
整個(gè)mobilenet v1的論文結(jié)構(gòu)代碼為:
class MobileNet(nn.Module):
def __init__(self):
super(MobileNet, self).__init__()
def conv_bn(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, oup, 3, stride, 1, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True)
)
def conv_dw(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, inp, 3, stride, 1, groups=inp, bias=False),
nn.BatchNorm2d(inp),
nn.ReLU(inplace=True),
nn.Conv2d(inp, oup, 1, 1, 0, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True),
)
self.model = nn.Sequential(
conv_bn( 3, 32, 2),
conv_dw( 32, 64, 1),
conv_dw( 64, 128, 2),
conv_dw(128, 128, 1),
conv_dw(128, 256, 2),
conv_dw(256, 256, 1),
conv_dw(256, 512, 2),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 512, 1),
conv_dw(512, 1024, 2),
conv_dw(1024, 1024, 1),
nn.AvgPool2d(7),
)
self.fc = nn.Linear(1024, 1000)
def forward(self, x):
x = self.model(x)
x = x.view(-1, 1024)
x = self.fc(x)
return x
