1.介紹：

MediaCodec類(lèi)可用于訪問(wèn)Android底層的媒體編解碼器，也就是，編碼器/解碼器組件。它是Android底層多媒體支持基本架構(gòu)的一部分（通常與MediaExtractor, MediaSync, MediaMuxer, MediaCrypto, MediaDrm, Image, Surface, 以及AudioTrack一起使用）；MediaCodec作為比較年輕的Android多媒體硬件編解碼框架，在終端硬解方案中帶來(lái)了很大便利。Android源碼中的CTS部分也給出了很多可以關(guān)于Media編解碼的Demo。

2.解碼

Android的MediaCodec解碼需要分為視頻、音頻解碼。
首先獲取MediaCodec支持的數(shù)量，根據(jù)MediaCodec的句柄獲取MediaCodec支持的編解碼格式
MediaCodecList.getCodecCount()
MediaCodecList.getCodecInfoAt(i);
比如通過(guò)以下測(cè)試代碼，就可以知道終端MediaCodec的解碼能力：

        int n = MediaCodecList.getCodecCount();
        for (int i = 0; i < n; ++i) {
            MediaCodecInfo info = MediaCodecList.getCodecInfoAt(i);
            String[] supportedTypes = info.getSupportedTypes();
            boolean mime_support = false;
            if(info.isEncoder()){
                return;
            }
            for (int j = 0; j < supportedTypes.length; ++j) {
                Log.d(TAG, "codec info:" + info.getName()+" supportedTypes:" + supportedTypes[j]);
                if (supportedTypes[j].equalsIgnoreCase(mime)) {
                    mime_support = true;
                }
            }
        }

Android提供了MediaExtractor來(lái)分離本地/網(wǎng)絡(luò)視頻流的音視頻。
首先定義了一個(gè)統(tǒng)一音視頻處理的類(lèi)：

public class MediaDecoder extends Thread{
    
    protected String mVideoFilePath = null; 
    
    public static final long TIME_US = 10000;
    
    protected MediaExtractor mExtractor = null;
    protected MediaCodec mDecoder = null;
    protected MediaFormat mediaFormat;
    protected UpstreamCallback mCallback;
    protected Surface mSurface;
    
    public MediaDecoder(String videoFilePath, Surface surface,UpstreamCallback callback) {
        this.mVideoFilePath = videoFilePath;
        this.mSurface = surface;
        this.mCallback = callback;
    }
    
    @Override
    public void run() {
        // TODO Auto-generated method stub
        super.run();
        prepare();
    }
    
    public void prepare(){
        try {
            File videoFile = new File(mVideoFilePath);
            mExtractor = new MediaExtractor();
            mExtractor.setDataSource(videoFile.toString());
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    
}

① 視頻解碼：

首先創(chuàng)建視頻的解碼器：

            for (int i = 0; i < mExtractor.getTrackCount(); i++) {
                MediaFormat format = mExtractor.getTrackFormat(i);
                String mime = format.getString(MediaFormat.KEY_MIME);
                if (mime.startsWith("video/")) {
                    mExtractor.selectTrack(i);
                    mDecoder = MediaCodec.createDecoderByType(mime);
                    if(mCallback != null){
                        mDecoder.configure(format, null, null, 0);    //decode flag no output for surface
                    }else{
                        mDecoder.configure(format, mSurface, null, 0);    //decode flag output to surface
                    }
                    break;
                }
            }

            if (mDecoder == null) {
                Log.e(TAG, "Can't find video info!");
                return;
            }
            mDecoder.start(); 

當(dāng)MediaCodec的解碼buffer空閑時(shí)（mDecoder.dequeueInputBuffer），就可以把分離出的視頻數(shù)據(jù)填充到buffer中（mDecoder.queueInputBuffer），讓Decoder開(kāi)始解碼：

                    int inIndex = mDecoder.dequeueInputBuffer(TIME_US);
                    if (inIndex >= 0) {
                        ByteBuffer buffer = inputBuffers[inIndex];
                        int sampleSize = mExtractor.readSampleData(buffer, 0);
                        if (sampleSize < 0) {
                            // We shouldn't stop the playback at this point, just pass the EOS
                            // flag to mDecoder, we will get it again from the
                            // dequeueOutputBuffer
                            Log.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");
                            mDecoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                            isEOS = true;
                        } else {
                            mDecoder.queueInputBuffer(inIndex, 0, sampleSize, mExtractor.getSampleTime(), 0);
                            mExtractor.advance();
                        }
                    }

Decoder解碼出來(lái)的數(shù)據(jù)（mDecoder.dequeueOutputBuffer），解碼的數(shù)據(jù)就可以做該做的處理，比如再編碼，合成文件等。

                BufferInfo info = new BufferInfo();
                int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);
                switch (outIndex) {
                case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
                    Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");
                    outputBuffers = mDecoder.getOutputBuffers();
                    break;
                case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
                    Log.d(TAG, "New format " + mDecoder.getOutputFormat());
                    break;
                case MediaCodec.INFO_TRY_AGAIN_LATER:
                    Log.d(TAG, "dequeueOutputBuffer timed out!");
                    break;
                default:
                    //here erro?
                    Log.d(TAG, "outIndex:"+outIndex);
                    ByteBuffer buffer = outputBuffers[outIndex];
                    Log.d(TAG, "ByteBuffer limit:"+buffer.limit()+" info size:"+info.size);
                    final byte[] chunk = new byte[info.size];
                    buffer.get(chunk);
                    if(mCallback != null){
                        //mCallback.UpstreamCallback(chunk,info.size);
                    }
                    //clear buffer,otherwise get the same buffer which is the last buffer
                    buffer.clear();
                    if(DEBUG_VIDEO)Log.v(TAG, "We can't use this buffer but render it due to the API limit, " + buffer);
                    // We use a very simple clock to keep the video FPS, or the video
                    // playback will be too fast
                    while (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {
                        try {
                            sleep(10);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                            break;
                        }
                    }
                    mDecoder.releaseOutputBuffer(outIndex, true);
                    break;
                }

視頻解碼的完整代碼：

public class VideoDecoder extends MediaDecoder{
    private static final String TAG = "VideoDecode";  
    private static final boolean DEBUG_VIDEO = false;
    
    public VideoDecoder(String videoFilePath,Surface surface,UpstreamCallback callback){
        super(videoFilePath, surface, callback);
    }
    
    @Override
    public void run() {
        // TODO Auto-generated method stub
        super.run();
        VideoDecodePrepare();
    }
    
    public void VideoDecodePrepare() {  
        try {              
            for (int i = 0; i < mExtractor.getTrackCount(); i++) {
                MediaFormat format = mExtractor.getTrackFormat(i);
                String mime = format.getString(MediaFormat.KEY_MIME);
                if (mime.startsWith("video/")) {
                    mExtractor.selectTrack(i);
                    mDecoder = MediaCodec.createDecoderByType(mime);
                    if(mCallback != null){
                        mDecoder.configure(format, null, null, 0);  //decode flag no output for surface
                    }else{
                        mDecoder.configure(format, mSurface, null, 0);  //decode flag output to surface
                    }
                    break;
                }
            }
            
            if (mDecoder == null) {
                Log.e(TAG, "Can't find video info!");
                return;
            }
            mDecoder.start();  
            ByteBuffer[] inputBuffers = mDecoder.getInputBuffers();
            ByteBuffer[] outputBuffers = mDecoder.getOutputBuffers();
            boolean isEOS = false;
            long startMs = System.currentTimeMillis();
            while (!Thread.interrupted()) {
                if (!isEOS) {
                    int inIndex = mDecoder.dequeueInputBuffer(TIME_US);
                    if (inIndex >= 0) {
                        ByteBuffer buffer = inputBuffers[inIndex];
                        int sampleSize = mExtractor.readSampleData(buffer, 0);
                        if (sampleSize < 0) {
                            // We shouldn't stop the playback at this point, just pass the EOS
                            // flag to mDecoder, we will get it again from the
                            // dequeueOutputBuffer
                            Log.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");
                            mDecoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                            isEOS = true;
                        } else {
                            mDecoder.queueInputBuffer(inIndex, 0, sampleSize, mExtractor.getSampleTime(), 0);
                            mExtractor.advance();
                        }
                    }
                }
                BufferInfo info = new BufferInfo();
                int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);
                switch (outIndex) {
                case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
                    Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");
                    outputBuffers = mDecoder.getOutputBuffers();
                    break;
                case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
                    Log.d(TAG, "New format " + mDecoder.getOutputFormat());
                    break;
                case MediaCodec.INFO_TRY_AGAIN_LATER:
                    Log.d(TAG, "dequeueOutputBuffer timed out!");
                    break;
                default:
                    //here erro?
                    Log.d(TAG, "outIndex:"+outIndex);
                    ByteBuffer buffer = outputBuffers[outIndex];
                    Log.d(TAG, "ByteBuffer limit:"+buffer.limit()+" info size:"+info.size);
                    final byte[] chunk = new byte[info.size];
                    buffer.get(chunk);
                    if(mCallback != null){
                        //mCallback.UpstreamCallback(chunk,info.size);
                    }
                    //clear buffer,otherwise get the same buffer which is the last buffer
                    buffer.clear();
                    if(DEBUG_VIDEO)Log.v(TAG, "We can't use this buffer but render it due to the API limit, " + buffer);
                    // We use a very simple clock to keep the video FPS, or the video
                    // playback will be too fast
                    while (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {
                        try {
                            sleep(10);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                            break;
                        }
                    }
                    mDecoder.releaseOutputBuffer(outIndex, true);
                    break;
                }
                // All decoded frames have been rendered, we can stop playing now
                if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                    Log.d(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");
                    break;
                }
            }
            mDecoder.stop();
            mDecoder.release();
            mExtractor.release();
        } catch (Exception ioe) {  
           Log.d(TAG,"failed init decoder", ioe);  
        }  
    } 

注意video解碼時(shí)，如果使用surface來(lái)輸出，則outbuffer會(huì)被消耗掉，想要在視頻輸出的同時(shí)轉(zhuǎn)碼，建議使用OpenESGL來(lái)繪制窗口（這部分后面提供代碼），保留原有buffer。

② 音頻解碼：

基本和視頻解碼的方式一樣，只不過(guò)是通過(guò)MediaExtractor從文件中分離出音頻，然后使用指定的音頻MediaFormat來(lái)通知Decoder解碼。
音頻解碼器的創(chuàng)建如下：

          for (int i = 0; i < mExtractor.getTrackCount(); i++) {
                MediaFormat format = mExtractor.getTrackFormat(i);
                String mime = format.getString(MediaFormat.KEY_MIME);
                if (mime.startsWith("audio/")) {
                    mExtractor.selectTrack(i);
                    mSampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);
                    channel = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
                    mDecoder = MediaCodec.createDecoderByType(mime);
                    mDecoder.configure(format, null, null, 0);
                    break;
                }
            }

            if (mDecoder == null) {
                Log.e(TAG, "Can't find audio info!");
                return;
            }
            mDecoder.start();

在這里，我直接通過(guò)Android的AudioTrack來(lái)輸出音頻，完整代碼如下：

public class AudioDecoder extends MediaDecoder {
    private static final String TAG = "VideoDecode";
    private static final boolean DEBUG_AUDIO = false;

    private int mSampleRate = 0;
    private int channel = 0;

    public AudioDecoder(String videoFilePath, Surface surface,UpstreamCallback callback) {
        super(videoFilePath, surface, callback);
    }

    @Override
    public void run() {
        // TODO Auto-generated method stub
        super.run();
        AudioDecodePrepare();
    }

    public void AudioDecodePrepare() {
        try {
            for (int i = 0; i < mExtractor.getTrackCount(); i++) {
                MediaFormat format = mExtractor.getTrackFormat(i);
                String mime = format.getString(MediaFormat.KEY_MIME);
                if (mime.startsWith("audio/")) {
                    mExtractor.selectTrack(i);
                    mSampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);
                    channel = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
                    mDecoder = MediaCodec.createDecoderByType(mime);
                    mDecoder.configure(format, null, null, 0);
                    break;
                }
            }

            if (mDecoder == null) {
                Log.e(TAG, "Can't find audio info!");
                return;
            }
            mDecoder.start();
            ByteBuffer[] inputBuffers = mDecoder.getInputBuffers();
            ByteBuffer[] outputBuffers = mDecoder.getOutputBuffers();
            BufferInfo info = new BufferInfo();
            int buffsize = AudioTrack.getMinBufferSize(mSampleRate,
                    AudioFormat.CHANNEL_OUT_STEREO,
                    AudioFormat.ENCODING_PCM_16BIT);
            AudioTrack audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
                    mSampleRate, AudioFormat.CHANNEL_OUT_STEREO,
                    AudioFormat.ENCODING_PCM_16BIT, buffsize,
                    AudioTrack.MODE_STREAM);
            audioTrack.play();

            boolean isEOS = false;
            long startMs = System.currentTimeMillis();

            while (!Thread.interrupted()) {
                if (!isEOS) {
                    int inIndex = mDecoder.dequeueInputBuffer(TIME_US);
                    if (inIndex >= 0) {
                        ByteBuffer buffer = inputBuffers[inIndex];
                        int sampleSize = mExtractor.readSampleData(buffer, 0);
                        if (sampleSize < 0) {
                            // We shouldn't stop the playback at this point,
                            // just pass the EOS
                            // flag to mediaDecoder, we will get it again from
                            // the
                            // dequeueOutputBuffer
                            Log.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");
                            mDecoder.queueInputBuffer(inIndex, 0, 0, 0,
                                    MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                            isEOS = true;
                        } else {
                            mDecoder.queueInputBuffer(inIndex, 0,
                                    sampleSize, mExtractor.getSampleTime(), 0);
                            mExtractor.advance();
                        }
                    }
                }
                int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);
                switch (outIndex) {
                case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
                    Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");
                    outputBuffers = mDecoder.getOutputBuffers();
                    break;
                case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
                    MediaFormat format = mDecoder.getOutputFormat();
                    Log.d(TAG, "New format " + format);
                    audioTrack.setPlaybackRate(format
                            .getInteger(MediaFormat.KEY_SAMPLE_RATE));
                    break;
                case MediaCodec.INFO_TRY_AGAIN_LATER:
                    Log.d(TAG, "dequeueOutputBuffer timed out!");
                    break;
                default:
                    ByteBuffer buffer = outputBuffers[outIndex];
                    
                    if(DEBUG_AUDIO)Log.v(TAG,"We can't use this buffer but render it due to the API limit, "+ buffer);
                    final byte[] chunk = new byte[info.size];
                    buffer.get(chunk);
                    if(mCallback != null){
                        mCallback.UpstreamCallback(chunk,info.size);
                    }
                    //clear buffer,otherwise get the same buffer which is the last buffer
                    buffer.clear();                 
                    // We use a very simple clock to keep the video FPS, or the
                    // audio playback will be too fast
                    while (info.presentationTimeUs / 1000 > System
                            .currentTimeMillis() - startMs) {
                        try {
                            sleep(10);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                            break;
                        }
                    }
                    // AudioTrack write data
                    audioTrack.write(chunk, info.offset, info.offset
                            + info.size);
                    mDecoder.releaseOutputBuffer(outIndex, false);
                    break;
                }
                // All decoded frames have been rendered, we can stop playing now
                if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                    Log.d(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");
                    break;
                }
            }
            mDecoder.stop();
            mDecoder.release();
            mExtractor.release();
            audioTrack.stop();
            audioTrack.release();
        } catch (Exception ioe) {
            throw new RuntimeException("failed init encoder", ioe);
        }
    }
    
}

通過(guò)以上處理，就可以使用硬解方案實(shí)現(xiàn)音視頻播放了。

3.結(jié)束語(yǔ)

解碼的知識(shí)先介紹這么多，下一篇會(huì)介紹MediaCodec來(lái)實(shí)現(xiàn)編碼。感謝持續(xù)關(guān)注！