Capture and Transfer Video Frames

Encoding

I420 (YUV), RGB24: http://blog.csdn.net/leixiaohua1020/article/details/12234821

1. I420        A frame in I420 (YUV 4:2:0) is size = width * height * 1.5Byte

                    I420 is YYYY…UUUUVVVV

                    NV21 is YYYY…UVUVUVUV (Maybe used in the deprecated Camera API)

                    YV12 IS YYYY…VVVVUUUU (Maybe used in the deprecated Camera API)

2. RGB24    A frame in RGB24 is size = width * height * 3Byte

Camera2 API

Usage

1. Official guide: https://developer.android.com/guide/topics/media/camera

2. Official samples

        a. https://github.com/googlesamples/android-Camera2Basic/

        b. https://github.com/googlesamples/android-Camera2Raw/

        c. https://github.com/googlesamples/android-Camera2Video/

Format

1. Image class: https://www.polarxiong.com/archives/Android-Image%E7%B1%BB%E6%B5%85%E6%9E%90-%E7%BB%93%E5%90%88YUV_420_888.html

2. YUV_420_888 to I420: https://www.polarxiong.com/archives/Android-YUV_420_888%E7%BC%96%E7%A0%81Image%E8%BD%AC%E6%8D%A2%E4%B8%BAI420%E5%92%8CNV21%E6%A0%BC%E5%BC%8Fbyte%E6%95%B0%E7%BB%84.html

Open Source Libs

1. libyuv

        It is used to rotate, crop and scale the captured video frames before transfer. The frames must be head-up no matter the phone orientation. The best order of image processing for downscaling the captured image is: scale -> crop -> rotate.

        Use Neon to optimize the libyuv when compiling, so that we can do parallel processing.

2. libtheora

        Use the unofficial libtheora 1.2 with neon optimization. Need to change the instruction “BLX” in lib/arm/armenquant.s to “BL” to avoid crush issue.

        1).    Get the tools:

                a.    sudo apt-get install autoconf

                b.    sudo apt-get install libtool

                c.    sudo apt-get install libogg-dev

        2).    set up the ndk build evn:

                a.    NDK=/build/toolchain/lin64/android-ndk-r12b

                b.    SYSROOT=$NDK/platforms/android-19/arch-arm (Use API Level 19 because the libogg and libtheora are too old to be compiled on newer platform)

                c.    PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64

        3).    Build the libogg:

                a.    ./autogen.sh --prefix=$HOME/tmp/libogg --host=arm-linux --enable-shared CC="$PREBUILT/bin/arm-linux-androideabi-gcc --sysroot=$SYSROOT" LDFLAGS="-Wl,-O2,--as-needed" CFLAGS="-march=armv7-a -mfpu=neon"

                b.    make install

        4).    Build the libtheora

                ARM compile options https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html

                Into Assambly: http://www.cnblogs.com/goodhacker/p/3206405.htmlhttp://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0204ic/Cihfddaf.html

                a.    ./autogen.sh --prefix=$HOME/tmp/libtheora --host=arm-linux --disable-examples CC="$PREBUILT/bin/arm-linux-androideabi-gcc --sysroot=$SYSROOT" LDFLAGS="-Wl,-O2,-g,-L$HOME/tmp/libogg/lib" CPPFLAGS="-I$HOME/tmp/libogg/include" CFLAGS="-march=armv7-a -mfpu=neon -mthumb "

                b.    make install