【Android】局域网屏幕共享与反向控制功能的实现

---

前言

之前用了一下QQ电脑版的远程协助，发现这个功能很方便实用，于是就想开发一款类似功能的APP，无奈本人只会一点点Android和Java，开发过程中爬了很多坑，但是经过不懈努力，终于把基本功能实现了。

---

一、功能介绍

1.屏幕共享

这个APP主要有屏幕共享和反向控制两个功能。屏幕共享功能的实现需要两台手机，一台手机作为服务端，共享屏幕；另一台手机做客户端，显示屏幕。服务端与客户端需要在同一局域网或热点连接。服务端主要是通过MediaProjection实时截屏，通过TCP把图片数据发送给客户端；客户端则把TCP接收的图片数据通过SurfaceView渲染显示。

2.反向控制

反向控制的功能主要是结合了ADB。这个功能的实现需要手机服务端先开启 开发者模式及USB调试，然后用USB连接电脑端。共享屏幕时，在电脑端运行Python或其他语言编写的脚本，客户端的SurfaceView会侦听用户的触摸事件，并通过服务端TCP传输给电脑端，电脑端则发送ADB命令给服务端，从而实现客户端反向控制服务端的功能。

二、功能原理

1.原理框图

2.工作原理

(1)MediaProjection截屏

MediaProjection是Google在Android5.0之后给开发者提供的截屏或录屏方法。在使用MediaProjection之前需要先申请权限。

    private void Request_Media_Projection_Permission() {
        MediaProjectionManager mediaProjectionManager = (MediaProjectionManager) this.getSystemService(Context.MEDIA_PROJECTION_SERVICE);
        Intent intent = mediaProjectionManager.createScreenCaptureIntent();
        startActivityForResult(intent, REQUEST_MEDIA_PROJECTION_CODE);
    }
    
    @Override
    public void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);
        if (requestCode == REQUEST_MEDIA_PROJECTION_CODE) {
            if (resultCode != Activity.RESULT_OK) {
                Toast.makeText(this, "Media Projection Permission Denied", Toast.LENGTH_SHORT).show();
                return;
            }
            MyUtils.setResultCode(resultCode);
            MyUtils.setResultData(data);
        }
    }
    
    private ScreenCapture(Context context, int resultCode, Intent data) {
        MediaProjectionManager mMediaProjectionManager = (MediaProjectionManager) context.getSystemService(Context.MEDIA_PROJECTION_SERVICE);
        mMediaProjection = mMediaProjectionManager.getMediaProjection(resultCode, data);
        screen_width = MyUtils.getScreenWidth();
        screen_height = MyUtils.getScreenHeight();
        screen_density = MyUtils.getScreenDensity();
        mImageReader = ImageReader.newInstance(
                screen_width,
                screen_height,
                PixelFormat.RGBA_8888,
                2);
    }

    public static ScreenCapture getInstance(Context context, int resultCode, Intent data) {
        if(screenCapture == null) {
            synchronized (ScreenCapture.class) {
                if(screenCapture == null) {
                    screenCapture = new ScreenCapture(context, resultCode, data);
                }
            }
        }
        return screenCapture;
    }

MediaProjection通过createVirtualDisplay来截屏，我们可以通过ImageReader的setOnImageAvailableListener把截屏数据转为Bitmap数据。

    private void setUpVirtualDisplay() {
        mVirtualDisplay = mMediaProjection.createVirtualDisplay(
                "ScreenCapture",
                screen_width,
                screen_height,
                screen_density,
                DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
                mImageReader.getSurface(),
                null,
                null);

        mImageReader.setOnImageAvailableListener(this, null);
    }

    @Override
    public void onImageAvailable(ImageReader imageReader) {
        try {
            Image image = imageReader.acquireLatestImage();
            if(image != null) {
                Image.Plane[] planes = image.getPlanes();
                ByteBuffer buffer = planes[0].getBuffer();
                int pixelStride = planes[0].getPixelStride();
                int rowStride = planes[0].getRowStride();
                int rowPadding = rowStride - pixelStride * screen_width;
                Bitmap bitmap = Bitmap.createBitmap(screen_width + rowPadding / pixelStride, screen_height, Bitmap.Config.ARGB_8888);
                bitmap.copyPixelsFromBuffer(buffer);
                MyUtils.setBitmap(bitmap);
                image.close();
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

(2)SurfaceView显示

SurfaceView渲染图片是在独立线程里进行的，所以它显示大图片会更快更流畅。我们可以新建一个View来继承它，并在这个View里实现我们想要的功能，比如显示Bitmap。侦听用户的触摸事件主要是通过View的OnTouchListener来实现的。

    public void drawBitmap() {
        Canvas canvas = surfaceHolder.lockCanvas();
        if (canvas != null) {
            bitmap = getBitmap();
            if (bitmap != null) {
                canvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
                Rect rect = new Rect(0, 0, viewWidth, viewHeight);
                canvas.drawBitmap(bitmap, null, rect, null);
            }
            surfaceHolder.unlockCanvasAndPost(canvas);
        }
    }
    
    @Override
    public boolean onTouch(View view, MotionEvent motionEvent) {
        switch (motionEvent.getAction()) {
            case MotionEvent.ACTION_DOWN:
                int staX = (int) (motionEvent.getX() * getWidthConvert());
                int staY = (int) (motionEvent.getY() * getHeightConvert());
                MyUtils.setStartX(staX);
                MyUtils.setStartY(staY);
                touchClientRunnable.setTouchDown(true);
                break;
            case MotionEvent.ACTION_UP:
                int endX = (int) (motionEvent.getX() * getWidthConvert());
                int endY = (int) (motionEvent.getY() * getHeightConvert());
                MyUtils.setEndX(endX);
                MyUtils.setEndY(endY);
                touchClientRunnable.setTouchUp(true);
                break;
        }
        return true;
    }

    @Override
    public void run() {
        while (isDraw) {
            try {
                drawBitmap();
                setOnTouchListener(this);
                Thread.sleep(10);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

(3)TCP传输Bitmap

由于截屏的图片很大，直接传输会很慢，所以我们需要对图片进行压缩处理，这里采用的是缩放压缩。

    public static Bitmap BitmapMatrixCompress(Bitmap bitmap) {
        Matrix matrix = new Matrix();
        matrix.setScale(0.5f, 0.5f);
        return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
    }

服务端发送Bitmap

    private final static byte[] PACKAGE_HEAD = {(byte)0xFF, (byte)0xCF, (byte)0xFA, (byte)0xBF, (byte)0xF6, (byte)0xAF, (byte)0xFE, (byte)0xFF};

    public static byte[] BitmaptoBytes(Bitmap bitmap) {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
        return baos.toByteArray();
    }

    private void ServerTransmitBitmap() {
        try {
            DataOutputStream dataOutputStream = new DataOutputStream(socket.getOutputStream());
            if (bitmap != null) {
                byte[] bytes = MyUtils.BitmaptoBytes(bitmap);
                dataOutputStream.write(PACKAGE_HEAD);
                dataOutputStream.writeInt(MyUtils.getScreenWidth());
                dataOutputStream.writeInt(MyUtils.getScreenHeight());
                dataOutputStream.writeInt(bytes.length);
                dataOutputStream.write(bytes);
            }
            dataOutputStream.flush();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

客户端接收Bitmap

    private final static byte[] PACKAGE_HEAD = {(byte)0xFF, (byte)0xCF, (byte)0xFA, (byte)0xBF, (byte)0xF6, (byte)0xAF, (byte)0xFE, (byte)0xFF};

    public static Bitmap BytestoBitmap(byte[] b) {
        if(b.length != 0) {
            return BitmapFactory.decodeByteArray(b, 0, b.length);
        } else {
            return null;
        }
    }

    private void ClientReceiveBitmap() {
        try {
            InputStream inputStream = socket.getInputStream();
            boolean isHead = true;
            for (byte b : PACKAGE_HEAD) {
                byte head = (byte) inputStream.read();
                if (head != b) {
                    isHead = false;
                    break;
                }
            }
            if (isHead) {
                DataInputStream dataInputStream = new DataInputStream(inputStream);
                int width = dataInputStream.readInt();
                int height = dataInputStream.readInt();
                int len = dataInputStream.readInt();
                byte[] bytes = new byte[len];
                dataInputStream.readFully(bytes, 0, len);
                Bitmap bitmap = MyUtils.BytestoBitmap(bytes);
                if (bitmap != null && width != 0 && height != 0) {
                    if (listener != null) {
                        listener.onClientReceiveBitmap(bitmap, width, height);
                    }
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

(4)ADB端口转发

反向控制主要是用到了adb forward命令进行端口转发，其实也是TCP通信。使用这种方法主要是手机不用ROOT。

import json
import os
import socket
isConnect = False
isTouch = False
ack = os.popen('adb forward tcp:50003 tcp:50004').read()
if ack.find('error') == 0:
    isConnect = False
    print('no device')
else:
    isConnect = True
if isConnect:
    client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    client.connect(('127.0.0.1', 50003))
    while True:
        try:
            msg = client.recv(2048)
            data = json.loads(msg.decode('utf-8'))
            staX = data.get('staX')
            staY = data.get('staY')
            endX = data.get('endX')
            endY = data.get('endY')
            action = data.get('action')
            if action != 0:
                isTouch = True
            if isTouch:
                cmd = ''
                if action == 1:
                    cmd = 'adb shell input tap {} {}'.format(staX, staY)
                elif action == 2:
                    cmd = 'adb shell input swipe {} {} {} {}'.format(staX, staY, endX, endY)
                elif action == 3:
                    cmd = 'adb shell input keyevent 4'
                os.system(cmd)
                isTouch = False
                action = 0
                print(cmd)
        except Exception:
            continue

以上是部分代码片段。

三、效果演示

---

四、项目链接

Screen Projection
RemoteAssistance

总结

现阶段主要是实现了基本功能，还存在很多缺陷，现在只支持在局域网或热点下共享屏幕，屏幕显示有很明显的延迟，反向控制需要连接电脑等。