X-Git-Url: https://gerrit.automotivelinux.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2F06_Component_Documentation%2F11_Unified_HMI.md;fp=docs%2F06_Component_Documentation%2F11_Unified_HMI.md;h=fcd8c0169ef059baacac0c4afc77ed49a70c3777;hb=923fdc80a22c571025e408560aa58d58e52deff0;hp=0000000000000000000000000000000000000000;hpb=fa46fde3c3567d41cb609085c7ed43565c03e4c5;p=AGL%2Fdocumentation.git diff --git a/docs/06_Component_Documentation/11_Unified_HMI.md b/docs/06_Component_Documentation/11_Unified_HMI.md new file mode 100644 index 0000000..fcd8c01 --- /dev/null +++ b/docs/06_Component_Documentation/11_Unified_HMI.md @@ -0,0 +1,165 @@ +--- +title: Unified HMI +--- +# Unified HMI + +## Introduction + +This document describes the design and usage of the **Unified HMI**, which is a common platform technology for UX innovation in integrated cockpit by flexible information display on multiple displays of various applications. Applications can be rendered to any display via a unified virtual display. + +### Remote Virtio GPU (RVGPU) + +The RVGPU rendering engine, operating on a client-server model, creates 3D rendering commands on a client device and transmits them to a server device via the network, where the server-side device performs GPU-based rendering and displays the results. The meta-rvgpu currently supports x86, Raspberry Pi 4, and Renesas Boards platforms. + +### RVGPU Architecture + +![RVGPU_Architecture](images/Unified-HMI/RVGPU_Architecture.png) + +## How to Build + +Follow the [AGL documentation](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/01_Build_Process_Overview/) for the build process, and set up the "[Initializing Your Build Environment](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/04_Initializing_Your_Build_Environment/)" section as described below to enable the AGL feature 'agl-rvgpu'. For example: +``` +$ cd $AGL_TOP/master +$ source ./meta-agl/scripts/aglsetup.sh -m qemux86-64 -b qemux86-64 agl-demo agl-devel agl-rvgpu +``` +After adding the feature, execute the command: +``` +$ bitbake +``` +**Note**: The operation has been confirmed with the **agl-demo-platform**. If you wish to specify a different `` other than agl-demo-platform, create a file named `.bbappend` in the directory +``` +$AGL_TOP/master/meta-agl-devel/meta-uhmi/meta-rvgpu/recipes-platform/images +``` +, and then add the following line to the file: + +``` +IMAGE_INSTALL:append = " packagegroup-rvgpu" +``` + +After making this change, you will need to execute the following commands: + +``` +$ cd $AGL_TOP/master +$ source agl-init-build-env +$ bitbake +``` + +## How to Setup and Boot + +For Environment setup instructions for each platform, refer to the following link in the AGL Documentation. +[Building for x86(Emulation and Hardware)](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/07_Building_for_x86_%28Emulation_and_Hardware%29/) +[Building for Raspberry Pi 4](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/08_Building_for_Raspberry_Pi_4/) +[Building for Supported Renesas Boards](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/09_Building_for_Supported_Renesas_Boards/) + +For Raspberry Pi 4 and Supported Renesas Boards, refer to the above URL for boot methods. +For x86 emulation (qemu), network bridge is required to enable communication with other devices when using RVGPU. Here’s an example procedure for your reference. +``` +$ sudo ip link add type bridge +$ sudo ip addr add dev +$ sudo ip link set dev master +$ sudo ip link set dev up +``` +Replace the placeholders with the appropriate values: +- ``: You can assign any name, for example: `br0` +- ``: Enter an available IP address, for example: `192.168.0.100/24` +- ``: Specify the network interface, for example: `eth0` + +To enable the use of the bridge, create or append /etc/qemu directory and /etc/qemu/bridge.conf file. +``` +allow +``` +Make sure /etc/qemu/ has 755 permissions. +Create the following bash file named **run_qemu_bridge.sh** in any ``. +``` +#!/bin/bash + +KERNEL_PATH=$1 +DRIVE_PATH=$2 +BRIDGE_NAME="" + +printf -v macaddr "52:54:%02x:%02x:%02x:%02x" $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff )) $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff )) + +qemu-system-x86_64 -enable-kvm -m 2048 \ + -kernel ${KERNEL_PATH} \ + -drive file=${DRIVE_PATH},if=virtio,format=raw \ + -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt \ + -vga virtio -show-cursor \ + -device virtio-net-pci,netdev=net0,mac=$macaddr \ + -netdev bridge,br=$BRIDGE_NAME,id=net0 \ + -serial mon:stdio -serial null \ + -soundhw hda \ + -append 'root=/dev/vda rw console=ttyS0,115200n8 fstab=no' +``` +Save the file and run the following to start QEMU. +``` +sudo /run_qemu_bridge.sh /tmp/deploy/images/qemux86-64/bzImage /tmp/deploy/images/qemux86-64/agl-demo-platform-qemux86-64.ext4 +``` +When QEMU boot, assign an IP address. For example: +``` +ifconfig 192.168.0.10 netmask 255.255.255.0 +``` + +## How to run RVGPU remotely +Prepare two images, one as the Sender and the other as the Receiver. +It is necessary for the Sender and Receiver to be within the same network. + +**Receiver side** +``` +$ export XDG_RUNTIME_DIR=/run/user/ +$ rvgpu-renderer -b @0,0 -p & +``` + +Replace the placeholders with the appropriate values: +- ``: Specify according to your environment, for example:1001 +- ``: Enter an usable area for example: 1080x1488 + With the following command, you can know the usable area. + ``` + $ journalctl | grep -i "usable area" + # Example Output: + Nov 29 11:42:53 qemux86-64 agl-compositor[259]: [11:42:53.166] Usable area: 1080x1488+0,216 + ``` +- ``: Enter an available port number, for example: 55555 + + +**Sender side** +Create the following shell script **run_remote_app.sh** in any `` for a smooth experience. +``` +#!/bin/bash + +export XDG_RUNTIME_DIR=/tmp +export LD_LIBRARY_PATH=/usr/lib/mesa-virtio + +# ------------- +# launch app +# ------------- +$@ +``` + +Save the file and run the following to start weston. + +``` +$ rvgpu-proxy -s 1080x1488@0,0 -n : & +$ /run_remote_app.sh weston --backend drm-backend.so -Swayland-0 --seat=seat_virtual -i 0 & +``` +Replace the placeholders with the appropriate values: +- ``: Port set in the renderer. + + +After completing these steps, the Weston screen from the Sender will be transferred and displayed on the Receiver using rvgpu-proxy and rvgpu-renderer. You can verify that everything is functioning properly by launching wayland applications on the Sender side, such as: +``` +$ weston-simple-egl -f +``` + +You can also verify the touch or keyboard operation support of the RVGPU by using app such as +``` +$ weston-smoke +$ weston-editor +``` +**Note**: There are known issues with mouse, such as the cursor becoming invisible and occasional flickering of a green screen. + +**Appendix** +- By building the RVGPU on Ubuntu, it is possible to enable bidirectional remote rendering between the agl-demo-platform and Ubuntu. +For the build procedure on Ubuntu, see the following URL: https://github.com/unified-hmi/remote-virtio-gpu +The figure below shows an example where Ubuntu is used as the Sender and AGL as the Receiver, running RVGPU. + +![Using_RVGPU_from_Ubuntu_to_AGL](images/Unified-HMI/Using_RVGPU_from_Ubuntu_to_AGL.png) \ No newline at end of file