docs/06_Component_Documentation/11_Unified_HMI.md

   1 ---
   2 title: Unified HMI
   3 ---
   4 # Unified HMI
   5
   6 ## Introduction
   7
   8 This document describes the design and usage of the **Unified HMI**, which is a common platform technology for UX innovation in integrated cockpit by flexible information display on multiple displays of various applications. Applications can be rendered to any display via a unified virtual display.
   9
  10 ### Remote Virtio GPU (RVGPU)
  11
  12 The RVGPU rendering engine, operating on a client-server model, creates 3D rendering commands on a client device and transmits them to a server device via the network, where the server-side device performs GPU-based rendering and displays the results. The meta-rvgpu currently supports x86, Raspberry Pi 4, and Renesas Boards platforms.
  13
  14 ### RVGPU Architecture
  15
  16 ![RVGPU_Architecture](images/Unified-HMI/RVGPU_Architecture.png)
  17
  18 ## How to Build
  19
  20 Follow the [AGL documentation](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/01_Build_Process_Overview/) for the build process, and set up the "[Initializing Your Build Environment](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/04_Initializing_Your_Build_Environment/)" section as described below to enable the AGL feature 'agl-rvgpu'. For example:
  21 ```
  22 $ cd $AGL_TOP/master
  23 $ source ./meta-agl/scripts/aglsetup.sh -m qemux86-64 -b qemux86-64 agl-demo agl-devel agl-rvgpu
  24 ```
  25 After adding the feature, execute the command:
  26 ```
  27 $ bitbake <image_name>
  28 ```
  29 **Note**: The operation has been confirmed with the **agl-demo-platform**. If you wish to specify a different `<image_name>` other than agl-demo-platform, create a file named `<image_name>.bbappend` in the directory
  30 ```
  31 $AGL_TOP/master/meta-agl-devel/meta-uhmi/meta-rvgpu/recipes-platform/images
  32 ```
  33 , and then add the following line to the file:
  34
  35 ```
  36 IMAGE_INSTALL:append = " packagegroup-rvgpu"
  37 ```
  38
  39 After making this change, you will need to execute the following commands:
  40
  41 ```
  42 $ cd $AGL_TOP/master
  43 $ source agl-init-build-env
  44 $ bitbake <image_name>
  45 ```
  46
  47 ## How to Setup and Boot
  48
  49 For Environment setup instructions for each platform, refer to the following link in the AGL Documentation.
  50 [Building for x86(Emulation and Hardware)](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/07_Building_for_x86_%28Emulation_and_Hardware%29/)
  51 [Building for Raspberry Pi 4](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/08_Building_for_Raspberry_Pi_4/)
  52 [Building for Supported Renesas Boards](https://docs.automotivelinux.org/en/master/#01_Getting_Started/02_Building_AGL_Image/09_Building_for_Supported_Renesas_Boards/)
  53
  54 For Raspberry Pi 4 and Supported Renesas Boards, refer to the above URL for boot methods.
  55 For x86 emulation (qemu), network bridge is required to enable communication with other devices when using RVGPU. Here’s an example procedure for your reference.
  56 ```
  57 $ sudo ip link add <bridge_name> type bridge
  58 $ sudo ip addr add <IP address> dev <bridge_name>
  59 $ sudo ip link set dev <interface> master <bridge_name>
  60 $ sudo ip link set dev <bridge_name> up
  61 ```
  62 Replace the placeholders with the appropriate values:
  63 - `<bridge_name>`: You can assign any name, for example: `br0`
  64 - `<IP_address>`: Enter an available IP address, for example: `192.168.0.100/24`
  65 - `<interface>`: Specify the network interface, for example: `eth0`
  66
  67 To enable the use of the bridge, create or append /etc/qemu directory and /etc/qemu/bridge.conf file.
  68 ```
  69 allow <bridge_name>
  70 ```
  71 Make sure /etc/qemu/ has 755 permissions.
  72 Create the following bash file named **run_qemu_bridge.sh** in any `<WORKDIR>`.
  73 ```
  74 #!/bin/bash
  75
  76 KERNEL_PATH=$1
  77 DRIVE_PATH=$2
  78 BRIDGE_NAME="<bridge_name>"
  79
  80 printf -v macaddr "52:54:%02x:%02x:%02x:%02x" $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff )) $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff ))
  81
  82 qemu-system-x86_64 -enable-kvm -m 2048 \
  83     -kernel ${KERNEL_PATH} \
  84     -drive file=${DRIVE_PATH},if=virtio,format=raw \
  85     -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt \
  86     -vga virtio -show-cursor \
  87     -device virtio-net-pci,netdev=net0,mac=$macaddr \
  88     -netdev bridge,br=$BRIDGE_NAME,id=net0 \
  89     -serial mon:stdio -serial null \
  90     -soundhw hda \
  91     -append 'root=/dev/vda rw console=ttyS0,115200n8 fstab=no'
  92 ```
  93 Save the file and run the following to start QEMU.
  94 ```
  95 sudo <WORKDIR>/run_qemu_bridge.sh <build_directory>/tmp/deploy/images/qemux86-64/bzImage <build_directory>/tmp/deploy/images/qemux86-64/agl-demo-platform-qemux86-64.ext4
  96 ```
  97 When QEMU boot, assign an IP address. For example:
  98 ```
  99 ifconfig <your environment> 192.168.0.10 netmask 255.255.255.0
 100 ```
 101
 102 ## How to run RVGPU remotely
 103 Prepare two images, one as the Sender and the other as the Receiver.
 104 It is necessary for the Sender and Receiver to be within the same network.
 105
 106 **Receiver side**
 107 ```
 108 $ export XDG_RUNTIME_DIR=/run/user/<your_UID>
 109 $ rvgpu-renderer -b <your_Area>@0,0 -p <Port_Number> &
 110 ```
 111
 112 Replace the placeholders with the appropriate values:
 113 - `<your_UID>`: Specify according to your environment, for example:1001
 114 - `<your_Area>`: Enter an usable area for example: 1080x1488
 115  With the following command, you can know the usable area.
 116   ```
 117   $ journalctl | grep -i "usable area"
 118   # Example Output:
 119   Nov 29 11:42:53 qemux86-64 agl-compositor[259]: [11:42:53.166] Usable area: 1080x1488+0,216
 120   ```
 121 - `<Port_Number>`: Enter an available port number, for example: 55555
 122
 123
 124 **Sender side**
 125 Create the following shell script **run_remote_app.sh** in any `<WORKDIR>` for a smooth experience.
 126 ```
 127 #!/bin/bash
 128
 129 export XDG_RUNTIME_DIR=/tmp
 130 export LD_LIBRARY_PATH=/usr/lib/mesa-virtio
 131
 132 # -------------
 133 # launch app
 134 # -------------
 135 $@
 136 ```
 137
 138 Save the file and run the following to start weston.
 139
 140 ```
 141 $ rvgpu-proxy -s 1080x1488@0,0 -n <IP_address_of_Receiver>:<Port_Number> &
 142 $ <WORKDIR>/run_remote_app.sh weston --backend drm-backend.so -Swayland-0 --seat=seat_virtual -i 0 &
 143 ```
 144 Replace the placeholders with the appropriate values:
 145 - `<Port_Number>`: Port set in the renderer.
 146
 147
 148 After completing these steps, the Weston screen from the Sender will be transferred and displayed on the Receiver using rvgpu-proxy and rvgpu-renderer. You can verify that everything is functioning properly by launching wayland applications on the Sender side, such as:
 149 ```
 150 $ weston-simple-egl -f
 151 ```
 152
 153 You can also verify the touch or keyboard operation support of the RVGPU by using app such as
 154 ```
 155 $ weston-smoke
 156 $ weston-editor
 157 ```
 158 **Note**: There are known issues with mouse, such as the cursor becoming invisible and occasional flickering of a green screen.
 159
 160 **Appendix**
 161 - By building the RVGPU on Ubuntu, it is possible to enable bidirectional remote rendering between the agl-demo-platform and Ubuntu.
 162 For the build procedure on Ubuntu, see the following URL: https://github.com/unified-hmi/remote-virtio-gpu
 163 The figure below shows an example where Ubuntu is used as the Sender and AGL as the Receiver, running RVGPU.
 164
 165 ![Using_RVGPU_from_Ubuntu_to_AGL](images/Unified-HMI/Using_RVGPU_from_Ubuntu_to_AGL.png)