7 When the AGL project was started, weston was chosen as the compositor, which is
8 the reference implementation of a Wayland compositor, while for window management
9 functionality it relied on *ivi-shell* (In-Vehicle Infotainment) together
10 with an extension, called [wayland-ivi-exension](https://github.com/GENIVI/wayland-ivi-extension).
12 A demo platform image of AGL comes with a handful of demo applications, done
13 with the Qt, which abstracts the protocol communication between the client and
14 the compositor. Additional functionality was in place under the form of
15 library, to control and signal back to the compositor when applications were
16 started, among other things.
18 Management of applications, starting, running and stopping them is done in AGL
19 with AFM [Application Framework Management](5_appfw.md),
20 which is an umbrella name to denote the suite of tools and daemons that handle
21 all of that. It is integrated with systemd and with the current security model,
22 SMACK (Simplified Mandatory Access Control Kernel), a Linux kernel security
23 module. Applications can use AFM to hang off data, and to pass it down to
24 other services. Together with AFM, and with the help of a library,
25 applications could tell the compositor which application to activate or to
29 ## Simplifying the graphical stack
31 Trimming down these abstractions, simplifying the way clients interact with the
32 compositor, and avoid using modules that aren't really maintained upstream were
33 the reasons behind looking at alternatives to ivi-shell. On the desktop,
34 [xdg-shell](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/stable/xdg-shell/xdg-shell.xml)
35 is currently de-facto protocol for handling all window management related
38 Wayland protocol has a window-like interface embedded into its protocol (called
39 wl_shell), but *xdg-shell* has long time ago deprecated it and instead
40 of adding it in the wayland protocol namespace, it was integrated
41 together with some other useful protocols, into
42 [wayland-protocols](https://gitlab.freedesktop.org/wayland/wayland-protocols)
43 project. The whole purpose of wayland-protocols is to enhance the Wayland
44 protocol with new functionality and bring new extensions entirely. Compositors
45 are free to implement, modify, enhance, and add new extensions to
46 wayland-protocols but they need to do so in consensus.
48 Besides the core wayland protocol and extended functionality from
49 wayland-protocols, a compositor can provide and implement additional protocol
50 extensions (custom to that compositor). By using such private extensions we
51 align with the AGL project and its requirements, without compromising specific
52 functionality and allows to add or improve the current ones. With that in mind,
53 the approach was to create a new compositor, called
54 [agl-compositor](https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-compositor)
55 and implement dedicated private extensions, rather than trying to modify weston
56 itself, which AGL project would have been required to keep and maintain for
59 ## A compositor based on libweston
61 The compositor used currently in AGL, just like weston, is built on top of
62 *libweston* and *libweston-desktop*. The latter, among other things, is required
63 as it provides the server side implementation of the xdg-shell protocol which
64 underlying toolkits (like Qt/Chromium project) makes use of to deliver
65 desktop-like functionality. The former is used to provide back-ends and
66 rendering support, effectively managing the HW, besides implementing the
69 The high-level goal of [libweston](https://wayland.pages.freedesktop.org/weston/toc/libweston.html) is
70 to decouple the compositor from the shell implementation.
72 Traditionally, clients were entirely separated from the window manager, the
73 desktop environment and the display server. In wayland all these are
74 conceptually under the same entity though they are implemented as different
75 (UNIX) processes, or as a different namespaces with front and back-end APIs,
76 exposed by libraries. The compositor and the shell driving the UI should be
77 seen as one and the same, and in practice, this happens on desktop
78 environments. For AGL, the shell client can be represented under different
79 forms, as well as the fact that the process management has another layer
80 baked-in to handle MAC (Mandatory Access Control) labels and use the
81 above-mentioned Application Framework Management. These are all tightly
82 integrated and therefore, the AGL compositor will not automatically start the
83 shell client, although there's code to handle that. One can modify the
84 configuration file, add the shell client path, and the compositor will attempt
89 Compositors can define and implement custom extensions to further control
90 application behaviour. For AGL, we have two private extensions defined.
91 One targeted at defining surface roles commonly found in desktop environments
92 (like panels, and backgrounds), which a shell client would bind to, and one
93 targeted at regular application(s) that might require additional functionality:
94 being able to display/activate its own surface or other's application surface,
95 implement some kind of split screen management of windows, or
96 dialog/pop-ups that exhibit always-on-top property even if the active
97 surface has been changed.
99 
101 Clients can make use of these private extensions to define other kind of roles
102 for instance dialog/pop-ups or full-screen roles, and split windows vertically or
103 horizontally. It includes the ability to activate other applications, assuming
104 that the surfaces have been created, and the capability of delaying
105 presentation for the client shell. Doing so, all the information is displayed
106 at once, rather than waiting for the toolkit to map/show the surface.
108 An application identification mechanism was required to be able to activate
109 other clients windows/surfaces. A string-based identifier name was chosen
110 which can be used by the client to set an application-based identifier using
111 the xdg-shell protocol. While there's nothing stopping the client to avoid
112 doing that, specifically, to avoid assigning an application identifier,
113 the compositor won't be able to find which surfaces matches to a particular
114 client, if one would want to activate/display it at some time in the future.
118 Client shellls can make use of this protocol to define panels and background
119 roles for different surfaces. It includes to ability to activate other
120 applications, assuming that those are already running. Activation happens by
121 using using the app_id, respectively using set_app_id request as defined by the
122 xdg-shell protocol. Established client-side implementation of the xdg-shelll
123 protocol will have a function exposed which can be used to set an application
124 identifier. Further more, the compositor will not present/display anything to
125 the user as long the `ready()` is not requested. So, after creating the surfaces
126 assigning them panel and/or background roles, and they're fully loaded,
127 the client can then issue `ready()` request and the compositor will start
130 Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell.xml;hb=refs/heads/master)
131 as that is the authoritative way of getting the latest version.
133 ### agl-shell-desktop
135 This extension is targeted at keeping some of the functionally already
136 established in AGL as to a) allow applications display/activate other
137 surfaces/application window, and b) set further roles, specially dialog/window
138 pop-ups and split-type of surfaces.
140 Clients can make use of this protocol to set further roles, like independently
141 positioned pop-up dialog windows, split type of surfaces or fullscreen ones.
142 Additional roles, and implicitly functionality can be added by extending the
143 protocol. These roles serve as hints for the compositor and should be used
144 before the actual surface creation takes place, such that the compositor can
145 take the necessary steps to satisfy those requirements.
147 Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell-desktop.xml;hb=refs/heads/master)
148 as that is the authoritative way of getting the latest version.
150 #### Additional surface roles in agl-shell-desktop
152 Like mentioned earlier, the compositor is already making use of some (internal)
153 roles, and with this extension we add some further ones. These are:
155 * split (there's vertical and a horizontal one)
159 Internally these are encoded with different values such that there's a
160 translation needed, between the protocol values and the internal ones. Besides
161 the roles, additional data can to be passed on, but only depending on the role.
162 It is highly recommend **to avoid** using the protocol to pass down information
163 between different applications using this communication channel. It is only
164 intended to help out with demo applications. Other sharing mechanism are
165 available in the AGL project that can satisfy those requirements.
167 #### Receiving application state events from (other) applications
169 agl-shell-desktop exposes two events which client can install handlers for, one
170 that signals when regular xdg application have been created, and one that
171 signals state changes (active/hidden) as well as destroyed/no longer present
172 surfaces. These events can be useful to add additional functionality if
175 #### Activating (other) applications
177 Both agl-shell and agl-shell-desktop have requests to activate other
178 application based on their xdg-shell app_id. In case the application is
179 present/running, it will attempt to make the surface backing that application
180 the current activate one, with each output having independently active
185 The activation and setting surface roles requires passing a Wayland output
186 (wl_output). The output is the wayland interface representation of an output
187 and is **mandatory** to pass it down to the compositor when activating a surface.
188 Clients can retrieve it (the output) if they wish to place the surface on other
189 outputs by using the toolkits that expose the Wayland objects. A human-like
190 representation is provided by either the toolkit, or by using other extensions
191 implemented by the client, for instance [xdg-output](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/unstable/xdg-output/xdg-output-unstable-v1.xml)
192 is the one recommended way and provides a mapping between a human
193 representation of the output and the wayland one.
195 One can also choose the output where the application can start, by configuring
196 directly the AGL compositor. Under the `[output]` section one can use
197 `agl-shell-app-id=appid` restart the AGL compositor unitd systemd service and
198 start the application. Currently this *only* applies to regular applications, the
199 client shell having to handle it in the code.
201 ## Available toolkits, application conversions and available eco-systems
203 Users and OEM vendors alike have the possibility, depending on their use-cases,
204 to either use some kind of a toolkit (Qt/GTK) for writing application,
205 or use plain C and the wayland library to communicate with the compositor
206 directly, without any toolkit abstraction.
208 Currently, the demo applications in AGL use the Qt platform with Chromium being
209 at this phase, a second-class citizen, being currently in the works of
210 achieving the same level of integration as QtWayland (the underlying library that
211 abstracts the Wayland interaction) has at this moment. The Qt platform has
212 long been favoured in embedded systems so it feels natural why AGL project
213 chose it over other alternatives. In the same time, as web applications are
214 for quite some time now permeating the application development scene, it also
215 felt natural to add support for a runtime that gives that option, which in AGL
216 was achieved with the help of the Chromium project.
218 For normal applications, not needing the ability to activate or displaying
219 other's application surface, would basically mean that it would use what the
220 toolkit has to offer, simplifying the application handling even more. Under
221 Qt, the client shell can use QPA (Qt Platform Abstraction) to gain access to
222 Wayland primitives, and implicitly is being able use the private extensions.
224 
226 On the Chromium side of things, that happens indirectly, as Chromium doesn't
227 expose the Wayland primitives. Not only that, but on the Chromium platform,
228 there's another mid-layer component, called [WAM](https://github.com/webosose/wam)
229 (WebApplicationManager) with the purpose of handling web applications life-cycle.
231 So, controlling and passing information from a web application, that resembles
232 that of a shell client, has to travel more than a few levels in the software
233 stack, until it reaches the lower layers in Chromium where the Wayland
234 communication and interaction takes place. Support for the private extension
235 was done at the Ozone interface abstraction, which Chromium projects uses now
236 to handle the display/graphical interaction with the lower stack levels.
238 ## Streaming buffers and receiving events to and from remote outputs
240 Quite a common feature, in the infotainment market, is the ability to stream
241 out buffers/data to remote outputs. For instance, super-imposing the navigation
242 application, between the speedometer and tachometer, in the IC (Instrument
243 Cluster) of a car is such a scenario. Just like weston, the AGL compositor is
244 capable of loading up libweston modules and make use of them. And just like
245 weston, the AGL compositor loads up the remoting-plugin to achieve the same
248 The remoting-plugin uses the DRM virtual output API from libweston together
249 with gstreamer pipeline to capture, using DMA buffers, the DRM output and to
250 stream it, remotely to another machine. They can be over the network, or
253 Further more, to cope with situations where the output is just a
254 panel/display, without some kind of compositor driving it, the necessity of
255 handling input events is an important feature to have, giving the user to
256 possibility to manipulate the application/environment as he or she seems fit.
257 The compositor loads a plug-in that streams out the buffers to an output
258 remotely, with [another plug-in](2_waltham-receiver_waltham-transmitter.md)
259 handling the input events. The events, which are sent back from the display to
260 the compositor, are generated with the help of wayland-eque protocol that works
261 over the network, called [Waltham](https://github.com/waltham/waltham).
263 Together, they provide proper means to achieve a seamless integration with
264 other display devices in the car cabin.
266 ## Policies and Role Base Arbitration
268 The compositor contains an API useful for implementing user-defined policies.
269 It contains a policy engine, and installs by default an allow-all kind of
270 policy. The policy engine controls if the client using the private extensions
271 is permitted to perform those calls. Not only that, but with some policies, it
272 won't allow the client to bind to the interfaces in the first place. That
273 happens with the deny-all policy, which is able to retrieve the client's
274 SMACK label and compares it with the ones statically defined.
276 In the works, there's a new policy model, called [Role Based
277 Arbitration](https://gerrit.automotivelinux.org/gerrit/admin/repos/staging/rba).
278 Internally, how it works, should be found at [RBA](3_rba.md).
279 While the other two policies are embedded into the compositor, the RBA policy
280 model is an off the-shell policy. Obviously, vendors and users can hook up
281 their own policies, just like RBA did. These all work towards satisfying
282 the driver distraction mitigation requirement for the AGL project, as to avoid
283 overwhelming the driver with too much information.
285 Users wanting to create their own policy should create a specialized version
286 of the callbacks defined in `struct ivi_policy_api`.
288 As there's no dynamic loading of policies you'll need to recompile the compositor
289 with that policy in mind, specifically like the following:
291 $ meson -Dprefix=/path/to/install-compositor/ -Dpolicy-default=my_policy build_directory
293 The default policy found in src/policy-default.c should more than sufficient to
294 get started on creating new ones. Users can either re-purpose the default
295 policy or create a new one entirely different, based on their needs.
297 These are hooks in place by the policy engine control the creation, committing
298 and activation of surfaces (`ivi_policy_api::surface_create()`,
299 `ivi_policy_api::surface_commited()`, `ivi_policy_api::surface_activate()`),
300 among other situations.
302 Users can customize the hooks by using some sort of database to retrieve the
303 application name to compare against, or incorporate some kind of policy rule
304 engine. Alternatively, one can use the deny-all policy engine which allows the
305 top panel applications to be used/displayed as permitted applications.
309 The policy engine is stateful, and allows the ability to inject back events,
310 such that it allows the user to add custom rules into a policy and, depending
311 on the event received by the policy engine, to execute a rule match for that
312 event. Further more, the framework allows adding new states and events and the
313 default implementation has code for handling events like showing or hiding the
314 application specified in the policy rule. The most common example to exemplify
315 this feature is the ability to show a custom application, like displaying the
316 rear view camera application, when the automobile has been put in reverse.
318 For deadling with these kind of rules, `ivi_policy_api::policy_rule_allow_to_add()`
319 can be used to control if policy rules could be added or not. Finally, we have
320 `ivi_policy_api::policy_rule_try_event()` which is executed for each policy
321 rule added, by using the policy API `ivi_policy_add()` function.
323 By default the policy framework it will add the 'show', and 'hide' events and
324 the 'start', 'stop' and 'reverse' states. An special type, assigned by default
325 is 'invalid'. A **state change** has to be propagated to the compositor, which can
326 happen by using `ivi_policy_state_change()` function, and which signals the
327 compositor the state change took place, in order to apply the policy rules, and
328 implicitly to call the event handler `ivi_policy_api::policy_rule_try_event()`.
330 ## Back-ends and specific options for agl-compositor
332 The compositor has support for the following back-ends:
334 * **DRM/KMS** - runs a stand-alone back-end, uses Direct Rendering Manager/Kernel
335 Modesetting and evdev, that is utilizes and runs on real or virtualized HW
337 * **Wayland** - runs as a Wayland application, nested in another Wayland compositor
339 * **X11** - run as a x11 application, nested in a X11 display server instance
341 ### Building and running the compositor on different platforms
343 The compositor can run on desktop machines as easily as it does on AGL
344 platform. It should infer, depending on the environment, if it is being
345 compiled with the AGL SDK, or with the host build system. Running would also
346 be inferred from the environment.
348 The compositor has some additional configuration options like:
350 * `--debug` - enables the screenshooter interface, useful if one would want to
351 take a screenshot using `agl-screenshooter` client. This might be seen as a
352 security risk to it only be enabled in the AGL platform if built with agl-devel
355 Additional configuration ini options have been added to help with the CI
356 integration. Worth mentioning are:
358 * `activate-by-default=[true]` - if the surface of the client should be
359 displayed when the application started. Present in the `[core]` section.
360 By default set to `true`. Setting it to `false` will not activate,
361 by default, the client's surface when started.
362 * `hide-cursor=[false]` - do not advertise pointer/cursor to clients. Present
363 in the `[core]` section.
365 ## Running with software rendering
367 By default the compositor will attempt to use the GL-renderer, and implicitly
368 the GPU. One could instead use the CPU, by making use of the Pixman library. To
369 use it in the compositor append `--use-pixman` to the command line. This purely
370 software approach has the benefit that would not rely at all on any GL
371 implementatation or library. In constrast, even if the GL-renderer is used,
372 in some situations it won't be able to use the GPU supported implementation
373 and fallback to sofware based one, and for instance that might happen when
374 running in virtualized environments.
376 Both approaches could end up not actually using the GPU, but the latter does
377 actually use the GL library and perform the operations in software, while the
378 former does not use any GL whatsover. All back-ends support disabling the
379 GL-render to make sure it does not interfere with the composing process.
381 ## Multiple output set-up and touch input devices
383 There's no deterministic way in which the compositor enables the outputs and
384 depending on the input devices, specifically touch input devices, and the way
385 the connectors are wired, a touch input device might be associated with a
386 different output than the one intended.
388 A consistent way, that survives a reboot, is to use
389 [udev rules](https://man7.org/linux/man-pages/man7/udev.7.html), which
390 libweston would be able to use such that a particular output is tied/associated
391 to a particular touch input device.
393 For instance, assuming that you have a set-up consisting of 4 outputs, a 4
394 touch input devices, when the outputs are being enabled the compositor
395 front-end will associate all 4 touch input device -- if they haven't been
396 previously being associated to a particular output, to the first enabled
399 In order to avoid that, and associate each touch input device to
400 their respective output an udev rule can be installed, for the default
401 seat (named `seat0`).
403 Example of a udev rule:
406 SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004a", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-1"
407 SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004b", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-2"
408 SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004c", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-3"
409 SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004d", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-4"
412 Add the following under `/etc/udev/rules.d/91-output.rules` and reload udev
413 rules for these changes to take effect:
415 $ udevadm control --reload-rules && udevadm trigger
417 Note that in the above example, we use physical seat, named `seat0` which is
418 the default physical seat. You can verify that these changes have been applied by
419 checking the compositor logs (under `/run/platform/display/compositor.log` file)
420 You should be seeing `CONNECTOR-NO by udev` message like the following:
423 associating input device event0 with output HDMI-A-1 (HDMI-A-1 by udev)
429 associating input device event0 with output HDMI-A-2 (none by udev)
432 where the rules are either incorrect or badly written.
434 Retrieving device attributes could be done archaically using `lsusb` or `lspci`
435 or using `udevadm info -a /dev/input/event*` which can provide with a multitude
436 of attributes to use. In our above example we only relied `idVendor` and
437 `idProduct` but potentially other attributes might be used.
Code Review / AGL / documentation.git / blob