1 # **Window Manager Application Guide**
3 <div align="right">Revision: 0.7</div>
4 <div align="right">TOYOTA MOTOR CORPORATION</div>
5 <div align="right">12th/Oct/2018</div>
9 <div id="Table\ of\ content"></div>
13 - [Target reader of this document](#Target\ reader\ of\ this\ document)
14 - [Overview](#Overview)
15 - [Supported usecase](#Supported\ usecase)
16 - [Getting Started](#Getting\ Started)
20 - [Enable to call Window Manager](#Enable\ to\ call\ Window\ Manager)
21 - [Software Architecture](#Software\ Architecture)
22 - [Sequence](#Sequence)
23 - [API reference](#API\ reference)
24 - [Request to Window Manager](#Request\ to\ Window\ Manager)
25 - [Event from Window Manager](#Event\ from\ Window\ Manager)
26 - [Client Library](#Client\ library)
27 - [Sample code](#Sample\ code)
28 - [Policy Manager](#Policy\ Manager)
29 - [Enabling split](#Enabling\ split)
30 - [Release Note](#Release\ Note)
34 <div id="Target\ reader\ of\ this\ document"></div>
36 ## Target reader of this document
38 This document is intended for developers and system integrators who
39 need to know, how Window manager works and how it is to be used.
41 ### Scope of this Document
43 This document covers Window manager that was implemented for TMC and
44 delivered to the Automotive Grade Linux (AGL) project. It includes its
45 implementation details, concepts of operation, configuration and usage.
49 - document of the underlying architecture, see
50 [HMI-Framework](https://wiki.automotivelinux.org/hmiframework).
52 - document of the AGL application framework and its technologies,
54 Framework](https://wiki.automotivelinux.org/agl-distro/app-framework).
56 - document of HomeScreen, see
57 [HomeScreen](http://docs.automotivelinux.org/docs/apis_services/en/dev/reference/hmi-framework/3_1-HomeScreen-Guide.html).
59 It is highly recommended to have a good understanding of these documents
60 and projects before using Window manager.
64 <div id="Overview"></div>
68 Window Manager is the service process which provides **window management based on policy**.
69 And implements a layout switching of applications on
70 multiple layers and with different layer layouts.
71 Window Manager is based on ivi layer management from GENIVI and AGL application framework.
73 Window Manager consists of
76 - service binding library
77 - shared library for policy management
80 In order to understand Window Manager, the below figure shows the one of typical usecases.
81 In this example, there are two mode for window management.
83 1. Window Management in `Car Stops`
84 1. Window Management in `Car Runs`
86 ![Figure: Typical usecase](parts/state_change_example.png)
88 The important points are:
90 - **Window transition should be done by Window Manager**
91 Window Manager switch application displayed on top layer by user operation(touch shortcut button).
92 In this example, when an user touches `navigation` shortcut button, Window Manager displays `navigation` and hide `launcher`. Next, when an user touches `videoplayer` shortcut button, Window Manager divides a screen into two parts and display two applications.
94 - **There is a priority `role` for each application.**
95 Window Manager realizes state transition change based on the policy which consists of `role`.
96 According to the state transition table, it controls the visibility of application window, layout change, and so on.
97 The reasons why this is necessary are
99 - to support user driving
100 - not to disturb a driver concerns on driving for safety
102 In this example, for safety, when the car starts running, Window Manager set up the role `videoplayer`
103 to be masked and uncontrollable by user not to disturb driver concerns.
104 And, for supporting driving, set up `navigation` role to be displayed 3 seconds after the car ran.
105 In `Car Run` state, the user can't switch to other application from Navigation application until car stops.
107 <div id="Supported\ usecase"></div>
112 - When an user chooses a different application, Window Manager changes the layout and then displays the application.
113 - When an user chooses a different application, Window Manager changes the layout and then hides the displayed application.
115 - Window Manager changes layout according to policy table based on `role`.
116 3. Define Layout by `area` configuration
117 - Window Manager realizes the abstracted `area` and can resize the window by using it. User can easily edit this configuration.
121 <div id="Getting\ Started"></div>
125 <div id="Build"></div>
130 git clone https://gerrit.automotivelinux.org/gerrit/apps/agl-service-windowmanager
131 cd agl-service-windowmanager
134 source <your SDK path> // normally this is /opt/agl-sdk/environment
140 The widget package is populated in the 'package' directory.
144 root windowmanager-service.wgt
147 <div id="Install"></div>
151 Copy windowmanager-service.wgt to the file system then execute the following command.
154 afm-util install windowmanager-service.wgt
157 <div id="Bitbake"></div>
161 You can make Window Manager object files with the following two stage operations.
168 repo init -u https://gerrit.automotivelinux.org/gerrit/AGL/AGL-repo
175 source meta-agl/scripts/aglsetup.sh -m m3ulcb agl-demo hmi-framework
176 bitbake agl-demo-platform
181 <div id="Enable\ to\ call\ Window\ Manager"></div>
183 ## Enable to call Window Manager
185 To call Window Manager, it is important to enable the permission from security framework.
186 To use Window Manager API, an application or a service shall add the following configuration definition into "config.xml" of your application.
189 <feature name="urn:AGL:widget:required-api">
190 <param name="windowmanager" value="ws" />
194 To call Window Manager function easily, Window Manager provides a library which is called "libwindowmanager".
195 This library provides a function style API calling interface.
196 So you can include the libwindowmanager.hpp header file, and can link against this library.
197 Please also refer to the sample application.
199 See also our [Sample code](#Sample\ code).
203 <div id="Software\ Architecture"></div>
205 # Software Architecture
207 The static relationship with other components is shown below.
208 The actual logic of Window Manager is the binding in Binder(afb-daemon).
209 Window Manager is based on AGL application framework,
210 so the IPC via websocket is protected by AGL application framework.
212 The upper binder is for the application side security context.
213 The lower binder is the Window Manager for the server side security context.
214 Usually an application side binder has some business logic for each application, so the number of binders depend on the number of applications which use Window Manager.
215 On the other hand, regarding lower binder there is only one module in the system. This binder receives messages from multiple applications.
217 An application can use libwindowmanager.so to call Window Manager API simply.
219 Window Manager is based on the GENIVI layer management system.
221 ![software-stack.png](parts/software-stack.png)
225 Layers are entities that means the application stack group defined in `layers.json`.
226 This doesn't mean layer ID defined in GENIVI ivi layer.
227 The layer ID is used for application itself in Window Manager.
228 Currently, application can't have multiple surfaces in Window Manager.
230 <div id="Layers"></div>
234 Surfaces are *placed* on layers . The surface
235 will then be resized to dimensions, according to the name of `areas.db`
236 application requests by `activateWindow` or policy management.
237 As default, `normal.full` is set by libwindowmanager for native application.
239 <div id="Configuration"></div>
243 The window manager is configured with the *layers.json*, *areas.db*, *roles.db* configuration
244 files. By default they are searched in `${AFM_APP_INSTALL_DIR}/etc/`.
246 Sample configurations are provided with the window manager
247 implementation, these samples are installed to ${AFM_APP_INSTALL_DIR}/etc/ .
249 This configuration is supposed to be configured by policy designer which means OEM or Tier1.
253 `layers.json` has three roles.
254 First, to create application containers `Layer`.
255 Second, to set id range for applications.
256 Third, to attach application to `Layer` according to the role application requests.
258 The sample configuration is here
262 "comment": "Surface ID to Layer ID mapping",
265 "surface_role": "HomeScreen",
266 "comment": "This surface should never be made invisible (The HomeScreen)"
271 "role": "BackGroundLayer",
272 "name": "BackGroundLayer",
274 "comment": "Single BackGround layer map for the map, radio, music and video"
277 "role": "homescreen",
278 "name": "FarHomeScreen",
280 "comment": "FarHomeScreen is the part of HomeScreen. The z order of this layer is lower than NearHomeScreen"
283 "role": "music|video|browser|radio|phone|map|hvac|settings|dashboard|poi|mixer|sdl|launcher|fallback",
286 "comment": "Range of IDs that will always be placed on layer 1001"
289 "role": "^on_screen.*",
292 "comment": "Range of IDs that will always be placed on the OnScreen layer, that gets a very high 'dummy' id of 9999"
298 Each mapping defines the following items to map corresponding surfaces
301 - `role` defines what kind of ability the application has. And the application will be attached to `Layer` according to the `role`.
302 A regular expression that application drawing names
303 are matched against. If applications match this regular expression,
304 the surface will be visible on this layer.
306 - `name` is just a name definition for `Layer`, it has no
307 functional use apart from identifying a layer with a name.
308 - `layer_id` is the id used in GENIVI IVI layer management control.
310 `Layer` stacks from beginning to end.
311 The above `Layer` example image is below.
313 ![wm_layer_stack.png](parts/wm_layer_stack.png)
316 "fallback" role is the special role. This role is set if the role application requests doesn't exist
317 in `layers.json`. Then, Window Manager will accept any applications.
318 If the "fallback" is not set in layers.json, window manager blocks the application displaying in such case.
319 In such a situation, you have to add your role(application name) at "role" in layers.json.
322 `BackGroundLayer` name of `Layer` is exception for work around. This layer is fallback layer not to stop event loop of application when it becomes invisible.
323 The problem is issued in <https://jira.automotivelinux.org/browse/SPEC-1484>.
325 <div id="Configuration\ Items"></div>
329 Area means abstract expressions of 2-dimensional size and position.
330 areas.db defines the area which an application is set.
336 "name": "fullscreen",
345 "name": "normal.full",
354 "name": "split.main",
375 The image of the above setting is described below.
376 ![wm_area.png](parts/wm_area.png)
378 - `name` is an abstract data of rectangle.
380 - `rect` has 4 arguments. `x`, `y` means the offset from (0, 0) of screen.`w` means the width of the area, and `h` means the height of the area. The dimensions can be specified relative to the screen dimensions.
382 The dimensions can be specified absolute to the screen dimensions. But if `fullscreen` is not suitable to screen dimensions, Window Manager scales the area automatically.
385 `fullscreen` must be set because this is the base size of scaling in Window Manger.
388 The direction of the coordinates depends on `transform` in weston.ini.
389 Currently, agl-demo-platform set `transform=270`.
390 This suppose to use screen vertically.
396 <div id="Sequence"></div>
400 To understand the sequence between application and window manager, refer to the [spec document](https://wiki.automotivelinux.org/hmiframework).
402 The typical sequence to render your application, follow the sequence below.
404 1. Register your role (and request surfaceID)
406 ![request_role.png](parts/request_role.png)
408 The above sequence is the initialization phase of your application to use Window Manager.
409 An Application has to register your `role` to Window Manager. For ivi-shell application, Window Manager generates surfaceID to input it into the function
411 And also it is important for synchronization to get `syncDraw` event for receiving the request for resize and redraw, and notifying Window Manager of `endDraw`, so register callback function with setEventHandler for `syncDraw`.
413 [requestSurface](#requestSurface)
414 [setEventHandler](#wm_subscribe)
416 setEventHandler is API of libwindowmanager. This calls wm_subscribe internally.
418 2. Display your window
420 ![hmi_framework_designed_seq_toyota.png](parts/hmi_framework_designed_seq_toyota.png)
422 To display your window, your application has to request `activateWindow` with `role` and `area` to Window Manager.
423 Window Manager checks the app should be visible on the `area` according to the policy table using `role` .
424 If it is accepted, afb_req_success will be returned, and next Window Manager
425 will push the event `syncDraw` to applications which will be displayed.
426 If it is denied, afb_req_fail will be returned.
427 In this sample sequence, `syncDraw` is emitted to the apps who requested only,
428 but this shall be emitted to other applications whose size shall be changed.
430 [activateWindow](#activateWindow)
431 [syncDraw](#syncDraw)
433 [flushDraw](#flushDraw)
435 3. Activate OnScreen Window
437 ![deactivate_window.png](parts/deactivate_window.png)
439 [deactivateWindow](#deactivateWindow)
440 [See sample code for more detail about OnScreen Window.](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps%2Fonscreenapp.git;a=summary)
442 The above sequence shows the sample of OnScreen Window.
443 If the role is high priority than NormapApp, Window Manager rejects NormalApp
444 request when OnScreenApp is displayed.
446 Note : Above repository is currently empty, so please refer to the sandbox branch.
450 <div id="API\ reference"></div>
454 ## Request to Window Manager
456 | Use | verb | version |
458 | Initialize | requestSurface | from 0.7 |
459 | | wm_subscribe | from 0.7 |
460 | | requestSurfaceXDG | from 0.7 |
461 |Activate/Deactivate| activateWindow | from 0.7 |
462 | | deactivateWindow | from 0.7 |
463 | | endDraw | from 0.7 |
464 | Get Infomation | getDisplayInfo | from 0.7 |
466 Note: We created this table from 0.7
468 The data of IPC via websocket consists of JSON.
469 This section describes the verb of API and key.
470 Normally, the body of requesting API will be here.
472 <div id="Request\ to\ Widnow \Manager"></div>
476 <div id="requestSurface"></div>
480 Register your role to Window Manager and get surfaceID for ivi-shell.
481 The role is used for policy management.
482 SurfaceID is supposed to be set to the API `ivi_application_surface_create` of ivi-application protocol or set it to environment variable `QT_IVI_SURFACE_ID` if your app is Qt and integrate ivi-shell.
484 - verb : "requestSurface"
485 - argument : {"drawing_name":"your role"}
487 the value must be selected in layers.json.
493 "drawing_name" : "navigation"
497 ### *requestSurfaceXDG*
499 This API is for XDGLauncher, so it is not necessary for normal application.
500 XDGLauncher is created for XDG application for desktop app without editing for HMI-Framework.
501 Please see the repository in detail.
502 <https://gerrit.automotivelinux.org/gerrit/gitweb?p=staging%2Fxdg-launcher.git;a=summary>
504 <div id="wm_subscribe"></div>
508 Subscribe the Window Manager's event.
509 Application must subscribe `syncDraw` event.
511 - verb : "wm_subscribe"
512 - argument : {"event" : *event number*}
522 The event is abstracted with a number (enumeration).
532 | 6 | "screenUpdated" |
534 ## Activate/Deactivate
536 <div id="activateWindow"></div>
540 Request to display your application with `role` on the `area` to Window Manager.
541 Window Manager checks the app should be visible on the `area` and change layout according to the policy table using `role` .
542 If it is accepted, afb_req_success will be returned, and next Window Manager
543 will push the event `syncDraw` to applications which will be displayed.
544 If it is denied, afb_req_fail will be returned.
546 - verb : "activateWindow"
547 - argument : {"drawing_name" : "your role", "drawing_area" : "your area"}
549 the value must be selected among layers.json.
555 "drawing_name" : "navigation",
556 "drawing_area" : "normal.full"
560 <div id="deactivateWindow"></div>
562 ### *deactivateWindow*
564 Request to hide your application to Window Manager.
565 This verb is supposed to be used by high priority application which
566 are for example popup application or system UI application such like alert.
567 If Window Manager set the priority of popup high in the policy, Window Manager may not hide the popup even if normal applications
568 send `activateWindow` until popup application send `deactivateWindow` . This behavior depends on the policy table.
569 After this request, Window Manager checks which app should be visible
570 and change layout according to the policy table.
572 - verb : "deactivateWindow"
575 <div id="endDraw"></div>
579 Notify Window Manager of application finishes drawing.
580 This function must be sent in event `syncDraw`.
581 Otherwise, Window Manager will roll back to previous state and reject your request `activateWindow` .
584 - argument : {"drawing_name" : "your role"}
590 "drawing_name" : "navigation",
598 Get screen information such as resolution.
600 - verb : "getDisplayInfo"
603 Return : The screen information will return.
615 "jtype" : "afb-reply",
619 "uuid":"05ae219a-0e56-4f46-af9f-3186a18cb110"
625 "width_mm", "height_mm" is from output which is one of the wayland object.
626 These items lack reliability, so recommend not to use.
628 <div id="Event\ from\ Window \Manager"></div>
630 ## Event from Window Manager
632 | Number | Event | version |
634 | 0 | "active" | from 0.7|
635 | 1 | "inactive" | from 0.7 |
636 | 2 | "visible" | from 0.7 |
637 | 3 | "invisible" | from 0.7 |
638 | 4 | "syncDraw" | from 0.7 |
639 | 5 | "flushDraw" | from 0.7 |
640 | 6 | "screenUpdated" | from 0.7 |
642 Events also consists of JSON.
643 The data of event is contained in `data` such like
647 "event":"windowmanager\/active",
649 "drawing_name":"navigation"
655 "event" is the event name.
656 "data" is the data object from Window Manager and contains
657 the message of event.
658 This section describes "event" and the contents of "data".
662 This event means when the application becomes active state.
668 "event":"windowmanager\/active",
670 "drawing_name":"launcher"
679 This event means when the application becomes inactive state.
685 "event":"windowmanager\/inactive",
687 "drawing_name":"launcher"
696 This event is issued when the application is visible state.
702 "event":"windowmanager\/visible",
704 "drawing_name":"launcher"
713 This event is issued when the application is invisible state.
719 "event":"windowmanager\/invisible",
721 "drawing_name":"launcher"
728 <div id="syncDraw"></div>
732 This event is issued by Window Manager state change operation in policy to the following cases.
734 - Your app requested `activateWindow` then your application will be resized or visible.
735 - Other app requested `activateWindow` then your application will be resized or visible.
736 - Window Manager change layout due to vehicle condition.
738 This event is the requests from Window Manager to
740 - request your app to callback `endDraw` to Window Manager.
741 - request your app to resize and redraw according to "drawing_area".
743 This is the abstract word then the real size is given in "drawing_rect".
749 "event":"windowmanager\/syncDraw",
751 "drawing_name":"radio",
752 "drawing_area":"normal.full",
764 An application which gets this event must send `endDraw`.
765 For details, please see the sequence.
767 <div id="flushDraw"></div>
771 This event is issued after Window Manager receives all `endDraw` from applications who recieved `syncDraw` .
772 After this event, Window Manager expects applications to update its surface.
778 "event":"windowmanager\/flushDraw",
780 "drawing_name":"launcher"
789 This event is issued after the visible application changes as a state transition change. This contains resized applications and visible applications. This event is issued to all subscriber.
790 Typical usecase is only for HomeScreen. If HomeScreen has an animation until the started application is visible such as progress bar, this signal may become a switch to stop the animation.
794 "event":"windowmanager\/screenUpdated",
805 "ids" is the application_id described in config.xml of application.
807 <div id="Client library"></div>
811 A client library implementation that internally uses the *libafbwsc*, is
812 provided in the `libwindowmanager`.
813 This library is for C++ native application.
815 Regarding more detail, please refer to <https://gerrit.automotivelinux.org/gerrit/gitweb?p=src%2Flibwindowmanager.git;a=summary>
819 <div id="Sample\ Code"></div>
823 In order to enable application to activate application(render on layer),
824 above described steps need to be implemented.
826 As a minimal example the usage and initialization can look like the
829 Repo: `git clone https://gerrit.automotivelinux.org/gerrit/src/libhomescreen`
830 Path: `sample/simple-egl/main.c`
831 Typical implementation of C++ application.
833 Repo: `git clone https://gerrit.automotivelinux.org/gerrit/apps/radio`
834 Typical implementation of Qt application.
836 Repo: `git clone https://gerrit.automotivelinux.org/gerrit/apps/videoplayer`
837 This is the good example to write more simply for Qt application using QtAGLExtra.
839 <div id="Policy Manager"></div>
845 Policy Manager decides next layout by using input event data and current state
846 based on the policy table.
848 And PolicyManager is plugin for WindowManager.
849 Therefore the OEMs can replace it.
851 <div id="Enabling\ split"></div>
855 Window Manager supports split layout to change policy and `areas.db`.
856 This section describes how to play split layout.
857 The sample image is here.
859 ![example_split.png](parts/example_split.png)
861 To play the split layout,
863 1. Edit in `policy_manager/CMakeLists.txt` as follows:
866 This results in using source code generated by ZIPC.
867 1. Set bool value "ON" to TRY_SPLIT_LAYOUT at line 28 in policy_manager/CMakeLists.txt as follows:
868 set(TRY_SPLIT_LAYOUT ON CACHE BOOL "Enable to show split layout")
870 1. copy window manager to your board
871 1. re-install windowmanager and reboot
873 As a result, if application requests `navi` with `activateWindow` when current layout is `video` or `mediaplayer`, Window Manager change layout to split window. The reverse is true.
876 Currently, the policy manager force application change the size even if the application which has the role doesn't have the split design.
877 In that case, the view of application may be ugly.
878 Window Manager supposes that applications may have multi designs according to system design by OEM.
879 For example, if OEM sets 2 pattern layout for `navi`, the application which requests `navi` should have 2 pattern designs.
883 <div id="Release\ Note"></div>
895 - Only single-surface Qt applications are support through the
896 libwindowmanager library. This is a limitation of how Qt creates surface
897 IDs for the ivi-application interface.
899 - Currenly, Window Manager supports only one screen. Dual display is not supported.
900 - As implemented in sample code, Qt application has to wait requesting `activateWindow` until `frameSwapped` is emitted.
901 - Qt application conceals, wayland and openGL processes, so it is difficult to call `swapBuffer` after `flushDraw` event described in the architecture document. But no problem if you use toolkit such as Qt, because it is automatically processed between applications and compositor(weston).
902 - Editing ZIPC is difficult for open source developer due to licence.