X-Git-Url: https://gerrit.automotivelinux.org/gerrit/gitweb?a=blobdiff_plain;f=docs%2F06_Component_Documentation%2F02_agl_compositor.md;h=67b73da84a42504e53ea65e44c4d520b1c4af920;hb=refs%2Ftags%2Fpike_16.0.5;hp=9e62890f035febbcb32acd7beb979afdaf6835c2;hpb=fe5a8e87f9f2c57ced90e898ebb1af10f92bcb35;p=AGL%2Fdocumentation.git diff --git a/docs/06_Component_Documentation/02_agl_compositor.md b/docs/06_Component_Documentation/02_agl_compositor.md index 9e62890..67b73da 100644 --- a/docs/06_Component_Documentation/02_agl_compositor.md +++ b/docs/06_Component_Documentation/02_agl_compositor.md @@ -191,7 +191,81 @@ handle any window management interaction on its own. Further more, with this protocol update we also includes some events already present in the agl-shell-desktop protocol like deactivate and terminate. -### agl-shell-desktop +#### V4 updates + +Version 4 of the agl-shell protocol brings a new request, +"set_activate_region". This is a hint for the compositor to use a custom area +for activation, rather than inferring the activation area from the panels. +This is a work-around designed for toolkits which can't handle multiple +surfaces at the same, such that it can this achieve the same functionality as +having multiple surfaces, but instead have just a single main surface to +handle. The compositor will place other applications in the designated +activation region, and will activate/deactivate surfaces only that in region. +With a custom activation area that leaves portions of the output available to +the toolkit this means it can achieve a similar graphical outcome as if there +are panels. All the toolkits, including Qt are now using this approach, with only +CI using distinct surfaces with panels. + +#### V5 updates + +Version 5 of the agl-shell protocol brings a new request, "deactivate_app". +Similar to the activate request, this will hide the current active application. +Depending on the window role, this request will either display the previously +active window (or the background in case there's no previously active surface) +or temporarily (or until a 'activate_app' is called upon) hide the surface. + +#### V6 updates + +Version 6 of the agl-shell protocol, brings two new request, "set_app_float" and +"set_app_normal". The first one is a method to change the surface role, either +on the fly, or at start-up to behave like a dialog/pop-up window, which can be +positioned anywhere on the output. The later one, is useful to return the main +original surface role, as maximized. + +#### V7 updates + +Version 7 of the agl-shell protocol, brings one new request, "set_app_fullscreen". +This is similar to "set_app_float", but it changes the surface role to fullscreen. +Going back to the orignal, maximized role should be with "set_app_normal". + +#### V8 updates + +Version 8 of the agl-shell protocol, adds a new request, "set_app_output", and +a new event, "app_on_output". This request/event is useful to migrate +applications from one output to another and signal out when that happens. + +#### V9 updates + +Version 9 of the agl-shell protocol, adds a new request, "set_app_position". This +request only makes sense for float type of surface, which was previously set with +"set_app_float". + +#### V10 updates + +Version 10 of the agl-shell protocol, adds a new request, "set_app_scale". +Similar to the "set_app_position", the surface role must be a floating type, +before using this request. The app scale request is actually a resize +hint for the client to change its dimensions to the one specified in the +request arguments. The client should use [wp_viewporter +interface](https://wayland.app/protocols/viewporter#wp_viewporter) to perform +cropping and scaling. + +#### V11 updates + +Version 11 of the agl-shell protocol, adds a new request "set_app_split". This +request only handles a single level of tiling for practical reasons as to keep +implementation simple and straight forward. Apart from the usual, tile +horizontally, and vertically, two arguments can be used to customize handling. +One defines the width of the split window, such that clients can specify, in +absolute values, the size of the split window. The other is about keeping the +split window always present when activating other windows. + +By default when activating a new windows, the split window roles are destroyed +and new activated window will be put in front. Making the split window sticky +would avoid doing that. Obviously trying to activate a window already active, +which has the sticky role won't be possible (it is already displayed). + +### agl-shell-desktop (deprecated, will be removed in future, see gRPC proxy) This extension is targeted at keeping some of the functionally already established in AGL as to a) allow applications display/activate other @@ -241,6 +315,158 @@ present/running, it will attempt to make the surface backing that application the current activate one, with each output having independently active surfaces. +## gRPC proxy for window management + +The gRPC proxy is an alternative to allow management of windows, +without the need for clients to "speak" native Wayland C code. Most major +languages have RPC bindings, and in AGL clients do not touch Wayland code +natively, using most of the time toolkits. It seemed reasonably to use gRPC +as a way to interact with the compositor. The way this works is that +there's proxy daemon started by the compositor which translates the gRPC +calls to Wayland ones, using agl-shell protocol. Events coming from the +compositor are handled by subscribing to RPC stream events, such that +clients can act upon as well, so it is not just one way interaction. + +With respect to window management: placement, movement, and surface role +changes all should, and can be done with RPC. Some of the calls, specifically +background set-up or activation area, are still required to happen prior or +require wayland primitives which can't be passed between processes, so those +can't be used with RPC proxy. + +In terms of API, the following are available: activation/deactivation, +role management, and events. All the message arguments are self-explanatory, +requiring always an app_id, in form of string and possibly an output, in +form a string as well. Toolkits can set/retrieve this information so refer +to the respective toolkit. At the wayland level, the app_id is set/get +with [xdg-shell](https://wayland.app/protocols/xdg-shell#xdg_toplevel:request:set_app_id) +whereas output from [wl_output interface](https://wayland.app/protocols/wayland#wl_output:event:name) + +Activation and deactivation of applications: + +``` +rpc ActivateApp(ActivateRequest) returns (ActivateResponse) {} +rpc DeactivateApp(DeactivateRequest) returns (DeactivateResponse) {} +``` +With the following message arguments: + +``` +message ActivateRequest { + string app_id = 1; + string output_name = 2; +} + +message ActivateResponse { +}; + + +message DeactivateRequest { + string app_id = 1; +}; + +message DeactivateResponse { +}; +``` + +Management of surface roles can be done with these requests: + + +``` +rpc SetAppSplit(SplitRequest) returns (SplitResponse) {} +rpc SetAppFloat(FloatRequest) returns (FloatResponse) {} +rpc SetAppFullscreen(FullscreenRequest) returns (FullscreenResponse) {} +rpc AppStatusState(AppStateRequest) returns (stream AppStateResponse) {} +rpc GetOutputs(OutputRequest) returns (ListOutputResponse) {} +rpc SetAppNormal(NormalRequest) returns (NormalResponse) {} +rpc SetAppOnOutput(AppOnOutputRequest) returns (AppOnOutputResponse) {} +rpc SetAppPosition(AppPositionRequest) returns (AppPositionResponse) {} +rpc SetAppScale(AppScaleRequest) returns (AppScaleResponse) {} +``` + +Message arguments are: + +``` +message SplitRequest { + string app_id = 1; + int32 tile_orientation = 2; + int32 width = 3; + int32 sticky = 4; + string output_name = 5; +} + +message SplitResponse { +}; + +message FloatRequest { + string app_id = 1; + int32 x_pos = 2; + int32 y_pos = 3; +}; + +message FloatResponse { +}; + +message NormalRequest { + string app_id = 1; +}; + +message NormalResponse { +}; + +message FullscreenRequest { + string app_id = 1; +}; + +message FullscreenResponse { +}; + +message AppOnOutputRequest { + string app_id = 1; + string output = 2; +}; + +message AppOnOutputResponse { +}; + +message AppPositionRequest { + string app_id = 1; + int32 x = 2; + int32 y = 3; +}; + +message AppPositionResponse { +}; + +message AppScaleRequest { + string app_id = 1; + int32 width = 2; + int32 height = 3; +}; + +message AppScaleResponse { +}; +``` + +Events which clients can subscribe to: + +``` +rpc AppStatusState(AppStateRequest) returns (stream AppStateResponse) {} +``` + +With the message arguments: + +``` +message AppStateRequest { +}; + +message AppStateResponse { + int32 state = 1; + string app_id = 2; +}; +``` + +A C++ client implementation the gRPC API can be found in [agl-shell-activator](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-shell-activator.git;a=tree;h=refs/heads/master;hb=refs/heads/master) +or in [window-management-client-grpc](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/window-management-client-grpc.git;a=tree;h=refs/heads/master;hb=refs/heads/master). + ## Explicit output The activation and setting surface roles requires passing a Wayland output @@ -296,6 +522,48 @@ communication and interaction takes place. Support for the private extension was done at the Ozone interface abstraction, which Chromium projects uses now to handle the display/graphical interaction with the lower stack levels. +## How to integrate or incorporate your own UI + +The Minimum Viable Product (MV) to be able to switch/change/replace the current +UI, depending on toolkit is to call `set_background()` and `set_ready()` requests +from the agl-shell client protocol. + +This means that the toolkits need to provides access to Wayland primitives, +exposing them in a such that they can reach the client code. For instance, +Qt uses [QPA](https://wiki.qt.io/Qt_Platform_Abstraction), while +GTK can also expose it through similar ways. +Chromium/CEF and flutter platform do not explicitly expose the windowing system +(and implictly Wayland) and have that implemented at a lower level. + +Further more, depending on the needs, one would also need to either use the +gRPC proxy API or just agl-shell protocol on its own. For instance for Qt and +flutter we now use a combination of both. In Qt, we use QPA and Wayland native +code to set the [background surface](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/homescreen.git;a=blob;f=homescreen/src/main.cpp;h=a98a15bb0113f3b28c1766e79c5d3f2d0b20fac4;hb=refs/heads/master#l255), +while activation/deactivation and anything else is handled using gRPC API, +but in the [same application](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/homescreen.git;a=blob;f=homescreen/src/main.cpp;h=a98a15bb0113f3b28c1766e79c5d3f2d0b20fac4;hb=refs/heads/master#l355). + +In flutter because we can't reach Wayland code +from within the client, we handle the Wayland part in the +[flutter embedder](https://github.com/toyota-connected/ivi-homescreen/blob/agl/shell/wayland/window.cc#L95-L121) +whereas the activation is handled in [flutter-ics-homescreen](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/flutter-ics-homescreen.git;a=blob;f=lib/data/data_providers/app_launcher.dart;h=8762643dba7f5a6e3ad2051749e30239743e759a;hb=HEAD) + +Similarly, CEF/Chromium has the same thing, the background and ready to present +request is handled at the lower levels. + +### Simple shell client examples + +An alternative to using toolkits is to avoid using any of them and +instead use native Wayland C code to create a simple shell client. This +means the client needs to manage everything, including redrawing the +background surface. + +An example of that is +[native-shell-client](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/native-shell-client.git;a=summary) +that is being used in the kvm images as a barebone shell client. Because that +just sets a black background matching the entire output you can have a +dedicated client that basically displays or mimics being fullscreen +(although technically it's set to maximized). + ## Streaming buffers and receiving events to and from remote outputs Quite a common feature, in the infotainment market, is the ability to stream @@ -315,11 +583,14 @@ Further more, to cope with situations where the output is just a panel/display, without some kind of compositor driving it, the necessity of handling input events is an important feature to have, giving the user to possibility to manipulate the application/environment as he or she seems fit. + The compositor loads a plug-in that streams out the buffers to an output -remotely, with [another plug-in](03_waltham_receiver_transmitter.md) -handling the input events. The events, which are sent back from the display to -the compositor, are generated with the help of wayland-eque protocol that works -over the network, called [Waltham](https://github.com/waltham/waltham). +remotely, with another plug-in was used to handle the input events. The events, +which are sent back from the display to the compositor, are generated with +the help of wayland-eque protocol that works over the network, called +[Waltham](https://github.com/waltham/waltham). This solution was deprecated +and removed a while back, with an alternative in the works to achieve +the same functionality. Together, they provide proper means to achieve a seamless integration with other display devices in the car cabin.