src/qa-testdefinitions.git
9 months agoAdd gitreview file for Prickly Pike branch pike
Jan-Simon Moeller [Mon, 31 Jul 2023 15:17:27 +0000 (17:17 +0200)]
Add gitreview file for Prickly Pike branch

Update the .gitreview file.

Bug-AGL: SPEC-4867
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I2136f2415ce7da647e3fd8a59dd6640f9a962e27

10 months agoAdapt screenshot tests after session removal 15/29015/1
Jan-Simon Moeller [Tue, 13 Jun 2023 21:22:25 +0000 (23:22 +0200)]
Adapt screenshot tests after session removal

Adapt the scripts.

Bug-AGL: SPEC-4813
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ief1ca6d35ed4f891082660500c0ab08b307e7674

11 months agoUse full path to reference image
Jan-Simon Moeller [Sun, 4 Jun 2023 13:34:10 +0000 (15:34 +0200)]
Use full path to reference image

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I664757283777cc79f3b40bc043b07294a724b785

11 months agoWork in /home/agl-driver
Jan-Simon Moeller [Sun, 4 Jun 2023 13:12:23 +0000 (15:12 +0200)]
Work in /home/agl-driver

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I364ee1531fbd6444166b3404e3402d5321b1afa9

11 months agoUpdate screenshot test for multiple images.
Jan-Simon Moeller [Fri, 2 Jun 2023 13:33:38 +0000 (15:33 +0200)]
Update screenshot test for multiple images.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ifa5952e05b463fecf50d9e6a4f84a324cb0f8f76

11 months agoDisable color output in pytest 36/28936/1
Jan-Simon Moeller [Wed, 31 May 2023 09:34:52 +0000 (11:34 +0200)]
Disable color output in pytest

Disable the color output to avoid escape sequences in LAVA logs.

Bug-AGL: SPEC-4817

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Iff38a79b72952416792c4c66d1e482736d308544

11 months agoSPEC-4816: Fix upload in agl-extend-test.sh 35/28935/1
Corentin LABBE [Wed, 24 May 2023 14:35:21 +0000 (14:35 +0000)]
SPEC-4816: Fix upload in agl-extend-test.sh

agl-extend-test.sh fail to upload a zip file, since it give the
directory instead of the file to upload.
While fixing this, add some extra test and debug around upload of files.

Bug-AGL: SPEC-4816
Fixes: 2e5067e302bc ("qa-testdefinitions: add yaml and script for agl extend test")
Change-Id: Ieeefa239398a82b2156391dfe8f58f654a6a29b3
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
11 months agoEnable multi-screen and list screens
Jan-Simon Möller [Wed, 10 May 2023 13:55:24 +0000 (15:55 +0200)]
Enable multi-screen and list screens

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I315657ac59d0a2aee20974215e426810b412985c

12 months agoAdapt path
Jan-Simon Moeller [Fri, 5 May 2023 16:07:25 +0000 (18:07 +0200)]
Adapt path

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ib9d15959a4ac1de4642ea2070f325dd6cd02ac08

12 months agodebug
Jan-Simon Moeller [Fri, 5 May 2023 13:23:15 +0000 (15:23 +0200)]
debug

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Icc2938efb7568ff65630a316a624b3372409d4d6

12 months agoAdd image upload to qa-testdefinition.
Jan-Simon Moeller [Fri, 5 May 2023 13:07:33 +0000 (15:07 +0200)]
Add image upload to qa-testdefinition.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Id5902477f8b946757427d344f47082b673031ce0

12 months agoUse fallback IPs for pyartiproxy.
Jan-Simon Moeller [Thu, 20 Apr 2023 17:31:48 +0000 (19:31 +0200)]
Use fallback IPs for pyartiproxy.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ia31e5a29e46ee4fca45b4d8ddd8e8cd8e7347133

15 months agoAdd extended test with redirect
Jan-Simon Moeller [Wed, 25 Jan 2023 16:08:37 +0000 (17:08 +0100)]
Add extended test with redirect

try with a logfile

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ia6124760784614e9965770ea688bcc87eee5c5f2

15 months agoRevert "For master do not check connman.service" 10/28410/1
Jan-Simon Moeller [Fri, 20 Jan 2023 08:42:39 +0000 (09:42 +0100)]
Revert "For master do not check connman.service"

With systemd-network-generator masked, we can start up connman again.

This reverts commit bd2c66954f5f0139cfd7610aeb3989965f4b3f7d.

Change-Id: I80edf1a036c1fd418375f8e4cb9b8a77fd354ca8

15 months agoUpdate skiplists in qa-testdefinitions 09/28409/1
Jan-Simon Moeller [Fri, 20 Jan 2023 08:41:57 +0000 (09:41 +0100)]
Update skiplists in qa-testdefinitions

This updates the skiplists. Only pi4 does currently skip the screenshooter.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I12d276b4268d80392dee3464e4ec5280c2e950ea

15 months agoFor master do not check connman.service
Jan-Simon Moeller [Mon, 16 Jan 2023 17:21:02 +0000 (18:21 +0100)]
For master do not check connman.service

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Id05a2e1e78bad13b53edd77feff5c10068ba3208

20 months agoqa-testdefinitions: add yaml and script for agl extend test 99/28099/2
duerpei [Sat, 6 Aug 2022 03:47:36 +0000 (11:47 +0800)]
qa-testdefinitions: add yaml and script for agl extend test

In qa-testdefinitions, add lava testdefinition (yaml+script)
for agl extend test

Bug-AGL: SPEC-4345

Signed-off-by: duerpei <duep.fnst@fujitsu.com>
Change-Id: I2df480b5e880c62868c17192c827e8f7ccc2a749

19 months agoThe launcher is no longer needed for window activation
Jan-Simon Moeller [Mon, 26 Sep 2022 10:15:57 +0000 (12:15 +0200)]
The launcher is no longer needed for window activation

Instead we rely now on the homescreen. So remove the launcher.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I77d8adea7080e19fb3bffe63a029632c4f4d195a

23 months agoagl-screenshot-test.sh: Save the contents of the screenshooter 24/27524/1 13.93.0 needlefish/13.93.0 needlefish_13.93.0
Marius Vlad [Tue, 24 May 2022 13:18:39 +0000 (16:18 +0300)]
agl-screenshot-test.sh: Save the contents of the screenshooter

We're getting a checksum error when comparing the reference image
against the one done at boot, so let's have to store it, for later
retrieval and investigation.

Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Change-Id: I7a532d78fc840064d74c6f385ddbf2537390b1ad

23 months agoagl-screenshot-test.sh: Redo the compositor restart 23/27523/1
Marius Vlad [Tue, 24 May 2022 13:07:28 +0000 (16:07 +0300)]
agl-screenshot-test.sh: Redo the compositor restart

Stopping services and the client shell before restarting the compositor,
would avoid some large delays until the new compositor instance
is brought up.

With this approach, were we stop explicitly the client shell and the
launcher, we're basically have a hard stop, and should avoid those huge
delays for qemu-arm. Until we can confirm this is the case, we still
maintain that delay.

Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Change-Id: Ifa067ac40de162291d7d6cfb70035e87add07220

2 years agoFix screenshot test and user session 02/27202/1 12.93.0 marlin/12.93.0 marlin_12.93.0
Jan-Simon Moeller [Tue, 15 Feb 2022 16:54:16 +0000 (17:54 +0100)]
Fix screenshot test and user session

The user session has its own daemon that needs to be reloaded separately.
Thus execute a separate reload call.

Bug-AGL: SPEC-4165

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ie58423dfb29002660eea69462623d3f37614079f

2 years agoAdapt screenshot test to new framework 57/27057/2 12.91.0 12.92.0 marlin/12.91.0 marlin/12.92.0 marlin_12.91.0 marlin_12.92.0
Jan-Simon Moeller [Tue, 21 Dec 2021 22:10:22 +0000 (23:10 +0100)]
Adapt screenshot test to new framework

Update the restart command and adapt to the user session.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I92ee001ded6d49e83078ce13e31e29cebe69d967

2 years agoExtend delay before taking screenshot 66/26966/1
Jan-Simon Möller [Mon, 6 Dec 2021 15:14:25 +0000 (16:14 +0100)]
Extend delay before taking screenshot

On slow emulated targets, we do need more time to spin up.
Extend the timeout.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I90231ad23846e77191626ccaebd657b53ee4b897

2 years agoscreenshot-test: Extend startup delay and use no pager 62/26962/1
Jan-Simon Moeller [Fri, 3 Dec 2021 20:58:23 +0000 (21:58 +0100)]
screenshot-test: Extend startup delay and use no pager

We need more time on e.g. qemuarm to finish processing
and also have no pager enabled.

Bug-AGL: SPEC-4165

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I1f1d4adcc2b9b702b09e33005559d2d061f7472f

2 years agoUpdate agl-screenshot-test for master 20/26920/1
Jan-Simon Möller [Wed, 24 Nov 2021 14:31:19 +0000 (15:31 +0100)]
Update agl-screenshot-test for master

The latest changes on the master branch require an update
to the screenshot test.

The script has been changed to use the journal and use the
new settings file locations.

Bug-AGL: SPEC-4150
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: If321c62940bb1809e38585c7dcdb08b273af8221

2 years agoLimit service check after appfw rework
Jan-Simon Möller [Tue, 9 Nov 2021 10:01:11 +0000 (11:01 +0100)]
Limit service check after appfw rework

These services are disabled now and will re-enter as part of the appfw rework.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I716a1c437c0d50f1756bb0a058b5b1036ca95430

2 years agoIgnore screenshot on arm 32 - known flakey 12.90.1 marlin/12.90.1 marlin_12.90.1
Jan-Simon Moeller [Wed, 22 Sep 2021 15:31:36 +0000 (17:31 +0200)]
Ignore screenshot on arm 32 - known flakey

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I4a16b8f0449fb7413777f08bbc569d5cd76527e0

2 years agoWait for screenshot until display is up
Jan-Simon Möller [Wed, 15 Sep 2021 14:50:18 +0000 (16:50 +0200)]
Wait for screenshot until display is up

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: Ie74175ab892f12a4eca439350bdd043bb282f931

2 years agoExtend the waiting time for emulated platforms lamprey 12.1.0 12.1.1 12.1.10 12.1.11 12.1.12 12.1.13 12.1.14 12.1.15 12.1.16 12.1.17 12.1.18 12.1.19 12.1.2 12.1.3 12.1.4 12.1.5 12.1.6 12.1.7 12.1.8 12.1.9 lamprey/12.1.0 lamprey/12.1.1 lamprey/12.1.10 lamprey/12.1.11 lamprey/12.1.12 lamprey/12.1.13 lamprey/12.1.14 lamprey/12.1.15 lamprey/12.1.16 lamprey/12.1.17 lamprey/12.1.18 lamprey/12.1.19 lamprey/12.1.2 lamprey/12.1.3 lamprey/12.1.4 lamprey/12.1.5 lamprey/12.1.6 lamprey/12.1.7 lamprey/12.1.8 lamprey/12.1.9 lamprey_12.1.0 lamprey_12.1.1 lamprey_12.1.10 lamprey_12.1.11 lamprey_12.1.12 lamprey_12.1.13 lamprey_12.1.14 lamprey_12.1.15 lamprey_12.1.16 lamprey_12.1.17 lamprey_12.1.18 lamprey_12.1.19 lamprey_12.1.2 lamprey_12.1.3 lamprey_12.1.4 lamprey_12.1.5 lamprey_12.1.6 lamprey_12.1.7 lamprey_12.1.8 lamprey_12.1.9
Jan-Simon Möller [Wed, 15 Sep 2021 12:09:13 +0000 (14:09 +0200)]
Extend the waiting time for emulated platforms

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I3d8cf2dd5227edabf8dc39eb50f5fa0560a5d221

2 years agoMerge "SPEC-4003: Use pyartiproxy for uploading a file to artyfactory" 12.0.0 12.0.1 lamprey/12.0.0 lamprey/12.0.1 lamprey_12.0.0 lamprey_12.0.1
Jan-Simon Moeller [Sun, 4 Jul 2021 19:05:52 +0000 (19:05 +0000)]
Merge "SPEC-4003: Use pyartiproxy for uploading a file to artyfactory"

2 years agoSPEC-4003: Use pyartiproxy for uploading a file to artyfactory 64/26464/3
Corentin LABBE [Tue, 30 Mar 2021 09:11:33 +0000 (09:11 +0000)]
SPEC-4003: Use pyartiproxy for uploading a file to artyfactory

Dumpjournal will now dump to artyfactory by default.

Bug-AGL: SPEC-4003
Change-Id: I2ae89be507062bceb84a6a7cbd054dcce09b6874
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2 years agoAdd skiplist for bbe 11.91.0 11.92.0 12.90.0 lamprey/11.91.0 lamprey/11.92.0 lamprey_11.91.0 lamprey_11.92.0 marlin/12.90.0 marlin_12.90.0
Jan-Simon Möller [Wed, 12 May 2021 19:16:26 +0000 (21:16 +0200)]
Add skiplist for bbe

bbe has no sound. skip sound tests.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: Ief9254c5c856b19df063985bce3b2de7d4b6ac87

2 years agoExtend wait time for compositor restart
Jan-Simon Möller [Mon, 10 May 2021 20:27:10 +0000 (22:27 +0200)]
Extend wait time for compositor restart

This fixes slow (e.g. mesa) startup.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I10ef1e14ca97370c59984b5d848b90e6cbb9d40b

3 years agoapplication-lifecycle: replace continue by return 53/26053/1
Corentin LABBE [Wed, 17 Feb 2021 08:53:15 +0000 (08:53 +0000)]
application-lifecycle: replace continue by return

Lot of code was moved in do_release_test(), but since it is not in a
loop anymore, continue are noop.
Replace them by returns.

Fixes: 1aee04806027c9496f5860a0dfcdbd4aafde148d ("SPEC-3484: create new application tests using new wgt types")
Change-Id: I1dcd7b389dd859d8aadb3608c6e5255d13a470b8
Bug-AGL: SPEC-3808
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoMerge "SPEC-3764: add screenshoot test" 10.93.0 koi/10.93.0 koi_10.93.0
Jan-Simon Moeller [Wed, 20 Jan 2021 15:01:50 +0000 (15:01 +0000)]
Merge "SPEC-3764: add screenshoot test"

3 years agoSPEC-3764: add screenshoot test 27/25927/5
Corentin LABBE [Fri, 15 Jan 2021 12:47:16 +0000 (12:47 +0000)]
SPEC-3764: add screenshoot test

Add basic screenshooter test.

Change-Id: I8be9c6b4f621dc757c7e8ab846384c0fb32561df
Bug-AGL: SPEC-3764
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoAdd AGL_CAN_INTERFACE=vcan0 to application-lifecycle 10.92.0 koi/10.92.0 koi_10.92.0
Jan-Simon Moeller [Fri, 15 Jan 2021 22:51:34 +0000 (23:51 +0100)]
Add AGL_CAN_INTERFACE=vcan0 to application-lifecycle

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ic0b2e75122835786899aae2979feb8d29edefd5c

3 years agoUpdate pyagl testdefinition to set AGL_CAN_INTERFACE 05/25905/1
Jan-Simon Möller [Wed, 13 Jan 2021 16:34:39 +0000 (17:34 +0100)]
Update pyagl testdefinition to set AGL_CAN_INTERFACE

We need to use vcan0 for the canreplay.

Bug-AGL: SPEC-3756
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I8e3e8c7d6eadf081014e129cfd59ad217004f46d

3 years agoDisable colors in journal dump 56/25656/1 10.91.0 koi/10.91.0 koi_10.91.0
Jan-Simon Möller [Fri, 27 Nov 2020 14:09:28 +0000 (15:09 +0100)]
Disable colors in journal dump

Avoid escaping characters.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I50b1895cc148b43a73e8730f76a39c29382973cb

3 years agoDo not require tests that enforce internet to be available in CI
Jan-Simon Möller [Wed, 25 Nov 2020 11:25:28 +0000 (12:25 +0100)]
Do not require tests that enforce internet to be available in CI

Do not enforce tests that require the internet connection to be up.
This is the case for geoclue and weather.

We might be able to revert this in the future after
fixes to the systemd dependencies and lab setup.

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I1f53e6bfda5be7c9f9613b485efb4f2e25839a63

3 years agoFlip order in veth setup
Jan-Simon Möller [Thu, 22 Oct 2020 09:20:42 +0000 (11:20 +0200)]
Flip order in veth setup

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I2da141b438122eee2768582c5a31054cb8a68964

3 years agoSPEC-3597: prevent conman to break routes 62/25462/1
Corentin LABBE [Wed, 21 Oct 2020 09:37:32 +0000 (11:37 +0200)]
SPEC-3597: prevent conman to break routes

conman break routes after adding veth, prevent this.

Change-Id: Ic674a8ad609783950ac2ca95a9176d427c59b6ad
Bug-AGL: SPEC-3597
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoMerge "Extend skiplists and fix typo in export"
Jan-Simon Moeller [Mon, 19 Oct 2020 11:04:58 +0000 (11:04 +0000)]
Merge "Extend skiplists and fix typo in export"

3 years agoExtend skiplists and fix typo in export 45/25445/1
Jan-Simon Möller [Mon, 19 Oct 2020 11:02:28 +0000 (13:02 +0200)]
Extend skiplists and fix typo in export

We the can test 1_pyagl-tests.can_write_wo_auth can easily fail on
the 2nd invocation. Work this around until fixed.

Bug-AGL: SPEC-3648
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I25af2c86604672f2e439fef19b400c83fff358d9

3 years agoSPEC-3631: Add veth test 31/25431/1
Corentin LABBE [Wed, 14 Oct 2020 07:18:58 +0000 (09:18 +0200)]
SPEC-3631: Add veth test

Add veth test which adds an extra virtual network card for testing.

Change-Id: I4e6fef99009830cd016b2124b45b6815192b5ced
Bug-AGL: SPEC-3631
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoAdd AGL_AVAILABLE_INTERFACES to application test 29/25429/1
Jan-Simon Möller [Tue, 13 Oct 2020 20:49:53 +0000 (22:49 +0200)]
Add AGL_AVAILABLE_INTERFACES to application test

The standalone run did have it defined, but the application test
did mit the setting of
AGL_AVAILABLE_INTERFACES=ethernet.

If necessary it has to be redefined in the test for bluetooth and wifi
per board.

Bug-AGL: SPEC-3597
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Change-Id: I0772f9a6cc449406785fb167f575d6ca65e6267c

3 years agoDisable afm-test execution 03/25403/1
Jan-Simon Moeller [Wed, 7 Oct 2020 16:00:52 +0000 (18:00 +0200)]
Disable afm-test execution

agl-service-homescreen crashes when a testwgt is executed.
Until resolved, disable the test wgt execution.

Bug-AGL: SPEC-3608

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I415b2fe80338f4090ef037c77d61af8205fbb77e

3 years agoSPEC-3484: Use the new gcovr-wrapper 02/25402/1
Corentin LABBE [Fri, 25 Sep 2020 11:41:41 +0000 (13:41 +0200)]
SPEC-3484: Use the new gcovr-wrapper

For using gcov, a new gcovr-wrapper was created.
Let's use it, capture its output ang generate some metrics.

Bug-AGL: SPEC-3484
Change-Id: Idbd9aa34d37d31414d18341b035440e28df5dc83
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoMerge "SPEC-3484: install and basicly test coverage package"
Jan-Simon Moeller [Mon, 14 Sep 2020 16:58:43 +0000 (16:58 +0000)]
Merge "SPEC-3484: install and basicly test coverage package"

3 years agoMerge "Change regular pyagl invocation to nohw"
Jan-Simon Moeller [Mon, 14 Sep 2020 16:17:58 +0000 (16:17 +0000)]
Merge "Change regular pyagl invocation to nohw"

3 years agoChange regular pyagl invocation to nohw 56/25256/1
Jan-Simon Moeller [Mon, 14 Sep 2020 13:53:21 +0000 (15:53 +0200)]
Change regular pyagl invocation to nohw

In CI we will execute the 'not hwrequired' flag.
Make this the default case in the test-suites.

Bug-AGL: SPEC-3572
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I370bb3c12953d7455a1a4c06c573cbc10ee17bda

3 years agoSPEC-3484: install and basicly test coverage package 49/25249/1
Corentin LABBE [Mon, 31 Aug 2020 11:58:41 +0000 (13:58 +0200)]
SPEC-3484: install and basicly test coverage package

Install and test coverage the same way than normal package.
Handling coverage generated files will be handled later.

Bug-AGL: SPEC-3484
Change-Id: I02d4ea1aa77954a7c85317accb6b565ece8676c0
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3550: fix parsing of id 47/25247/1
Corentin LABBE [Mon, 14 Sep 2020 08:12:28 +0000 (10:12 +0200)]
SPEC-3550: fix parsing of id

The id could be not the first element, and so the parsing is wrong.

Change-Id: I325aac6de4733a166c7be8e3515b85e9edff4365
Bug-AGL: SPEC-3550
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3550: name could be in uppercase 31/25231/1 9.99.4 jellyfish/9.99.4 jellyfish_9.99.4
Corentin LABBE [Thu, 10 Sep 2020 12:37:20 +0000 (14:37 +0200)]
SPEC-3550: name could be in uppercase

In config.xml, name could be in uppercase, so let's use id which is
lowercase.

Change-Id: Ibc3e9e145692c7773f70a1308c7ff0de27e9f0bd
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoAdd not hwrequired to regular script case 27/25227/1
Jan-Simon Moeller [Wed, 9 Sep 2020 14:30:30 +0000 (16:30 +0200)]
Add not hwrequired to regular script case

Qemu by default has no hardware attached. Limit the run to not
execut hardware-specific tests.

Bug-AGL: SPEC-3572

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I3305d0c34737cf026989e2bde197d8ce3d96db19

3 years agoAdd test definitions for pyagl 26/25226/1
Jan-Simon Moeller [Wed, 9 Sep 2020 12:35:45 +0000 (14:35 +0200)]
Add test definitions for pyagl

We can now call pytest using the pyagl/tests module.

These expose -k regular  and -k hwrequired, thus create two caller scripts.

Also create a sample testsuite entry using the regular variant with limited AGL_AVAILABLE_INTERFACES.

Bug-AGL: SPEC-3572

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ic6b977da959f946ca88348715d5249748c70f736

3 years agoSPEC-3550: handle package with service name different than their filename 58/25158/1
Corentin LABBE [Thu, 27 Aug 2020 08:39:05 +0000 (10:39 +0200)]
SPEC-3550: handle package with service name different than their filename

Some package has their service name different than their filename.
This patch handle this situation

Bug-AGL: SPEC-3550

Change-Id: I99c102f411ce5a0636fa14528cd2890694d986c6
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3484: create new application tests using new wgt types 11/25111/7 9.99.3 jellyfish/9.99.3 jellyfish_9.99.3
Corentin LABBE [Thu, 13 Aug 2020 09:26:34 +0000 (11:26 +0200)]
SPEC-3484: create new application tests using new wgt types

The wgt-test and wgt-coverage needs to be handled differently.
This patchs splits the application tests to handle all files
encountered.
First we download all files, then we use them in a specific order.

Change-Id: Idc71ecfe36496db314b288f2b8ca7c2d61887de7
Bug-AGL: SPEC-3484
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3482: handle new package name 09/25109/1
Corentin LABBE [Wed, 12 Aug 2020 14:44:19 +0000 (16:44 +0200)]
SPEC-3482: handle new package name

Now we build all package type, we need to handle coverage/test packages.

Bug-AGL: SPEC-3482

Change-Id: Idb6dc2d453f26183094d6679dcc5c1f553780a16
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoPrint the syslog when the application does not start 94/25094/1
Jan-Simon Moeller [Mon, 10 Aug 2020 18:22:20 +0000 (20:22 +0200)]
Print the syslog when the application does not start

Dump the journal when the app does not start.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ia886e87599721deca57dd8664c6637c3cd1f9b3f

3 years agoSPEC-3533: unzip wgt files in a dedicated directory 90/25090/1
Corentin LABBE [Mon, 10 Aug 2020 08:54:38 +0000 (10:54 +0200)]
SPEC-3533: unzip wgt files in a dedicated directory

For finding wgtname, we download an index.html and the WGT archive could
contains a file with the same name and unzip complain about replacing a file.
The solution is to unzip in an empty dedicated directory.

Change-Id: I9680eb8909256e749a2b5177ed1d942add841af5
BUG-AGL: SPEC-3533
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3495: fix greping of systemd units 26/25026/1 9.99.2 jellyfish/9.99.2 jellyfish_9.99.2
Corentin LABBE [Thu, 16 Jul 2020 07:30:47 +0000 (09:30 +0200)]
SPEC-3495: fix greping of systemd units

The grep pattern for systemd units is not enough restrictive.
So let's made grep a little more restrictive.

Change-Id: If0f75919a89c39da63e5375e3b3cdd020c6ea324
Bug-AGL: SPEC-3495
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3495: handle better all package name 02/25002/1
Corentin LABBE [Mon, 13 Jul 2020 12:43:53 +0000 (14:43 +0200)]
SPEC-3495: handle better all package name

Package could have -debug in their filename, but their service name is
without it.
Some package could exists with a basename-extension, so we need to grep
better to not catch them all.
Example: agl-service-bluetooth catch agl-service-bluetooth-map.

Change-Id: Ibc750e7893a8cbe3fc22a0d8f13a32c28a898bd1
Bug-AGL: SPEC-3495
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoSPEC-3427: check if RID is null 66/24766/2
Corentin LABBE [Mon, 15 Jun 2020 12:29:06 +0000 (14:29 +0200)]
SPEC-3427: check if RID is null

RID could be null if process failed to start.
In that case, we will retry to start it.
On a second fail, we will not continue the test suite and fail.

In th same time add a second status check, but after a delay to be sure
the service is still working.

Bug-AGL: SPEC-3427
Change-Id: I566f1aefa06cbb791674373340ce87be220b18a2
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
3 years agoMake the script runnable locally and adapt ps call 9.99.1 jellyfish/9.99.1 jellyfish_9.99.1
Jan-Simon Moeller [Thu, 28 May 2020 14:21:08 +0000 (16:21 +0200)]
Make the script runnable locally and adapt ps call

The script can be run for testing locally now and afm-util ps needs to start as the target user.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: Ief0f785a46439531d4257c2eef9ced15be1b20fb

3 years agoRemove versioning of wgts from test
Jan-Simon Möller [Thu, 28 May 2020 09:46:57 +0000 (11:46 +0200)]
Remove versioning of wgts from test

This was removed thus there is no '@?.?' .

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
4 years agoUpdate skiplists. 80/24280/1
Jan-Simon Moeller [Tue, 24 Mar 2020 13:24:41 +0000 (14:24 +0100)]
Update skiplists.

lava-read-feedback fail on qemu due to a sequence issues.
This is merely a timing issue on the lava side that we mask here.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I19070931704e7b9bf0a27e9370fdffd5adb899ba

4 years agoDisable the can module unload/load test 54/23454/1 8.99.5 icefish/8.99.5 icefish_8.99.5
Jan-Simon Möller [Fri, 20 Dec 2019 23:27:20 +0000 (00:27 +0100)]
Disable the can module unload/load test

And also fix a logic bug in the test_can_modules.

The test can stay disabled. There is no need to unload the modules.

Bug-AGL: SPEC-3070
.

Change-Id: Ic94d5bb9cad3d6cdcbc6534cfc4003213ad82aaa
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
4 years agoSPEC-3064: handle removal of can_bcm 07/23407/2 8.99.4 icefish/8.99.4 icefish_8.99.4
Corentin LABBE [Tue, 17 Dec 2019 12:54:46 +0000 (13:54 +0100)]
SPEC-3064: handle removal of can_bcm

The test removing can module fail on DRA7 due to can_bcm using it.
This patchs handle this case.

Bug-AGL: SPEC-3064

Change-Id: I272e0ea62786fb43c4ea68601f2fb12864a279b6
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoMerge "Skip the can test" 8.99.3 icefish/8.99.3 icefish_8.99.3
Jan-Simon Moeller [Tue, 10 Dec 2019 13:39:37 +0000 (13:39 +0000)]
Merge "Skip the can test"

4 years agoSPEC-2970: Do not fail ofone and bluetooth services if bluetooth tag is non present 06/23306/1
Corentin LABBE [Mon, 9 Dec 2019 13:53:24 +0000 (14:53 +0100)]
SPEC-2970: Do not fail ofone and bluetooth services if bluetooth tag is non present

When no bluetooth dongle is present, there is a timing issue which let
the service fail.

So if no bluetooth tag is present, skip the service test instead of
failling.

Bug-AGL: SPEC-2907
Change-Id: I10755ec698ab632ffcf0eec18ebe6bc41b1b5ec1
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSkip the can test 86/23286/2
Khouloud Touil [Mon, 9 Dec 2019 10:41:42 +0000 (11:41 +0100)]
Skip the can test

As the behaviour of the test not determined, to avoid the fail of the
test, we will skip it if the value of no_transmission not found.

AGL-bug: SPEC-3025

Change-Id: If24e512f2cc83ee7018a4309cb64434cf3a2b938
Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
4 years agoSPEC-2970: Add bluetooth and ofono on qemu ARM/ARM64 skiplists 81/23181/1 8.99.2 icefish/8.99.2 icefish_8.99.2
Corentin LABBE [Wed, 4 Dec 2019 14:40:23 +0000 (15:40 +0100)]
SPEC-2970: Add bluetooth and ofono on qemu ARM/ARM64 skiplists

Since bluetooth is still failling on qemu ARM and ARM64 (and bring down
ofono by dependency), let's add them to the skiplist.

Change-Id: I1ac3485f339763f16d49e7527faa74f6efe71b30
Bug-AGL: SPEC-2970
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoAlso call kill in the right context 18/23118/1
Jan-Simon Moeller [Thu, 28 Nov 2019 22:30:48 +0000 (23:30 +0100)]
Also call kill in the right context

depending on the app.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I7208497e81fb3fcb2636bf7559b27ff2b5ce7676

4 years agoCall info ps and status in the context of the app user 17/23117/1
Jan-Simon Moeller [Thu, 28 Nov 2019 22:19:48 +0000 (23:19 +0100)]
Call info ps and status in the context of the app user

We need to call the info and status call under the context of user under which the app runs.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I25abff89db22c5effb6addfbc59ec30575aae2ed

4 years agoMerge "SPEC-2996: handle rcar_can"
Jan-Simon Moeller [Thu, 28 Nov 2019 15:06:15 +0000 (15:06 +0000)]
Merge "SPEC-2996: handle rcar_can"

4 years agoSPEC-2996: handle rcar_can 83/23083/1
Corentin LABBE [Thu, 28 Nov 2019 09:10:49 +0000 (10:10 +0100)]
SPEC-2996: handle rcar_can

The test remove can_dv module fail on H3 due to rcar_can using it.
This patchs handle this case.

Change-Id: I452a61813567ad09f3f819ce73c1c11753326514
Bug-AGL: SPEC-2996
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSPEC-2996: Fix ip check 82/23082/1
Corentin LABBE [Thu, 28 Nov 2019 09:05:10 +0000 (10:05 +0100)]
SPEC-2996: Fix ip check

BusyBox ip print its version stderr, so we need to redirect it to stdin.
Furthermore, using set -x reset the error code, we need to store it before.

Bug-AGL: SPEC-2996
Change-Id: I4b80475b0f9291fc55e661362a981ac14dbdb9b0
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSPEC-2996: rework the CAN modules tests 74/23074/1
Corentin LABBE [Wed, 27 Nov 2019 09:58:26 +0000 (10:58 +0100)]
SPEC-2996: rework the CAN modules tests

The CAN modules tests have lots of redundant code, let's add some
function for probing/removing modules.

Furthermore, some tests cannot work with busybox ip, I adds some tests
for it.

When reworking the tests, I have removed the "lsmod |grep can" tests
which was of no value.
I have also removed all sleep which was undocumented(probably for some
log flushing), if something need them I will re-add it with comment.

Bug-AGL: SPEC-2996
Change-Id: I2e8a109d799c58726d848814fd4c21a9fbd2eef5
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoVerbose output
Jan-Simon Möller [Tue, 26 Nov 2019 21:12:07 +0000 (22:12 +0100)]
Verbose output

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
4 years agoMerge "test_can_loopback_mode: Add ip util version check"
Jan-Simon Moeller [Tue, 26 Nov 2019 16:58:11 +0000 (16:58 +0000)]
Merge "test_can_loopback_mode: Add ip util version check"

4 years agotest_can_loopback_mode: Add ip util version check 51/23051/1
Corentin LABBE [Tue, 26 Nov 2019 10:43:06 +0000 (11:43 +0100)]
test_can_loopback_mode: Add ip util version check

CAN tests fail on M3 and H3 because their build includes busybox version
of ip (and not iproute one).
This patchs adds a test for detecting if the right ip util is installed.

Change-Id: I5909bc0af23271626dec9c1ed7003bdfc9129041
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSPEC-2970: Improove logging size 50/23050/1
Corentin LABBE [Tue, 26 Nov 2019 09:07:37 +0000 (10:07 +0100)]
SPEC-2970: Improove logging size

When TERM is dump, the width of logging is cropped at 80columns which
made debugging harder.
This patchs increase columns size to 1000.

Bug-AGL: SPEC-2970
Change-Id: Ia70136b6aa6242362b1286631ae4b00ab567ae2e
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoCynagora: Replace cynara with cynagora 75/22875/3 8.99.1 icefish/8.99.1 icefish_8.99.1
Jose Bollo [Wed, 6 Nov 2019 15:06:47 +0000 (16:06 +0100)]
Cynagora: Replace cynara with cynagora

Currently we disable ptests/socket tests
to be enabled in SPEC-2940 and SPEC-2941.

Shift from the permission database cynara
to cynagora permission database with a compatibility
library.

Bug-AGL: SPEC-2844
Bug-AGL: SPEC-2941

Signed-off-by: Jose Bollo <jose.bollo@iot.bzh>
Change-Id: I673e47ae237a83926f15d003dd2d2623a08eefb5

4 years agoMask the bluetooth test until issues are resolved
Jan-Simon Moeller [Wed, 23 Oct 2019 12:03:12 +0000 (14:03 +0200)]
Mask the bluetooth test until issues are resolved

Currently the bluetooth service fails exclusively on m3ulcb-nogfx.
We need to adapt the CI infra (lava labs) either with additional hw
or adapt the dtb used conditionally. This needs further work.
Meanwhile mask the bluetooth service test on the m3ulcb-nogfx target
to avoid false-positives being reported.

Bug-AGL: SPEC-2921

.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I592ea7f8a6b8ddc4ff10eac8428162c44a1d3ac4

4 years agoremove debuging information of do_afm_util 52/22752/2
Corentin LABBE [Wed, 23 Oct 2019 09:04:02 +0000 (11:04 +0200)]
remove debuging information of do_afm_util

This printing breaks 'rid' parsing, remove it.

Bug-AGL: SPEC-2903
Bug-AGL: SPEC-2925

Change-Id: I505ab9d2994f983de53849fce25d87c8af44362d
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSPEC-2903: fix the grep test method 48/22748/1
Corentin LABBE [Tue, 22 Oct 2019 12:11:15 +0000 (14:11 +0200)]
SPEC-2903: fix the grep test method

While inspecting if application test is working with our latest
SPEC-2903 changes I see in job 8845 the following errors:
+ grep urn:AGL:widget:provided-api config.xml
+ '<feature' 'name=\"urn:AGL:widget:provided-api\">'
./common/scripts/application-lifecycle.sh: line 74: <feature: command not found

So let's change the grep method.
Furthermore, add a debug for the do_afm_util for being sure it work as
needed.

Bug-AGL: SPEC-2903
Change-Id: I5a8a6d98cd9dc6c1b2d1626fc6841eb2c2b16d53
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoSPEC-2903: Use a wrapper instead of constructing a command 40/22740/2
Corentin LABBE [Mon, 21 Oct 2019 14:00:34 +0000 (16:00 +0200)]
SPEC-2903: Use a wrapper instead of constructing a command

This patch change the way we ran afm-util commands by using a simple
wrapper.
Furthermore, since we are in a loop, let's reset all variable
(SERVICE_PLATFORM/etc..) to 0 at the beginning of the loop.

Extra-furthermore: restore the unzip error code test, but in non-fatal
way, since busybox unzip seems to dont like our CRC.

Bug-AGL: SPEC-2903

Change-Id: I5d7357fd74307ed69a6675beac57d8601fdcdb52
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoImprove lifecycle test 12/22712/1
Jan-Simon Moeller [Fri, 18 Oct 2019 22:30:33 +0000 (00:30 +0200)]
Improve lifecycle test

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I9076278096e15829b4bca52f6b5ef9e9a3b39e9e

4 years agoMerge "Differ platform and user scope in service and application"
Jan-Simon Moeller [Thu, 17 Oct 2019 19:06:00 +0000 (19:06 +0000)]
Merge "Differ platform and user scope in service and application"

4 years agoDiffer platform and user scope in service and application 05/22705/1
Jan-Simon Moeller [Thu, 17 Oct 2019 18:57:59 +0000 (20:57 +0200)]
Differ platform and user scope in service and application

When executing the lifecycle test, we need to take into account if
the service or app runs in platform or user scope.

Bug-AGL: SPEC-2903

.

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Change-Id: I49a53a5a2b14666757b65b0f8203eb73ed2c7b45

4 years agoMerge "Inverse the fail/pass logic"
Jan-Simon Moeller [Wed, 16 Oct 2019 07:52:13 +0000 (07:52 +0000)]
Merge "Inverse the fail/pass logic"

4 years agoMerge "qa-testdefinitions: Split can tests to avoid failures"
Jan-Simon Moeller [Wed, 16 Oct 2019 07:51:31 +0000 (07:51 +0000)]
Merge "qa-testdefinitions: Split can tests to avoid failures"

4 years agoInverse the fail/pass logic 49/22649/1
Khouloud Touil [Tue, 15 Oct 2019 14:54:15 +0000 (16:54 +0200)]
Inverse the fail/pass logic

This test should be failing, so it should be set to pass when it
fails, and should be set to fail when it passes.

AGL-bug: SPEC-2878

Change-Id: I43921717eea77d654921eca4691c936bb2901e54
Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
4 years agoqa-testdefinitions: Split can tests to avoid failures 16/22616/7
Khouloud Touil [Fri, 11 Oct 2019 10:50:11 +0000 (12:50 +0200)]
qa-testdefinitions: Split can tests to avoid failures

Split the can tests in two, one for basic tests that doesn't need the
two can interfaces to be related, and the other one is for a board with
two related interfaces.

Modify the test_can_modules to be run on a board with only one can if at
least.

AGL-bug: SPEC-2878

Change-Id: I6f761458c3e191aa8000d19cb05b0592566cb6c9
Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
4 years agoMerge "Avoid some test cases failures"
Jan-Simon Moeller [Sun, 6 Oct 2019 13:27:40 +0000 (13:27 +0000)]
Merge "Avoid some test cases failures"

4 years agoAvoid some test cases failures 61/22561/1
Khouloud Touil [Thu, 3 Oct 2019 09:17:04 +0000 (11:17 +0200)]
Avoid some test cases failures

To make sure that the can interface could be configured, we have to make
it dowm each time before the start of the config step.

Change-Id: I7bebea65072ffd822ee30241c68a0a89123b1576
Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
4 years agoMerge "Use the new arg --build-tags and the arg --device-tags"
Jan-Simon Moeller [Wed, 11 Sep 2019 22:57:43 +0000 (22:57 +0000)]
Merge "Use the new arg --build-tags and the arg --device-tags"

4 years agoSPEC-2740: add weston in the qemuarm/qemuarm64 skiplist 73/22373/1
Corentin LABBE [Tue, 10 Sep 2019 12:22:21 +0000 (14:22 +0200)]
SPEC-2740: add weston in the qemuarm/qemuarm64 skiplist

weston does not start on qemuarm/qemuarm64, let's add it to the test skiplist.

Change-Id: I885b8fe0e6521a1e707c4a3298a9f7d9aa384728
Bug-AGL: SPEC-2740
Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
4 years agoUse the new arg --build-tags and the arg --device-tags 19/22119/5
Khouloud Touil [Wed, 7 Aug 2019 15:08:30 +0000 (17:08 +0200)]
Use the new arg --build-tags and the arg --device-tags

Use the  new arg --build-tags along with the arg --device-tags to decide
the tests to be run and the tests to be skiped.
The test to be tested in this patch is for the weston service.

Bug-AGL: SPEC-2721

Change-Id: Ia34680c5e60e27dfe286626fedc5775efff87ba3
Signed-off-by: Khouloud Touil <ktouil@baylibre.com>