Merge pull request #5360 from nextcloud/testDocumentation

Test documentation

.github/ISSUE_TEMPLATE/pull_request_template.md

@@ -0,0 +1,6 @@
+### Testing 
+Writing tests is very important. Please try to write some tests for your PR. If you need help, please do not hesitate to
+ask in this PR for help.
+- [ ] [unit tests](https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#unit-tests)
+- [ ] [instrumented tests](https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#instrumented-tests)
+- [ ] [UI tests](https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#ui-tests)

CONTRIBUTING.md

@@ -23,6 +23,7 @@
         1. Backport pull request
         1. Pull requests that also need changes on library
         1. Adding new files
+        1. Testing
 	1. File naming
         1. Menu files
     1. Translations
@@ -238,6 +239,52 @@ Source code of app:
 -->
 ```
 
+### 6. Testing
+- testing is very important, but is lacking a lot on this project. Starting with 2020 we aim to write tests for every
+  new pull request.
+- Code coverage can be found [here](https://codecov.io/gh/nextcloud/android).
+
+#### 1. Unit tests
+- small, isolated tests, with no need of Android SDK
+- code coverage can be directly shown via right click on test and select "Run Test with Coverage"
+
+#### 2. Instrumented tests
+- tests to see larger code working in correct way
+- tests that require parts of Android SDK
+- best to avoid server communication, see https://github.com/nextcloud/android/pull/3624
+
+- run all tests ```./gradlew createGplayDebugCoverageReport -Pcoverage=true```
+- run selective test class: ```./gradlew createGplayDebugCoverageReport -Pcoverage=true
+  -Pandroid.testInstrumentationRunnerArguments.class=com.owncloud.android.datamodel.FileDataStorageManagerTest```
+- run multiple test classes:
+  -   separate by ","
+  - ```./gradlew createGplayDebugCoverageReport -Pcoverage=true -Pandroid.testInstrumentationRunnerArguments.class=com.owncloud.android.datamodel.FileDataStorageManagerTest,com.nextcloud.client.FileDisplayActivityIT```
+- run one test in class: ```./gradlew createGplayDebugCoverageReport -Pcoverage=true
+  -Pandroid.testInstrumentationRunnerArguments.class=com.owncloud.android.datamodel.FileDataStorageManagerTest#saveNewFile```
+- JaCoCo results are shown as html: firefox ./build/reports/coverage/gplay/debug/index.html
+
+
+#### 3. UI tests
+We use [shot](https://github.com/Karumi/Shot) for taking screenshots and compare them 
+- check screenshots: ```./gradlew executeScreenshotTests ```
+- update/generate new screenshots: ```scripts/updateScreenshots.sh ``` 
+    - in this script are samples how to only execute a given class/test
+    - this will fire up docker & emulator to ensure that screenshots look the same
+- creating own UI comparision tests: 
+    - add IntentsTestRule for launching activity directly:
+    ```java
+ @Rule public IntentsTestRule<SettingsActivity> activityRule = new IntentsTestRule<>(SettingsActivity.class,
+                                                                                        true,
+                                                                                        false);
+```
+    -  in test method:
+    ```java 
+Activity activity = activityRule.launchActivity(null); 
+…do something, e.g. navigate, create folder, etc. … 
+  Screenshot.snapActivity(activity).record();
+  ```
+    - best practise is to first create test with emulator too see behaviour and then create screenshots
+
 ## File naming
 
 The file naming patterns are inspired and based on [Ribot's Android Project And Code Guidelines](https://github.com/ribot/android-guidelines/blob/c1d8c9c904eb31bf01fe24aadb963b74281fe79a/project_and_code_guidelines.md).

build.gradle

@@ -192,6 +192,7 @@ android {
         testOptions {
             unitTests.returnDefaultValues = true
             execution 'ANDROIDX_TEST_ORCHESTRATOR'
+            animationsDisabled true
         }
     }
 

scripts/updateScreenshots.sh

@@ -1,7 +1,13 @@
 #!/usr/bin/env bash
 
+if ( [[ $(grep NC_TEST_SERVER_BASEURL ~/.gradle/gradle.properties   | grep -v "#" -c) -gt 0 ]] ); then
+    echo "This will not use server in docker. Please comment in .gradle/gradle.properties. Aborting!"
+    exit 1
+fi
+
 ## emulator
 if ( [[ $(emulator -list-avds | grep uiComparison -c) -eq 0 ]] ); then
+    avdmanager delete avd -n uiComparison
     (sleep 5; echo "no") | avdmanager create avd -n uiComparison -c 100M -k "system-images;android-27;google_apis;x86" --abi "google_apis/x86"
 fi
 
@@ -9,7 +15,7 @@ emulator -avd uiComparison -no-snapshot -gpu swiftshader_indirect -no-window -no
 PID=$(echo $!)
 
 ## server
-docker run --name=uiComparison nextcloudci/server:server-3 1>/dev/null &
+docker run --name=uiComparison nextcloudci/server 1>/dev/null &
 sleep 5
 IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' uiComparison)
 
@@ -23,7 +29,22 @@ cp gradle.properties gradle.properties_
 sed -i s"/server/$IP/" gradle.properties
 scripts/wait_for_emulator.sh
 scripts/wait_for_server.sh ${IP}
+
+## update all screenshots
 ./gradlew executeScreenshotTests -Precord
+
+## update screenshots in a class
+#./gradlew executeScreenshotTests \
+#-Precord \
+#-Pandroid.testInstrumentationRunnerArguments.class=\
+#com.owncloud.android.ui.dialog.SyncFileNotEnoughSpaceDialogFragmentTest
+
+## update single screenshot within a class
+#./gradlew executeScreenshotTests \
+#-Precord \
+#-Pandroid.testInstrumentationRunnerArguments.class=\
+#com.owncloud.android.ui.dialog.SyncFileNotEnoughSpaceDialogFragmentTest#showNotEnoughSpaceDialogForFile
+
 mv gradle.properties_ gradle.properties
 
 # tidy up