Skip to content

Commit a2d7824

Browse files
SanjulaGanepolaSanjulaGanepola
authored
Merge pull request #91 from codefori/feature/test-enhancements
Document enhancements in IBM i Testing `v1.3.2`
2 parents 9b7e10e + 3887379 commit a2d7824

File tree

6 files changed

+111
-11
lines changed

6 files changed

+111
-11
lines changed

‎src/content/docs/developing/testing/Testing_CancelledTestResult.png‎

242 KB
Loading

‎src/content/docs/developing/testing/Testing_CancellingTest.png‎

70.7 KB
Loading

‎src/content/docs/developing/testing/Testing_InlineAssertionError.png‎

272 KB
Loading

‎src/content/docs/developing/testing/configuring.mdx‎

Lines changed: 85 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,15 @@ Use the following steps to create a default configuration for your project:
8585
"cOption": [
8686
"*EVENTF"
8787
]
88+
},
89+
"rucalltst": {
90+
"order": "*API",
91+
"libl": "*CURRENT",
92+
"jobD": "*DFT",
93+
"detail": "*BASIC",
94+
"output": "*ALLWAYS",
95+
"rclRsc": "*NO",
96+
"onFailure": "*ABORT"
8897
}
8998
},
9099
"codecov": {
@@ -120,7 +129,8 @@ Here is an example with all execution parameters set in the `testing.json` file:
120129
"jobD": "*DFT",
121130
"detail": "*BASIC",
122131
"output": "*ALLWAYS",
123-
"rclRsc": "*NO"
132+
"rclRsc": "*NO",
133+
"onFailure": "*ABORT"
124134
}
125135
}
126136
}
@@ -132,6 +142,10 @@ Here is an example with all execution parameters set in the `testing.json` file:
132142
<TabItem label="RUCRTRPG" >
133143
|Parameter |Name |Description |Default |
134144
|-------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|
145+
|`tstPgm` |Test program |Specifies the name of the test program (`*SRVPGM`) to create. By default, this will be set to `&OPENLIB/&OPENMBR` for source members and `&CURLIB/&SHORTNAME` for local files. | |
146+
|`srcFile` |Source file |Specifies the name of the source file containing a member with the same name as `TSTPGM`. By default, this will be set to `&OPENLIB/&OPENSPF` for source members. | |
147+
|`srcMbr` |Source member |Specifies the name of the member containing the source code to be compiled. By default, this will be set to `&OPENMBR` for source members. | |
148+
|`srcStmf` |Source stream file |Specifies the path name of the stream file containing the source code to be compiled. By default, this will be set to `&FULLPATH` for local files. | |
135149
|`text` |Text description |Allows you to enter text that briefly describes the RPGUnit test case and its function. | |
136150
|`cOption` |Compile options |Refer to the `OPTION` parameter in `CRTRPGMOD` command help. |*EVENTF |
137151
|`dbgView` |Debugging views |Refer to the `DBGVIEW` parameter in `CRTRPGMOD` command help. |*SOURCE |
@@ -153,6 +167,10 @@ Here is an example with all execution parameters set in the `testing.json` file:
153167
<TabItem label="RUCRTCBL" >
154168
|Parameter |Name |Description |Default |
155169
|-------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|
170+
|`tstPgm` |Test program |Specifies the name of the test program (`*SRVPGM`) to create. By default, this will be set to `&OPENLIB/&OPENMBR` for source members and `&CURLIB/&SHORTNAME` for local files. | |
171+
|`srcFile` |Source file |Specifies the name of the source file containing a member with the same name as `TSTPGM`. By default, this will be set to `&OPENLIB/&OPENSPF` for source members. | |
172+
|`srcMbr` |Source member |Specifies the name of the member containing the source code to be compiled. By default, this will be set to `&OPENMBR` for source members. | |
173+
|`srcStmf` |Source stream file |Specifies the path name of the stream file containing the source code to be compiled. By default, this will be set to `&FULLPATH` for local files. | |
156174
|`text` |Text description |Allows you to enter text that briefly describes the RPGUnit test case and its function. | |
157175
|`cOption` |Compile options |Refer to the `OPTION` parameter in `CRTCBLMOD` command help. |*EVENTF |
158176
|`dbgView` |Debugging views |Refer to the `DBGVIEW` parameter in `CRTCBLMOD` command help. |*SOURCE |
@@ -178,18 +196,75 @@ Here is an example with all execution parameters set in the `testing.json` file:
178196
|`exclude` |Exclude |Objects to exclude | |
179197
</TabItem>
180198
<TabItem label="RUCALLTST" >
181-
|Setting |Description |Default |
182-
|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
183-
|Run Order |Specifies the order for running the test procedures. Useful to check that there is no dependencies between test procedures. |*API |
184-
|Library List |Specifies the library list for executing the specified unit test. |*CURRENT |
185-
|Job Description |Specifies the name of the job description that is used to set the library list, when the `IBM i Testing: Library List` setting is set to `*JOBD`. `*DFT` can be used here to indicate the library of the unit test suite (service program) is searched for job description `RPGUNIT`. |*DFT |
186-
|Report Detail |Specifies how detailed the test run report should be. |*BASIC |
187-
|Create Report |Specifies whether a report is created. |*ALLWAYS |
188-
|Reclaim Resources |Specifies when to reclaim resources. Resources, such as open files, can be reclaimed after each test case or at the end of the test suite. This option is useful if the test suite calls OPM programs, which do not set the `*INLR` indicator. |*NO |
199+
|Parameter |Name |Description |Default |
200+
|-------------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
201+
|`tstPgm` |Test program |Specifies the name of the test program (`*SRVPGM`). By default, this will be set to `&OPENLIB/&OPENMBR` for source members and `&CURLIB/&SHORTNAME` for local files. | |
202+
|`order` |Run Order |Specifies the order for running the test procedures. Useful to check that there is no dependencies between test procedures. |*API |
203+
|`libl` |Library List |Specifies the library list for executing the specified unit test. |*CURRENT |
204+
|`jobD` |Job Description |Specifies the name of the job description that is used to set the library list, when the `IBM i Testing: Library List` setting is set to `*JOBD`. `*DFT` can be used here to indicate the library of the unit test suite (service program) is searched for job description `RPGUNIT`. |*DFT |
205+
|`detail` |Report Detail |Specifies how detailed the test run report should be. |*BASIC |
206+
|`output` |Create Report |Specifies whether a report is created. |*ALLWAYS |
207+
|`rclRsc` |Reclaim Resources |Specifies when to reclaim resources. Resources, such as open files, can be reclaimed after each test case or at the end of the test suite. This option is useful if the test suite calls OPM programs, which do not set the `*INLR` indicator. |*NO |
208+
|`onFailure` |On Failure |Specifies test execution behavior after a failing assertion. |*ABORT |
209+
|`wrapperCmd` |Wrapper Command |Specifies a custom command to wrap the `RUCALLTST` command. | |
189210
</TabItem>
190211
</Tabs>
191212

192-
### Vendor Integration
213+
## Advanced Configuration
214+
215+
Some compile and execution configuration parameters (`TSTPGM`, `SRCFILE`, `SRCMBR`, `SRCSTMF`) are special because the extension typically manages and sets them for you. This is because they can be determined by the extension based on what tests are being compiled and executed. However, to provide maximum flexibility in configuring even these parameters, users can still do so via the `testing.json` file. A simple use case could be that you want to build your source member tests into your current library instead of the library containing the source.
216+
217+
Under the covers, the extension has default values for these special parameters along with supported variables based on whether you are working with local files or source members. The defaults below can be overwritten in the `testing.json` file:
218+
219+
<Tabs>
220+
<TabItem label="Source Members">
221+
```json
222+
{
223+
"rpgunit": {
224+
"rucrtrpg": {
225+
"tstPgm": "&OPENLIB/&OPENMBR",
226+
"srcFile": "&OPENLIB/&OPENSPF",
227+
"srcMbr": "&OPENMBR"
228+
},
229+
"rucalltst": {
230+
"tstPgm": "&OPENLIB/&OPENMBR"
231+
}
232+
}
233+
}
234+
```
235+
Supported Variables:
236+
* `&CURLIB`: Current library, changeable in the **Library List**
237+
* `&OPENLIB`: Library name where the test source member lives
238+
* `&OPENSPF`: Source file name where the test source member lives
239+
* `&OPENMBR`: Name of the test source member
240+
241+
</TabItem>
242+
243+
<TabItem label="Local Files">
244+
```json
245+
{
246+
"rpgunit": {
247+
"rucrtrpg": {
248+
"tstPgm": "&CURLIB/&SHORTNAME"
249+
"srcStmf": "&FULLPATH",
250+
},
251+
"rucalltst": {
252+
"tstPgm": "&CURLIB/&SHORTNAME"
253+
}
254+
}
255+
}
256+
```
257+
258+
Supported Variables:
259+
* `&CURLIB`: Current library, changeable in the **Library List**
260+
* `&NAME`: Name of the test file
261+
* `&SHORTNAME`: Shortened name of the test file (based on rules [here](https://ibm.github.io/sourceorbit/#/./pages/general/rules?id=long-file-names))
262+
* `&FULLPATH`: Full path of the test file on the remote system
263+
</TabItem>
264+
</Tabs>
265+
266+
267+
## Vendor Integration
193268
The `wrapperCmd` parameter in the `RUCRTRPG`/`RUCRTCBL`/`RUCALLTST` specifications are not actual parameters of the RPGUnit commands, but rather are provided by the extension to allow wrappering them. This is primarily used when integrating RPGUnit with other vendor tools.
194269

195270
**Example `testing.json` configuration**:

‎src/content/docs/developing/testing/overview.mdx‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,14 @@ The following extensions can be installed from the Visual Studio Code Marketplac
3535

3636
### 2. RPGUnit
3737

38-
The `RPGUNIT` library ***(`5.1.0-beta.005` minimum)*** must be be installed on your IBM i in order to leverage the framework and assertions that it provides. It can be installed easily via the extension using the steps below which is the recommended approach or manually using the instructions [here](https://irpgunit.sourceforge.io/help/).
38+
The `RPGUNIT` library must be installed on your IBM i in order to leverage the framework and assertions that it provides. It can be installed easily via the extension using the steps below which is the recommended approach or manually using the instructions [here](https://irpgunit.sourceforge.io/help/).
39+
40+
The required RPGUnit version depends on which version of the **IBM i Testing** extension you have installed:
41+
42+
| Extension Version | Required RPGUnit Version |
43+
|-------------------|----------------------------|
44+
| ≤ 1.2.3 | 5.1.0 |
45+
| 1.3.2 | ≥ 6.0.0 |
3946

4047
#### Preconditions
4148
RPGUnit requires OS400 7.5 for the latest features. It can also be installed on 7.4 and 7.3 if the following PTFs have been installed:

‎src/content/docs/developing/testing/running.mdx‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ For tests to be discovered by the extension, you must first [connect to an IBM i
1818

1919
![](Testing_TestExplorer.png)
2020

21+
### Running Tests
22+
2123
From here you can hover on any test case, test suite, folder, library, or source file and use the inline **Run Test** button. Using this action will start the following process:
2224

2325
<Steps>
@@ -32,6 +34,16 @@ From here you can hover on any test case, test suite, folder, library, or source
3234
![](Testing_GutterActions.png)
3335
</Aside>
3436

37+
### Cancelling Tests
38+
39+
The current test deployment, compilation, and execution can be cancelled at anytime by navigating to the **Test Explorer** view and using the **Cancel Test Run** button.
40+
41+
![](Testing_CancellingTest.png)
42+
43+
Any test suites which were executed before cancellation will still have their test results shown in the **Test Results** view. Cancelled test suites will be marked accordingly and a total count of cancelled test suites and cases will be included in the test summary.
44+
45+
![](Testing_CancelledTestResult.png)
46+
3547
## Test Results
3648

3749
After running any test, the results will be displayed in detail in the **Test Results** view. This will include the number of passed, failed, and errored test cases along with any relevant messages. The results can also be viewed at a high level in the `Testing` view itself where the icon will be updated to reflect the status of the test. Failures associated with a specific line will also be displayed in the editor.
@@ -42,6 +54,12 @@ After running any test, the results will be displayed in detail in the **Test Re
4254

4355
![](Testing_TestResults.png)
4456

57+
### Diagnosing Errors
58+
59+
When there is an assertion failure, you can view the actual and expected results (depending on the type of assertion) along with the failure message in the **Test Results** view. In addition, you can scroll through the editor and see the same message inline right next to the failing assertion. Upon clicking on this message, you will get an expanded inline view of the error which also includes a diff of the actual and expected result along with the stack trace.
60+
61+
![](Testing_InlineAssertionError.png)
62+
4563
## Code Coverage
4664

4765
You can also run the tests with code coverage to see which lines of code were executed during the test run. This is useful for identifying untested code paths and ensuring that your tests cover all critical areas of your application.

0 commit comments

Comments
 (0)