1 | #### Where to put a test in the repository
|
2 |
|
3 | - **ASM** mode:
|
4 | - Test file should be put here - "/test/commands/"
|
5 | - Add a reference to the testfile in "/test/testlist.txt"
|
6 | - **ARM** mode:
|
7 | - Test file should be put here - "/test/commands/arm/{category-of-the-command}"
|
8 | - For example: All the tests related to the group category ```azure group -h" were put here "/test/commands/arm/group"
|
9 | - Add a reference to the testfile in "/test/testlist**arm**.txt"
|
10 |
|
11 | #### Structure of a test
|
12 | A sample test to explain the test structure. Please pay close attention to the comments in the following test snippet.
|
13 |
|
14 | ```javascript
|
15 | 'use strict';
|
16 | //"should.js" (http://unitjs.com/guide/should-js.html) is used for asserting the outcomes.
|
17 | var should = require('should');
|
18 |
|
19 | //"/test/framework/arm-cli-test.js" is the suite used for writing tests in "ARM" mode.
|
20 | //"/test/framework/cli-test.js" is the suite used for writing tests in "ASM" mode.
|
21 | var CLITest = require('../../../framework/arm-cli-test');
|
22 |
|
23 | //Always provide a testPrefix. This would be the name of the directory
|
24 | //in which the test recordings would be stored for playback purposes
|
25 | //for example: "/test/recordings/arm-cli-location-tests/*"
|
26 | var testprefix = 'arm-cli-location-tests';
|
27 |
|
28 | var sitename;
|
29 | var createdSites = [];
|
30 |
|
31 | //List of requiredEnvironment variables for this test. If the envt. variable is not set
|
32 | //and a default value is provided then it will be used in the test, else the test will
|
33 | //throw an error letting the user know the list of required envt variables that need to be set.
|
34 | var requiredEnvironment = [
|
35 | { name: 'AZURE_SITE_TEST_LOCATION', defaultValue: 'East US'},
|
36 | 'AZURE_STORAGE_ACCESS_KEY'
|
37 | ];
|
38 |
|
39 | //We are using a poplular javascript testing framework named "mocha" (http://mochajs.org/) for writing tests.
|
40 | //As per mocha, describe() defines a "test-suite" and it() defines a "test" in a test-suite.
|
41 | describe('arm', function () {
|
42 | describe('location', function () {
|
43 | var suite;
|
44 | //before is executed once at the start of the this test-suite.
|
45 | before(function (done) {
|
46 | suite = new CLITest(this, testprefix, requiredEnvironment);
|
47 | //setupSuite is a hook provided for the developer to perform steps that
|
48 | //need to be performed once before the first test gets executed.
|
49 | //A. If nothing needs to be performed then setupSuite() needs to be called as follows:
|
50 | suite.setupSuite(done);
|
51 |
|
52 | //B. Let us assume that a new site needs to be created once, that will be used by every test.
|
53 | //Then we shall do something like this:
|
54 | suite.setupSuite(function () {
|
55 |
|
56 | //During RECORD mode, generateId will write the random test id to the recording file.
|
57 | //This id will be read from the file during PLAYBACK mode
|
58 | sitename = suite.generateId(
|
59 | "test-site" /*some good site prefix for you to identify the sites created by your test*/,
|
60 | createdSites /*An array to maintain the list of created sites.
|
61 | This is useful to delete the list of created sites in teardown*/ );
|
62 |
|
63 | suite.execute("site create --location %s %s --json" /*Azure command to execute*/,
|
64 | process.env.AZURE_SITE_TEST_LOCATION,
|
65 | sitename,
|
66 | function (result) {
|
67 | //test to verify the successful execution of the command
|
68 | result.exitStatus.should.equal(0);
|
69 | //done is an important callback that signals mocha that the current phase in the
|
70 | //test is complete and the mocha runner should move to the next phase in the test
|
71 | done();
|
72 | });
|
73 | });
|
74 | });
|
75 |
|
76 | //after is execute once at the end of this test-suite
|
77 | after(function (done) {
|
78 | //teardownSuite is a hook provided for the developer to perform steps that
|
79 | //need to be performed once after the execution of the entire test-suite is complete.
|
80 | //A. If nothing needs to be performed then setupSuite() needs to be called as follows:
|
81 | suite.teardownSuite(done);
|
82 |
|
83 | //B. The created artifacts in setupSuite() need to be deleted, so that the suite leaves the
|
84 | //environment in a consistent state.
|
85 | //Then we shall do something like this:
|
86 | suite.teardownSuite(function () {
|
87 | //delete all the artifacts that were created during setup
|
88 | createdSites.forEach(function (item) {
|
89 | suite.execute('site delete %s -q --json', item, function (result) {
|
90 | result.exitStatus.should.equal(0);
|
91 | });
|
92 | });
|
93 | done();
|
94 | });
|
95 | });
|
96 |
|
97 | //beforeEach is executed everytime before the test starts
|
98 | beforeEach(function (done) {
|
99 | //setupTest is a hook provided for the developer to perform steps that
|
100 | //need to be performed before every test
|
101 | //Mechanism to add custom steps for setupTest() is the same that is explained above in setupSuite()
|
102 | suite.setupTest(done);
|
103 | });
|
104 |
|
105 | //afterEach is executed everytime after the test execution is complete,
|
106 | //irrespective of success or failure
|
107 | afterEach(function (done) {
|
108 | //teardownTest is a hook provided for the developer to perform steps that
|
109 | //need to performed after every test
|
110 | //Mechanism to add custom steps for teardownTest() is the same that is explained above in teardownSuite()
|
111 | suite.teardownTest(done);
|
112 | });
|
113 |
|
114 | describe('list', function () {
|
115 | //positive test
|
116 | it('should work', function (done /*Always provide the done callback as a parameter*/) {
|
117 | //execute the command
|
118 | //It is very important to use the --json switch as it becomes easy to parse the output.
|
119 | suite.execute('location list --json', function (result) {
|
120 | //check for zero exit code if you are expecting a success or 1 if you are expecting a failure
|
121 | result.exitStatus.should.equal(0);
|
122 | //parse the expected output from the text property of the result
|
123 | var allResources = JSON.parse(result.text);
|
124 | allResources.some(function (res) {
|
125 | return res.name.match(/Microsoft.Sql\/servers/gi);
|
126 | }).should.be.true;
|
127 | //do not forget the done() callback.
|
128 | done();
|
129 | });
|
130 | });
|
131 |
|
132 | //negative test
|
133 | it('should fail when an invalid resource group is provided', function (done /*Always provide the done callback as a parameter*/) {
|
134 | suite.execute('group log show -n %s --json', 'random_group_name', function (result) {
|
135 | result.exitStatus.should.equal(1);
|
136 | //errorText property of result will contain the expected error message. Doing a Regex match
|
137 | //is always the best option, as one need not change the test if there is a minor modification
|
138 | //in the error message from the server side.
|
139 | result.errorText.should.match(/.*Resource group \'random_group_name\' could not be found.*/ig);
|
140 | //do not forget the done() callback.
|
141 | done();
|
142 | });
|
143 | });
|
144 | });
|
145 | });
|
146 | });
|
147 | ```
|
148 |
|
149 | #### Test Recording Structure
|
150 |
|
151 | Steps to set the **RECORD** mode and selective test recording can be found [here](./TestModes.md).
|
152 |
|
153 | - **Suite**
|
154 | - There is a **"suite-*"** recording file for the test file.
|
155 | - Ids of the artifacts generated during setupSuite() are recorded in this file and retrieved during playback
|
156 | - Test Recording is **not done** during setupSuite() and teardownSuite(). Random ids generated for the created artifacts during this phase, are stored in the suite recording file and are retrieved during playback.
|
157 | - **Test**
|
158 | - Every test is recorded in a separate file. This makes it easier to re-record selective tests due to failures or server side changes.
|
159 | - **Note** If the test is using an artifact, created during **setupSuite()** then all the tests in that suite will have to be re-recorded
|