Skip to content

Commit 939eadb

Browse files
wasin-waeosriwasin-waeosri
committed
add mocking and rdp auth fail case
1 parent ccf5476 commit 939eadb

File tree

3 files changed

+291
-22
lines changed

3 files changed

+291
-22
lines changed

‎README.md‎

Lines changed: 282 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ Please find more detail about the unittest framework from the following resource
7878

7979
The [Refinitiv Data Platform (RDP) APIs](https://developers.refinitiv.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis) provide various Refinitiv data and content for developers via easy-to-use Web-based API.
8080

81-
RDP APIs give developers seamless and holistic access to all of the Refinitiv content such as Historical Pricing, Environmental Social and Governance (ESG), News, Research, etc, and commingled with their content, enriching, integrating, and distributing the data through a single interface, delivered wherever they need it. The RDP APIs delivery mechanisms are the following:
81+
RDP APIs give developers seamless and holistic access to all of the Refinitiv content such as Environmental Social and Governance (ESG), News, Research, etc, and commingled with their content, enriching, integrating, and distributing the data through a single interface, delivered wherever they need it. The RDP APIs delivery mechanisms are the following:
8282
* Request - Response: RESTful web service (HTTP GET, POST, PUT or DELETE)
8383
* Alert: delivery is a mechanism to receive asynchronous updates (alerts) to a subscription.
8484
* Bulks: deliver substantial payloads, like the end-of-day pricing data for the whole venue.
@@ -275,14 +275,16 @@ The ```setUpClass()``` is a class method that is called only onces for the whole
275275

276276
Note: The counterpart method of ```setUpClass()``` method is ```tearDownClass()``` which is called after all tests in the class have run.
277277

278-
The ```test_login_rdp_success()``` is a test case that for the success RDP Authentication login scenario. It just send the RDP Auth Service URL and RDP credentials to the ```rdp_authentication()``` method and check the response token information. Please noticed that a unit test just focusing on if the rdp_authentication() returns none empty/zero token information only. The token content validation would be in a system test (or later) phase.
278+
The ```test_login_rdp_success()``` is a test case for the success RDP Authentication login scenario. It just send the RDP Auth Service URL and RDP credentials to the ```rdp_authentication()``` method and check the response token information. Please noticed that a unit test just focusing on if the rdp_authentication() returns none empty/zero token information only. The token content validation would be in a system test (or later) phase.
279279

280280
```
281281
self.assertIsNotNone(access_token) # Check if access_token is not None or Empty
282282
self.assertIsNotNone(refresh_token) # Check if refresh_token is not None or Empty
283283
self.assertGreater(expires_in, 0) # Check if expires_in is greater then 0
284284
```
285285

286+
Please see more detail about the supported assertion methods on the [unittest framework](https://docs.python.org/3.9/library/unittest.html) page.
287+
286288
The ```test_login_rdp_none_empty_params()``` is a test case that check if the ```rdp_authentication()``` method handles empty or none parameters as expected (throws the TypeError exception and not return token information to a caller).
287289

288290
```
@@ -330,19 +332,292 @@ Ran 2 tests in 0.816s
330332
FAILED (errors=1)
331333
```
332334

333-
However, the test suite above make HTTP requests to RDP APIs in every run. It is not a good idea to flood HTTP requests to RDP APIs repeatedly every time developers run test suite after they updated the code or configurations.
335+
However, the test suite above make HTTP requests to RDP APIs in every run. It is not a good idea to flood requests to external services every time developers run test suite when they have updated the code or configurations.
334336

335-
Unit test cases should be able to run independently without rely on external services, APIs, or components. Those external dependencies add uncontrolled factors (network, data reliability, behaviors, etc) to the unit test scenarios. Those components-to-components testing should be done in integration testing phase.
337+
Unit test cases should be able to run independently without rely on external services or APIs. The external dependencies add uncontrolled factors (such as network connection, data reliability, etc) to unit test cases. Those components-to-components testing should be done in an integration testing phase.
336338

337339
So, how can we unit test HTTP request methods calls without sending any HTTP request messages to an actual server? Fortunately, developers can simulate the HTTP request and response messages with a Mock object.
338340

339341
## Mocking Python HTTP API call with Responses
340342

341-
A mock is a fake object that you construct to look and act like real data within a testing environment. The purpose of mocking is to isolate and focus on the code being tested and not on the behavior or state of external dependencies.
343+
A mock is a fake object that is constructed to look and act like real data within a testing environment. We can simulates various scenario of the real data with mock object, then use a mock library to trick the system into thinking that that mock is the real one.
344+
345+
The purpose of mocking is to isolate and focus on the code being tested and not on the behavior or state of external dependencies. By mocking out external dependencies, developers can run tests as often without being affected by any unexpected changes or irregularities of those dependencies. Mocking also helps developers save time and computing resources if they have to test HTTP requests that fetch a lot of data.
346+
347+
This example project uses the [Responses](https://github.com/getsentry/responses) library for mocking the Requests library.
348+
349+
So, I will start off with a mock object for testing a success RDP login case. Firstly, create a *rdp_test_auth_fixture.json* fixture file with a dummy content of the RDP authentication success response message in a *tests/fixtures* folder.
350+
351+
```
352+
{
353+
"access_token": "access_token_mock1mock2mock3mock4mock5",
354+
"refresh_token": "refresh_token_mock1mock2mock3mock4mock5",
355+
"expires_in": "600",
356+
"scope": "test1 test2 test3 test4 test5",
357+
"token_type": "Bearer"
358+
}
359+
```
360+
361+
Next, load this *rdp_test_auth_fixture.json* file in the ```setUpClass()``` method to a ```mock_valid_auth_json``` class variable. The other test cases can use this mock json object for the dummy access token information.
362+
363+
```
364+
#test_rdp_http_controller.py
365+
366+
import unittest
367+
import requests
368+
import json
369+
import sys
370+
import os
371+
from dotenv import dotenv_values
372+
config = dotenv_values("../.env.test")
373+
374+
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
375+
from rdp_controller import rdp_http_controller
376+
377+
class TestRDPHTTPController(unittest.TestCase):
378+
379+
# A class method called before tests in an individual class are run
380+
@classmethod
381+
def setUpClass(cls):
382+
# Create an app object for the RDPHTTPController class
383+
cls.app = rdp_http_controller.RDPHTTPController()
384+
# Getting the RDP APIs https://api.refinitiv.com base URL.
385+
cls.base_URL = config['RDP_BASE_URL']
386+
# Loading Mock RDP Auth Token success Response JSON
387+
with open('./fixtures/rdp_test_auth_fixture.json', 'r') as auth_fixture_input:
388+
cls.mock_valid_auth_json = json.loads(auth_fixture_input.read())
389+
390+
```
391+
392+
The Responses library lets developers register mock responses to the Requests library and cover test method with ```responses.activate``` decorator. Developers can specify the endpoint URL, HTTP method, status response, response message, etc of that request via a ```responses.add()``` method.
393+
394+
Example Code:
395+
396+
```
397+
#test_rdp_http_controller.py
398+
399+
import unittest
400+
import responses
401+
import requests
402+
import json
403+
import sys
404+
import os
405+
406+
...
407+
408+
class TestRDPHTTPController(unittest.TestCase):
409+
410+
# A class method called before tests in an individual class are run
411+
@classmethod
412+
def setUpClass(cls):
413+
...
414+
415+
416+
@responses.activate
417+
def test_login_rdp_success(self):
418+
"""
419+
Test that it can logged in to the RDP Auth Service (using Mock)
420+
"""
421+
auth_endpoint = self.base_URL + config['RDP_AUTH_URL']
422+
423+
mock_rdp_auth = responses.Response(
424+
method= 'POST',
425+
url = auth_endpoint,
426+
json = self.mock_valid_auth_json,
427+
status= 200,
428+
content_type= 'application/json'
429+
)
430+
responses.add(mock_rdp_auth)
431+
432+
username = config['RDP_USERNAME']
433+
password = config['RDP_PASSWORD']
434+
client_id = config['RDP_CLIENTID']
435+
access_token = None
436+
refresh_token = None
437+
expires_in = 0
438+
439+
# Calling RDPHTTPController rdp_authentication() method
440+
access_token, refresh_token, expires_in = self.app.rdp_authentication(auth_endpoint, username, password, client_id)
441+
442+
# Assertions
443+
...
444+
```
445+
446+
The code above set a Responses mock object with the *https://api.refinitiv.com/auth/oauth2/v1/token* URL and HTTP *POST* method. The Requests library then returns a ```mock_valid_auth_json``` JSON message with HTTP status *200* and Content-Type *application/json* to the application for all HTTP *POST* request messages to *https://api.refinitiv.com/auth/oauth2/v1/token* URL without any network operations between the machine and the actual RDP endpoint.
447+
448+
### Testing Invalid RDP Authentication Request-Response
449+
450+
This mock object also useful for testing the false cases such as invalid login too.
451+
452+
```
453+
#test_rdp_http_controller.py
454+
455+
...
456+
457+
class TestRDPHTTPController(unittest.TestCase):
458+
459+
# A class method called before tests in an individual class are run
460+
@classmethod
461+
def setUpClass(cls):
462+
...
463+
464+
@responses.activate
465+
def test_login_rdp_success(self):
466+
...
467+
468+
469+
@responses.activate
470+
def test_login_rdp_invalid(self):
471+
"""
472+
Test that it handle some invalid credentials
473+
"""
474+
auth_endpoint = self.base_URL + config['RDP_AUTH_URL']
475+
476+
mock_rdp_auth_invalid = responses.Response(
477+
method= 'POST',
478+
url = auth_endpoint,
479+
json = {
480+
'error': 'invalid_client',
481+
'error_description':'Invalid Application Credential.'
482+
},
483+
status= 401,
484+
content_type= 'application/json'
485+
)
486+
responses.add(mock_rdp_auth_invalid)
487+
username = 'wrong_user1'
488+
password = 'wrong_password1'
489+
client_id = 'XXXXX'
490+
access_token = None
491+
refresh_token = None
492+
expires_in = 0
493+
with self.assertRaises(requests.exceptions.HTTPError) as exception_context:
494+
access_token, refresh_token, expires_in = self.app.rdp_authentication(auth_endpoint, username, password, client_id)
495+
496+
self.assertIsNone(access_token)
497+
self.assertIsNone(refresh_token)
498+
self.assertEqual(expires_in, 0)
499+
self.assertEqual(exception_context.exception.response.status_code, 401)
500+
self.assertEqual(exception_context.exception.response.reason, 'Unauthorized')
342501
343-
We can simulates various scenario of the real data with mock object, then use a mock library to trick the system into thinking that that mock is the real data. By mocking out external dependencies, developers can run tests as often without being affected by any unexpected changes or irregularities of those dependencies. Mocking also helps developers save time and computing resources if they have to test HTTP requests that fetch a lot of data.
502+
json_error = json.loads(exception_context.exception.response.text)
503+
self.assertIn('error', json_error)
504+
self.assertIn('error_description', json_error)
344505
345-
This example project uses the [Responses](https://github.com/getsentry/responses) library which is specifically for mocking the Requests library.
506+
```
507+
508+
The ```test_login_rdp_invalid()``` method is a test case for the failure RDP Authentication login scenario. We set a Responses mock object for the *https://api.refinitiv.com/auth/oauth2/v1/token* URL and HTTP *POST* method with the expected error response message and status (401 - Unauthorized).
509+
510+
Once the ```rdp_authentication()``` method is called, the test case verify if the method raises the ```requests.exceptions.HTTPError``` exception with the expected error message and status. The test case also make assertions to check if the method not return token information to a caller.
511+
512+
With mocking, a test case never need to send actual request messages to the RDP APIs, so we can test more scenarios for other RDP services too.
513+
514+
## <a id="rdp_get_data"></a>Unit Testing for RDP APIs Data Request
515+
516+
That brings us to requesting the RDP APIs data. All subsequent REST API calls use the Access Token via the *Authorization* HTTP request message header as shown below to get the data.
517+
- Header:
518+
* Authorization = ```Bearer <RDP Access Token>```
519+
520+
Please notice *the space* between the ```Bearer``` and ```RDP Access Token``` values.
521+
522+
The application then creates a request message in a JSON message format or URL query parameter based on the interested service and sends it as an HTTP request message to the Service Endpoint. Developers can get RDP APIs the Service Endpoint, HTTP operations, and parameters from Refinitiv Data Platform's [API Playground page](https://api.refinitiv.com/) - which is an interactive documentation site developers can access once they have a valid Refinitiv Data Platform account.
523+
524+
The example console application consume content from the following the RDP Services:
525+
- ESG Service ```/data/environmental-social-governance/<version>/views/scores-full``` endpoint that provides full coverage of Refinitiv's proprietary ESG Scores with full history for consumers.
526+
- Discovery Search Explore Service ```/discover/search/<version>/explore``` endpoint that explore Refinitiv data based on searching options.
527+
528+
However, this development article covers the ESG Service test cases only. The Discovery Search Explore Service's test cases have the same test logic as the ESG's test cases.
529+
530+
## Testing ESG Data
531+
532+
Now let me turn to testing the Environmental Social and Governance (ESG) service endpoint.
533+
534+
### Testing a valid RDP ESG Request-Response
535+
536+
I will begin by creating a fixture file with a valid ESG dummy response message. A file is *rdp_test_esg_fixture.json* in a *tests/fixtures* folder.
537+
538+
```
539+
{
540+
"links": {
541+
"count": 5
542+
},
543+
"variability": "variable",
544+
"universe": [
545+
{
546+
"Instrument": "TEST.RIC",
547+
"Company Common Name": "TEST ESG Data",
548+
"Organization PermID": "XXXXXXXXXX",
549+
"Reporting Currency": "USD"
550+
}
551+
],
552+
"data": [
553+
[
554+
"TEST.RIC",
555+
"2021-12-31",
556+
99.9999999999999,
557+
99.9999999999999,
558+
...
559+
],
560+
...
561+
],
562+
...
563+
,
564+
"headers": [
565+
....
566+
{
567+
"name": "TEST 1",
568+
"title": "ESG Score",
569+
"type": "number",
570+
"decimalChar": ".",
571+
"description": "TEST description"
572+
}...
573+
]
574+
}
575+
```
576+
Next, create the ```test_request_esg()``` method in test_rdp_http_controller.py file to test the valid ESG data request-response test case.
577+
578+
```
579+
#test_rdp_http_controller.py
580+
...
581+
582+
class TestRDPHTTPController(unittest.TestCase):
583+
584+
# A class method called before tests in an individual class are run
585+
@classmethod
586+
def setUpClass(cls):
587+
...
588+
589+
...
590+
591+
@responses.activate
592+
def test_request_esg(self):
593+
"""
594+
Test that it can request ESG Data
595+
"""
596+
esg_endpoint = self.base_URL + config['RDP_ESG_URL']
597+
598+
#Mock RDP ESG View Score valid response JSON
599+
with open('./fixtures/rdp_test_esg_fixture.json', 'r') as esg_fixture_input:
600+
mock_esg_data = json.loads(esg_fixture_input.read())
601+
602+
mock_rdp_esg_viewscore = responses.Response(
603+
method= 'GET',
604+
url = esg_endpoint,
605+
json = mock_esg_data,
606+
status= 200,
607+
content_type= 'application/json'
608+
)
609+
responses.add(mock_rdp_esg_viewscore)
610+
611+
esg_endpoint = self.base_URL + config['RDP_ESG_URL']
612+
universe = 'TEST.RIC'
613+
response = self.app.rdp_getESG(esg_endpoint, self.mock_valid_auth_json['access_token'], universe)
614+
615+
# verifying basic response
616+
self.assertIn('data', response) #Check if return JSON has 'data' fields
617+
self.assertIn('headers', response) #Check if return JSON has 'headers' fields
618+
self.assertIn('universe', response) #Check if return JSON has 'universe' fields
619+
620+
```
346621

347622
## Project Structure
348623

‎tests/fixtures/rdp_test_auth_fixture.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"access_token": "",
3-
"refresh_token": "",
2+
"access_token": "access_token_mock1mock2mock3mock4mock5",
3+
"refresh_token": "refresh_token_mock1mock2mock3mock4mock5",
44
"expires_in": "600",
55
"scope": "test1 test2 test3 test4 test5",
66
"token_type": "Bearer"

‎tests/test_rdp_http_controller.py‎

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@
1414
import responses
1515
import requests
1616
import json
17-
import string
18-
import secrets
1917
import sys
2018
import os
2119
from dotenv import dotenv_values
@@ -33,15 +31,9 @@ def setUpClass(cls):
3331
cls.app = rdp_http_controller.RDPHTTPController()
3432
# Getting the RDP APIs https://api.refinitiv.com base URL.
3533
cls.base_URL = config['RDP_BASE_URL']
36-
# Creating mock access_token and refresh_token
37-
alphabet = string.ascii_letters + string.digits
38-
mock_access_token = ''.join(secrets.choice(alphabet) for i in range(250))
39-
mock_refresh_token = ''.join(secrets.choice(alphabet) for i in range(36))
40-
# Mock RDP Auth valid Response JSON
34+
# Loading Mock RDP Auth Token success Response JSON
4135
with open('./fixtures/rdp_test_auth_fixture.json', 'r') as auth_fixture_input:
4236
cls.mock_valid_auth_json = json.loads(auth_fixture_input.read())
43-
cls.mock_valid_auth_json['access_token'] = f'{mock_access_token}'
44-
cls.mock_valid_auth_json['refresh_token'] = f'{mock_refresh_token}'
4537

4638
# Mock RDP Auth Token Expire Response JSON
4739
with open('./fixtures/rdp_test_token_expire_fixture.json', 'r') as auth_expire_fixture_input:
@@ -93,8 +85,10 @@ def test_login_rdp_refreshtoken(self):
9385
auth_endpoint = self.base_URL + config['RDP_AUTH_URL']
9486

9587
#Create new access_token
96-
alphabet = string.ascii_letters + string.digits
97-
self.mock_valid_auth_json['access_token'] = ''.join(secrets.choice(alphabet) for i in range(250))
88+
# alphabet = string.ascii_letters + string.digits
89+
# self.mock_valid_auth_json['access_token'] = ''.join(secrets.choice(alphabet) for i in range(250))
90+
91+
self.mock_valid_auth_json['access_token'] = 'new_access_token_mock1mock2mock3mock4mock5mock6'
9892

9993
mock_rdp_auth = responses.Response(
10094
method= 'POST',
@@ -136,8 +130,8 @@ def test_login_rdp_invalid(self):
136130
content_type= 'application/json'
137131
)
138132
responses.add(mock_rdp_auth_invalid)
139-
username = 'User2'
140-
password = 'password1'
133+
username = 'wrong_user1'
134+
password = 'wrong_password1'
141135
client_id = 'XXXXX'
142136
access_token = None
143137
refresh_token = None

0 commit comments

Comments
 (0)