Skip to content

Commit bb552a9

Browse files
committed
Merge branch 'rebrand_2' into master
1. Rebrand products 2. Fix Typo errors 3. Fix links
2 parents 5015f02 + 2af6e36 commit bb552a9

File tree

3 files changed

+70
-63
lines changed

3 files changed

+70
-63
lines changed

README.md

Lines changed: 42 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,58 @@
11
# WebSocket API Machine Readable News Example with Python
2-
- Last update: September 2020
2+
- Last update: Jan 2021
33
- Environment: Windows and Linux OS
44
- Compiler: Python
5-
- Prerequisite: ADS and ADH servers version 3.2.1 and above, MRN service
5+
- Prerequisite: Refinitiv Real-Time Advanced Data Hub and Refinitiv Real-Time Advanced Distribution servers version 3.2.1 and above, MRN service
66

77
## Overview
88

9-
This example shows how to writing the [Websocket API for Pricing Streaming and Real-Time Service](https://developers.refinitiv.com/elektron/websocket-api) aka Websocket API application to subscribe Machine Readable News (MRN) from Refinitiv Real-Time Distribution System (ADH and ADS servers). The example just connects to Refinitiv Real-Time via a WebSocket connection, then subscribes and display MRN News data in a console or classic Jupyter Notebook. The project are implemented with Python language for both console and Jupyter Notebook applications, but the main concept for consuming and assembling MRN News messages are the same for all technologies.
9+
This example shows how developers may use the [Websocket API for Pricing Streaming and Real-Time Service](https://developers.refinitiv.com/en/api-catalog/elektron/refinitiv-websocket-api) aka Websocket API application to subscribe Machine Readable News (MRN) from Refinitiv Real-Time Distribution System (Refinitiv Real-Time Advanced Data Hub and Refinitiv Real-Time Advanced Distribution Server). The example just connects to Refinitiv Real-Time via a WebSocket connection, then subscribes and displays MRN News data in a console or classic Jupyter Notebook. The project is implemented with Python language for both console and Jupyter Notebook applications, but the main concept for consuming and assembling MRN News messages are the same for all technologies.
1010

1111
For example of MRN from Refinitiv Real-Time Optimized (formerly known as ERT in Cloud), please switch to [ERT-in-Cloud](https://github.com/Refinitiv-API-Samples/Example.WebSocketAPI.Python.MRN/tree/ERT-in-Cloud) branch.
1212

13-
Please see a full documentation of this example application in [this article](https://developers.refinitiv.com/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
13+
Please see the full documentation of this example application in [this article](https://developers.refinitiv.com/en/article-catalog/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
1414

15-
*Note:* The news message is in UTF-8 JSON string format. Some news messages that contains special unicode character may not be able to show in Windows OS console (cmd, git bash, powershell, etc) due to the OS limitation. Those messages will be print as ```UnicodeEncodeError exception. Cannot decode unicode character``` message in a console instead.
15+
*Note:* The news message is in UTF-8 JSON string format. Some news messages that contain special Unicode character may not be able to show in Windows OS console (cmd, git bash, powershell, etc) due to the OS limitation. Those messages will be print as ```UnicodeEncodeError exception. Cannot decode Unicode character``` message in a console instead.
1616

1717
## Prerequisite
1818

1919
This example is focusing on the Refinitiv Machine Readable News (MRN) data processing only. I highly recommend you check the [WebSocket API Tutorials](https://developers.refinitiv.com/en/api-catalog/elektron/refinitiv-websocket-api/tutorials) page if you are not familiar with WebSocket API.
2020

21-
The Tutorials page provide a step by step guide (connect, login, request data, parse data, etc) for developers who interested in developing a WebSocket application to consume real-time data from Refinitiv Real-Time.
21+
The Tutorials page provides a step-by-step guide (connect, log in, request data, parse data, etc) for developers who are interested in developing a WebSocket application to consume real-time data from Refinitiv Real-Time.
2222

2323
## Machine Readable News Overview
2424

25-
Refinitiv Machine Readable News (MRN) is an advanced service for automating the consumption and systematic analysis of news. It delivers deep historical news archives, ultra-low latency structured news and news analytics directly to your applications. This enables algorithms to exploit the power of news to seize opportunities, capitalize on market inefficiencies and manage event risk.
25+
Refinitiv Machine Readable News (MRN) is an advanced service for automating the consumption and systematic analysis of news. It delivers deep historical news archives, ultra-low latency structured news and news analytics directly to your applications. This enables algorithms to exploit the power of news to seize opportunities, capitalize on market inefficiencies, and manage event risk.
2626

2727
### MRN Data model
28-
MRN is published over Refinitiv Real-Time using an Open Message Model (OMM) envelope in News Text Analytics domain messages. The Real-time News content set is made available over MRN_STORY RIC. The content data is contained in a FRAGMENT field that has been compressed, and potentially fragmented across multiple messages, in order to reduce bandwidth and message size.
28+
29+
MRN is published over Refinitiv Real-Time using an Open Message Model (OMM) envelope in News Text Analytics domain messages. The Real-time News content set is made available over MRN_STORY RIC. The content data is contained in a FRAGMENT field that has been compressed and potentially fragmented across multiple messages, to reduce bandwidth and message size.
2930

3031
A FRAGMENT field has a different data type based on a connection type:
31-
* RSSL connection (RTSDK [C++](https://developers.refinitiv.com/elektron/elektron-sdk-cc)/[Java](https://developers.refinitiv.com/elektron/elektron-sdk-java)): BUFFER type
32-
* WebSocket connection: Base64 ascii string
32+
* RSSL connection (RTSDK [C++](https://developers.refinitiv.com/en/api-catalog/elektron/elektron-sdk-cc)/[Java](https://developers.refinitiv.com/en/api-catalog/elektron/elektron-sdk-java)): BUFFER type
33+
* WebSocket connection: Base64 ASCII string
3334

3435
The data goes through the following series of transformations:
3536

3637
1. The core content data is a UTF-8 JSON string
3738
2. This JSON string is compressed using gzip
38-
3. The compressed JSON is split into a number of fragments (BUFFER or Base64 ascii string) which each fit into a single update message
39+
3. The compressed JSON is split into several fragments (BUFFER or Base64 ASCII string) which each fit into a single update message
3940
4. The data fragments are added to an update message as the FRAGMENT field value in a FieldList envelope
4041

4142
![Figure-1](diagram/mrn_process.png "MRN data compression process")
4243

43-
Therefore, in order to parse the core content data, the application will need to reverse this process. The WebSocket application also need to convert a received Base64 string in a FRAGMENT field to bytes data before further process this field. This application uses Python [base64](https://docs.python.org/3/library/base64.html) and [zlib](https://docs.python.org/3/library/zlib.html) modules to decode Base64 string and decompress JSON string.
44+
Therefore, to parse the core content data, the application will need to reverse this process. The WebSocket application also needs to convert a received Base64 string in a FRAGMENT field to bytes data before further process this field. This application uses Python [base64](https://docs.python.org/3/library/base64.html) and [zlib](https://docs.python.org/3/library/zlib.html) modules to decode Base64 string and decompress JSON string.
4445

4546
If you are not familiar with MRN concept, please visit the following resources which will give you a full explanation of the MRN data model and implementation logic:
4647
* [Webinar Recording: Introduction to Machine Readable News](https://developers.refinitiv.com/news#news-accordion-nid-12045)
47-
* [Introduction to Machine Readable News (MRN) with Enterprise Message API (EMA)](https://developers.refinitiv.com/article/introduction-machine-readable-news-mrn-elektron-message-api-ema).
48-
* [MRN Data Models and Refinitiv Real-Time SDK Implementation Guide](https://developers.refinitiv.com/elektron/elektron-sdk-java/docs?content=8736&type=documentation_item).
49-
* [Introduction to Machine Readable News with WebSocket API](https://developers.refinitiv.com/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
48+
* [Introduction to Machine Readable News (MRN) with Enterprise Message API (EMA)](https://developers.refinitiv.com/en/article-catalog/article/introduction-machine-readable-news-mrn-elektron-message-api-ema).
49+
* [MRN Data Models and Refinitiv Real-Time SDK Implementation Guide](https://developers.refinitiv.com/en/api-catalog/elektron/elektron-sdk-java/documentation#mrn-data-models-implementation-guide).
50+
* [Introduction to Machine Readable News with WebSocket API](https://developers.refinitiv.com/en/article-catalog/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
5051

5152
## Example Prerequisite
52-
This example requires the following dependencies softwares and libraries.
53-
1. ADH and ADS servers version 3.2.x with WebSocket connection and MRN Service.
53+
This example requires the following dependencies software and libraries.
54+
1. Refinitiv Real-Time Advanced Data Hub and Refinitiv Real-Time Advanced Distribution
55+
Server version 3.2.x with WebSocket connection and MRN Service.
5456
2. [Python](https://www.python.org/) compiler and runtime
5557
3. Python's [requests 2.x](https://pypi.org/project/requests/) library.
5658
4. Python's [websocket-client](https://pypi.org/project/websocket-client/) library (*version 0.49 or greater*).
@@ -61,24 +63,25 @@ This example requires the following dependencies softwares and libraries.
6163
- The Python example has been qualified with Python versions 3.6.5 and Python 3.7.4 (Docker 19.03.1 - CentOS 7)
6264
- Please refer to the [pip installation guide page](https://pip.pypa.io/en/stable/installing/) if your environment does not have the [pip tool](https://pypi.org/project/pip/) installed.
6365
- If your environment already have a websocket-client library installed, you can use ```pip list``` command to verify a library version, then use ```pip install --upgrade websocket-client``` command to upgrade websocket-client library.
64-
- It is not advisable to change the ADH/ADS configuration, if you are not familiar with the configuration procedures. Please consult your Market Data administrator for any questions regarding ADS/ADH-MRN service configuration.
66+
- It is not advisable to change the Refinitiv Real-Time Distribution System configuration if you are not familiar with the configuration procedures. Please consult your Market Data administrator for any questions regarding ADS/ADH-MRN service configuration.
6567

6668

6769
## Application Files
6870
This example project contains the following files and folders
6971
1. *mrn_console_app.py*: The example application file
7072
2. *notebook_python/mrn_notebook_app.ipynb*: The example Jupyter Notebook application file
7173
3. *Dockerfile*: The example application Dockerfile
72-
3. *requirements.txt*: The application dependencies configurationf file
74+
3. *requirements.txt*: The application dependencies configuration file
7375
4. LICENSE.md: Project's license file
7476
5. README.md: Project's README file
7577

7678
## How to run this example
7779

78-
Please be informed that your ADS/ADH server should have a Service that contain MRN data. The first step is unzip or download the example project folder into a directory of your choice, then choose how to run application based on your environment below.
80+
Please be informed that your Refinitiv Real-Time Advanced Data Hub and Refinitiv Real-Time Advanced Distribution
81+
Server should have a Service that contains MRN data. The first step is to unzip or download the example project folder into a directory of your choice, then choose how to run the application based on your environment below.
7982

8083
### A console example
81-
1. Go to project folder in console
84+
1. Go to the project folder in the console
8285
2. Run ```$> pip install -r requestments.txt``` command in a console to install all the dependencies libraries.
8386
3. Then you can run mrn_console_app.py application with the following command
8487
```
@@ -87,21 +90,21 @@ Please be informed that your ADS/ADH server should have a Service that contain
8790
Optionally, the application subscribes ```MRN_STORY``` RIC code from ADS by default. You can pass your interested MRN RIC code to ```--ric``` parameter on the application command line. The supported MRN RIC codes are ```MRN_STORY```, ```MRN_TRNA```, ```MRN_TRNA_DOC``` and ```MRN_TRSI``` only. the application
8891
8992
### Docker example
90-
1. Go to project folder in console
93+
1. Go to the project folder in the console
9194
2. Run ```$> docker build -t <project tag name> .``` command in a console to build an image from a Dockerfile.
9295
```
93-
$> docker build -t esdk_ws_mrn_python .
96+
$> docker build -t rtsdk_ws_mrn_python .
9497
```
95-
3. Once the build is success, you can create and run the container with the following command
98+
3. Once the build is a success, you can create and run the container with the following command
9699
```
97-
$> docker run esdk_ws_mrn_python --hostname <ADS server IP Address/Hostname> --port <WebSocket Port> --ric <MRN RIC name>
100+
$> docker run rtsdk_ws_mrn_python --hostname <ADS server IP Address/Hostname> --port <WebSocket Port> --ric <MRN RIC name>
98101
```
99102
### Classic Jupyter Notebook example
100103
101-
Please be informed that Python [Ananconda](https://www.anaconda.com/distribution/) or [MiniConda](https://docs.conda.io/en/latest/miniconda.html) distribution/package manager is highly recommend for running Jupyter Notebook example.
104+
Please be informed that Python [Ananconda](https://www.anaconda.com/distribution/) or [MiniConda](https://docs.conda.io/en/latest/miniconda.html) distribution/package manager is highly recommended for running the Jupyter Notebook example.
102105
103-
1. Open Anaconda Prompt and go to project's folder
104-
2. Run the following command in a Anaconda Prompt to create Conda environment named *mrn_python_notebook* for the project.
106+
1. Open Anaconda Prompt and go to the project's folder
107+
2. Run the following command in an Anaconda Prompt to create a Conda environment named *mrn_python_notebook* for the project.
105108
```
106109
(base) $>conda create --name mrn_python_notebook python=3.7
107110
```
@@ -117,15 +120,15 @@ Please be informed that Python [Ananconda](https://www.anaconda.com/distributio
117120

118121
(mrn_python_notebook) $>pip install -r requestments.txt
119122
```
120-
5. Go to project's notebook folder in console
121-
6. Run the following command in a console to start classic Jupyter Notebook in the notebook folder.
123+
5. Go to the project's notebook folder in the console
124+
6. Run the following command in a console to start the classic Jupyter Notebook in the notebook folder.
122125
```
123126
(mrn_python_notebook) $>jupyter notebook
124127
```
125128
7. Open *mrn_notebook_app.ipynb* Notebook document, then follow through each notebook cell.
126129
127130
*Note:*
128-
- You can install a classic Jupyter Notebook on your local machine and then test the example on the machine. The alternate choice is a free Jupyter Notebook on cloud environment such as [Azure Notebook](https://notebooks.azure.com/) provided by Microsoft. You can find more details from [this tutorial](https://docs.microsoft.com/en-us/azure/notebooks/tutorial-create-run-jupyter-notebook). If you are not familiar with Jupyter Notebook, the following [tutorial](https://www.datacamp.com/community/tutorials/tutorial-jupyter-notebook) created by DataCamp may help.
131+
- You can install a classic Jupyter Notebook on your local machine and then test the example on the machine. The alternate choice is a free Jupyter Notebook on cloud environments such as [Azure Notebook](https://notebooks.azure.com/) provided by Microsoft. You can find more details from [this tutorial](https://docs.microsoft.com/en-us/azure/notebooks/tutorial-create-run-jupyter-notebook). If you are not familiar with Jupyter Notebook, the following [tutorial](https://www.datacamp.com/community/tutorials/tutorial-jupyter-notebook) created by DataCamp may help.
129132
130133
## Example Results
131134
### Send MRN_STORY request to ADS
@@ -219,13 +222,15 @@ News = {'altId': 'nIdw5d8Hwd', 'audiences': ['NP:CNRA', 'NP:IDXN'], 'body': 'Lap
219222
```
220223
221224
## References
222-
* [Refinitiv Real-Time SDK Family page](https://developers.refinitiv.com/elektron) on the [Refinitiv Developer Community](https://developers.refinitiv.com/) web site.
223-
* [WebSocket API page](https://developers.refinitiv.com/websocket-api).
225+
226+
For further details, please check out the following resources:
227+
* [Refinitiv Real-Time & Distribution Family page](https://developers.refinitiv.com/en/use-cases-catalog/refinitiv-real-time) on the [Refinitiv Developer Community](https://developers.refinitiv.com/) web site.
228+
* [WebSocket API page](https://developers.refinitiv.com/en/api-catalog/elektron/refinitiv-websocket-api).
224229
* [Developer Webinar Recording: Introduction to Electron WebSocket API](https://www.youtube.com/watch?v=CDKWMsIQfaw).
225-
* [Introduction to Machine Readable News with WebSocket API](https://developers.refinitiv.com/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
226-
* [Machine Readable News (MRN) & N2_UBMS Comparison and Migration Guide](https://developers.refinitiv.com/article/machine-readable-news-mrn-n2_ubms-comparison-and-migration-guide).
227-
* [Introduction to Machine Readable News (MRN) with Enterprise Message API (EMA)](https://developers.refinitiv.com/article/introduction-machine-readable-news-mrn-elektron-message-api-ema).
228-
* [MRN Data Models and Real-Time SDK Implementation Guide](https://developers.refinitiv.com/elektron/elektron-sdk-java/docs?content=8736&type=documentation_item).
230+
* [Introduction to Machine Readable News with WebSocket API](https://developers.refinitiv.com/en/article-catalog/article/introduction-machine-readable-news-elektron-websocket-api-refinitiv).
231+
* [Machine Readable News (MRN) & N2_UBMS Comparison and Migration Guide](https://developers.refinitiv.com/en/article-catalog/article/machine-readable-news-mrn-n2_ubms-comparison-and-migration-guide).
232+
* [Introduction to Machine Readable News (MRN) with Enterprise Message API (EMA)](https://developers.refinitiv.com/en/article-catalog/article/introduction-machine-readable-news-mrn-elektron-message-api-ema).
233+
* [MRN Data Models and Real-Time SDK Implementation Guide](https://developers.refinitiv.com/en/api-catalog/elektron/elektron-sdk-java/documentation#mrn-data-models-implementation-guide).
229234
* [MRN WebSocket JavaScript example on GitHub](https://github.com/Refinitiv-API-Samples/Example.WebSocketAPI.Javascript.NewsMonitor).
230235
* [MRN WebSocket C# NewsViewer example on GitHub](https://github.com/Refinitiv-API-Samples/Example.WebSocketAPI.CSharp.MRNWebSocketViewer).
231236

mrn_console_app.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ def processMRNUpdate(ws, message_json): # process incoming News Update messages
7575
guid = None
7676

7777
try:
78-
# Get data for all requried fields
78+
# Get data for all required fields
7979
fragment = base64.b64decode(fields_data["FRAGMENT"])
8080
frag_num = int(fields_data["FRAG_NUM"])
8181
guid = fields_data["GUID"]

0 commit comments

Comments
 (0)