Script Generator for Distributed Testing: The ASAP Engine Test Suite (needs revision)

General

This page serves as a user manual for using the program stored in the package src/ASAPEngineTestSuite.

---

Introduction

The script generator's main goal is the output of command lists for test actors to execute. As of now (August 2025), the generator covers two network topologies using TCP, namely the star and chain formations. The former consists of a central node, communicating directly with other nodes. In the latter, all peers, except for the first peer, connect to the previous peer, forming a linear path spanning over all the nodes.

Test Structure:

Terminology:

Test Scenario Script:

A list of commands that are executed by the primary actors of the test scenarios. A test script is a text file containing a list of commands to be passed to the SharkNetMessenger CLI.

Note: Commands passed this way will be executed in a cascade. No interference is possible due to the speed of execution.

Example:

sendMessage HelloWorld
openTCP 9999
connectTCP 123.4.56.000 4444
lsMessages
closeTCP 9999
exit

Messenger Log:

The SharkNetMessenger provides users with an interactive command-line interface. For testing and evaluation purposes, we redirect standard output to a file called the 'messenger log'.

ASAP Log:

This log is created and written to by the messenger process by the ASAP Engine. It keeps track of the processes regarding the ASAPPeer, ASAPEncounter, SharkPKI, other communication operations and more.

Actors and Roles

The test is intended to contain three main types of actors.

1. Test Host

The test host is the entity responsible for the generation of the test scripts and distribution of necessary information such as IP addresses and scenario scripts containing the commands to be executed by the wrapper and subject peers. The host also collects the ASAP Engine and SharkNetMessenger logs when a test run concludes.

2. Peer Wrapper

The peer wrapper is the entity, which is responsible for filtering the correct scenario to be passed to the subject peer and sequentially the collection of the ASAP and SharkNetMessenger logs to be forwarded to the test host. There is one wrapper peer directory per subject peer.

3. Primary test peers (P1-P<n>)

The subject peers are the nodes which execute the test scenarios. The logs they produce during the execution of the scenario make up the result of each test.

In addition to the communication between the subject peers, the file transfer between the wrapper peers and the test host also carries out the file transfer using SharkNetMessenger.

Processes and Phases

There is only one precondition for a test run on TCP to be initialised, which is that a node participating in a test needs to possess the IP address of the host. There is currently no other way for a participant to receive the files using SNM.

There are three phases of a test run.

Phase 1:

The first phase is the communication between the test host and the wrapper peers.

• The test host starts the messenger and opens a port for incoming connections.
• The wrapper peers connect to the test host through its TCP port (default port number: 9999).
• Each participating node writes its IP address to a file. The wrapper peers transfer this file over to the test host.
• The gossip protocol ensures that every node receives each file containing the IP addresses of a test participant.

Note: It is not necessary in every scenario that all peers are provided with each address. The wrapper peer is responsible for passing the right file name onto the subject peer to guarantee the correct execution.

• The host, on the other hand, sends all the command lists of the scenarios the wrapper peers and the test subject peers will execute.
• As soon as the file transfer is completed on both sides, the first phase concludes.
• The host and the wrappers stay active during the upcoming phases.

Phase 2:

• The wrapper peer starts its subject peer using the correct scenario. The correct scenario can be identified by the index, which must match that of the executing peer.
• The subject peer starts its own process running the SNM to execute the commands listed in the scenario.
  • This process automatically creates a log file named asapLogs<peerName>.txt. The output of the CLI of the Messenger should be redirected onto a file called snm_<PeerName>.txt for the correct usage of the generated scripts in the next phase.
• As soon as the execution of the test scenarios is completed by the subject peers, this phase is concluded.

Phase 3:

• The final phase plays out between the wrapper peers and the test host.
• The wrapper peers transfer the ASAP engine and SNM logs to the test host for evaluation.
• The final phase and thus the test concludes, as soon as all the file transfer attempts by the wrapper peers finalize, regardless of success.

It is then recommended to archive the test host directory, in order to store test results of each execution separately.

Attention: File transfer using SNM does not differentiate between files with identical names. This may disturb test results if not regarded, since only the first file under the same name is saved in the root of the running directory and is not overwritten by subsequent identically named files.

It is regarded as a success if both log files of each peer reach the test host. As of August 2025, the evaluation and detailed analysis of the logs are left to the user. This page will be updated in the case of the release of an evaluation helper tool.

Script Generator: Components and Usage

Creating the Executable:

The executable .jar file can be generated with the following script (UNIX):

rm -rf bin
mkdir bin
javac -d bin $(find src/ASAPEngineTestSuite -name "*.java")
echo "Manifest-Version: 1.0" > manifest.mf
echo "Main-Class: asapEngineTestSuite.TestScriptOutput" >> manifest.mf
jar cfm scriptgenerator.jar manifest.mf -C bin .
rm manifest.mf

Input

The script generator requires the following parameters as input:

• the host IP address
• scenario index (As of August 2025 only TCP star, as index 0 and TCP chain, as index 1 are implemented. (See Introduction above)).
• peer count
• numerical size of file to be transferred in the test scenario
• the unit of the file size (currently supported units: Bytes, KB, MB, GB)
• name of the aforementioned file

... to generate the following ...

Output

• the command list the wrapper peers will execute to receive the test scenario (called "receiveCommands.txt")
• the specific test scenario customized to each peer (referred to as "scenarioScript<PeerName>.txt")
• Host command list to distribute the test scenarios and receive the test logs (referred to as "host command list")
• Wrapper peer command list to send the subject peer logs to host runScenarioP<n>.txt
• A file with the specified size and name, if no file under the given name exists in the directory root. (referred to as "content file")

The structure and purpose of the output files will be explained in the following sections.

receiveCommands.txt

Example (with filler added by the script generator):

sendMessage ipAddress_FILLER_PEERNAME.txt sn/file  
wait 1000  
connectTCP <hostIPAddress> 9999  
wait 500  
lsMessages  
wait 300000  
exit

This file is executed by the wrapper peer using the SharkMessenger. Its purpose is to send a file with the peer's IP address and to receive the scenario scripts. The wrapper peer keeps the session running, to be able to collect the logs after the test has been completed.

This file needs to be edited after its creation. The filler in the first line needs to be replaced by the peer name (P1, P2, ...). The host IP address is passed during the execution of the script generator and is just highlighted here, as it is not a constant value. One possibility would be to execute the following:

peerName=$(basename $(pwd))  

sed -i "s/FILLER_PEERNAME/$peerName/g" receiveCommands.txt

Afterward, the command list is ready for use.

Host Command List:

Example:

sendMessage runScenario1.txt sn/file  
sendMessage runScenario2.txt sn/file  
sendMessage runScenario3.txt sn/file  
openTCP 9999  
wait 18000  
lsMessages  
wait 90000  
exit

The host command list sends the scenario scripts and keeps the session running for later phases. It is executed by the test host.

Log Sending Commands: runScenario.txt

Example:

wait 500  
sendMessage snm_P1.txt sn/file  
sendMessage asapLogsP1.txt sn/file  
connectTCP <HostIpAddress> 9999  
wait 1000  
exit

This command list is executed by the wrapper peers. It is used to send the logs created by the inner (subject) peers during the test execution.

scenarioScriptP<n>.txt

This command list is executed by the subject peers. It consists of the scenario to be tested in the run. Thus, each scenario has a different type and order of events. Example scenarioScript to test the chain topology:

sendMessage 5_200Kb.txt sn/file  
wait 800000  
connectTCP FILLER_PREVIOUS_PEER 4447  
wait 800000  
lsMessages  
exit

The filler has to be replaced by the IP address of the previous peer, which was received during the first phase of execution. E.g. using:

peerName=$(basename $(pwd))  
peerIndex=$(echo "$peerName" | sed 's/[^0-9]*//g')
  
previousPeerindex=$((peerIndex - 1))  
targetIPAddress=$(cat IPAddress_"$previousPeerindex".txt)  
  
sed -i "s/FILLER_PREVIOUS_PEER/$targetIPAddress/g" scenarioScriptP"$peerIndex".txt

Afterwards the file is ready for further use.

Content File:

This file is automatically created in the desired size and name.

---

Running the Script Generator:

Variant 1: Default Values

java -jar ScriptGenerator.jar default

See the default values below.

Parameter	Value
Host IP address	localhost
Scenario Index	0 (TCP, star topology)
Peer Count	5
File Size (num)	100
File Size (unit)	KB
File Name	100KB.txt

Variant 2: Info sheet

One option is to provide the path to a file containing the parameters in the form of:

key : value

The generator parses the file and assigns the correct test parameters. The listing order of the parameters is irrelevant. For possible keys: see the table of default values.

Variant 3: Command line arguments

Another way to input parameters is to pass them as command line arguments while calling the program on the console. This variant requires the parameters to be passed in a certain order, which matches the order listed under Input. An example program call on the console should look like:

java -jar ScriptGenerator.jar localhost 1 4 200 MB testFile.txt

Due to the fixed order, it is not possible to skip an intermediate parameter in this variant. However, a parameter can be left empty if no other parameters follow it.

In any case, if information is detected as missing, it will be replaced by default values.

Example Test Run Using the Script Generator

A complete execution of a test scenario is done in a few separate steps.

Prerequisites

The participating peers must be named correctly in numerical order. The generator assumes all wrapper peers are named P1T, P2T, P<n>T, ..., while all test peers are named P1, P2, P<n>, etc. If this rule is not adhered to (nor the source code changed), the command lists and scripts will not work as intended. Due to the distributed nature of the testing framework, it is each node's responsibility to know their order and name entities accordingly.

Preparation

In preparation each peer is required to write their IP address to a file called ipAddress_P1.txt. Example one-liners for Peer 1 (P1) on different OS:

on macOS:
ipconfig getifaddr en0 > ipAddress_P1.txt

on Linux: 
ip -4 addr show eno1 | grep -oP 'inet \K[\d.]+' > ipAddress_P1.txt

on Windows:
netsh interface ip show address "Ethernet" | findstr "IP Address" > ipAddress_P1.txt

Step 1:

• Generate the script generator according to the instructions.
• Determine the input parameters and either create a file or pass them as command line arguments.
• Run the script generator with preferred input
• Create the file with the node's IP address.

You should have generated a collection of all necessary files to run a distributed test scenario:

• (Depending on the topology) The IP addresses of all participants
• Command lists for the execution of the scenario and for the logs to be sent to the host

Visit [this]([[#Log Sending Commands runScenario.txt]]) for suggestions on how to edit the files containing fillers. It is mandatory for the fillers to be replaced, in order for the tests to run successfully.

Step 2:

1. The Test host is started with a script

cat hostCommandList.txt | java -jar SharkMessenger host 2 > hostLogs.txt

Note that the host will be kept running during the rest of the test.

2. Next, the wrapper peers are started by executing the receiveCommands.txt file.

cat receiveCommands.txt | java -jar SharkMessenger.jar P1T 2 > P1TLogs.txt

The wrapper peer should have received all scenarioScript.txt. Only the one that matches its own index is relevant.

Step 3:

After the wrapper peer has received the scenario scripts, it executes the one corresponding to its order, creating the actual test peer in the process. For example, the corresponding test peer to P1T is P1:

cat scenarioScript.txt | java -jar SharkMessenger.jar P1 2 > snm_P1.txt

It is imperative that the test peers are time-coordinated to ensure the test runs successfully. This can be done by integrating longer or shorter wait periods. The length of the pauses are currently hard coded and are calculated according to the peer order. This can be changed, or this step of the execution phase can be individually delayed.

Step 4:

During the execution of the scenarioScript.txt the messenger log is written so snm_P<n>.txt. This is one of the two files that the (still active) wrapper peer will collect and send to the (still active) host.

For this, the file should be copied from the test peer's directory to the wrapper peer's directory. Afterward, the wrapper peer should run again to execute the runScenario.txt commands, which send the log files of the test peer to the host.

Once the scenarioScript.txt has terminated for each peer, the test concludes. All sessions may be terminated.

Check the peer directories for the content file. If the test was successful, all peers should have received it, including the host. Furthermore, check the messenger log for the correct hopping list depending on the topology.

Check the host directory for all files named (n for peer index):

• snm_P<n>txt
• asapLogsP<n>.txt

If all log files exist, the scenario was executed successfully.

Evaluation:

Further evaluation is left to the tester. One possibility is the evaluation of the log files. The messenger provides thorough logging of the processes in the ASAP log. Errors in use or execution are reported in the second log named snm_P1 etc.

Further notes:

Depending on the size of the content file, the wait periods might have to be modified. Other anomalies connected to different sized content files were also reported. If anything unreported occurs, please file an issue on github.