Notes on running processor.run() externally + using WSS for latency

	modified:   docs/cloud/resources/best-practices.md
	modified:   docs/sdk/faq.md
	modified:   docs/sdk/how-to-start/squid-development.mdx
	modified:   docs/sdk/troubleshooting.mdx

Author: abernatskiy

docs/cloud/resources/best-practices.md

@@ -16,6 +16,8 @@ Here is a list of items to check out before you deploy your squid for use in pro
 Batch handler data filtering used to be compulsory before the release of `@subsquid/evm-processor@1.13.0`. Now it is optional but highly recommended.
 :::
 
+* For low block time, high data rate networks (e.g. Arbitrum) use a WSS RPC for best latency.
+
 * If your squid [saves its data to a database](/sdk/resources/persisting-data/typeorm), make sure your [schema](/sdk/reference/schema-file) has [`@index` decorators](/sdk/reference/schema-file/indexes-and-constraints) for all entities that will be looked up frequently.
   - Follow the [queries optimization procedure](/cloud/resources/query-optimization) for best results.
 

docs/sdk/faq.md

@@ -18,9 +18,19 @@ Here is an incomplete list:
 
 The SQD Network only serves finalized blocks and is typically ~1000 blocks behind the tip. The most recent blocks, as well as the unfinalized blocks are seamlessly handled by the SDK from a complementary RPC data source, set by the `chain` config. Potential chain reorgs are automatically handled under the hood. See [Indexing unfinalized blocks](/sdk/resources/unfinalized-blocks) for details.
 
+### I want to call `processor.run()` from somewhere inside my app. Is that possible?
+
+`processor.run()` is designed to be the top-level call and will try to control execution at the process level. For example, if a finite block range is specified in the processor config it will shutdown the parent process with `process.exit()` when it reaches the end.
+
+For this reason we do not recommend to use `processor.run()` as a regular function in your applications. It is usually preferable to run the processor as a child process.
+
+If you absolutely have to do it, please set the `SQUID_PROCESSOR_EXIT_DISABLED` environment variable to `false`.
+
 ### What is the latency for the data served by the squid? 
 
-Since the ArrowSquid release, the Squid SDK has the option to ingest unfinalized blocks directly from an RPC endpoint, making the indexing real-time. 
+Since the ArrowSquid release, the Squid SDK has the option to ingest unfinalized blocks directly from an RPC endpoint, making the indexing real-time.
+
+Typically this is achieved by polling HTTP RPCs, but that becomes unfeasible for some low block time, high data rate networks such as Arbitrum. For latency-critical applications on such networks please use WSS RPC endpoints.
 
 ### How do I enable GraphQL subscriptions for local runs?
 

docs/sdk/how-to-start/squid-development.mdx

@@ -15,7 +15,7 @@ Feel free to also use the template-specific `sqd` scripts defined in [`commands.
 
 ## Prepare the environment
 
-- Node v16.x or newer
+- Node v20.x or newer
 - Git
 - [Squid CLI](/squid-cli/installation)
 - Docker (if your squid will store its data to PostgreSQL)
@@ -324,6 +324,7 @@ Edit the definition of `const processor` to
    - It is possible to [use RPC](/sdk/reference/processors/evm-batch/general/#set-rpc-endpoint) as the only data source, but [adding](/sdk/reference/processors/evm-batch/general/#set-gateway) a [SQD Network](/subsquid-network/reference/networks/#evm--ethereum-compatible) data source will make your squid sync much faster.
    - RPC is a hard requirement if you're building a real-time API.
    - If you're using RPC as one of your data sources, make sure to [set the number of finality confirmations](/sdk/reference/processors/evm-batch/general/#set-finality-confirmation) so that [hot blocks ingestion](/sdk/resources/unfinalized-blocks) works properly.
+   - On low block time, high data rate networks (e.g. Arbitrum) use WSS endpoints if latency is critical.
 2. Request all [event logs](/sdk/reference/processors/evm-batch/logs/), [transactions](/sdk/reference/processors/evm-batch/transactions/), [execution traces](/sdk/reference/processors/evm-batch/traces) and [state diffs](/sdk/reference/processors/evm-batch/state-diffs/) that your task requires, with any necessary related data (e.g. parent transactions for event logs).
 
 3. [Select all data fields](/sdk/reference/processors/evm-batch/field-selection) necessary for your task (e.g. `gasUsed` for transactions).

docs/sdk/troubleshooting.mdx

@@ -14,7 +14,23 @@ Many issues can be resolved by following the [best practices guide](/cloud/resou
 ## Processor
 
 <details>
-<summary>`Error: data out-of-bounds` `ethers` errors on EVM</summary>
+<summary>I'm running a processor against a low block time, high data rate network (e.g. Arbitrum) and the latency is no good</summary>
+
+The default HTTP polling mechanism occasionally fails to keep up with the chain head on such networks. For latency-critical applications please use WSS RPC endpoints.
+
+</details>
+
+<details>
+<summary>I'm calling `processor.run()` from inside of my app and it shuts down the parent process</summary>
+
+`processor.run()` is designed to be used as a top level call and we don't recommend using it in other ways. Running the processor as a child process is usually preferable.
+
+If you absolutely have to do it, set the `SQUID_PROCESSOR_EXIT_DISABLED` to `false`.
+
+</details>
+
+<details>
+<summary>`Error: data out-of-bounds` errors on EVM</summary>
 
 * Usually this means that you're using the decoder on the wrong data. Make sure that the decoder receives _only_ the data you intend it to.