Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/e2e-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: E2E Tests

on:
push:
branches: [master]
pull_request:
branches: [master]
workflow_dispatch:
inputs:
e2e_tests_ref:
description: 'Branch or ref of sdk-e2e-tests to use'
required: false
default: 'main'

jobs:
e2e-tests:
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
runs-on: ubuntu-latest

steps:
- name: Checkout SDK
uses: actions/checkout@v4
with:
path: sdk

- name: Checkout sdk-e2e-tests
uses: actions/checkout@v4
with:
repository: segmentio/sdk-e2e-tests
ref: ${{ inputs.e2e_tests_ref || 'main' }}
token: ${{ secrets.E2E_TESTS_TOKEN }}
path: sdk-e2e-tests

- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.2'

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'

- name: Run E2E tests
working-directory: sdk-e2e-tests
run: |
./scripts/run-tests.sh \
--sdk-dir "${{ github.workspace }}/sdk/e2e-cli" \
--cli "ruby ${{ github.workspace }}/sdk/e2e-cli/main.rb"

- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: e2e-test-results
path: sdk-e2e-tests/test-results/
if-no-files-found: ignore
39 changes: 39 additions & 0 deletions .github/workflows/publish-e2e-cli.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Publish E2E CLI

on:
push:
branches: [master]
paths:
- 'e2e-cli/**'
- 'lib/**'
schedule:
- cron: '0 0 1 * *'
workflow_dispatch:

jobs:
publish:
runs-on: ubuntu-latest
steps:
- name: Checkout SDK
uses: actions/checkout@v4

- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.2'

- name: Prepare artifact
run: |
mkdir -p artifact
cp e2e-cli/main.rb artifact/
cp e2e-cli/e2e-config.json artifact/
cp e2e-cli/run-e2e.sh artifact/
cp e2e-cli/Gemfile artifact/
cp -r lib artifact/lib

- name: Upload CLI artifact
uses: actions/upload-artifact@v4
with:
name: e2e-cli-ruby
path: artifact/
retention-days: 90
3 changes: 3 additions & 0 deletions e2e-cli/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# frozen_string_literal: true
source 'https://rubygems.org'
gemspec path: '..'
112 changes: 112 additions & 0 deletions e2e-cli/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# analytics-ruby e2e-cli

A small CLI tool for end-to-end testing of the [analytics-ruby](https://github.com/segmentio/analytics-ruby) SDK. It accepts a JSON description of event sequences and SDK configuration, sends those events through the real SDK, and reports the outcome as JSON on stdout.

## Requirements

- Ruby 2.6+
- No extra gems beyond `analytics-ruby` itself (the script adds `../lib` to `$LOAD_PATH` automatically)

## Usage

```bash
ruby main.rb --input '<json>'
```

### Example

```bash
ruby main.rb --input '{
"writeKey": "YOUR_WRITE_KEY",
"apiHost": "https://api.segment.io",
"sequences": [
{
"delayMs": 0,
"events": [
{"type": "track", "event": "Test Event", "userId": "user-1", "properties": {"foo": "bar"}},
{"type": "identify", "userId": "user-1", "traits": {"name": "Alice"}},
{"type": "page", "userId": "user-1", "name": "Home", "category": "Nav"},
{"type": "screen", "userId": "user-1", "name": "Main"},
{"type": "alias", "userId": "new-id", "previousId": "user-1"},
{"type": "group", "userId": "user-1", "groupId": "group-1", "traits": {"plan": "pro"}}
]
}
],
"config": {
"flushAt": 15,
"flushInterval": 1000,
"maxRetries": 3,
"timeout": 10
}
}'
```

## Input JSON format

| Field | Type | Description |
|-------|------|-------------|
| `writeKey` | string | Segment write key |
| `apiHost` | string | Full API base URL (e.g. `https://api.segment.io`) |
| `sequences` | array | List of event sequences (processed in order) |
| `sequences[].delayMs` | number | Milliseconds to sleep before processing this sequence |
| `sequences[].events` | array | List of events to send |
| `config.flushAt` | number | Max events per batch (`batch_size`) |
| `config.flushInterval` | number | Flush interval in ms (informational, not applied) |
| `config.maxRetries` | number | Number of HTTP retries on failure |
| `config.timeout` | number | HTTP timeout in seconds (informational, not applied) |

### Supported event types

All event keys use camelCase in the JSON input; they are converted to snake_case before being passed to the SDK.

| type | Required keys | Optional keys |
|------|--------------|---------------|
| `track` | `userId` or `anonymousId`, `event` | `properties`, `context`, `integrations`, `messageId`, `timestamp` |
| `identify` | `userId` or `anonymousId` | `traits`, `context`, `integrations`, `messageId`, `timestamp` |
| `page` | `userId` or `anonymousId` | `name`, `category`, `properties`, `context`, `integrations`, `messageId`, `timestamp` |
| `screen` | `userId` or `anonymousId` | `name`, `properties`, `context`, `integrations`, `messageId`, `timestamp` |
| `alias` | `userId`, `previousId` | `context`, `integrations`, `messageId`, `timestamp` |
| `group` | `userId` or `anonymousId`, `groupId` | `traits`, `context`, `integrations`, `messageId`, `timestamp` |

## Output JSON format

On success (exit code 0):

```json
{"success": true, "sentBatches": 1}
```

On failure (exit code 1):

```json
{"success": false, "sentBatches": 1, "error": "status=400 error=Invalid write key"}
```

Debug information (event enqueue/flush progress) is written to **stderr** and does not affect the stdout JSON output.

## Running the full E2E test suite

```bash
./run-e2e.sh
```

This requires the [sdk-e2e-tests](https://github.com/segmentio/sdk-e2e-tests) repository to be checked out alongside the SDK root (i.e. at `../../sdk-e2e-tests` relative to this directory). Override with:

```bash
E2E_TESTS_DIR=/path/to/sdk-e2e-tests ./run-e2e.sh
```

Extra arguments are forwarded to `run-tests.sh`:

```bash
./run-e2e.sh --suite basic
```

## How it works

1. The script adds `../lib` to `$LOAD_PATH` so no gem installation is needed.
2. `apiHost` is parsed with `URI` to extract hostname, port, and SSL flag, which are passed to the SDK's `Transport` layer.
3. Each event's camelCase keys are converted to snake_case symbols before calling the corresponding SDK method (`track`, `identify`, etc.).
4. `timestamp` values are parsed from ISO8601 strings into `Time` objects.
5. After all events are enqueued, `client.flush` blocks until all batches have been sent.
6. Any errors reported via the `on_error` callback cause a failure result.
7 changes: 7 additions & 0 deletions e2e-cli/e2e-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"sdk": "ruby",
"test_suites": "basic",
"auto_settings": false,
"patch": null,
"env": {}
}
Loading
Loading