Skip to content

Conversation

@kfaraz
Copy link
Contributor

@kfaraz kfaraz commented Dec 29, 2025

Description

Kafka ITs come in 2 flavors - serialized and parallelized.
Serialized tests affect the cluster state and thus must not be run concurrently with any other test.
Parallelized tests are pretty much sandboxed since they work on a fresh datasource/supervisor/topic, as applicable.

In the context of embedded-tests, all tests are currently run serially since only 1 cluster can be running at a time to avoid port conflicts. Parallelizing them does not seem necessary at the moment, since the tests are fairly quick.

Changes

  • Add KafkaResource.produceRecordsWithoutTransaction()
  • Add KafkaFaultToleranceTest to replace the following:
    • ITKafkaIndexingServiceTransactionalParallelizedTest
    • ITKafkaIndexingServiceNonTransactionalParallelizedTest
    • ITKafkaIndexingServiceTransactionalSerializedTest
    • ITKafkaIndexingServiceNonTransactionalSerializedTest
  • Add FaultyClusterTest to replace ITFaultyClusterTest. Getting this test to work required the following changes:
    • Override TaskLockbox.cleanupPendingSegments() in FaultyTaskLockbox
    • Bind FaultyTaskActionClientFactory on Indexers
    • Allow deseriazation of KafkaSupervisorReportPayload
  • Add KafkaMultiSupervisorTest
  • Add new methods to EmbeddedKafkaSupervisorTest
  • Remove the following IT groups from GHA workflow
    • kafka-index
    • kafka-index-slow
    • kafka-transactional-index
    • kafka-transactional-index-slow

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@kfaraz kfaraz requested a review from abhishekrb19 December 30, 2025 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant