MD-TRT Support, Compile/Export, C++ and Python #4183

Open

narendasan wants to merge 21 commits intomainfrom

push-vqqzkszwrvyx

Commits on Apr 12, 2026

Multi-Device TensorRT Runtime with Native NCCL Collectives

apbose
authored and
narendasan
committed
removing the try-except block in TRTengine.cpp and correcting the typis

apbose
authored and
narendasan
committed
Redesign distributed inference API: auto-detect rank, lazy NCCL setup, ABI v9

apbose
authored and
narendasan
committed
remove nccl.h dependancy

apbose
authored and
narendasan
committed
clean up import and add comment

apbose
authored and
narendasan
committed
moving setup_nccl_library call to example script

apbose
authored and
narendasan
committed
work on the save/load export part-add is_md flag, guard export tracing and enable DTensor decomposition

apbose
authored and
narendasan
committed
refactor: Adjusting how we use NCCL
narendasan
committed
fix: enable torch.compile(backend='tensorrt') for LLMs with dynamic shapes
narendasan
committed
test: add torch.compile(backend='tensorrt') integration test for Llama 3.2 1B
narendasan
committed
feat: llama3.2 working with MD-TRT
narendasan
committed
feat: Support exported and serialization workflows for MD-TRT
narendasan
committed

Commits on Apr 13, 2026

ci: fix nccl builds in CI
narendasan
committed

Commits on Apr 14, 2026

Commits on Apr 17, 2026

Broaden NCCL skip guards to include native TRT collectives and fix distributed context usage in some tests
apbose
committed