Senior Technical Lead - Post Silicon Validation
HCLTech
Santa Clara, California, United StatesPosted 21 days agoDiscoveredMatch locked
RemoteFull-time
Job Summary
You will be responsible for
developing, executing, and maintaining tests for collective communication operations
(AllReduce, AllGather, ReduceScatter, AllToAll) on custom networking silicon — validating that the ASIC correctly enables large-scale distributed AI training workloads.
C/C++
Expert (Must-have)
Writing test frameworks, RDMA verbs test suites, driver-level test development, loopback and traffic tests
Python
Strong (Must-have)
Test automation, CI/CD integration, orchestration of multi-node test scenarios, emulation test infrastructure
Bash/Shell Scripting
Proficient (Good-to-have)
Test execution scripts, environment setup, multi-host coordination
Key Responsibilities
•
Develop test suites for collective operations
(AllReduce, AllGather, ReduceScatter, AllToAll) targeting Trantor ASIC across emulation, FPGA, and silicon platforms
•
Write RDMA verbs-level tests
using the RoCE Verbs Testing Framework (rdma-core Verbs API) — covering positive, negative, and error-injection scenarios
•
Validate multi-node, multi-NIC collective communication
patterns, ensuring correct behavior under various topologies (rail-aligned, cross-rail, multiplanar)
•
Develop traffic generation and validation tools
for RDMA collectives at scale — covering data integrity, performance, and error handling
•
Integrate tests into CI/CD pipelines
for regression prevention on every code change and nightly builds
•
Collaborate with driver, firmware, architecture, and modeling teams
to define test plans and ensure complete coverage of networking features
•
Run and analyze performance benchmarks
(NCCL-tests, perftest, rdma_gen) to identify regressions and validate throughput/latency targets
•
Debug and root-cause failures
across the full stack — ASIC RTL, firmware, driver, rdma-core provider, and user-space collectives
Must-Have
•
RDMA (Remote Direct Memory Access)
— Deep understanding of RDMA operations: READ, WRITE, SEND, RECEIVE; Queue Pairs (QPs), Completion Queues (CQs), Memory Regions (MRs), Protection Domains (PDs)
•
RoCE v2
— Understanding of RDMA over Converged Ethernet protocols, transport-level behavior, and conformance requirements
•
Collective Communication Operations
— AllReduce, AllGather, ReduceScatter, AllToAll; ring/tree algorithms; understanding of how collectives map to network traffic patterns
•
Ethernet / L2 Networking
— Layer 2 fundamentals, MTU, multiport networking, VLANs
•
PCIe Architecture
— PCIe endpoint/switch topology, Gen5/Gen6, BAR regions, MSI-X interrupts, SR-IOV, multi-function devices
•
NCCL / Communication Libraries
— Familiarity with NVIDIA Collective Communications Library or equivalent; understanding of how training jobs use collectives over RDMA NICs
Good-to-Have
•
InfiniBand / IB Verbs API
— Experience with libibverbs, rdma-core, ibv_* APIs
•
Network Topologies for AI Training
— Rail-optimized fabrics, fat-tree, multi-planar designs, PXN
•
Traffic Congestion & Flow Control
— PFC (Priority Flow Control), ECN, congestion management for lossless fabrics
•
DMA & Memory Subsystems
— GDR (GPUDirect RDMA), host memory registration, IOMMU
•
Protocol Conformance Testing
— ANVL or similar automated conformance testing methodologies
Preferred Qualifications
- Background in
silicon validation
for networking chips (switches, NICs, DPUs)
- Experience with
pre-silicon validation
environments (emulation, FPGA prototyping, software models/QEMU)
- Familiarity with
infrastructure
or large-scale hyperscaler networking
- Contributions to
open-source RDMA/networking projects
(rdma-core, Linux kernel networking, NCCL)
- Understanding of
AI/ML training workloads
and how network performance impacts training efficiency
- Experience with
build systems
and test infrastructure at scale
Education
- B.S./M.S. in Computer Science, Electrical Engineering, Computer Engineering, or related field
- Advanced degree preferred but not required with equivalent industry experience
Not included in the source posting: benefits.
Skills
artificial-intelligenceci-cdlinuxmachine-learningnode-jsbashcpython
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.