Synthetic Data Generation

Phase 1 builds the 3D content library that Replicator will randomize. We audit the target perception task — object detection, defect classification, pose estimation — and identify the visual diversity requirements: object categories, material variations, environmental contexts, and lighting conditions. 3D assets are created or sourced from existing CAD models, applying physically-based materials (PBR) with measured BRDF properties so that rendered images match real-world appearance under varied illumination. Environment stages represent deployment contexts — warehouse shelves, conveyor lines, outdoor loading docks — with modular construction allowing rapid composition of novel layouts. We author USD-based scene templates that separate background, foreground objects, and lighting rigs into independent layers, enabling Replicator to randomize each axis independently. Material libraries include parametric wear-and-tear variations — scratches, dust, oil stains — that reflect in-service conditions the perception model must handle. Asset validation checks polygon budgets, UV mapping quality, and material response under extreme lighting to prevent rendering artifacts. Deliverables include the validated asset library, scene templates, material catalogs, and a content-gap analysis mapping asset coverage against the deployment domain's visual diversity.

Phase 2 defines the randomization strategy that transforms static scenes into a diverse training distribution. Lighting randomization varies HDR environment maps, area-light positions, color temperatures (2700K–6500K), and intensity ranges to simulate indoor fluorescent, outdoor daylight, and mixed-lighting conditions. Texture randomization applies distractors — random patterns on background surfaces — to prevent the model from overfitting to environment textures. Object pose randomization samples from task-relevant distributions: parts scattered on a conveyor follow gravity-settled placements, while shelf-stocked items follow grid patterns with jitter. Camera randomization varies intrinsics (focal length, sensor size), extrinsics (mounting height, tilt), and post-processing (exposure, white balance, motion blur) to match the range of cameras the model will encounter in deployment. We implement stratified sampling to ensure uniform coverage across randomization axes, avoiding the mode collapse that naive random sampling can produce. Randomization schedules are version-controlled and linked to experiment IDs, enabling reproducibility. Deliverables include randomization configuration files, axis-coverage visualizations, a sampling-strategy document, and validation renders demonstrating the visual diversity achieved across lighting, pose, texture, and camera parameters.

Phase 3 executes large-scale rendering and labeling. Replicator orchestrates multi-GPU rendering farms to produce datasets of tens of thousands to millions of images, each accompanied by pixel-perfect annotations — bounding boxes, instance segmentation masks, depth maps, surface normals, and 6-DOF object poses — generated without human labeling effort. Multi-sensor rendering produces synchronized camera, lidar, and radar outputs when the downstream model fuses heterogeneous inputs. We implement dataset validation pipelines that check label consistency (no zero-area bounding boxes, no overlapping instance IDs), class balance (flagging underrepresented categories for additional rendering passes), and visual quality (detecting rendering artifacts like z-fighting or texture seams). Statistical analysis compares the synthetic distribution against available real-world data along feature dimensions — object scale, aspect ratio, occlusion level — to identify and fill coverage gaps. Output formats are configurable — COCO JSON, KITTI, TFRecord, custom schemas — and datasets are versioned in object storage with full provenance linking each image to its scene template, randomization seed, and rendering parameters. Deliverables include the validated dataset, distribution-analysis reports, a label-quality dashboard, and rendering pipeline scripts ready for incremental dataset expansion.

The final phase connects synthetic datasets to model training and measures real-world impact. We configure TAO Toolkit transfer-learning pipelines that fine-tune pre-trained backbones (ResNet, EfficientNet, YOLO, DINO) on synthetic data, applying progressive domain-adaptation techniques — style transfer, feature alignment, curriculum mixing — that blend synthetic and real samples to maximize deployment accuracy. Training experiments sweep synthetic-to-real mixing ratios (100% synthetic → 80/20 → 50/50) to identify the optimal blend for the target task. Model evaluation uses a held-out real-world test set with metrics aligned to operational KPIs — mAP at deployment-relevant IoU thresholds, per-class recall for safety-critical categories, inference latency on target hardware. We implement feedback loops where model failure modes — missed detections, false positives on novel objects — trigger targeted asset creation and additional rendering passes, closing the data flywheel. TensorRT optimization produces deployment-ready engines profiled on Jetson or T4 inference hardware. Deliverables include trained model checkpoints, training experiment reports, synthetic-vs-real analysis, TensorRT engine packages, and a data-flywheel run-book that teams use to continuously improve model performance through iterative synthetic data refinement.