Description
Objective: The objective is to develop a capability for generating geo-specific, sensor-independent scenes for multimodal (RF and EO/IR) synthetic data generation by leveraging geo-spatial information, time-of-day, seasonal data, and measured databases, overcoming limitations in existing models and radiometric data. Description: The DoD requires large-scale, high-fidelity background scenes to advance autonomous systems and Artificial Intelligence and Machine Learning (AI/ML) capabilities. These scenes are critical for providing realistic, context-rich environments that enable AI/ML and/or autonomous systems to learn, adapt, and perform effectively in real-world, dynamic conditions. A critical component of this effort is the ability to generate dynamic, high-fidelity background scenes that realistically model operational environments. Unlike traditional synthetic data generation, which often focuses on isolated sensor outputs, scene generation must create a coherent, interactive world where autonomous agents can navigate, perceive, and process imagery based on their movement and decision-making. This presents several challenges. First, scene generation requires accurate modeling of complex environmental factors such as terrain variation, urban structures, vegetation, weather conditions, and electromagnetic propagation—all of which impact sensor performance. Additionally, ensuring spatial and temporal consistency across multimodal data (e.g., RF and EO/IR) is far more demanding than simply generating independent synthetic datasets. Autonomous systems rely on their ability to interpret changes in the environment dynamically, requiring realistic physics-based interactions between sensors and the scene. Further, aligning RF and EO/IR perspectives within the same scenario for sensor fusion introduces an added layer of complexity, demanding precise calibration of sensor viewpoints, occlusions, and atmospheric effects. To accurately model such complex environments, scene generation tools must not only produce synthetic RF and EO/IR data but also ensure that these representations align with real-world sensor measurements. When the underlying environment is well-characterized, scene generation tools can generate multimodal imagery alongside ground truth labels, providing ready-made datasets for AI/ML models and autonomous agents. However, their effectiveness is often constrained by the availability of accurate models and measured databases that capture the necessary radiometric and electromagnetic characteristics of the environment. Addressing these limitations requires the development of software that integrates geospatial data, time-of-day, seasonal variations, measured databases, and land cover data to generate detailed representations of the environment. Furthermore, this software must support standardized scene formats compatible with existing simulation tools such as FLITES (EO/IR) and Xpatch (RF), allowing for flexible resolution and fidelity adjustments based on scenario requirements. Finally, a structured approach should be proposed to refine synthetic scene renderings as real-world measurements become available, improving realism and scene fidelity over time. Keywords: Scene generation; artificial intelligence; radio frequency; electro optical; infrared; multimodal; synthetic data; geo-specific CMMC Level: Level 2 (Self)