First paper on Semi-Autonomous 3D aerial mapping published!

Our Immersive Technology team which consists of students and scientists from the Departments of Electrical Engineering and Computer Science and Nuclear Engineering at UC Berkeley and some of our ANP scientists recently published a first paper on “Immersive Operation of a Sem-Autonomous Aerial Platform for Detecting and Mapping Radiation”.

This is a wonderful accomplishment by one of our multi-disciplinary teams and demonstrates for the first time the remote operation and visualization of a 3D Scene-Data Fusion enabled aerial platform. Please check it out!

—————–

Figures and captions:

Fig. 3. - Workflow of deploying a radiation-mapping mission between an sUAS (left) and the VR interface (right). Waypoints are placed on the 3-D map in VR and transmitted over WiFi as geolocated flight targets for the sUAS to reach semi-autonomously. Operators can view the real-time LAMP radiation and surface reconstruction from the sUAS in VR.

Workflow of deploying a radiation-mapping mission between an sUAS (left) and the VR interface (right). Waypoints are placed on the 3-D map in VR and transmitted over WiFi as geolocated flight targets for the sUAS to reach semi-autonomously. Operators can view the real-time LAMP radiation and surface reconstruction from the sUAS in VR.

User can equip a VR headset and control the sUAS with VR controllers.

Fig. 6. - Our VR interface displays a textured 3-D view of the environment derived from prior satellite imagery. The operator sets waypoints through the VR interface to mark important flight path geo-locations. The sUAS (red) follows the flight path (yellow) around the building, detecting and streaming radiation and environment reconstruction data back to the interface as a colorized 3-D mesh (not shown here). The operator uses their virtual hands to select from the left-controller’s sensors and flight-control menus via the cyan pointer to control sensor data visualizations and sUAS operation, respectively.

Our VR interface displays a textured 3-D view of the environment derived from prior satellite imagery. The operator sets waypoints through the VR interface to mark important flight path geo-locations. The sUAS (red) follows the flight path (yellow) around the building, detecting and streaming radiation and environment reconstruction data back to the interface as a colorized 3-D mesh (not shown here). The operator uses their virtual hands to select from the left-controller’s sensors and flight-control menus via the cyan pointer to control sensor data visualizations and sUAS operation, respectively.

Our system streams radiation and environment reconstruction data back to the VR interface, shown here, as colorized 3-D meshes. The environment reconstruction (top, in purple) is generated by the LAMP’s LiDAR and is used for the radiation reconstruction. We overlay the environment reconstruction on the baseline map to provide an accurate, real-time view of the sUAS’s surroundings to the operator. The radiation-only visualization (bottom) shows reconstructed intensities of 662-keV reflecting measurements of the gamma-ray energies with the CLLBC-based NG-LAMP. Higher gamma-ray intensities are denoted by darker shades of red. Note we display only the five (of six) most intense gamma-ray isosurfaces for visualization purposes. The operator can therefore intuitively localize the source to the darkest part of the mesh. For reference, the square building has a side length of about 8 m.