In the drone industry, it is taken nearly as gospel that small uncrewed aircraft systems (UAS) will allow people to do their jobs better, safer, and faster than whatever the traditional method happens to be—especially when it comes to public safety. However, if you are making decisions that have financial consequences or lives on the line, it isn’t enough to believe it. You must be able to prove it. That’s the job of researchers like Dr. Joseph Cerreta at the Embry-Riddle Aeronautical University (ERAU) Worldwide Campus: to build the case through the scientific process that drones really can deliver results comparable to legacy systems, which have demonstrated their reliability through decades of service.
In this synthetic image, created using the FARO laser scanner, the individual points where the unit was set up are identified. The need to move LiDAR-type sensors to multiple locations within an environment, such as a crash scene, contributes substantially to the time required.
This synthetic view of the test area represents a three-dimensional point cloud, rendered in Pix4D based on imagery gathered by a DJI Mavic Pro Platinum.
Specifically, Dr. Cerreta wanted to see how 3D models created by Pix4D using drone imagery would stack up against a portable, tripod-mounted laser scanner from FARO. Established in 1981, FARO is a global leader in precision 3D measurement and is widely used by law enforcement to create accurate representations of crime scenes and automobile accidents that can be submitted as evidence in court.
Dr. Cerreta selected Gunsite, a civilian firearms training facility in Paulden, Arizona as the location to conduct his research. The 3,000-acre facility is dotted with outdoor shooting ranges, including one that is home to bullet-riddled cars where students practice their vehicle defense skills.
“We selected this site because it has a lot of vehicles in a small area,” said Dr. Cerreta. “It combines elements you would expect to see both at a crime scene, like bullet holes, and an automobile accident, with parts strewn around the location. Also, it’s an austere location in uncontrolled airspace, so doing our work here reduces the risk both to bystanders and other air traffic.”
Laser Battle
The Embry-Riddle Worldwide Campus faculty provides classes for law enforcement officials, who want to understand how to integrate drones into their work, according to Dr. Cerreta.
“They have been asking us a lot of questions about which types of drones are the most accurate, and how they should fly them in order to construct an accurate point cloud,” he said. “We needed to do this research so that we could answer those questions with confidence.”
Dr. Cerreta needed to test three specific points: First, which of the small, commercial UAS widely used by law enforcement could provide the most accurate results; second, what combination of altitude and flight path yielded the best results; and third, could drones compete with the accuracy of established alternatives like the FARO laser scanner.
The results from the terrestrial scanner were the baseline for the test. Researchers set up the scanner at multiple locations around the site, conducting a triple-redundant scan: verifying every single point with three separate sweeps and measuring the distance between points to an accuracy of 6.1 mm at a range of 10 meters. In addition, the FARO unit took 66 visible light images at each location, so it would be able to render a complete 3D model.
Dr. Joseph Cerreta launches a DJI Mavic Pro Platinum, one of the platforms he tested against the FARO laser scanner to determine the accuracy of drone-based photogrammetry versus a terrestrial-based LiDAR system.
Up and Away
To see how drones compared to each other and to the laser scanner, Dr. Cerreta gathered up a fleet of common commercial platforms and deployed them over the range. They included a DJI Mavic Pro, Mavic 2 Enterprise Dual, Phantom 4 Pro, Inspire 1, Inspire 2, a Parrot Anafi, and a Parrot Bebop 2. Using Pix4Dcapture, he tasked each drone with completing the same series of flights: a single grid, a double grid, a circle, and a double grid plus a circle. Then, he performed those same patterns at five different altitudes: 82, 100, 150, 200, and 250 feet above ground level (AGL).
The lone problem child among the assembled drones was the Mavic 2 Enterprise Dual. Embry-Riddle purchased the Dual with DJI’s smart controller, which features an integrated, Android-based tablet. Dr. Cerreta wasn’t able to load Pix4D onto the controller, so he did his best to replicate the same flight profile using the DJI Pilot app.
Operating autonomously, each of the drones performed the same flight, capturing dozens of individual still images. Although these were two-dimensional photographs, the advanced photogrammetry tools within the Pix4D software suite allowed it to construct a 3D model of the site from the data.
This yielded a “point cloud”: millions of individual points, their coordinates in three-dimensional space inferred from the photographs. Using a fundamentally different technique, the FARO laser scanner also created a point cloud from the scene—making it possible to compare the results from the drone and the scanner, head to head.
The FARO Focus S is employed by law enforcement to document crime scenes and car accidents using LiDAR technology. It offers unrivaled accuracy, but acquiring a comprehensive scan is a time-consuming process, and the unit itself costs more than $35,000.
Pix4Dcapture is an app available for both Android and iOS devices. Capable of operating drones from multiple manufacturers autonomously, it acquires the images necessary to create 2D and 3D models of the area surveyed.
A Game of Inches
Dr. Cerreta presented his results in a peer-reviewed paper entitled “UAS for Public Safety Operations: A Comparison of UAS Point Clouds to Terrestrial LiDAR Point Cloud Data Using a FARO Scanner,” co-authored with Dr. Scott Burgess and UAS undergraduate student Jeremy Coleman from the Embry-Riddle Prescott campus. The accuracy of the drone data was derived by employing Aeropoints ground control points and comparing the point cloud that Pix4D rendered for each flight with the point cloud created using the FARO scanner data.
Overall, the two data sets were quite similar. On average, the drone data varied between 30 to 35 millimeters (just slightly more than one inch) from the results derived from the laser scanner. The best results were delivered by the DJI Phantom 4 Pro, which had a 26-millimeter variation from the FARO data, whereas the venerable Parrot Bebop 2 performed the worst overall, with a 39-millimeter variation.
“Even though UAS are not as accurate as a terrestrial LiDAR system according to our findings, they are, practically speaking, within a reasonable level of accuracy for law enforcement to be able to use drones to document crime scenes or automobile accidents,” Dr. Cerreta explained.
Just as important as the variation between different types of aircraft was the impact of different flight patterns on the final results. Of all the altitudes he tested, Dr. Cerreta found that 100 feet AGL yielded the best results. Flying at 82 feet or 150 feet AGL had a small but measurable impact on the data, while flying at 200 feet AGL and above had a much more significant negative impact. Also, the double grid plus a circle proved to be the best flight pattern at every altitude, making it the preferred alternative for public safety drone operations.
In addition to testing the accuracy of different small commercial uncrewed aircraft systems (UAS), Dr. Joseph Cerreta’s study also compared the results from different types of search patterns, including a grid, a double grid, a circle, and a double grid plus a circle.
Advantage: Drones!
Beyond their proven accuracy relative to their ground-based competitors, drones have some key benefits. According to Dr. Cerreta’s paper, capturing a scene with a drone takes 80 percent less time than using a ground-based scanner like the FARO. That means police officers spend less time in harm’s way, such as standing on the side of a busy road to document a crash.
In addition, drones are much, much less expensive than LiDAR systems. A Phantom 4 Pro with a nice assortment of accessories can be purchased for less than $2,000, whereas a FARO laser scanner can cost upward of $50,000. That puts detailed crime scene investigation capabilities into the hands of smaller law enforcement agencies that might not be able to afford a terrestrial system.
Thanks to the hard work of Dr. Cerreta and his colleagues, we no longer need to take it on faith that drones make it safer, easier, and faster to perform this one type of law enforcement task using drones. We’ve got the peer-reviewed paper to prove it.
LiDAR: How it works
In the same way the common word radar is actually an acronym for “radio detection and ranging,” LiDAR is an acronym for “light detection and ranging.” The basic principle behind LiDAR is that the speed of light is a well-known, universal constant. If you measure exactly how long it takes a beam of light to reach an object and then reflect back to its source, and you multiply that result by the speed of light, you know the distance between those two objects with high precision.
Typically, lasers, rather than ordinary light, are used in LiDAR systems to deliver pinpoint accuracy. Commercial LiDAR systems, like those built by FARO, employ an array of spinning lasers that sweep across the environment and rapidly build up an accurate 3D model. Of course, objects in the environment shadow objects located behind them, so scans need to be taken from several different positions to build up a complete model—a time-consuming process.
When all of the data have been collected, those billions of individual measurements are loaded into a computer that processes them into a model of the environment that is accurate down to the millimeter. This model is referred to as a “point cloud” because it defines shapes through the locations of millions of individual points in three-dimensional space. To aid human comprehension, point clouds are often overlaid with photographs from the scene to create a recognizable virtual environment.
Photogrammetry: How it works
Initially conceived by Albrecht Meydenbauer in 1867, photogrammetry is the science of extracting reliable measurements and information about the physical world from photographs, such as the distance between two points on the ground, or the height of a building. A common example of photogrammetry using drones is orthomosaic mapping.
In this application, the drone flies a fixed pattern over the landscape, taking still photographs with its camera at regular intervals while fixed in a nadir orientation—that is, pointing straight down at the ground. By capturing overlap between neighboring images, a computer is able to match up the individual photos and assemble them into a single picture, like a tile mosaic. The result resembles a conventional, two-dimensional map with a consistent scale.
Using stereoscopic techniques, it is also possible to build a three-dimensional model using photogrammetry. This technique was done by hand during World War II, to present British Prime Minister Winston Churchill with an accurate model of a V-2 rocket sitting on the launch pad, convincing him of the danger posed by the Nazi “vengeance weapons.” The process involves complex mathematics, but is well within the capabilities of modern computers.
Text & photos by Patrick Sherman
