Performance Tests for Drones, Pilots

Wednesday, December 5, 2018 @ 12:12 PM gHale

An emergency response drone approaches a bucket-shaped target on the NIST performance test course. The capabilities of the robot and the skills of its pilot can be evaluated using the standardized system.
Source: NIST

Tracking the spread of a wildfire, searching for a missing person, or detecting toxic fumes after an explosion before sending in rescue workers are all tasks falling on a drone these days.

In these critical situations, the performance of the flying robots known as small unmanned aircraft systems (sUAS) and the pilot remotely controlling it must be flawless.

RELATED STORIES
Facial Recognition Software Quality Advancing
Game Theory to Quantify Grid Cyber Threats
Human Side of Cyber-Physical Systems
Next-Gen Security for IoT

To help ensure that capability, a new standard ended up published based on test methods developed by the National Institute of Standards and Technology (NIST).

Performance test methods developed by NIST measure the system capabilities of aerial emergency response drones and the proficiency of their pilots. The test methods are low-cost and easy to fabricate and use, so they can be replicated by any group interested in evaluating its aircraft and remote pilots before they face a real-life emergency.

The National Fire Protection Association (NFPA), a U.S. organization that creates and maintains more than 300 codes and standards to reduce loss of life and property due to fire, electrical and related hazards, issued NFPA 2400, Standard for Small Unmanned Aircraft Systems (sUAS) Used for Public Safety Operations. The new standard includes a suite of 10 aerial test methods developed by NIST that quantitatively measures the system capabilities of the drone and the proficiency of the pilot in carrying out five basic maneuvers, including accurate landing, vertical climbing, and straight and level flying. There also are five functionality test methods, including circular orbits to identify objects from afar and spiral maneuvers to conduct close-range inspections.

Pilot Proficiency
“NFPA 2400 references our test methods as a practical way to guide training and measure remote pilot proficiency,” said Adam Jacoff, NIST robotics research engineer. Jacoff leads the agency’s efforts to develop, refine and promote adoption and use of consensus test standards for response robots that function on land, sea and air.

“The test methods are low-cost and easy to fabricate and use, so they can be replicated by any group interested in evaluating its aircraft and remote pilots,” he said.

“Emergency response organizations can quantitatively establish reliability and confidence during training before drones and remote pilots are deployed in real-life situations,” said NIST mechanical engineer Kamel Saidi, who works with Jacoff on metrics for response robots. “For the aircraft, it’s like testing a car on a speedway track to make sure it works as designed. For the pilot, it’s like a driving test to prove he or she can operate a specialized vehicle that’s not for everyone, such as a commercial truck.”

Many of the test methods involve aligning the drone simultaneously with two cylindrical targets in a series of designated positions and orientations. “Primarily, these tests are implemented using white buckets with black rings inscribed inside them,” Jacoff said. “The pilot uses onboard cameras to align with and identify the targets, perform a task such as reading a hazardous material label or detecting motion, and then move on to the next drill.”

Another test has the remote pilot prove his or her skills at avoiding obstacles by performing figure eight transits through a course of hurdle-like gates with vertical posts and horizontal crossbars.

Test Methods in Development
More than 20 additional aerial test methods are being developed by NIST. These are all being standardized through the ASTM International Standards Committee on Homeland Security Applications: Response Robots (E54.09), which is chaired by Jacoff. Included are tests for navigating hallways within buildings, searching and mapping wide areas, and avoiding situations that may interfere with radio communications.

The new standard and its planned expansions are being welcomed by first responders.

“The tests give you specific numbers that quantify the whole process: the aircraft, the pilot, everything,” said Victor Galladora, a Montgomery County, Maryland, police officer who has helped validate the NIST test method suite operating emergency response drones at NIST’s Robotics Test Facility. “These metrics allow you to say with confidence that a drone works well, or someone is a really good pilot. And when a test becomes standardized, it means you can be in Los Angeles and I can be here in Maryland and we can both get the same training, optimize our performances and then compare results.”



Leave a Reply

You must be logged in to post a comment.