Proximity outdoor robustness tests#

Summary#

This report summarises testing of the Novelda Proximity Outdoor in various outdoor environments and shows how robust the product is against common outdoor motions such as rain and animals.

Note

UWB in outdoor applications may have regional limitations. The tests performed in this report is based on a variant of the Novelda Proximity algorithm tuned for outdoor applications. For more information about this, please contact Novelda for support.

Abstract#

This report summarizes testing of the Novelda Proximity Outdoor algorithm in various outdoor environments and shows how robust the product is against common outdoor motions such as rain and animals.

For optimum product performance, it is necessary to ensure the false positive rate is as low as possible and the Proximity Outdoor algorithm is designed with this in mind.

As can be seen, rain and water droplets flowing on the product housing in front of the sensor are fully rejected in this testing. Furthermore, the ability the sensor can reject most movement by small animals such as birds and cats. Larger animals such as dogs are typically detected since these are closer to the physical size of humans.

Results#

Weather#

The robustness of the algorithm against rain was tested with two tests, recordings (raw data) and logging (presence/no presence). In case of below 0 deg conditions, the algorithm has been tested against snow and ice packed in front of the sensor.

Rain recordings (Proximity Standalone FW)#

During the 48 hours of recordings, no false alarms were measured due to weather triggers.

Rain logging (Proximity Outdoor algorithm)#

During the 1.5 months of logging, the Novelda X4 F103 rev 1.2 (and rev 1.3) sensors have not been triggered by rain as seen in the following figure. As pointed out in the following figure, at beginning of the logging period, window cleaners triggered the sensor (confirmed with photos)

../../_images/outdoor_rain_log.png — Outdoor log from 01/09-22 - 19/10-22. Rain in (negative blue bars) mm and the number of detections (positive green bars) on the y-axis over time.#

Snow and ice#

To analyse the impact of snow and ice, recordings with packed snow and ice of thicknesses 0.5 - 4 cm and 0.5, respectively, in front of the sensor were done. The maximum observed attenuation with 4 cm of snow was 2 dB. For ice, an enhancement of ~5 dB occurred. Both values are close to the measurement tolerances in the setup and have negligible impact on the detection performance.

../../_images/snow_logging_setup.png — Test setup with snow#

../../_images/ice_logging_setup.png — Sketch of the test setup with ice#

Overnight recordings (16 hours) of light-heavy falling snow showed no impact on the signal.

Long-term logging(same as for rain) has been running for XX days with the proximity outdoor v.0.10.50, with a total of 0 registered detections. The maximum amount of snow registered during 1h was 5 mm.

../../_images/outdoor_log_snow.png — Snowfall end December-22 - mid January-23#

Conclusion#

During 55 days, there were 27 days with rain. The maximum total amount of rain during a day was measured at 20.3 mm and a total of 6 days had more than 10 mm of rain in total. During this period there were no registered detections. Hence, the outdoor test shows that weather/artefact rejections work nicely for rejecting weather outdoors.

Snow and/or ice should not alter the performance by blocking the signal or creating false positives. Ice could alter the signal in some directions because of the possible focusing effect.

Note

The change in signal power in the ice and snow recordings could be caused by instabilities in the test setup. In stable test setups, its normal with a ~ 1 dB varaiaty between recordings

Animals#

Birds#

4 birds were observed during 70 hours of recordings. A bird event is here loosely defined as one event per bird visit. During the period, 3 detections were registered. In many events, the birds were detectable but rejected by the Proximity application logic.

../../_images/bird_1.png — Magpie runs over radome generated hits, not detections.#

../../_images/bird_2.png — Fighting magpies generated detections.#

../../_images/bird_3.png — Magpie landing in foresight, generated detections.#

Cat#

Data is processed with outdoor sensitivity 3 variant of the Proximity algorithm and resulted in a rejection of 5 out of 6 cases. Examples of three test cases are given in the three images below.

../../_images/cat_1.png — 30 deg azimuth approach, detected in both sensitivities, rejected by application logic.#

../../_images/cat_2.png — 10 deg azimuth approach, detected in both sensitivities, rejected by application logic.#

../../_images/cat_3.png — 45 deg azimuth approach, detected in both sensitivities, rejected by application logic at sensitivity 3.#

Dog#

Data is processed with the outdoor sensitivity 5 variant of the Proximity algorithm, where 1 out of 12 cases was rejected. Examples of three cases are given below.

../../_images/dog_2.png — Dog walking along detection zone border. Rejected at all sensitivities.#

../../_images/dog_1.png — Dog approaching ca 90 deg azimuth. Dog detected at all sensitivity levels.#

../../_images/dog_3.png — Dog crossing detection zone. Dog rejected at sensitivity level 1 and detected at 3 and 5.#

Conclusion#

In some cases, the birds were detectable and exceeded threshold values but were rejected by the application logic. Only in major events such as “fights” and birds in very close proximity to the sensor yielded detections. The cat is small and is in most cases either not detected or rejected by the algorithm. Because a dog - like a human user - is a relatively large approaching body, it is very difficult to reject with the simple Proximity algorithm. The sensitivity level can be used to filter out many cases but not all.

Plants#

Data is processed with the outdoor sensitivity 3 variant of the Proximity algorithm, where 14 out of 22 cases were rejected. Examples of cases are given below.

../../_images/bushes_2.png — Bushes in fan wind had hits filtered out by application logic.#

../../_images/bushes.png — Tree branch in fan wind had hits filtered out by application logic.#

../../_images/tree.png — Tree branch in wind. Branch swaying and leaves vibrating resulted in detections.#

Conclusion#

In many cases, the plant signal was detectable and exceeded threshold values but was rejected by the application logic. Larger tree branches were detected.

Test setup#

Three different test setups were used.

Test setup for weather and birds#

The setup used to execute rain and bird recordings in “recording mode”. The same setup was used for a long-term log in “normal mode”. The description of each mode can be found in the X4 Sensor Library User Guide (internal link).

Waterproof plastic box (W x H x L) angled at 7 deg mounted on the railing of a balcony. No objects were near the setup (at least 1.8 m away).
Sensor hot glued to lid facing outwards. The distance between the sensor and the lid is > 1 mm.
Sensor version: X4F103 rev 1.2

../../_images/box_with_sensor.jpeg — Sketch of the plastic box with sensor glued to the inside of the lid.#

Test setup for cat and dog#

The setup used to execute recordings of cats and dogs. * Plastic box (see illustration below) with the sensor mounted on a tripod elevated 1.15 m above the floor. * The sensor was placed in the void in the green area. * Sensor version: X4F103 rev 1.2

../../_images/aruco_cube.png — Sketch of the plastic box where the sensor is mounted on.#

Test setup for plants#

The setup used to execute recordings of plants and trees. The water gutter pipes used the same tripod mount placed 10 cm from the pipe without any other equipment.

Tripod mount in plastic (see illustration below) with radar, mounted on a tripod. The sensor placement was in the rectangular void.
A fan with 3 intensity levels is used to create artificial wind. Placed at a 1.8 m distance from the sensor to reduce interference to a minimum.
Distance between plants and radar: 0.8 m.
Sensor version: X4 F103 rev 1.3

../../_images/tripod.png — Sensor/radar mount in plastic.#