Active Safety Envelopes using Light Curtains with Probabilistic Guarantees
Active Safety Envelopes using Light Curtains
with Probabilistic Guarantees
Carnegie Mellon University

Published at RSS, 2021


Talk (5min)

In order to safely navigate unknown environments, robots must accurately perceive dynamic obstacles. Instead of directly measuring the scene depth with a LiDAR sensor, we explore the use of a much cheaper and higher resolution sensor: programmable light curtains. Light curtains are controllable: they sense depth only along a surface that a user selects. We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles. We show that generating light curtains to sense random locations can quickly discover the safety envelope of unknown objects. Importantly, we present a novel analytical method to compute the probability of detecting an obstacle using random curtains. We combine random curtains with a machine learning based model that forecasts and tracks the motion of the safety envelope efficiently. Our method accurately estimates safety envelopes while providing probabilistic safety guarantees that can be used to certify the efficacy of a robot perception system to detect and avoid obstacles.


What is a Safety Envelope?



The safety envelope is an imaginary, vertically ruled surface that separates the robot from all obstacles in its environment. The region between the envelope and the robot is free space and is safe for the robot to occupy without colliding with any objects. Furthermore, the safety envelope "hugs" the closest object surfaces to maximize the amount of free space between the robot and the envelope. As long as the robot never intersects the safety envelope, it is guaranteed to not collide with any obstacle!


Programmable Light Curtains

LiDAR
Light Curtain
A programmable light curtain is a recently-invented controllable sensor that can measure the depth of any user-specified 2D vertical surface in the environment. It is relatively inexpensive, faster and of much higher resolution compared to LiDAR. To learn more about light curtains, please look at our light curtain website that contains an overview of light curtains, explains how they work, and lists our previous work on light curtains. In this work, we use light curtains to directly estimate the safety envelope of a scene.


Random Light Curtains


Suppose we have a scene with no prior knowledge of where the obstacles are. How should we place light curtains to find them? We place curtains at random locations in the scene, which we refer to as "random curtains". It turns out to be incredibly hard for obstacles to avoid getting detected by random curtains! We place random curtains to quickly discover unknown objects and estimate the safety envelope of the scene.

We also present a novel analytical technique to compute the probability of random curtains intersecting and detecting obstacles. This provides probabilistic safety guarantees for our perception system towards detecting and avoiding obstacles. The next section shows a demo of this technique.



Random Curtain Analysis Demo

Click here for Random Curtain Demo!


Check out our interactive web-demo for the probabilistic analysis of random curtains! The demo lets the user design the shape, size and location of an obstacle in the top-down view. It simulates random curtains, and visualizes points at which the curtain intersects and detects the object. The demo runs our dynamic programming technique that analytically and efficiently computes the probability of detecting the given object in a couple of seconds. Finally, we provide an analysis of the detection probability as a function of the time required to place mutliple random curtains.


Forecasting Safety Envelopes


Assume that we have estimated the safety envelope in the current timestep. As objects move and the scene changes with time, we wish to estimate the envelope for the next timestep. In this case, it may be inefficient to use random curtain placements. Instead, we use machine learning to forecast how the safety envelope will move in the next timestep.


Active Light Curtain Placement Pipeline


Our pipeline for placing light curtains to estimate and track the safety envelope is as follows. Given previous light curtain measurements, we train a neural network to forecast how the safety envelope of the scene will evolve in the next timestep. We then place light curtains to sense the predicted locations. At the same time, we place random light curtains to discover obstacles and update our predictions. Finally, the light curtain measurements are input to the forecasting method, closing the loop.


Qualitative Results

The videos below show the results of our method estimating the safety envelope in a real-world environment. The scene consists of multiple people walking in a range of different motions. The left video shows the RGB video of the scene. The middle video shows the light curtain in black, along with a LiDAR point cloud in red. The LiDAR points are only to aid visualization; they are not used by our algorithm. The right video shows the intersection points of the light curtain and object surfaces in green. The middle and right videos are taken from the top-down view.



Brisk walking

RGB Scene Video
Top-down view with LiDAR points
Top-down view of intersection points

Relaxed walking

RGB Scene Video
Top-down view with LiDAR points
Top-down view of intersection points

Many people (structured walking)

RGB Scene Video
Top-down view with LiDAR points
Top-down view of intersection points

Many people (haphazard, occluded walking)

RGB Scene Video
Top-down view with LiDAR points
Top-down view of intersection points

Fast motion

RGB Scene Video
Top-down view with LiDAR points
Top-down view of intersection points


Source Code

We have released our implementation and source code on Github. Try our code!


Paper and Bibtex

[Paper] [ArXiv] [Appendix]

Citation
 
Siddharth Ancha, Gaurav Pathak, Srinivasa G. Narasimhan,
and David Held. "Active Safety Envelopes using Light Curtains with Probabilistic Guarantees."
In Proceedings of Robotics: Science and Systems (RSS), July 2021.

[Bibtex]
@inproceedings{Ancha-RSS-21, 
  author    = {Siddharth Ancha
                AND Gaurav Pathak
                AND Srinivasa Narasimhan
                AND David Held}, 
  title     = {{Active Safety Envelopes using Light Curtains
                with Probabilistic Guarantees}}, 
  booktitle = {Proceedings of Robotics: Science and Systems}, 
  year      = {2021}, 
  address   = {Virtual}, 
  month     = {July}, 
  doi       = {10.15607/rss.2021.xvii.045} 
}


Acknowledgements

We thank Adithya Pediredla, N. Dinesh Reddy and Zelin Ye for help with real-world experiments. This material is based upon work supported by the National Science Foundation under Grants No. IIS-1849154, IIS-1900821 and by the United States Air Force and DARPA under Contract No. FA8750-18-C-0092.

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, United States Air Force and DARPA.