Talk (5min)

In order to safely navigate unknown environments, robots must accurately perceive dynamic obstacles. Instead of directly measuring the scene depth with a LiDAR sensor, we explore the use of a much cheaper and higher resolution sensor: programmable light curtains. Light curtains are controllable: they sense depth only along a surface that a user selects. We use light curtains to estimate the safety envelope of a scene: a hypothetical surface that separates the robot from all obstacles. We show that generating light curtains to sense random locations can quickly discover the safety envelope of unknown objects. Importantly, we present a novel analytical method to compute the probability of detecting an obstacle using random curtains. We combine random curtains with a machine learning based model that forecasts and tracks the motion of the safety envelope efficiently. Our method accurately estimates safety envelopes while providing probabilistic safety guarantees that can be used to certify the efficacy of a robot perception system to detect and avoid obstacles.

The safety envelope is an imaginary, vertically ruled surface that separates the robot from all obstacles in its environment. The region between the envelope and the robot is free space and is safe for the robot to occupy without colliding with any objects. Furthermore, the safety envelope "hugs" the closest object surfaces to maximize the amount of free space between the robot and the envelope. As long as the robot never intersects the safety envelope, it is guaranteed to not collide with any obstacle!

Suppose we have a scene with no prior knowledge of where the obstacles are. How should we place light curtains to find them? We place curtains at random locations in the scene, which we refer to as "random curtains". It turns out to be incredibly hard for obstacles to avoid getting detected by random curtains! We place random curtains to quickly discover unknown objects and estimate the safety envelope of the scene.

We also present a novel analytical technique to compute the probability of random curtains intersecting and detecting obstacles. This provides probabilistic safety guarantees for our perception system towards detecting and avoiding obstacles. The next section shows a demo of this technique.

Check out our interactive web-demo for the probabilistic analysis of random curtains! The demo lets the user design the shape, size and location of an obstacle in the top-down view. It simulates random curtains, and visualizes points at which the curtain intersects and detects the object. The demo runs our dynamic programming technique that analytically and efficiently computes the probability of detecting the given object in a couple of seconds. Finally, we provide an analysis of the detection probability as a function of the time required to place mutliple random curtains.

Assume that we have estimated the safety envelope in the current timestep. As objects move and the scene changes with time, we wish to estimate the envelope for the next timestep. In this case, it may be inefficient to use random curtain placements. Instead, we use machine learning to forecast how the safety envelope will move in the next timestep.

Our pipeline for placing light curtains to estimate and track the safety envelope is as follows. Given previous light curtain measurements, we train a neural network to forecast how the safety envelope of the scene will evolve in the next timestep. We then place light curtains to sense the predicted locations. At the same time, we place random light curtains to discover obstacles and update our predictions. Finally, the light curtain measurements are input to the forecasting method, closing the loop.

The videos below show the results of our method estimating the safety envelope in a real-world environment. The scene consists of multiple people walking in a range of different motions. The left video shows the RGB video of the scene. The middle video shows the light curtain in black, along with a LiDAR point cloud in red. The LiDAR points are only to aid visualization; they are not used by our algorithm. The right video shows the intersection points of the light curtain and object surfaces in green. The middle and right videos are taken from the top-down view.

Brisk walking

Relaxed walking

Many people (structured walking)

Many people (haphazard, occluded walking)

Fast motion