Stereo Cameras

Here are notes of various Stereo Cameras I've used and built. I used lots of them on Primo to recreate the outside world.

An machine vision hardware company called E-Conn Systems makes an inexpensive and really great stereo camera called Tara. It is a USB 3.0 Stereo Vision Camera link. It has excellent accurate PCL creation fromranges of 0.5 meters to 3 meters. Below are outputs from two Tara cameras used on Primo. An custom self-filter and a voxel-grid-filter were used with the raw PCL output from the cameras.

• 1/3” MT9V024 Monochrome image sensor
• 8-bit/10-bit Monochrome output format
• WVGA 752(H) x 480(V) (360690 pixels)
• Depth ranges from 50cm to 300cm
• Baseline distance of 60mm
• S-mount lens holder (M12 P0.5)

What a pain in the ass. E-Con systems has a package for the Tara stereo camera that has the package name of uvc_camera. This conflicts with the friggin uvc_camera package that I use for the monochrome camera. Wow. Friggen, wow.

I have been testing the E-Con systems package and it is the way to go.

This driver reads calibration data that is store on flash on the camera and then automatically generates yaml calibration files for the left and right cameras.

Brightness is from 1 to 7 Expo is from 10 to 100,000 (can go higher but useless)

The uvc_camera driver requires a left and right video device. The tara usb cam shows up as one device and for this reason the uvc_camera library doesn't work.

I tried the following which makes a symlink for the camera:

KERNEL=="video*", SUBSYSTEM=="video4linux", ATTRS{idProduct}=="c114", SYMLINK+="video-tara"

Whenever I launch the tara_node of the e-con systems uvc_package referencing this symlink I would get an error saying:

================================================================================
REQUIRED process [stereo/uvc_camera-2] has died!
process has died [pid 15906, exit code -11, cmd
/home/paul/TD1/td1_ros/catkin_ws/devel/lib/uvc_camera/tara_node __name:=uvc_camera
__log:=/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2.log].
log file:
/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2*.log
Initiating shutdown!
================================================================================

Oddly enough that log file would not get created. There is some complicated stuff that the tara stereo camera does with the USB device that gets created that handles some settings for the camera. My guess is that it uses the device name to reference the USB device and that's why it's not working.

Moral of the story: symlinks don't work with the tara driver.

The Tara stereo USB camera also creates a hidraw device that is used to talk to the IMU, get the onboard calibration and I think set exposure.

This device will be mounted as hidraw* with * being a number.

cat /sys/class/hidraw/hidraw2/device/uevent

Here is a sample output of the Tara USB device:

    
DRIVER=hid-generic
HID_ID=0003:00002560:0000C114
HID_NAME=e-con Systems See3CAM_Stereo
HID_PHYS=usb-0000:00:14.0-2/input2
HID_UNIQ=19355304
MODALIAS=hid:b0003g0001v00002560p0000C114

The Tara device creates these USB interfaces (result of usb-devices).

T:  Bus=04 Lev=01 Prnt=01 Port=03 Cnt=02 Dev#=  3 Spd=5000 MxCh= 0
D:  Ver= 3.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS= 9 #Cfgs=  1
P:  Vendor=2560 ProdID=c114 Rev=00.00
S:  Manufacturer=e-con Systems
S:  Product=See3CAM_Stereo
S:  SerialNumber=1A2E5304
C:  #Ifs= 4 Cfg#= 1 Atr=80 MxPwr=800mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=0e(video) Sub=01 Prot=00 Driver=uvcvideo
I:  If#= 1 Alt= 0 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
I:  If#= 2 Alt= 0 #EPs= 2 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbhid
I:  If#= 3 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbhid

Trying to debug with DDD. The gui is pretty helpful.

Zeroing in on the problem.

Here is a dump of the DeviceInstances:

print DeviceInstances ->listVidDevices[0]@6

device = 0xa4c650 "/dev/video2", 
friendlyname = 0xa31e70 "See3CAM_Stereo", 
bus_info = 0xa31e90 "usb-0000:02:00.0-3", 
vendor = 0xa55520 "2560", 
product = 0xa55780 "c114", 
deviceID = 2
 
device = 0xa4c450 "/dev/video4", 
friendlyname = 0xa4c470 "See3CAM_Stereo", 
bus_info = 0xa4c490 "usb-0000:02:00.0-4"
vendor = 0xa53d50 "2560"
product = 0xa53fd0 "c114"
deviceID = 4
 
device = 0xa502a0 "/dev/video5"
friendlyname = 0xa502c0 "HD Pro Webcam C920"
bus_info = 0xa502e0 "usb-0000:00:14.0-5.4.4.2"
vendor = 0xa51bf0 "046d"
product = 0xa51e50 "082d"
deviceID = 5
 
device = 0xa52290 "/dev/video0"
friendlyname = 0xa522b0 "e-con's 1MP Monochrome Camera"
bus_info = 0xa522e0 "usb-0000:00:14.0-1"
vendor = 0xa46a70 "2560"
product = 0xa4ac00 "c110"
deviceID = 0
 
 
device = 0xa4ac40 "/dev/video3" , 
friendlyname = 0xa4ac60 "ZED: ZED"
bus_info = 0xa58e20 "usb-0000:00:14.0-5.4.4.4.2"
vendor = 0xa59e10 "2b03"
product = 0xa59560 "f580"
deviceID = 3
 
device = 0xa59000 "/dev/video1"
friendlyname = 0xa59020 "e-con's 1MP Monochrome Camera"
bus_info = 0xa59050 "usb-0000:05:00.0-2"
vendor = 0xa61b60 "2560"
product = 0xa61de0 "c110"
deviceID = 1

I built a stereo camera from scratch using USB3 monocular cameras, and by building a variable hardware triggering circuit to sequence frame grabs. This allowed me to get a camera with a C-Mount lens holder, giving me access to quality and inexpensive optics, and also allowed me to vary the baseline to focus accuracy on where I needed it. I created my own driver from a See3Cam fork that allowed me to sync the two cameras and also dynamically reconfigure exposure and sensor brightness levels.

In the above image, the two mono cameras separated on a rail, along with a triggering circuit in the middle comprise the custom stereo camera. Two Taras are housed above looking down on the left and right corners.

Name: ZED

Baseline: 120mm

The following is a sample output dump from the ZED camera. It had RGB coloring which was nice, and was accurate to near ranges, but for points further than 3 meters wavy inaccuracies formed.

I used the ZED in Primo running Ubuntu 16.04. I used the ROS driver package provided.

• OpenCV was installed already
• Installed cuda using the debian. Ran into an X server issue and glxgears didn't work. Restarted and it fixed everything
• Downloaded the ZED sdk run file. Made it executable and ran it.
• It installs in /usr/local/zed
• Ran the ZED Explorer in the tools section of the install dir
• Updated the firmware from 922 to 1142. After I updated i did notice some frame glitching every now and then
• Went to the samples and built them using CMake

• Installed the source from github as a package in the catkin_ws workspace.
• I changed the FPS in the launch file to 15.
• Also can play with the confidence level.

Quality of the point cloud doesn't look great, and lots of “fall offs” on edges. With that said I didn't see any speckles and CPU usage is VERY low especially with the lower framerates. It is a very fast little device.

Found the resolution definitions here:

RESOLUTION_HD2K,   /**< 2208*1242, available framerates: 15 fps.*/
RESOLUTION_HD1080, /**< 1920*1080, available framerates: 15, 30 fps.*/
RESOLUTION_HD720,  /**< 1280*720, available framerates: 15, 30, 60 fps.*/
RESOLUTION_VGA,    /**< 672*376, available framerates: 15, 30, 60, 100 fps.*/
RESOLUTION_LAST

So with 15 FPS and VGA resolution, and a level 25 confidence (which is now and low) I'm still getting 120 mb/s throughput on the point cloud, which means i have to fucking voxel grid filter it. I did find a pcl_cuda based voxel grid filter though.

Ok I think i just spent a half hour trying to get PCL with Cuda to build and I am gonna go right ahead and give the fuck up. This will be a timesink like no other. Gonna figure out what the nodelet manager is and attach a CPU based voxel filter to and end of story.

Voxel grid filtering from the PCL library was used to significantly lower the data throughput on the PCLs generated by the stereo cameras. On the left image below is a raw PCL dump and on the right is a voxel grid filtered one.