====== Stereo Cameras ======
Here are notes of various Stereo Cameras I've used and built. I used lots of them on [[primo_robot|Primo]] to recreate the outside world.
{{::cleaned_up.gif?direct&600|}}
===== E-Con Tara =====
An machine vision hardware company called E-Conn Systems makes an inexpensive and really great stereo camera called Tara. It is a USB 3.0 Stereo Vision Camera
[[https://www.e-consystems.com/3D-USB-stereo-camera.asp#key-features|link]]. It
has excellent accurate PCL creation fromranges of 0.5 meters to 3 meters. Below are outputs from two Tara cameras used on [[primo_robot|Primo]]. An custom self-filter and a voxel-grid-filter were used with the raw PCL output from the cameras.
{{::stereo_rig.png?direct&600|}}
==== Specs ====
* 1/3” MT9V024 Monochrome image sensor
* 8-bit/10-bit Monochrome output format
* WVGA 752(H) x 480(V) (360690 pixels)
* Depth ranges from 50cm to 300cm
* Baseline distance of 60mm
* S-mount lens holder (M12 P0.5)
==== General Notes ====
What a pain in the ass. E-Con systems has a package for the Tara stereo camera that
has the package name of ''%%uvc_camera%%''. This conflicts with the friggin
''%%uvc_camera%%'' package that I use for the monochrome camera. Wow. Friggen, wow.
I have been testing the E-Con systems package and it is the way to go.
This driver reads calibration data that is store on flash on the camera and then
automatically generates yaml calibration files for the left and right cameras.
Brightness is from 1 to 7
Expo is from 10 to 100,000 (can go higher but useless)
==== uvc_camera ====
The uvc_camera driver requires a left and right video device. The tara usb cam shows
up as one device and for this reason the uvc_camera library doesn't work.
==== udev Rules ====
I tried the following which makes a symlink for the camera:
KERNEL=="video*", SUBSYSTEM=="video4linux", ATTRS{idProduct}=="c114", SYMLINK+="video-tara"
Whenever I launch the tara_node of the e-con systems uvc_package referencing this
symlink I would get an error saying:
================================================================================
REQUIRED process [stereo/uvc_camera-2] has died!
process has died [pid 15906, exit code -11, cmd
/home/paul/TD1/td1_ros/catkin_ws/devel/lib/uvc_camera/tara_node __name:=uvc_camera
__log:=/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2.log].
log file:
/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2*.log
Initiating shutdown!
================================================================================
Oddly enough that log file would not get created. There is some complicated stuff
that the tara stereo camera does with the USB device that gets created that handles
some settings for the camera. My guess is that it uses the device name to reference
the USB device and that's why it's not working.
Moral of the story: symlinks don't work with the tara driver.
==== USB hidraw device ====
The Tara stereo USB camera also creates a ''%%hidraw%%'' device that is used to talk to the
IMU, get the onboard calibration and I think set exposure.
This device will be mounted as ''%%hidraw*%%'' with * being a number.
cat /sys/class/hidraw/hidraw2/device/uevent
Here is a sample output of the Tara USB device:
DRIVER=hid-generic
HID_ID=0003:00002560:0000C114
HID_NAME=e-con Systems See3CAM_Stereo
HID_PHYS=usb-0000:00:14.0-2/input2
HID_UNIQ=19355304
MODALIAS=hid:b0003g0001v00002560p0000C114
The Tara device creates these USB interfaces (result of ''%%usb-devices%%'').
T: Bus=04 Lev=01 Prnt=01 Port=03 Cnt=02 Dev#= 3 Spd=5000 MxCh= 0
D: Ver= 3.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS= 9 #Cfgs= 1
P: Vendor=2560 ProdID=c114 Rev=00.00
S: Manufacturer=e-con Systems
S: Product=See3CAM_Stereo
S: SerialNumber=1A2E5304
C: #Ifs= 4 Cfg#= 1 Atr=80 MxPwr=800mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=0e(video) Sub=01 Prot=00 Driver=uvcvideo
I: If#= 1 Alt= 0 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
I: If#= 2 Alt= 0 #EPs= 2 Cls=03(HID ) Sub=00 Prot=00 Driver=usbhid
I: If#= 3 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=00 Prot=00 Driver=usbhid
==== Debug Notes ====
Trying to debug with DDD. The gui is pretty helpful.
Zeroing in on the problem.
Here is a dump of the DeviceInstances:
print DeviceInstances ->listVidDevices[0]@6
device = 0xa4c650 "/dev/video2",
friendlyname = 0xa31e70 "See3CAM_Stereo",
bus_info = 0xa31e90 "usb-0000:02:00.0-3",
vendor = 0xa55520 "2560",
product = 0xa55780 "c114",
deviceID = 2
device = 0xa4c450 "/dev/video4",
friendlyname = 0xa4c470 "See3CAM_Stereo",
bus_info = 0xa4c490 "usb-0000:02:00.0-4"
vendor = 0xa53d50 "2560"
product = 0xa53fd0 "c114"
deviceID = 4
device = 0xa502a0 "/dev/video5"
friendlyname = 0xa502c0 "HD Pro Webcam C920"
bus_info = 0xa502e0 "usb-0000:00:14.0-5.4.4.2"
vendor = 0xa51bf0 "046d"
product = 0xa51e50 "082d"
deviceID = 5
device = 0xa52290 "/dev/video0"
friendlyname = 0xa522b0 "e-con's 1MP Monochrome Camera"
bus_info = 0xa522e0 "usb-0000:00:14.0-1"
vendor = 0xa46a70 "2560"
product = 0xa4ac00 "c110"
deviceID = 0
device = 0xa4ac40 "/dev/video3" ,
friendlyname = 0xa4ac60 "ZED: ZED"
bus_info = 0xa58e20 "usb-0000:00:14.0-5.4.4.4.2"
vendor = 0xa59e10 "2b03"
product = 0xa59560 "f580"
deviceID = 3
device = 0xa59000 "/dev/video1"
friendlyname = 0xa59020 "e-con's 1MP Monochrome Camera"
bus_info = 0xa59050 "usb-0000:05:00.0-2"
vendor = 0xa61b60 "2560"
product = 0xa61de0 "c110"
deviceID = 1
===== Custom Stereo Camera =====
I built a stereo camera from scratch using USB3 monocular cameras, and by
building a variable hardware triggering circuit to sequence frame grabs. This
allowed me to get a camera with a C-Mount lens holder, giving me access to
quality and inexpensive optics, and also allowed me to vary the baseline to
focus accuracy on where I needed it. I created my own driver from a See3Cam fork
that allowed me to sync the two cameras and also dynamically reconfigure exposure
and sensor brightness levels.
{{::img_6559.jpg?direct&600|}}
In the above image, the two mono cameras separated on a rail, along with a triggering circuit in the middle comprise the custom stereo camera. Two Taras are housed above looking down on the left and right corners.
{{::fwd_cam_expo.gif?direct&800|}}
===== ZED Camera =====
==== Specs ====
**Name:** ZED
**Baseline:** 120mm
The following is a sample output dump from the ZED camera. It had RGB coloring which was nice, and was accurate to near ranges, but for points further than 3 meters wavy inaccuracies formed.
{{::zed_1.png?600|}}
==== Setup ====
I used the ZED in Primo running Ubuntu 16.04. I used the ROS driver package
provided.
* OpenCV was installed already
* Installed cuda using the debian. Ran into an X server issue and ''%%glxgears%%'' didn't work. Restarted and it fixed everything
* Downloaded the ZED sdk run file. Made it executable and ran it.
* It installs in ''%%/usr/local/zed%%''
* Ran the ZED Explorer in the tools section of the install dir
* Updated the firmware from 922 to 1142. After I updated i did notice some frame glitching every now and then
* Went to the samples and built them using CMake
==== ROS Install ====
* Installed the source from github as a package in the catkin_ws workspace.
* I changed the FPS in the launch file to 15.
* Also can play with the confidence level.
==== General Notes ====
Quality of the point cloud doesn't look great, and lots of "fall offs" on edges.
With that said I didn't see any speckles and CPU usage is VERY low especially
with the lower framerates. It is a very fast little device.
==== Software ====
Found the resolution definitions here:
RESOLUTION_HD2K, /**< 2208*1242, available framerates: 15 fps.*/
RESOLUTION_HD1080, /**< 1920*1080, available framerates: 15, 30 fps.*/
RESOLUTION_HD720, /**< 1280*720, available framerates: 15, 30, 60 fps.*/
RESOLUTION_VGA, /**< 672*376, available framerates: 15, 30, 60, 100 fps.*/
RESOLUTION_LAST
So with 15 FPS and VGA resolution, and a level 25 confidence (which is now and
low) I'm still getting 120 mb/s throughput on the point cloud, which means i
have to fucking voxel grid filter it. I did find a ''%%pcl_cuda%%'' based voxel grid
filter though.
Ok I think i just spent a half hour trying to get PCL with Cuda to build and I
am gonna go right ahead and give the fuck up. This will be a timesink like no
other. Gonna figure out what the nodelet manager is and attach a CPU based voxel
filter to and end of story.
===== Filtering =====
Voxel grid filtering from the PCL library was used to significantly lower the data throughput on the PCLs generated by the stereo cameras. On the left image below is a raw PCL dump and on the right is a voxel grid filtered one.
{{::voxel_filter.png?600|}}