Differences

This shows you the differences between two versions of the page.

--- stereo_cameras [2019/02/28 17:17]
paul created
+++ stereo_cameras [2019/03/31 14:49] (current)
@@ Line 1: / Line 1: @@
-======= Stereo Cameras =======
+====== Stereo Cameras ======
+Here are notes of various Stereo Cameras I've used and built. I used lots of them on [[primo_robot|Primo]] to recreate the outside world.
+{{::cleaned_up.gif?direct&600|}}
+===== E-Con Tara =====
+An machine vision hardware company called E-Conn Systems makes an inexpensive and really great stereo camera called Tara. It is a USB 3.0 Stereo Vision Camera
+[[https://www.e-consystems.com/3D-USB-stereo-camera.asp#key-features|link]]. It
+has excellent accurate PCL creation fromranges of 0.5 meters to 3 meters. Below are outputs from two Tara cameras used on [[primo_robot|Primo]]. An custom self-filter and a voxel-grid-filter were used with the raw PCL output from the cameras.
+{{::stereo_rig.png?direct&600|}}
+==== Specs ====
+  * 1/3” MT9V024 Monochrome image sensor
+  * 8-bit/10-bit Monochrome output format
+  * WVGA 752(H) x 480(V) (360690 pixels)
+  * Depth ranges from 50cm to 300cm
+  * Baseline distance of 60mm
+  * S-mount lens holder (M12 P0.5)
+==== General Notes ====
+What a pain in the ass. E-Con systems has a package for the Tara stereo camera that
+has the package name of ''%%uvc_camera%%''. This conflicts with the friggin
+''%%uvc_camera%%'' package that I use for the monochrome camera. Wow. Friggen, wow.
+I have been testing the E-Con systems package and it is the way to go.
+This driver reads calibration data that is store on flash on the camera and then
+automatically generates yaml calibration files for the left and right cameras.
+Brightness is from 1 to 7
+Expo is from 10 to 100,000 (can go higher but useless)
+==== uvc_camera ====
+The uvc_camera driver requires a left and right video device. The tara usb cam shows
+up as one device and for this reason the uvc_camera library doesn't work.
+==== udev Rules ====
+I tried the following which makes a symlink for the camera:
+<code>
+KERNEL=="video*", SUBSYSTEM=="video4linux", ATTRS{idProduct}=="c114", SYMLINK+="video-tara"
+</code>
+Whenever I launch the tara_node of the e-con systems uvc_package referencing this
+symlink I would get an error saying:
+<code>
+================================================================================
+REQUIRED process [stereo/uvc_camera-2] has died!
+process has died [pid 15906, exit code -11, cmd
+/home/paul/TD1/td1_ros/catkin_ws/devel/lib/uvc_camera/tara_node __name:=uvc_camera
+__log:=/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2.log].
+log file:
+/home/paul/.ros/log/55ac4df8-708b-11e7-9301-507b9df0bf3c/stereo-uvc_camera-2*.log
+Initiating shutdown!
+================================================================================
+</code>
+Oddly enough that log file would not get created. There is some complicated stuff
+that the tara stereo camera does with the USB device that gets created that handles
+some settings for the camera. My guess is that it uses the device name to reference
+the USB device and that's why it's not working.
+Moral of the story: symlinks don't work with the tara driver.
+==== USB hidraw device ====
+The Tara stereo USB camera also creates a ''%%hidraw%%'' device that is used to talk to the
+IMU, get the onboard calibration and I think set exposure.
+This device will be mounted as ''%%hidraw*%%'' with * being a number.
+<code>
+cat /sys/class/hidraw/hidraw2/device/uevent
+</code>
+Here is a sample output of the Tara USB device:
+<code>
+DRIVER=hid-generic
+HID_ID=0003:00002560:0000C114
+HID_NAME=e-con Systems See3CAM_Stereo
+HID_PHYS=usb-0000:00:14.0-2/input2
+HID_UNIQ=19355304
+MODALIAS=hid:b0003g0001v00002560p0000C114
+</code>
+The Tara device creates these USB interfaces (result of ''%%usb-devices%%'').
+<code>
+T:  Bus=04 Lev=01 Prnt=01 Port=03 Cnt=02 Dev#=  3 Spd=5000 MxCh= 0
+D:  Ver= 3.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS= 9 #Cfgs=  1
+P:  Vendor=2560 ProdID=c114 Rev=00.00
+S:  Manufacturer=e-con Systems
+S:  Product=See3CAM_Stereo
+S:  SerialNumber=1A2E5304
+C:  #Ifs= 4 Cfg#= 1 Atr=80 MxPwr=800mA
+I:  If#= 0 Alt= 0 #EPs= 1 Cls=0e(video) Sub=01 Prot=00 Driver=uvcvideo
+I:  If#= 1 Alt= 0 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
+I:  If#= 2 Alt= 0 #EPs= 2 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbhid
+I:  If#= 3 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbhid
+</code>
+==== Debug Notes ====
+Trying to debug with DDD. The gui is pretty helpful.
+Zeroing in on the problem.
+Here is a dump of the DeviceInstances:
+<code bash>
+print DeviceInstances ->listVidDevices[0]@6
+</code>
+<code bash>
+device = 0xa4c650 "/dev/video2",
+friendlyname = 0xa31e70 "See3CAM_Stereo",
+bus_info = 0xa31e90 "usb-0000:02:00.0-3",
+vendor = 0xa55520 "2560",
+product = 0xa55780 "c114",
+deviceID = 2
+device = 0xa4c450 "/dev/video4",
+friendlyname = 0xa4c470 "See3CAM_Stereo",
+bus_info = 0xa4c490 "usb-0000:02:00.0-4"
+vendor = 0xa53d50 "2560"
+product = 0xa53fd0 "c114"
+deviceID = 4
+device = 0xa502a0 "/dev/video5"
+friendlyname = 0xa502c0 "HD Pro Webcam C920"
+bus_info = 0xa502e0 "usb-0000:00:14.0-5.4.4.2"
+vendor = 0xa51bf0 "046d"
+product = 0xa51e50 "082d"
+deviceID = 5
+device = 0xa52290 "/dev/video0"
+friendlyname = 0xa522b0 "e-con's 1MP Monochrome Camera"
+bus_info = 0xa522e0 "usb-0000:00:14.0-1"
+vendor = 0xa46a70 "2560"
+product = 0xa4ac00 "c110"
+deviceID = 0
+device = 0xa4ac40 "/dev/video3" ,
+friendlyname = 0xa4ac60 "ZED: ZED"
+bus_info = 0xa58e20 "usb-0000:00:14.0-5.4.4.4.2"
+vendor = 0xa59e10 "2b03"
+product = 0xa59560 "f580"
+deviceID = 3
+device = 0xa59000 "/dev/video1"
+friendlyname = 0xa59020 "e-con's 1MP Monochrome Camera"
+bus_info = 0xa59050 "usb-0000:05:00.0-2"
+vendor = 0xa61b60 "2560"
+product = 0xa61de0 "c110"
+deviceID = 1
+</code>
+===== Custom Stereo Camera =====
+I built a stereo camera from scratch using USB3 monocular cameras, and by
+building a variable hardware triggering circuit to sequence frame grabs. This
+allowed me to get a camera with a C-Mount lens holder, giving me access to
+quality and inexpensive optics, and also allowed me to vary the baseline to
+focus accuracy on where I needed it. I created my own driver from a See3Cam fork
+that allowed me to sync the two cameras and also dynamically reconfigure exposure
+and sensor brightness levels.
+{{::img_6559.jpg?direct&600|}}
+In the above image, the two mono cameras separated on a rail, along with a triggering circuit in the middle comprise the custom stereo camera. Two Taras are housed above looking down on the left and right corners.
+{{::fwd_cam_expo.gif?direct&800|}}
+===== ZED Camera =====
+==== Specs ====
+**Name:**       ZED
+**Baseline:**   120mm
+The following is a sample output dump from the ZED camera. It had RGB coloring which was nice, and was accurate to near ranges, but for points further than 3 meters wavy inaccuracies formed.
+{{::zed_1.png?600|}}
+==== Setup ====
+I used the ZED in Primo running Ubuntu 16.04. I used the ROS driver package
+provided.
+  * OpenCV was installed already
+  * Installed cuda using the debian. Ran into an X server issue and ''%%glxgears%%'' didn't work. Restarted and it fixed everything
+  * Downloaded the ZED sdk run file. Made it executable and ran it.
+  * It installs in ''%%/usr/local/zed%%''
+  * Ran the ZED Explorer in the tools section of the install dir
+  * Updated the firmware from 922 to 1142. After I updated i did notice some frame glitching every now and then
+  * Went to the samples and built them using CMake
+==== ROS Install ====
+  * Installed the source from github as a package in the catkin_ws workspace.
+  * I changed the FPS in the launch file to 15.
+  * Also can play with the confidence level.
+==== General Notes ====
+Quality of the point cloud doesn't look great, and lots of "fall offs" on edges.
+With that said I didn't see any speckles and CPU usage is VERY low especially
+with the lower framerates. It is a very fast little device.
+==== Software ====
+Found the resolution definitions here:
+<code c++>
+RESOLUTION_HD2K,   /**< 2208*1242, available framerates: 15 fps.*/
+RESOLUTION_HD1080, /**< 1920*1080, available framerates: 15, 30 fps.*/
+RESOLUTION_HD720,  /**< 1280*720, available framerates: 15, 30, 60 fps.*/
+RESOLUTION_VGA,    /**< 672*376, available framerates: 15, 30, 60, 100 fps.*/
+RESOLUTION_LAST
+</code>
+So with 15 FPS and VGA resolution, and a level 25 confidence (which is now and
+low) I'm still getting 120 mb/s throughput on the point cloud, which means i
+have to fucking voxel grid filter it. I did find a ''%%pcl_cuda%%'' based voxel grid
+filter though.
+Ok I think i just spent a half hour trying to get PCL with Cuda to build and I
+am gonna go right ahead and give the fuck up. This will be a timesink like no
+other. Gonna figure out what the nodelet manager is and attach a CPU based voxel
+filter to and end of story.
+===== Filtering =====
+Voxel grid filtering from the PCL library was used to significantly lower the data throughput on the PCLs generated by the stereo cameras. On the left image below is a raw PCL dump and on the right is a voxel grid filtered one.
+{{::voxel_filter.png?600|}}