Camera Design

Android uses an IPC framework called Binder that allows for inter process communication. Everyone in Android is connected together in Binder.

Android allows for processes to communicate with each other using Intents. You can send intents to other processes and then handle their callbacks.

Binder Structure

Binder allows us to communicate to other process through a managed way. It's pretty much roscore in ROS.

Theres a bunch of different stuff that will happen to get messages from senders to receivers. No need to worry about it but here is how it looks.

Here is an interesting example of how this plays out when an app requests a location service.

Compositions: models s relationship. Owner is responsible for the creation and destruction of this data.

Aggregation: models “has a” relationship. This data can be a member elsewhere. The class including this data is not responsible for its creation/deletion.

Association: models a bi-directional association relationship. For example a patient has a doctor, and a doctor has a patient. They both know about each other.

Reflexive association: models relationship of objects of the same type. For instance, a course may have prerequisites, which are themselves courses.

| Property                                 | Composition    | Aggregation    | Association                     |
| Relationship type                        | Whole/part     | Whole/part     | Otherwise unrelated             |
| Members can belong to multiple classes   | No             | Yes            | Yes                             |
| Members existence managed by class       | Yes            | No             | No                              |
| Directionality                           | Unidirectional | Unidirectional | Unidirectional or bidirectional |
| Relationship verb                        | Part-of        | Has-a          | Uses-a                          |

Abstract Class: Abstract classes contain pure virtual functions are cannot be instantiated. They can however contain constructors, and other code that gets inherited by derived classes.

Virtual Functions: in C++ a function marked virtual derives to the lowest derived function of that class. If you have a pointer of base class type, but it is a pointer to a derived class, the function will resolve to derived class.

Pure Virtual Function: A function marked with = 0 that has to be implemented by a derived class. Having a pure virtual function in a class makes the class abstract.

class A{
  int m_id;
  A(int id) : m_id{id}
  virtual void printId() = 0; // this make the function pure virtual, and the

Interfaces are pure virtual functions that have no member variables and contain all pure virtual functions.

class ICameraDriver {
  virtual capture() = 0;
  virtual stream() = 0;
  virtual ~ICameraDriver() {}

The following are different design patterns that can be implemented.

Strategy Pattern

I've used this pattern a lot. It's when you define a set of functionality and encapsulate them and then make it interchangeable.

Here is some implemented C++ code. To see some code, check out ex4 of oop1 in learn. Use the strategy pattern when you need to use one of several behaviors dynamically.

Observer Pattern

This very similar to the publisher subscriber model. An Observer

A 1920 x 1080 video at 30 fps with RGB8 (24 bits total) comes out to 1.5Gbits/s.

Encoding and decoding.

White Balance

Correct the colors based on a color temperature assumption.

Color Correction Matrix

Sensor based color correction matrix that is generated at the factory using a color chart that solves for the error between actual and produced color charts.

Gamma Correction

Color space is not linear in reality, but the sensor perceives it as such. This will lead to images that have colors compressed other side of the range, and make it harder for us to distinguish between them. Gamma correcection corrects for the exponent of the non linear nature of light and color.


Camera sensors will exhibit noise over the pixel array.

There are multiple de-noising techniques, such as temporal denoising that uses a burst of frames to average out the random noise pattern.

This can also be done on a single frame use a spatial denoising technique, that looks at the values of neighboring pixels.

Back Side Illumination

Final Pipeline example.

API for controlling a camera by Kari Pulli

You can share memory between forked processes using mmap. This gives you a pointer to shared memory.


Physically contiguous memory allocated in the kernal space.


Physically contiguous memory allocated in the kernal space. Initialized to zero.


Like malloc but initializes data requested to zeros.

sysfs is a virtual filesystem that is used to communicate information about the system to the user space.

  • camera_design.txt
  • Last modified: 2020/08/31 03:24
  • by paul