3.3. DesignCore® Discovery IMX678¶

D3 Embedded’s DesignCore® Discovery IMX678 camera modules feature Sony’s IMX678 image sensor. The IMX678 features high sensitivity, low dark current, and no smear with low power consumption. It can be used for various applications including as a security camera.

Warning

Due to limitations with GMSL chips, D3 Embedded’s IMX678 cannot be mixed with non-IMX678 sensors on the same deserializer. For example, 4 IMX678s on a single GMSL deserializer is a supported configuration but 3 IMX678s and 1 AR0234 is an unsupported configuration. Note that streaming 3 or 4 IMX678s simultaneously at full resolution on a single deserializer will likely exceed the deserializer’s bandwidth capabilities and cause corruption of the video streams.

3.3.1. Modes¶

D3 Embedded’s IMX678 driver supports the following modes on NVIDIA Jetson:

IMX678 Modes¶
Mode ID	Resolution	Framerate	HDR
0	3840x2160	30 fps	–
1	3840x2160	25 fps	–
2	3840x2160	25 fps	Digital Overlap HDR
3	1920x1080 (2x2 binning)	60 fps	–
4	1920x1080 (2x2 binning)	25 fps	–
5	1920x1080 (2x2 binning)	25 fps	Digital Overlap HDR

3.3.2. Controls¶

In addition to the standard V4L2 controls (gain, exposure, etc.), D3 Embedded’s IMX678 driver supports the following custom controls:

IMX678 Custom Controls¶
Control	Description	Values
`frame_sync`	Controls frame synchronization. When enabled, the carrier board must generate or forward a pulse signal to the IMX678’s XVS pin. Refer to your carrier board’s documentation. This setting is ‘sticky’ - it will persist after streaming stops. It must be set prior to starting a stream. If set during a stream, it will not be applied until the current stream is stopped and a new stream is started.	`0`: Frame synchronization disabled (default) `1`: Frame synchronization enabled
`gain_conversion`	Controls which gain conversion mode the sensor uses. High conversion gain helps lower noise and improve the signal-to-noise ratio in low light scenes. Low conversion gain increases dynamic range and provides more detail in bright light scenes. This setting is ‘sticky’ - it will persist after streaming stops. Unlike `frame_sync`, this can be set during a stream and takes effect immediately.	`0`: Low conversion gain (default) `1`: High conversion gain

These custom controls are accessible with the standard V4L2 utilities:

v4l2-ctl -d 0 -cframe_sync=1
v4l2-ctl -d 0 -cgain_conversion=1

3.3.3. Example Usage¶

3.3.3.1. GStreamer Pipelines¶

These pipelines fetch image data from the sensor and display it in a window:

# Mode 0
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=0 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=30/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! xvimagesink

# Mode 1 (with framerate overlay)
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=1 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=25/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! fpsdisplaysink

# Mode 2
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=2 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=3840,height=2160,framerate=25/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! xvimagesink

# Mode 3
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=3 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=60/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! xvimagesink

# Mode 4
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=4 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=25/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! xvimagesink

# Mode 5
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=4 \
   ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=25/1' \
   ! nvvidconv ! 'video/x-raw,format=I420' \
   ! queue ! xvimagesink

Note

sensor-id must be set to the desired IMX678’s Argus sensor ID. To determine this ID, refer to Deterministic Camera Nodes and IDs.

3.3.4. Miscellaneous Notes¶

3.3.4.1. Handling¶

Sony provides the following guidance for handling the IMX678 sensor:

Do not expose the sensor to strong light such as sunrays for extended periods, as the color filters will become discolored.
Care should be taken not to apply excessive mechanical shocks or force.
Imaging characteristics of the sensor may be affected when approaching strong electromagnetic waves or magnetic fields during operation.

3.3.4.2. White Pixels¶

Particle radiation such as cosmic rays may permanently distort pixels of image sensors using CMOS technology, including the IMX678. Distorted pixels may cause white points in dark signals of pictures (“white pixels”). Unfortunately, it is not possible with current CMOS technology to prevent white pixels from occurring.

White pixel occurrence fluctuates depending on the camera’s environment (altitude, geomagnetic latitude, building structure, etc.) as well as time (solar activity).