Use Case - Lattice FPGA based
The Lattice CrossLink FPGA Camera Emulator is intended to streamline early-stage camera bring-up on embedded platforms by providing a deterministic MIPI CSI-2 video source when a physical camera is unavailable or impractical.
Typical use cases include:
- Driver exploration and early bring-up on new Jetson platforms without depending on sensor availability.
- Debugging on custom carrier boards (device-tree/GPIO/lane mapping issues) using a known-good CSI-2 stream.
- Camera pipeline validation with repeatable test patterns and controlled timing.
Camera Emulation
This project emulates the essential behavior of a Sony IMX219 image sensor using a Lattice CrossLink FPGA, allowing an NVIDIA Jetson platform to detect and use the emulator as if a real camera were connected.
The emulator reproduces the two key aspects required by the Jetson camera stack:
- Sensor configuration and identification, handled through an I²C slave interface compatible with the IMX219 register map.
- Image data transmission, implemented as a MIPI CSI-2 compliant video stream carrying RAW10 image frames over two data lanes.
During system boot and camera initialization, the Jetson accesses the emulator through the I²C bus, reading identification registers and writing configuration values exactly as it would with a physical IMX219 sensor. These transactions enable mode selection and control of the video stream.
Once streaming is enabled, the FPGA internally generates deterministic Bayer test patterns, packs them into RAW10 format, and transmits complete CSI-2 frames with correct timing and packet structure expected by the host.
Block Design
The camera emulator is implemented on a Lattice CrossLink FPGA and is organized as a set of functional blocks that together reproduce the behavior of a MIPI CSI-2 image sensor. The high-level architecture is shown in the block diagram above.
At the top level, the FPGA connects to the NVIDIA Jetson platform through three interfaces:
- A two-lane MIPI CSI-2 data interface
- An I²C control interface (SDA/SCL)
- A reset signal
Internally, the design is composed of the following main blocks.
I²C Interface
The I²C Interface implements a soft I²C slave that responds to transactions issued by the Jetson during camera discovery and configuration. This block exposes a minimal, IMX219-compatible register map used for sensor identification and basic mode configuration. Register writes and reads received over I²C are forwarded to the Emulator Controller, which interprets them and updates the internal state of the emulator accordingly.
Emulator Controller
The Emulator Controller is the central coordination block of the design. It manages sensor-like behavior by:
- Interpreting configuration data received through the I²C interface
- Controlling frame and line sequencing
- Synchronizing the image generator with the CSI-2 transmission timing
This block drives the MIPI CSI-2 IP core, ensuring that frames are transmitted with the correct structure and timing expected by the Jetson camera stack.
Image Generator
The Image Generator produces synthetic image data internally, without relying on any external image source. It generates deterministic test patterns in Bayer format, which are then packed into RAW10 pixel data. The generated pixel stream is fed into the Emulator Controller, which schedules it for transmission over the CSI-2 interface.
MIPI CSI-2 Transmitter IP
The MIPI CSI-2 Transmitter is implemented using the Lattice CSI-2/DSI D-PHY Transmitter IP core. This block handles the low-level CSI-2 protocol and physical-layer signaling, transmitting RAW10 video data over two MIPI data lanes. The IP core is driven by the Emulator Controller, which provides the required packet sequencing and timing information.
Resolutions
The emulator supports a subset of the Sony IMX219 operating modes commonly used on NVIDIA Jetson platforms. Currently, the following modes are implemented and validated.
| Mode | Width | Height | FPS |
|---|---|---|---|
| 0 | 3264 | 2464 | 21 |
| 1 | 3264 | 1848 | 28 |
| 2 | 1920 | 1080 | 30 |
Width represents the horizontal pixel count per line, and height represents the total number of lines in the image. All supported modes output RAW10 image data over a two-lane MIPI CSI-2 interface.
Resolution calculations
In the RAW10 format, every 4 pixels are packed into 5 bytes. Therefore, the number of bytes per line is computed as:
The number of bytes per lane is obtained by dividing the total bytes per line by the number of CSI-2 lanes:
Example using mode 2:
Mode 2 uses 1920 pixels per line, so the number of bytes per line in RAW10 format is calculated as:
Since the sensor interface is configured with two CSI-2 data lanes, the total line payload is evenly distributed between them:
Limitations
FPGA programming and boot order
The CrossLink FPGA must be programmed before the Jetson platform boots. During system initialization, the Jetson camera stack probes the I²C bus and CSI interfaces only once. If the emulator is not present or not responding at that time, the sensor node will not be registered, and the camera will not be available until the next reboot.
Partial IMX219 register emulation
The emulator implements only a subset of the Sony IMX219 register map. This subset is sufficient to support sensor detection, mode configuration, and continuous video streaming. Registers related to advanced sensor features, calibration data, or runtime tuning are not implemented, as they are not required for driver bring-up and validation workflows.
ISP-induced image differences
Although the emulator generates deterministic RAW10 image patterns, the final image observed at the application level may differ due to image signal processing performed by the Jetson ISP through the Argus pipeline. This includes color conversion, scaling, and other post-processing stages that are outside the scope of the emulator.
Supported IMX219 operating modes
The emulator supports only a subset of the Sony IMX219 operating modes. Currently, operating modes 0, 1, and 2 are implemented and validated. Additional IMX219 operating modes are not supported, as the emulator uses a fixed exposure configuration and does not support multiple frame rates for the same resolution.
Unrecoverable CSI error states
In certain failure scenarios, the Jetson CSI/VI capture stack may enter an unrecoverable state. When this occurs, restarting the capture pipeline is insufficient, and a full system reboot is required to restore camera operation.