I have been involved with Linux kernel development since the late nineties, when I first implemented support for the PTP protocol it lacked at the time, and that I needed for my first digital camera. It was a quick and dirty hack that barely implemented basic picture transfer and that caused a number of stability issues. It certainly helped me to appreciate the difficulties of device driver and kernel development in general.
IEEE1394 Sony Camera V4L2 support
Professionally I began on a high level subsystem named Video4Linux (V4L), as our work in the University laboratory required support for one of the first Sony Firewire cameras (purporting advanced features such as zoom and focus control, embedded image control) that was completely unsupported, in what was little more back then than a modest component supporting a few webcams. After struggling with it for a little while I finally decided to start afresh with the still being drafted V4L2, that defined the required interfaces for the advanced controls I needed.
Most of the work amounted to understanding how Firewire messaging worked, and implementing the V4L2 user space interfaces: capabilities, video format, capturing and control
ioctls. Some additional, hard to debug pieces were memory buffer allocation and release, initialization and cleanup.
Fedetec TABIC PCI
Supporting commercial hardware seemed challenging just until you have to support custom one. My first experience in SW/HW co-design was developing a modular, FPGA based PCI card (PCI ID
fede:0003) that implemented multiple physical adaptor modules for digital and analog audio signals. The team was formed by a hardware designer for the both FPGA/VHDL and digital design, a contractor in charge of the layout (the PCB used twelve stacked layers) and myself for everything related to software.
The card was an evolution of a previous ISA card that was also supported by the driver. The type of protocols it dealt with was very diverse, from ISDN to Trunk Radio. The core of the hardware was a digital PBX, capable of mixing, uploading and downloading audio data via PCI DMA up to 128 channel. The driver was hardcoded to support a maximum of 8 simultaneous cards per PCI bus (due to memory bandwidth limitations) and it was successfully on systems with up to 12 simultaneous cards.
These are some of the main features I implemented over the course of the project. Please note that even though most were placed in kernel modules, some ran in user space.
- Modular kernel driver for Linux 2.4 and 2.6
- PCI/ISA DMA support
- Standard OSS and ALSA support for capturing and recording hundreds of simultaneous audio channels
- ISDN software stack support (layer 2/3, signalling)
- Radio protocol and signalling for a number of devices
- Complete support for SIP/VoIP stacks
- Factory-grade validation and power on self tests
As the strongest requirement was reliability and stability (as both the hardware and the software were used on emergency grade systems) the driver implementation was a robust as possible, using barely any lock and being basically I/O bounded. Systems based on this hardware are the main product the company produces still today.
My second experience with Linux-based custom hardware was the development of a MP3/DIVX video, WiFi and Bluetooth enabled player under the commercial name of ‘Blu:sens G14′. It was based on the Sigmatel STMP3600 SOC, for which the vendor only provided support for somel VxWorks-based firmware, that my company had used before in a previous device and that faced strong limitations.
The team I was part of was formed by two hardware developers (on the digital design and layout side) and myself (in charge of the software and prototype testing), the device being developed at the same time as the software. At a later stage, a second developer was added under my supervision to support in the creation of the QT-based UI. The internet connection was mainly used to access a remote drive the company made available to customers, although it also provided remote software updates capabilities.
The device, based on a 200MHz ARM926EJ-S core was able to play fullscreen (320×240) MPEG4 video content, stream audio and video from the 802.11b/g WiFi adaptor and to send stereo A2DP audio via Bluetooth to wireless headphones.
Most of the development effort was invested in porting the Linux kernel, given Sigmatel (mainly for licensing reasons) had decided to not use standard any of the standard, well supported ARM IP other than the core (such the AHB bus), but their own version instead (admittedly superior in clock gating capabilities), thus forcing a complete port from the ground up, including:
- Boot loader (MMU, DRAM setup)
- Console (UART and LCD) driver
- Power management (device gating, battery charger)
- I2C bus controller (battery monitor, FM RDS data)
- SDIO (used for the WiFi adaptor)
- High speed UART (Bluetooth chip)
- DMA subsystem (video, audio, NAND … )
- LCD controller (320x240x16 from Sharp, frame buffer interface)
- USB slave, master and gadget mode supporting low-level driver
- Audio controller (record and reproduction, ALSA based)
- NAND flash controller (supporting both SLC and MLC chips)
- Flash translation layer (small/big pages, round robin-based wear levelling)
- A/D converter (for the resistor ladder-based input controller)
An additional set of user space components I developed specifically for this product were:
- Heavily optimised ‘mplayer’ port, including device-specific plugins
- YUV to RGB ARM assembler video conversion (ARMv5TE, short cache lines)
- A2DP SBC codec assembler optimizations
- Parts of the QT-based UI
- Booting scripts (initd, ramdisk)
- WiFi support scripts for WEP/WPA
- Firmware packaging and updating software
- Optimised, portable toolchain building (Linux and Mac OS X)
Last but not least, as we were the first company to integrate the WiFi chip (developed by the swedish company Nanoradio, a spin off from Ericsson), I was also involved in supporting their team (by spending several weeks at their HQ in Stockholm) developing both the network and the SDIO interface driver.
In addition to the commercial launch, a case-less version of the device was produced and donated to the University laboratories and it still being used to support courses on embedded courses.