Fast Track with Linux

With ever increasing demands to quicken new product development time, Anton Aendenroomer & Ian McLoughlin explain how using an embedded Linux operating system can help reduce the pressure.

Working in consultancy has its benefits. A major one is that production deadlines are not nearly so much a life-and-death situation as compared to manufacturing companies in the final stages of product development. However, a big detraction is that companies tend to approach consultancies only when they foresee problems meeting their deadlines.

Another difference is that consultancies are often not tasked with developing new products at a costeffective price, but rather at a minimum consultancy cost: being pressed for prototypes in a ever shorter period of time. Indeed, working in consultancy has taught the authors all about customers asking for prototypes in ridiculously short periods of time, expecting to see results within a few months. But being embedded Linux enthusiasts, and knowing the key benefits of Linux as a rapidprototyping and development platform, we decided to look more closely into how embedded Linux can assist in the quest to rapidly develop product solutions.

Accelerating adoption

The worldwide adoption of embedded Linux is accelerating for appliances and gadgets such as smart phones. This is unsurprising considering the benefits of being free from paying runtime licenses. The opensource community, with the high availability of device drivers for the latest peripherals and protocol stacks for emerging communication standards, provides a perfect starting place for new product development: many ingredients are already available, and as we know, many recipes require a basic set of ingredients, with only a few special items (and of course good cooking skills).

How does open-source provide this? Well, the special licensing for open source software, GPL (GNU Public License), allows use, reuse and even modification of source code as long as the end product remains open source. A person receiving this end-product should by reasonable means also be able to retrieve the source code (and likewise be able to change it, enhance it, compile into a new product and sell it, while again making the source code available).

Detractors of this approach say that one may not be able to make a lot of money by selling open source products if you must provide the source code to anyone who asks! However, GPL does not specify that a product need be 100 percent open source. Rather, open source software, once compiled into an executable can be dynamically linked with any library, even closed source.

New products can thus mix components that require IP (intellectual property) protection as closed source, while those components that are not part of the company’s core business – but definitely need to be part of the product – remain open source. The final product package can then consist of an open source Linux kernel (which in fact is the operating system itself), open or closed source applications, which are dynamically linked to open and closed source libraries, to represent the common functionality (non-core IP) and company-specific (core IP) features of the product, respectively.

Does this model work? Ask the smart phone companies of Japan like NEC and Panasonic, ask Linksys, Google, HP, Samsung, Amazon, Philips and others. One telling example is the Linksys WRT54G wireless residential gateway. This product was on the market long before any competing product, due to the fact that the firmware was built on an embedded Linux platform. This gave the company the advantage to benefit from the huge source of device driver and networking code, available with documentation and templates.

After market share was established, Linksys could focus on reducing the hardware cost of the product by reducing the memory footprint. For this, it ported the firmware to VxWorks, which, like most commercial real-time operating systems (RTOSes), can be compiled together with applications into much smaller binaries than can be achieved with embedded Linux.

This typical example reveals how embedded Linux and other RTOSes can show their strength during product development and during the different phases of product maturity. Figure 1 shows how this concept can be worked out into a product, consisting of a Linux kernel, open source device drivers and open source libraries along with proprietary device drivers and libraries. These then cooperate and call each other’s functions in a seamless integrated environment. The arrows show that function calls between proprietary and open source modules can be realized via the concept of kernel modules and dynamically linked libraries.

Extending the hardware

Coming back to the issues faced in consultancy work, as compared to product development, being involved in prototyping there is not such a huge pressure for cost optimization in terms of materials. Instead, the main aim is to present a feasible, full-featured representation of a product to be developed.

In other words, a piece of hardware is required that can rapidly morph from pre-product to product prototype. This sounds good on paper, however, from a practical point-of-view, electronic hardware is not normally “morphable”!

One viable solution to this problem lies in incorporating an FPGA (field programmable gate array – a chip that is programmed in the field rather than in the semiconductor fab) on the pre-prototype platform.

This has been actualized in the form of the embedded Linuxbased Control Platform II (CP-II) – so named because the primary interest is in control tasks – developed at the Singapore Institute of Manufacturing Technology (SIMTech) and which can be used for smart appliances or scaled-up for use in industrial control applications.

The heart of the system comprises the ARM920T RISCbased AT91RM9200 microcontroller from Atmel, with 30 of its GPIO (General Purpose IO) pins one-to-one connected to programmable pins of a Spartan 400k FPGA, for reasons of offloading certain tasks from the main microcontroller to the FPGA co-processor.

By developing a microcontroller prototype board with an FPGA, which also has a number of its configurable I/O pins connected to an external connector, a configurable peripheral interface is created (CP-I did not have an FPGA on board, hence was less configurable in terms of hardware).

This extension concept is illustrated in Figure 2. Connecting either one of the extension cards (P1, P2) to the main controller (C) using connectors (M and F), and programming either the firmware F1 or F2 into the FPGA, creates drivers for the transceivers on P1 and P2, respectively.

For example, if requirements state that a prototype needs to be developed for a certain product containing a certain transceiver then quite likely a driver will be needed for the transceiver. One can opt to purchase this driver of course, provided that the MOQ (minimum order quantity) allows low-volume purchasing, and provided the driver is (still) available on the market.

This is one way of incorporating the transceiver; another way, which is provisioned for in Control Platform II, provides the option to build your own driver: write the driver code in Verilog and download into FPGA. This way the FPGA will contain the firmware of the driver for the transceiver.

The additional hardware required to fabricate the extension with transceiver will be minimum – basically, just the transceiver and a connector that can plug into the connector on the controller board. Figure 3 shows a close-up view of this concept, with the extension board connectable to configurable I/O pins of the FPGA.

Kernel is key

The AT91RM9200 microcontroller is capable of running an operating system, on top of which applications can run that represent the product’s IP. The operating system is embedded Linux, mainly motivated by Linux’s multitasking capabilities and features such as the device driver framework with option to create pluggable device drivers.

Settling for this choice allows, in one step, the addressing of a number of issues related to building a multipurpose system that can connect to the network and USB and run protocol stacks and core drivers for both. The Linux kernel communicates directly with the CPU, and the applications can only get hold of the CPU (and all other hardware) through kernel function calls. This way, all hardware access is guaranteed to go through a proper abstraction layer and potential damage or interruption of mission-critical tasks is avoided.

A large number of requirements for embedded appliances such as industrial controllers or digital cameras include the capability to connect to the LAN or the USB hotplug capability, so with these features already built-in a tremendous reduction in development time can be achieved. The Linux kernel manages resources, such as CPU time and memory, to prevent conflicts between different multitasking processes claiming access to them. Every process, selfish as it is, wants to make use of the CPU for as long as possible. Cooperative multitasking (where processes voluntarily relinquish their claims on CPU time) did not really work out in practice as we all remember from the Windows 3.1 days. These days, operating systems practice preemptive multitasking and the one in charge of overseeing that the preemption takes place on a fair basis with the more urgent and interactive tasks being carried out with higher priority and the less urgent and CPU hogging tasks with lower priority is the “scheduler”.

The Linux scheduler is a very sophisticated example of a scheduler with “almost” the same amount of overhead no matter if two or 200 tasks are running concurrently. The scheduler is dynamic in that it is able to punish CPU-hogging tasks by dynamically lowering their priority level and award interactive and fast completing tasks by dynamically elevating their priority level. Nevertheless, important (e.g. safetycritical) tasks can hog the CPU if required, e.g. by disabling all other interrupts when handling a safety-critical interrupt. The “virtual machine” concept in Linux, abstracts out the complexities related to multitasking from the viewpoint of all the processes running in that environment: Linux provides each process with a virtual machine, which is in fact only part of the physical machine on a time-sharing basis; each process thinks it has the whole machine for its own.

Memory management is another one of those concepts that relies on virtualization: each process presumably has a huge amount of memory; whereas it is actually only a addressing space in a virtual memory area managed by the MMU (Memory Management Unit) and mapped upon actual physical memory by the same MMU. Similarly, the virtual file system is another Linux concept which allows it to more flexibly manage the actual physical file system(s) on disc, in flash or elsewhere.

Driving devices

Along with file system management, the other important task of the Linux kernel is the management of peripherals such as monitor, keyboard, mouse, etc. This task, categorized as device management, is taken care of by special purpose kernel components called “device drivers”. The device driver concept involves abstracting out the hardware for the benefit of the processes in the application layer. This is to prevent potential conflicts between processes claiming access to the same hardware, and to control the mode of hardware access through a well-defined API, in order to avoid potential damage due to improperly written user-applications. Embedded Linux device drivers are even more of a distinguishing factor compared to “normal” Linux because embedded systems are normally built as a special piece of hardware interacting with unique hardware through device drivers. In Linux it is easy to write your own device driver.

The Linux framework for device drivers is based upon categories of device: character device drivers for peripherals such as keyboard or serial port which communicate through a serialized data stream; block device drivers for peripherals such for harddisks or CD-ROM which exchange data in chunks; and special device drivers which require a protocol stack and subsystem, such as Ethernet and USB device drivers. USB is a good example of a protocol that can rely on a sophisticated subsystem to undertake the most tedious part of development effort that would otherwise be required to provide USB support for the product, no matter whether it is going to be a USB gadget or a USB host to support certain gadgets.

Since the CP-II platform is designed for deployment as a generic controller, to provide connectivity with, and datacollection from field devices, only the USB host (A-type) connector was routed out, while host drivers were implemented for a number of USB gadgets, including an inhouse built range such as displays and datacollection devices, also for commercial gadgets such as thumbdrives or serial-to-USB converters.

The latter category benefited from the use of existing Linux USB drivers, and these are selected as a “module” option when compiling the kernel (so that they are not included in the kernel but can be “plugged-in” at runtime).

For the special-purpose gadgets, drivers were developed based on the framework provided with the Linux kernel source code. In Linux, custom drivers sit on top of the USB core API, which is built-in and takes care of the communication with the USB host controller (as can be seen in Figure 1). Also, configuring the hotplug features (for plug & play) is mainly a matter of selecting an option when compiling the kernel, then editing a few configuration files.

Different flavors of device drivers for the communication between ARM and FPGA were written: one for communication based on handshaking; another for asynchronous communication based on utilizing the FPGA block RAM; and a last one for unreliable communication (no handshaking).

A good mix

By configuring the Linux kernel, using other open-source tools for cross-compiling, file-system configuration, boot-loading, development and debugging, the Control Platform II configurable hardware (CP-II) can be customized into a prototype to suit many different purposes.

In case the hardware requires additional peripherals, then the extension feature, as shown in Figures 2 and 3, can provide customized features with minimum effort. The additional FPGA coding to provide the firmware for the extension can be as straightforward as providing the connection and synchronization between pins connected to the ARM and pins connected to the extension connector. If the extension requires some preprocessing, as is often the case with image sensors, then the FPGA again is the perfect candidate.

The end-product will thus become a colorful mix of open-source and closedsource hardware and software, as follows:

• The closed-source part naturally contains the core IP of the end-product

• The platform IP (CP-II) contains licensable closed-source as well as open-source

• There can possibly be licensable thirdparty IP in the product, e.g. IP core for the FPGA co-processor

• Finally, the product can contain newly generated open-source during the course of prototyping

Having taken care of the interface with the hardware, a fully-fledged product, whether it is a data-logger, gateway, Web server, safety monitor, industrial controller or all of the above, can be developed in an incredibly short time by leveraging the power of embedded Linux. So maybe now those ridiculously short prototyping phases are achievable after all?

Linksys

Figure 1

Figure 2

Figure 3

Embedded Systems Manufacturers Indicate Future Linux Commitment Recently published research by Venture Development Corporation (VDC), “Linux in the Embedded Systems Market”, indicates that a significant portion of embedded software and systems engineering teams plan to adopt Linux as their primary target operating system “Linux has proven itself to be well suited for a wide range of applications across various industries, and continues to gain market share, despite not being particularly well matched for certain embedded applications types. Changes in the way that systems are designed will allow Linux to continue to penetrate into segments of the market where the lack of capability was previously perceived as a barrier,” said Matt Volckmann of VDC’s Embedded Software Practice. “At the root of most of the challenges faced by commercial suppliers is the question of how to bring significant additional value to an operating system that is freely available, continually improving, increasingly supported by embedded system/ component suppliers, and in demand from companies interested in decreasing their overall development costs and/or bill of materials,” says Volckmann. While Linux has found its way into a number of embedded industries, one of the key opportunities for Linux over the next several years will be in the mobile phone market, where it is quickly becoming a leading application operating system, and manufacturers and telecom operators are rallying to drive more comprehensive, i n d u s t r y- s p e c i f i c L i n u x - b a s e d standards, says the VDC report. Operating System Planned for Next Project

Dr Anton Aendenroomer, Research Scientist, Singapore Institute of Manufacturing Technology (SIMTech), and Dr Ian McLoughlin, Associate Professor, School of Computer Engineering, Nanyang Technological University (NTU), Singapore, both have several years of academic and commercial experience of embedded Linux technology.