ES: System Programs in Embedded Linux

System programs form the user-space foundation of an embedded Linux system. They provide a convenient environment for development and execution, giving applications and operators access to operating system services through familiar command-line utilities.

Once the Linux kernel is running and the root filesystem is mounted, the next layer of the embedded system is system programs. These are the utilities and daemons that bridge the gap between the kernel and the application — providing the services, environment, and infrastructure that applications depend on.


What Are System Programs?

System programs are executables that run in user space and provide a convenient interface to operating system services. They are the tools available from the command line and the background services that keep the system running.

They form a critical part of the root filesystem — without them, the system cannot be managed, configured, or operated.


Categories of System Programs

System programs can be divided into several functional categories:

File Manipulation and Modification

Programs for working with the filesystem:

md
ls      - list directory contents
cp      - copy files
mv      - move or rename files
rm      - remove files
mkdir   - create directories
chmod   - change file permissions
chown   - change file ownership
cat     - display file contents
find    - search for files
grep    - search within files

Status Information

Programs for monitoring and inspecting the system:

md
ps      - list running processes
top     - live process and resource monitor
df      - disk space usage
free    - memory usage
uname   - kernel and system information
dmesg   - kernel ring buffer messages
ifconfig/ip - network interface status

Programming Language Support

Development tools, typically stripped down in production images:

  • Compilers and interpreters
  • Build utilities (make, cmake)
  • Script engines (sh, bash, python)

Program Loading and Execution

System utilities for managing processes:

  • Shell interpreters (sh, bash, ash)
  • Init systems (BusyBox init, systemd, SysV init)
  • exec, nohup, nice, taskset

Communication

Networking and IPC utilities:

  • ping, netstat, ss, wget, curl
  • scp, ssh
  • D-Bus utilities for inter-process communication

System Programs in Embedded Linux: BusyBox

In standard desktop Linux, each utility is a separate executable. On an embedded system, having hundreds of separate binaries would waste significant storage and RAM.

BusyBox solves this by combining the most essential Linux utilities into a single binary.

md
Standard Linux:
/bin/ls   (50KB)
/bin/cat  (30KB)
/bin/cp   (45KB)
/bin/sh   (120KB)
...total: many MB

BusyBox:
/bin/busybox  (one binary, ~500KB)
/bin/ls  --> symbolic link to busybox
/bin/cat --> symbolic link to busybox
/bin/cp  --> symbolic link to busybox
/bin/sh  --> symbolic link to busybox

BusyBox uses symbolic links to determine which utility is being called. When you run ls, the shell actually executes BusyBox with the name ls — BusyBox reads argv[0] to identify which utility to run.

Checking BusyBox Shared Library Dependencies

When deploying BusyBox on a target, you must ensure the required shared libraries are present:

bash
aarch64-none-linux-gnu-readelf -a /bin/busybox | grep "program interpreter"
aarch64-none-linux-gnu-readelf -a /bin/busybox | grep "Shared library"

These commands reveal which C library and other shared libraries BusyBox needs at runtime — they must be included in the root filesystem.


Role of System Programs in the Boot Process

System programs are heavily involved during system startup:

md
Kernel starts init (first user-space process)
    |
    v
init reads /etc/inittab or service files
    |
    v
Start essential daemons:
  - syslogd    (logging)
  - udevd      (device management)
  - dhcpcd     (network configuration)
  - sshd       (remote access)
    |
    v
System ready, application starts


Common Daemons in Embedded Linux

Daemons are background system programs that run continuously:

DaemonFunction
syslogdSystem logging
udevdDynamic device management (/dev)
dhcpcdDHCP client for network configuration
sshdSecure remote shell access
ntpd / chronydNetwork time synchronisation
watchdogdHardware watchdog management

System Programs vs Application

It is important to distinguish between system programs and the application:

AspectSystem ProgramsApplication
PurposeProvide OS services and environmentDeliver the device's end-user function
Examplesls, mount, sshd, syslogdNAS server, router UI, infotainment player
ReplaceabilityUsually from BusyBox or standard packagesCustom-developed for the product
PresenceAlways presentSpecific to the product

Final Thoughts

System programs form the operational foundation of an embedded Linux system. They are what makes the system manageable, maintainable, and useful beyond just running an application.

md
Embedded Linux User Space:
+----------------------------------+
|  Application                     |  (product function)
+----------------------------------+
|  System Programs                 |  (BusyBox, daemons)
+----------------------------------+
|  C Library (glibc / musl)        |  (POSIX interface)
+----------------------------------+
|  Linux Kernel                    |  (hardware management)
+----------------------------------+

In embedded Linux, the goal is to include exactly the system programs you need — no more, no less. BusyBox makes this practical by packing essential utilities into a minimal footprint.