BusyBox 官网 Frequently Asked Questions
http://www.busybox.net/FAQ.html#whatis
------------------------------------------------------------
Frequently Asked Questions
This is a collection of some of the more frequently asked questions about Busybox. Some of the questions even have answers. If you have additions/corrections to this document, we would love to add them.General questions
- What is Busybox?
- How can I get started using Busybox?
- How do I configure Busybox?
- How do I build Busybox with a cross-compiler?
- How do I build a Busybox-based system?
- Which Linux kernel versions are supported?
- Which architectures does Busybox run on?
- Which C libraries are supported?
- Where can I find other small utilities since Busybox does not include the features I want?
- I demand that you to add
right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems Right Now! - I need help with Busybox! What should I do?
- I need you to add
! Are the Busybox developers willing to be paid in order to fix bugs or add in ? Are you willing to provide support contracts? - Can I include Busybox as part of the software on my device?
- I want to use Busybox as part of the Linux-based firmware for a new device. Will it create any license issues in future?
Troubleshooting
- I think I found a bug in Busybox! What should I do?!
- I have trouble installing Busybox on my phone!
- I'm using an ancient version from the dawn of time and something's broken. Can you backport fixes for free?
- Busybox init isn't working!
- I am observing a bug in BusyBox on an obscure platform. Help.
- I can't configure Busybox on my system
- Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?
- sed "/CONFIG_FOO/s/.*/CONFIG_FOO=y/" -i .config; make; does not enable applet foo
- I installed a package on my Busybox system and now nothing works!
- I selected "standalone shell" option and now I have problems
Misc. questions
- How do I change the time zone in Busybox?
Programming questions
- What are the goals of Busybox?
- What is the design of Busybox?
- How is the source code organized?
- The applet directories
- The Busybox shared library (libbb)
- I want to make Busybox even smaller, how do I go about it?
- Adding an applet to Busybox
- What standards does Busybox adhere to?
- Portability
- Tips and tricks
- Encrypted Passwords
- Fork and vfork
- Short reads and writes
- Memory used by relocatable code, PIC, and static linking
- Including Linux kernel headers
- Who are the Busybox developers?
General questions
What is Busybox?
"I use Foomatic-9000 modem. In my modem, Busybox v1.23.4 is installed. I can reach my modem's Telnet interface and I can reach shell prompt. I want to change configuration, and add some tools. Can I enable FTP? SCP? If I edit a file on the device, I can't save it, I get 'filesystem is read only' error. How can I save a changed file?"
Questions like the example above is a sign that the person asking it does not fully understand what Busybox is. Busybox is not a complete solution for running modems, or wireless access points, etc. Busybox is only a set of programs. Even though it is a fairly comprehensive set of programs needed to run a Linux system, these programs per se can not "run a modem".
In order to make Busybox-based system useful, developers of this system add their own parts. At the very least, there are some shell scripts and configuration files, which control which tools are to be run, when, and how. Often, there are also some web pages and CGI scripts (in order to implement web-based configuration). Also, often developers add other tools beside Linux kernel and Busybox.
Since these scripts, configuration files and other tools are custom-designed for a specific device, Busybox mailing list is usually a wrong place to ask about inner workings of such a device.
For a specific example shown above, the answers most likely would look like this:
"Can I enable FTP?" - "We don't know whether Foomatic-9000 firmware comes with Busbox binary with enabled ftpd applet. (If you have access to shell prompt, try running "busybox ftpd --help"). If it does, you probably will be able to enable it. Unfortunately, we don't know how Foomatic-9000 firmware manages starting and stopping of various services (in case they already have ftpd sevice and it is just disabled), or how to create and enable a new service on Foomatic-9000 (if they dont). Try googling for it."
"Can I enable SCP?" - "Busybox-1.23.4 does not have scp applet. However, Foomatic-9000 might have scp installed (say, if they have dropbear installed), and it may be possible to enable it. If not, it may be possible to build and install your own scp tool. Try googling for it."
"If I edit a file on the device, I can't save it, I get 'filesystem is read only' error. How can I save a changed file?" - "Most embedded devices use flash-based filesystems, and often these filesystems are configured to read-only mode in normal operation. In order to modify some files, you need to find out how to switch the filesystem into read-write mode. Note that sometimes embedded devices use read-only filesystem formats such as cramfs, and therefore in order to replace or add even a single file (such as a new, or an additional Busybox binary), you need to re-create and write out the entire filesystem image. This operation is not simple for inexperienced users and may permanently disable your device if done incorrectly."
You are better off googling for a forum where users of this particular device discuss their findings about inner workings of the device, and joining it.
How can I get started using Busybox?
Obtaining and testing Busybox
First, download a prebuilt binary version from http://busybox.net/downloads/binaries/latest/ and save it under the name "busybox". The "busybox-i686" version should run on both 32-bit and 64-bit x86 PCs, and the armv4tl version is the most generic arm version (for smartphones), so those are probably good starting points.
This is a statically linked version of the Busybox "multiplexer", a single command that can perform multiple actions, the way a swiss army knife has multiple blades. To try it out, give Busybox the command line you'd like it to execute:
./busybox ls -l ./busybox ps ./busybox seq 1 5
To get a list of the commands supported by this instance of Busybox, run it without any arguments, or use the "--list" or :
./busybox
To see what an individual command does, use the --help option to that command:
./busybox zcip --help
Installing Busybox
If the Busybox executable is renamed to one of the commands it supports, it will act as that command automatically:
ln -s busybox pwd ./pwd
This allows you to create a bunch of symlinks or hardlinks to the Busybox executable, add them to your $PATH, and let a single Busybox provide a standard set of command line tools. The --list option to busybox gives the list of supported commands in an easily scriptable form. (The --list-full option gives full paths, such as usr/sbin/test, to help create a Busybox-based root filesystem.)
mkdir bbdir for i in $(busybox --list) do ln -s busybox bbdir/$i done
To launch Busybox's built in command shell with the $PATH giving access to just Busybox's built-in commands:
PATH=$(pwd)/bbdir bbdir/sh
Building Busybox from source
The prebuilt binaries are based on the default configuration of Busybox, which enables all functionality except special purpose things like selinux or debugging support which would reduce the portability of the resulting binary.
To build a defconfig Busybox from source, download the source code from http://busybox.net/downloads
Then configure and build it:
make defconfig make
This should result in a new "busybox" binary.
Busybox uses the same "menuconfig" infrastructure as the Linux kernel. You can start with "make defconfig" to enable almost everything, or "make allnoconfig" to disable everything, and then alter the selection with "make menuconfig" (which uses tab, cursor up and down, space, and escape keys to navigate, and the forward slash key to search for symbol name).
Cross compiling Busybox
Obtain and install a cross compiler for your target. (A few prebuilt ones are available from http://landley.net/aboriginal/downloads/binaries. See also the buildroot and crosstool-ng projects.) Add the cross compiler to your $PATH.
Cross compilers use prefixed tool names to avoid blocking the host's existing compiler, thus the tools your cross compiler provides are probably named things like "armv4tl-cc", "armv4tl-ld", and "armv4tl-strip". In this case, "armv4tl-" would be the prefix.
So to build Busybox with that cross compiler, go:
make CROSS_COMPILE=armv4tl-
(Note the trailing dash, if that's part of the prefix. This is a make variable override preventing Busybox from using its default value, which is why it has to come on the make command line instead of as an environment variable.)
To build a static version, set the environment variable "LDFLAGS=--static". And of course you can do a parallel SMP build with make's -j options. So to build a static cross compiled version of Busybox using a parallel build:
LDFLAGS="--static" make -j 4 CROSS_COMPILE="armv4tl-"
How do I configure Busybox?
Busybox is configured similarly to the linux kernel. Create a default configuration and then run "make menuconfig" to modify it. The end result is a .config file that tells the Busybox build process what features to include. So instead of "./configure; make; make install" the equivalent Busybox build would be "make defconfig; make; make install".
Busybox configured with all features enabled is a little under a megabyte dynamically linked on x86. To create a smaller Busybox, configure it with fewer features. Individual Busybox applets cost anywhere from a few hundred bytes to tens of kilobytes. Disable unneeded applets to save space, using "make menuconfig".
The most important Busybox configurators are:
-
make defconfig - Create the maximum "sane" configuration. This enables almost all features, minus things like debugging options and features that require changes to the rest of the system to work (such as selinux or devfs device names). Use this if you want to start from a full-featured Busybox and remove features until it's small enough.
-
make allnoconfig - Disable everything. This creates a tiny version of Busybox that doesn't do anything. Start here if you know exactly what you want and would like to select only those features.
-
make menuconfig - Interactively modify a .config file through a multi-level menu interface. Use this after one of the previous two.
Some other configuration options are:
-
make oldconfig - Update an old .config file for a newer version of Busybox.
-
make allyesconfig - Select absolutely everything. This creates a statically linked version of Busybox full of debug code, with dependencies on selinux, using devfs names... This makes sure everything compiles. Whether or not the result would do anything useful is an open question.
-
make randconfig - Create a random configuration for test purposes.
Menuconfig modifies your .config file through an interactive menu where you can enable or disable Busybox features, and get help about each feature.
To build a smaller Busybox binary, run "make menuconfig" and disable the features you don't need. (Or run "make allnoconfig" and then use menuconfig to add just the features you need. Don't forget to recompile with "make" once you've finished configuring.)
How do I build Busybox with a cross-compiler?
To build Busybox with a cross-compiler, specify CROSS_COMPILE=
CROSS_COMPILE specifies the prefix used for all executables used during compilation. Only gcc and related binutils executables are prefixed with $(CROSS_COMPILE) in the makefiles. CROSS_COMPILE can be set on the command line:
make CROSS_COMPILE=arm-linux-uclibcgnueabi-
Alternatively CROSS_COMPILE can be set in the environment. Default value for CROSS_COMPILE is not to prefix executables.
To store the cross-compiler in your .config, set the variable CONFIG_CROSS_COMPILER_PREFIX accordingly in menuconfig or by editing the .config file.
Example: here's the script which donloads and compiles Busybox using Rob's toolchain from Aboriginal Linux:
#!/bin/sh set -e test -f cross-compiler-armv5l.tar.bz2 \ || wget http://impactlinux.com/aboriginal/downloads/binaries/cross-compiler-armv5l.tar.bz2 rm -rf cross-compiler-armv5l tar xf cross-compiler-armv5l.tar.bz2 test -f busybox-1.17.2.tar.bz2 \ || wget http://busybox.net/downloads/busybox-1.17.2.tar.bz2 rm -rf busybox-1.17.2 tar xf busybox-1.17.2.tar.bz2 CROSS_COMPILE="$PWD/cross-compiler-armv5l/bin/armv5l-" cd busybox-1.17.2 make CROSS_COMPILE="$CROSS_COMPILE" defconfig make CROSS_COMPILE="$CROSS_COMPILE"
How do I build a Busybox-based system?
Busybox is a package that replaces a dozen standard packages, but it is not by itself a complete bootable system. Building an entire Linux distribution from source is a bit beyond the scope of this FAQ, but it understandably keeps cropping up on the mailing list, so here are some pointers.
Start by learning how to strip a working system down to the bare essentials needed to run one or two commands, so you know what it is you actually need. An excellent practical place to do this is the Linux BootDisk Howto, or for a more theoretical approach try From PowerUp to Bash Prompt.
To learn how to build a working Linux system entirely from source code, the place to go is the Linux From Scratch project. They have an entire book of step-by-step instructions you can read online or download. Be sure to check out the other sections of their main page, including Beyond Linux From Scratch, Hardened Linux From Scratch, their Hints directory, and their LiveCD project. (They also have mailing lists which are better sources of answers to Linux-system building questions than the Busybox list.)
If you want an automated yet customizable system builder which produces a Busybox and uClibc based system, try buildroot, which is another project by the maintainer of the uClibc (Erik Andersen). Download the tarball, extract it, unset CC, make. For more instructions, see the website.
Which Linux kernel versions are supported?
Full functionality requires Linux 2.4.x or better. (Earlier versions may still work, but are no longer regularly tested.) A large fraction of the code should run on just about anything. While the current code is fairly Linux specific, it should be fairly easy to port the majority of the code to support, say, FreeBSD or Solaris, or Mac OS X, or even Windows (if you are into that sort of thing).
Which architectures does Busybox run on?
Busybox in general will build on any architecture supported by gcc. Kernel module loading for 2.4 Linux kernels is currently limited to ARM, CRIS, H8/300, x86, ia64, x86_64, m68k, MIPS, PowerPC, S390, SH3/4/5, Sparc, v850e, and x86_64 for 2.4.x kernels.
With 2.6.x kernels, module loading support should work on all architectures.
Which C libraries are supported?
On Linux, Busybox releases are tested against uClibc (0.9.27 or later) and glibc (2.2 or later). Both should provide full functionality with Busybox, and if you find a bug we want to hear about it.
Linux-libc5 is no longer maintained (and has no known advantages over uClibc), dietlibc is known to have numerous unfixed bugs, and klibc is missing too many features to build Busybox. If you require a small C library for Linux, the Busybox developers recommend uClibc.
Some Busybox applets have been built and run under a combination of newlib and libgloss (see this thread). This is still experimental, but may be supported in a future release.
Where can I find other small utilities since Busybox does not include the features i want?
We maintain such a list on this site.
I demand that you to add right now! How come you don't answer all my questions on the mailing list instantly? I demand that you help me with all of my problems Right Now!
You have not paid us a single cent and yet you still have the product of many years of our work. We are not your slaves! We work on Busybox because we find it useful and interesting. If you go off flaming us, we will ignore you.
I need help with Busybox! What should I do?
If you find that you need help with Busybox, you can ask for help on the Busybox mailing list at [email protected].
In addition to the mailing list, Erik Andersen (andersee), Manuel Nova (mjn3), Rob Landley (landley), Mike Frysinger (SpanKY), Bernhard Reutner-Fischer (blindvt), and other long-time Busybox developers are known to hang out on the #busybox IRC channel on irc.freenode.net.
Please do not send private email to Rob, Erik, Manuel, or the other Busybox contributors asking for private help unless you are planning on paying for consulting services.
When we answer questions on the Busybox mailing list, it helps everyone since people with similar problems in the future will be able to get help by searching the mailing list archives. Private help is reserved as a paid service. If you need to use private communication, or if you are serious about getting timely assistance with Busybox, you should seriously consider paying for consulting services.
I need you to add ! Are the Busybox developers willing to be paid in order to fix bugs or add in ? Are you willing to provide support contracts?
Yes we are. The easy way to sponsor a new feature is to post an offer on the mailing list to see who's interested. You can also email the project's maintainer and ask them to recommend someone.
Can I include Busybox as part of the software on my device?
Yes. As long as you fully comply with the terms of the GPL Busybox license you can ship Busybox as part of the software on your device.
I want to use Busybox as part of the Linux-based firmware for a new device. Will it create any license issues in future?
If you use Busybox binary in your device's firmware, and if you (or the company you work for) is providing this device to others (selling, giving for free, etc), you have to provide users with means to build the same Busybox binary from source.
For example, you may do it by placing the following, or similar, text somewhere on the company's web site:
| This device's firmware includes the following open-source components: ... ... Busybox: We are using patched version of Busybox 1.6.2. In order to rebuild it from source, download http://company.site.com/firmware/source/busybox-1.6.2.tar.gz, unpack it to an empty directory, download http://company.site.com/firmware/source/busybox-1.6.2.patch and http://company.site.com/firmware/source/.config to the same directory and apply the patch with this command: patch -p1 |
Adjust the text to suit your actual situation.
You have to make the page easily accessible. In other words, users should not be forced to guess the URL. The most sensible choice is to put this text on the page where users normally download updated binary firmware images.
If you want to be extra nice, you may provide URLs to places where people may get ARM toolchains or read HOWTOs explaining how to build one. You may even provide the toolchain. But these "extra nice" things are not required by license. License also does not require you to provide any support for users which use firmware (or part of it) built from source.
What license does require, though, is that Busybox source which you provide actually can be built and that it will match Busybox binary which is found in your binary firmware. Match is meant as "functional match", not byte-by-byte match - binaries may slightly differ because of different compilers/linkers used, because of build timestamp and so on.
Troubleshooting
I think I found a bug in Busybox! What should I do?
If you simply need help with using or configuring Busybox, please submit a detailed description of your problem to the Busybox mailing list at [email protected]. Please do not send email to individual developers asking for private help unless you are planning on paying for consulting services. When we answer questions on the Busybox mailing list, it helps everyone, while private answers help only you...
Bug reports and new feature patches sometimes get lost when posted to the mailing list, because the developers of Busybox are busy people and have only so much they can keep in their brains at a time. You can post a polite reminder after 2-3 days without offending anybody. If that doesn't result in a solution, please use the Busybox Bug and Patch Tracking System to submit a detailed explanation and we'll get to it as soon as we can.
Note that bugs entered into the bug system without being mentioned on the mailing list first may languish there for months before anyone even notices them. We generally go through the bug system when preparing for new development releases, to see what fell through the cracks while we were off writing new features. (It's a fast/unreliable vs slow/reliable thing. Saves retransits, but the latency sucks.)
I have trouble installing Busybox on my phone!
Busybox mailing list and individual developers sometimes receive emails which ask for help with Busybox installation on their smartphones, tablets and the like.
Please note that at the time of this writing (2013-07), Busybox project does not provide prebuilt binary packages for any architecture or device - it provides the source code.
If you are downloading and installing a Busybox binary onto your device, it means that someone else made that Busybox binary. There are a number of people who do this (for example, Aboriginal Linux).
If you are experiencing problems with installing such a binary build, you may be better off if you contact a person or a project which provides it, not the Busybox development mailing list or an individual developer. The link to project website and/or an email address of the build's author is likely to be present on the same Web page where you downloaded the binary package.
If you still want to seek advice about your problem on the Busybox development mailing list, please provide as much information about your situation as possible:
- On what device you are trying to install the package?
- The URL to the place where you got the package from.
- The version of the package.
- The sequence of operations you are performing.
- What do you observe when you do that.
- In case it is not obvious from the previous description: What, in your opinion, is not happening correctly?
Email like this: "I tried downloading applets for all version of Busybox the problem remains consistent" will most likely be ignored, or you will be redirected to this FAQ. (In this particular email example, none of the above bits of information are present).
I'm using an ancient version from the dawn of time and something's broken. Can you backport fixes for free?
Variants of this one get asked a lot.
If you see a bug in an old version, it is recommended that you first check whether the problem still exists in the most recent release.
The purpose of the Busybox mailing list is to develop and improve Busybox, and we're happy to respond to our users' needs. But if you're coming to the list for free tech support we're going to ask you to upgrade to a current version before we try to diagnose your problem.
If you're building Busybox 0.50 with uClibc 0.9.19 and gcc 1.27 there's a fairly large chance that whatever problem you're seeing has already been fixed. To get that fix, all you have to do is upgrade to a newer version. If you don't at least _try_ that before asking for help on the mailing list, you're wasting developers' time.
If you don't want to take the risk of replacing all applets by newer versions, you can employ this trick:
Download most recent release, configure it with "make allnoconfig", then use "make menuconfig" to switch on just the applet you want to test and maybe a couple of tuning options. Then build Busybox.
Then, on target system, delete the old applet symlink that points to your old Busybox, and replace it with the new Busybox binary, renamed to applet's name. In other words, if you want to replace only, say, httpd, then delete, say, /bin/httpd symlink (which points to your old Busybox), then run "cp /path/to/new/busybox /bin/httpd". (For some applets such as login, you'll also need to "chmod u+s" your new busybox binary.)
Now you can test the new applet and post a more useful email to the mailing list, either "I see such and such bug even in latest release" or "I see such and such bug in release X.Y.Z, but it seems to be fixed in last release".
Deleting the old symlink still leaves the old functionality in your existing old Busybox binary, you just wouldn't be using it anymore. If things will get even worse with new version, you can always restore the symlink.
The volunteers are happy to fix any bugs you point out in the current versions because doing so helps everybody and makes the project better. We want to make the current version work for you. But diagnosing, debugging, and backporting fixes to old versions isn't something we do for free, because it doesn't help anybody but you. The cost of volunteer tech support is using a reasonably current version of the project.
If you don't want to upgrade, you have the complete source code and thus the ability to fix it yourself, or hire a consultant to do it for you. If you got your version from a vendor who still supports the older version, they can help you. But there are limits as to what the volunteers will feel obliged to do for you.
As a rule of thumb, volunteers will generally answer polite questions about a given version for about three years after its release before it's so old we don't remember the answer off the top of our head. And if you want us to put any _effort_ into tracking it down, we want you to put in a little effort of your own by confirming it's still a problem with the current version. It's also hard for us to fix a problem of yours if we can't reproduce it because we don't have any systems running an environment that old.
A consultant will happily set up a special environment just to reproduce your problem, and you can always ask on the list if any of the developers have consulting rates.
Busybox init isn't working!
Init is the first program that runs, so it might be that no programs are working on your new system because of a problem with your cross-compiler, kernel, console settings, shared libraries, root filesystem... To rule all that out, first build a statically linked version of the following "hello world" program with your cross compiler toolchain:
#includeint main(int argc, char *argv) { printf("Hello world!\n"); sleep(999999999); }
Now try to boot your device with an "init=" argument pointing to your hello world program. Did you see the hello world message? Until you do, don't bother messing with Busybox init.
Once you've got it working statically linked, try getting it to work dynamically linked. Then read the FAQ entry How do I build a Busybox-based system?, and the documentation for Busybox init (FIXME: dead link).
I am observing a bug in BusyBox on an obscure platform. Help.
A good thing to do would be to explain in details how this bug can be reproduced under emulation (such as qemu) by people who have only typical x86 machine at their disposal. This makes it possible for other people to independently verify that bug indeed exists, and work on fixes for it. Otherwise, they can only send you suggestions what _you_ can do on _your_ system, which is MUCH slower process (think "email round trip time" and "non-scalable").
Example of a recipe to reproduce a bug under qemu:
- Download and unpack arm-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2 to /usr/package/arm-2010q1 directory.
- Add /usr/package/arm-2010q1/bin to $PATH.
- Unpack pristine busybox-1.19.3 source tarball into an empty directory.
- Build dynamic busybox binary using this cross-compiler:
make defconfig make -j 4 CROSS_COMPILE=arm-none-linux-gnueabi- make CROSS_COMPILE=arm-none-linux-gnueabi- install - Make ext2 image with the resulting binary:
dd if=/dev/zero of=image_ext2 bs=1M count=16 mke2fs -F image_ext2 mkdir dir mount -o loop image_ext2 dir cp -a _install/* dir - Add to image_ext2 dynamic loader and libraries from toolchain:
cp -a /usr/package/arm-2010q1/arm-none-linux-gnueabi/libc/lib dir
- Unmount image_ext2:
sync umount -d dir - Download http://www.landley.net/aboriginal/downloads/binaries/system-image-armv6l.tar.bz2 and take zImage from there.
- Start this kernel with this file system image under qemu:
qemu-system-arm -cpu arm1136-r2 -M versatilepb -nographic -no-reboot \ -hda image_ext2 \ -kernel zImage \ -append 'rw init=/bin/sh panic=1 PATH=/bin:/sbin root=/dev/sda console=ttyAMA0' - (give further description here how to trigger the bug)
I can't configure Busybox on my system
Configuring Busybox depends on a recent version of sed. Older distributions (Red Hat 7.2, Debian 3.0) may not come with a usable version. Luckily Busybox can use its own sed to configure itself, although this leads to a bit of a chicken and egg problem. You can work around this by hand-configuring Busybox to build with just sed, then putting that sed in your path to configure the rest of Busybox with, like so:
tar xvjf sources/busybox-x.x.x.tar.bz2 cd busybox-x.x.x make allnoconfig make include/bb_config.h echo "CONFIG_SED=y" >> .config echo "#undef ENABLE_SED" >> include/bb_config.h echo "#define ENABLE_SED 1" >> include/bb_config.h make mv busybox sed export PATH=`pwd`:"$PATH"
Then you can run "make defconfig" or "make menuconfig" normally.
Why do I keep getting "sh: can't access tty; job control turned off" errors? Why doesn't Control-C work within my shell?
Job control will be turned off since your shell can not obtain a controlling terminal. This typically happens when you run your shell on /dev/console. The kernel will not provide a controlling terminal on the /dev/console device. You should run your shell on a normal tty such as tty1 or ttyS0 and everything will work.
Example: you booted into your machine with init=/bin/sh and got "sh: can't access tty" error because sh has its stdio opened to /dev/console. You want to reopen stdio to, say, /dev/tty1 and thus acquire a controlling tty.
# Let's try this:
exec /dev/tty1 2>&1
# No, doesn't work: even if opening /dev/tty1 gave sh the ctty,
# sh wouldn't know it - it checks for ctty just once at startup.
# Let's try re-execing sh:
exec /dev/tty1 2>&1
exec sh
# Got "sh: can't access tty" again. Why?
# The reason is somewhat obscure: kernel starts process with PID=1
# (in this case, shell) with SID=0 and PGID=0, not with SID=1 and PGID=1
# as you'd expect. IOW: our sh is not a session leader, and therefore
# cannot acquire ctty by opening /dev/tty1 (or any other tty).
# Let's try making us a session leader:
exec setsid sh
exec /dev/tty1 2>&1
exec sh
# Yes, this worked!
# This can be combined into one command,
# but need to be careful and perform these operations
# in the correct order:
# 1. make ourself session leader,
# 2. open /dev/tty1 and thus acquire a ctty,
# 3. re-execute the shell, allowing it to notice that it has a ctty:
exec setsid sh -c 'exec sh /dev/tty1 2>&1'
Note that above example talks about interactive shell with PID 1. Thus, it painstakingly uses "exec ..." to keep you in PID 1. If you have "sh: can't access tty" problem when you run interactive shell from, say, an init shell script (IOW: you boot with init=/some/shell/script.sh, and script.sh runs "sh" at some point in order to allow user to type in commands), then execing will be a wrong thing to do. You will need to drop it:
setsid sh -c 'exec sh /dev/tty1 2>&1'
# or using a hack tool:
setsid cttyhack sh
If you REALLY want your shell to run on /dev/console, then you can hack your kernel (if you are into that sort of thing) by changing drivers/char/tty_io.c to change the lines where it sets "noctty = 1;" to instead set it to "0". I recommend you instead run your shell on a real console.
sed "/CONFIG_FOO/s/.*/CONFIG_FOO=y/" -i .config; make; does not enable applet foo
In some common cases like "make allnoconfig", both .config and dependent generated files are updated, making their mtimes very close. If you have a build script which modifies .config immediately after this, and then rebuilds the Busybox, it may happen that .config's mtime will be still close the the generated files' mtimes. If filesystem time granularity is low (for example, 1 second), these mtimes may end up being equal, and dependent files wouldn't be rebuilt. The work around is to add "sleep 1" before sed.
I installed a package on my Busybox system and now nothing works!
"The system boots fine, but when I compile the latest e2fsprogs from sourceforge and add them, system stops with a kernel panic: attempt to kill init."
If /sbin/tune2fs is a link to /bin/busybox and e2fsprogs install process overwrites it instead of deleting it and then creating new /sbin/tune2fs executable, then /bin/busybox is not a Busybox binary anymore, it is a tune2fs binary. ALL /[s]bin/xxxx -> /bin/busybox links now point to it, including /sbin/init. When kernel runs /sbin/init, it runs tune2fs, which prints help text and exits.
I selected "standalone shell" option and now I have problems
Busybox has a feature called the "standalone shell", where the Busybox shell runs any built-in applets before checking the command path. This feature is not enabled by "make defconfig". To try it out, set CONFIG_FEATURE_PREFER_APPLETS and CONFIG_FEATURE_SH_STANDALONE to 'y' in .config (using text editor, or by running "make menuconfig" and setting these options interactively), rebuild Busybox, then run the command line "PATH= ./busybox ash". This will blank your command search path and run Busybox as your command shell, so the only commands it can find (without an explicit path such as /bin/ls) are the built-in Busybox ones. Note that the standalone shell also requires CONFIG_BUSYBOX_EXEC_PATH to be set appropriately, and the default value, /proc/self/exe, would work only if /proc filesystem is mounted. (So if you set it to /proc/self/exe, and happen to be able to chroot into your rootfs, you must mount /proc beforehand.) If you do not have /proc mouted, then point that config option to the location of your Busybox binary, usually /bin/busybox.
A typical indication that you set CONFIG_BUSYBOX_EXEC_PATH to /proc/self/exe but /proc is not mounted:
$ /bin/echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 $ echo $PATH /bin/sh: echo: not found
If this is a serious limitation for you, you may patch the kernel so that it always understands exec("/proc/self/exe"), even if /proc is not mounted. For example, in linux-2.6.30, you need to patch open_exec() function in fs/exec.c file:
file = do_filp_open(AT_FDCWD, name,
O_LARGEFILE | O_RDONLY | FMODE_EXEC, 0,
MAY_EXEC | MAY_OPEN);
- if (IS_ERR(file))
- goto out;
+ if (IS_ERR(file)) {
+ if ((PTR_ERR(file) == -ENOENT || PTR_ERR(file) == -EACCES)
+ && strcmp(name, "/proc/self/exe") == 0
+ ) {
+ struct file *sv = file;
+ struct mm_struct *mm;
+ mm = get_task_mm(current);
+ if (!mm)
+ goto out;
+ file = get_mm_exe_file(mm);
+ mmput(mm);
+ if (file)
+ goto ok;
+ file = sv;
+ }
+ goto out;
+ }
+ok:
err = -EACCES;
Since standalone shell option is not the default, it is less thoroughly tested. If you think you see a bug in standalone shell's behavior, first verify that the bug is indeed caused by this option: rebuild Busybox with CONFIG_FEATURE_PREFER_APPLETS and CONFIG_FEATURE_SH_STANDALONE turned off. If shell's behavior has changed, report the bug to the Busybox mailing list.
Misc. questions
How do I change the time zone in Busybox?
Busybox has nothing to do with the timezone. Please consult your libc documentation. (http://google.com/search?q=uclibc+glibc+timezone).
Development
What are the goals of Busybox?
Busybox aims to be the smallest and simplest correct implementation of the standard Linux command line tools. First and foremost, this means the smallest executable size we can manage. We also want to have the simplest and cleanest implementation we can manage, be standards compliant, minimize run-time memory usage (heap and stack), run fast, and take over the world.
What is the design of Busybox?
Busybox is like a swiss army knife: one thing with many functions. The Busybox executable can act like many different programs depending on the name used to invoke it. Normal practice is to create a bunch of symlinks pointing to the Busybox binary, each of which triggers a different Busybox function. (See getting started in the FAQ for more information on usage, and the Busybox documentation for a list of symlink names and what they do.)
The "one binary to rule them all" approach is primarily for size reasons: a single multi-purpose executable is smaller then many small files could be. This way Busybox only has one set of ELF headers, it can easily share code between different apps even when statically linked, it has better packing efficiency by avoding gaps between files or compression dictionary resets, and so on.
Work is underway on new options such as "make standalone" to build separate binaries for each applet, and a "libbb.so" to make the Busybox common code available as a shared library. Neither is ready yet at the time of this writing.
The applet directories
The directory "applets" contains the Busybox startup code (applets.c and Busybox.c), and several subdirectories containing the code for the individual applets.
Busybox execution starts with the main() function in applets/busybox.c, which sets the global variable applet_name to argv[0] and calls run_applet_and_exit() in applets/applets.c. That uses the applets[] array (defined in include/busybox.h and filled out in include/applets.h) to transfer control to the appropriate APPLET_main() function (such as cat_main() or sed_main()). The individual applet takes it from there.
This is why calling Busybox under a different name triggers different functionality: main() looks up argv[0] in applets[] to get a function pointer to APPLET_main().
Busybox applets may also be invoked through the multiplexer applet "busybox" (see Busybox_main() in libbb/appletlib.c), and through the standalone shell (grep for SH_STANDALONE in shell/*.c). See getting started in the FAQ for more information on these alternate usage mechanisms, which are just different ways to reach the relevant APPLET_main() function.
The applet subdirectories (archival, console-tools, coreutils, debianutils, e2fsprogs, editors, findutils, init, loginutils, miscutils, modutils, networking, procps, shell, sysklogd, and util-linux) correspond to the configuration sub-menus in menuconfig. Each subdirectory contains the code to implement the applets in that sub-menu, as well as a Config.in file defining that configuration sub-menu (with dependencies and help text for each applet), and the makefile segment (Kbuild) for that subdirectory.
The run-time --help is stored in usage_messages[], which is initialized at the start of applets/applets.c and gets its help text from usage.h. During the build this help text is also used to generate the Busybox documentation (in html, txt, and man page formats) in the docs directory. See adding an applet to Busybox for more information.
libbb
Most non-setup code shared between Busybox applets lives in the libbb directory. It's a mess that evolved over the years without much auditing or cleanup. For anybody looking for a great project to break into Busybox development with, documenting libbb would be both incredibly useful and good experience.
Common themes in libbb include allocation functions that test for failure and abort the program with an error message so the caller doesn't have to test the return value (xmalloc(), xstrdup(), etc), wrapped versions of open(), close(), read(), and write() that test for their own failures and/or retry automatically, linked list management functions (llist.c), command line argument parsing (getopt32.c), and a whole lot more.
I want to make Busybox even smaller, how do I go about it?
To conserve bytes it's good to know where they're being used, and the size of the final executable isn't always a reliable indicator of the size of the components (since various structures are rounded up, so a small change may not even be visible by itself, but many small savings add up).
The Busybox Makefile builds two versions of Busybox, one of which (Busybox_unstripped) has extra information that various analysis tools can use. (This has nothing to do with CONFIG_DEBUG, leave that off when trying to optimize for size.)
The "make bloatcheck" option uses Matt Mackall's bloat-o-meter script to compare two versions of Busybox (Busybox_unstripped vs Busybox_old), and report which symbols changed size and by how much. To use it, first build a base version with "make baseline". (This creates Busybox_old, which should have the original sizes for comparison purposes.) Then build the new version with your changes and run "make bloatcheck" to see the size differences from the old version.
The first line of output has totals: how many symbols were added or removed, how many symbols grew or shrank, the number of bytes added and number of bytes removed by these changes, and finally the total number of bytes difference between the two files. The remaining lines show each individual symbol, the old and new sizes, and the increase or decrease in size (which results are sorted by).
The "make sizes" option produces raw symbol size information for Busybox_unstripped. This is the output from the "nm --size-sort" command (see "man nm" for more information), and is the information bloat-o-meter parses to produce the comparison report above. For defconfig, this is a good way to find the largest symbols in the tree (which is a good place to start when trying to shrink the code). To take a closer look at individual applets, configure Busybox with just one applet (run "make allnoconfig" and then switch on a single applet with menuconfig), and then use "make sizes" to see the size of that applet's components.
The "showasm" command (in the scripts directory) produces an assembly dump of a function, providing a closer look at what changed. Try "scripts/showasm busybox_unstripped" to list available symbols, and "scripts/showasm busybox_unstripped symbolname" to see the assembly for a specific symbol.
Adding an applet to Busybox
To add a new applet to Busybox, first pick a name for the applet and a corresponding CONFIG_NAME. Then do this:
- Figure out where in the Busybox source tree your applet best fits, and put your source code there. Be sure to use APPLET_main() instead of main(), where APPLET is the name of your applet.
- Add your applet to the relevant Config.in file (which file you add it to determines where it shows up in "make menuconfig"). This uses the same general format as the linux kernel's configuration system.
- Add your applet to the relevant Kbuild file (in the same directory as the Config.in you chose), using the existing entries as a template and the same CONFIG symbol as you used for Config.in.
- Add your applet to "include/applets.h", using one of the existing entries as a template. (Note: they are in alphabetical order. Applets are found via binary search, and if you add an applet out of order it won't work.)
- Add your applet's runtime help text to "include/usage.h". You need at least appname_trivial_usage (the minimal help text, always included in the Busybox binary when this applet is enabled) and appname_full_usage (extra help text included in the Busybox binary with CONFIG_FEATURE_VERBOSE_USAGE is enabled), or it won't compile. The other two help entry types (appname_example_usage and appname_notes_usage) are optional. They don't take up space in the binary, but instead show up in the generated documentation (BusyBox.html, BusyBox.txt, and the man page busybox.1).
- Run menuconfig, switch your applet on, compile, test, and fix the bugs. Be sure to try both "allyesconfig" and "allnoconfig".
What standards does Busybox adhere to?
The standard we're paying attention to is the "Shell and Utilities" portion of the Open Group Base Standards (also known as the Single Unix Specification version 3 or SUSv3). Note that paying attention isn't necessarily the same thing as following it.
SUSv3 doesn't even mention things like init, mount, tar, or losetup, nor commonly used options like echo's '-e' and '-n', or sed's '-i'. Busybox is driven by what real users actually need, not the fact the standard believes we should implement ed or sccs. For size reasons, we're unlikely to include much internationalization support beyond UTF-8, and on top of all that, our configuration menu lets developers chop out features to produce smaller but very non-standard utilities.
Also, Busybox is aimed primarily at Linux. Unix standards are interesting because Linux tries to adhere to them, but portability to dozens of platforms is only interesting in terms of offering a restricted feature set that works everywhere, not growing dozens of platform-specific extensions. Busybox should be portable to all hardware platforms Linux supports, and any other similar operating systems that are easy to do and won't require much maintenance.
In practice, standards compliance tends to be a clean-up step once an applet is otherwise finished. When polishing and testing a Busybox applet, we ensure we have at least the option of full standards compliance, or else document where we (intentionally) fall short.
Portability
Busybox is a Linux project, but that doesn't mean we don't have to worry about portability. First of all, there are different hardware platforms, different C library implementations, different versions of the kernel and build toolchain... The file "include/platform.h" exists to centralize and encapsulate various platform-specific things in one place, so most Busybox code doesn't have to care where it's running.
To start with, Linux runs on dozens of hardware platforms. We try to test each release on x86, x86-64, arm, power pc, and mips. (Since qemu can handle all of these, this isn't that hard.) This means we have to care about a number of portability issues like endianness, word size, and alignment, all of which belong in platform.h. That header handles conditional #includes and gives us macros we can use in the rest of our code. At some point in the future we might grow a platform.c, possibly even a platform subdirectory. As long as the applets themselves don't have to care.
On a related note, we made the "default signedness of char varies" problem go away by feeding the compiler -funsigned-char. This gives us consistent behavior on all platforms, and defaults to 8-bit clean text processing (which gets us halfway to UTF-8 support). NOMMU support is less easily separated (see the tips section later in this document), but we're working on it.
Another type of portability is build environments: we unapologetically use a number of gcc and glibc extensions (as does the Linux kernel), but these have been picked up by packages like uClibc, TCC, and Intel's C Compiler. As for gcc, we take advantage of newer compiler optimizations to get the smallest possible size, but we also regression test against an older build environment using the Red Hat 9 image at http://busybox.net/downloads/qemu. This has a 2.4 kernel, gcc 3.2, make 3.79.1, and glibc 2.3, and is the oldest build/deployment environment we still put any effort into maintaining. (If anyone takes an interest in older kernels you're welcome to submit patches, but the effort would probably be better spent trimming down the 2.6 kernel.) Older gcc versions than that are uninteresting since we now use c99 features, although tcc might be worth a look.
We also test Busybox against the current release of uClibc. Older versions of uClibc aren't very interesting (they were buggy, and uClibc wasn't really usable as a general-purpose C library before version 0.9.26 anyway).
Other unix implementations are mostly uninteresting, since Linux binaries have become the new standard for portable Unix programs. Specifically, the ubiquity of Linux was cited as the main reason the Intel Binary Compatability Standard 2 died, by the standards group organized to name a successor to ibcs2: the 86open project. That project disbanded in 1999 with the endorsement of an existing standard: Linux ELF binaries. Since then, the major players at the time (such as AIX, Solaris, and FreeBSD) have all either grown Linux support or folded.
The major exceptions are newcomer MacOS X, some embedded environments (such as newlib+libgloss) which provide a posix environment but not a full Linux environment, and environments like Cygwin that provide only partial Linux emulation. Also, some embedded Linux systems run a Linux kernel but amputate things like the /proc directory to save space.
Supporting these systems is largely a question of providing a clean subset of Busybox's functionality -- whichever applets can easily be made to work in that environment. Annotating the configuration system to indicate which applets require which prerequisites (such as procfs) is also welcome. Other efforts to support these systems (swapping #include files to build in different environments, adding adapter code to platform.h, adding more extensive special-case supporting infrastructure such as mount's legacy mtab support) are handled on a case-by-case basis. Support that can be cleanly hidden in platform.h is reasonably attractive, and failing that support that can be cleanly separated into a separate conditionally compiled file is at least worth a look. Special-case code in the body of an applet is something we're trying to avoid.
Programming tips and tricks
Various things Busybox uses that aren't particularly well documented elsewhere.
Encrypted Passwords
Password fields in /etc/passwd and /etc/shadow are in a special format. If the first character isn't '$', then it's an old DES style password. If the first character is '$' then the password is actually three fields separated by '$' characters:
$type$salt$encrypted_password
The "type" indicates which encryption algorithm to use: 1 for MD5 and 2 for SHA1.
The "salt" is a bunch of ramdom characters (generally 8) the encryption algorithm uses to perturb the password in a known and reproducible way (such as by appending the random data to the unencrypted password, or combining them with exclusive or). Salt is randomly generated when setting a password, and then the same salt value is re-used when checking the password. (Salt is thus stored unencrypted.)
The advantage of using salt is that the same cleartext password encrypted with a different salt value produces a different encrypted value. If each encrypted password uses a different salt value, an attacker is forced to do the cryptographic math all over again for each password they want to check. Without salt, they could simply produce a big dictionary of commonly used passwords ahead of time, and look up each password in a stolen password file to see if it's a known value. (Even if there are billions of possible passwords in the dictionary, checking each one is just a binary search against a file only a few gigabytes long.) With salt they can't even tell if two different users share the same password without guessing what that password is and decrypting it. They also can't precompute the attack dictionary for a specific password until they know what the salt value is.
The third field is the encrypted password (plus the salt). For md5 this is 22 bytes.
The Busybox function to handle all this is pw_encrypt(clear, salt) in "libbb/pw_encrypt.c". The first argument is the clear text password to be encrypted, and the second is a string in "$type$salt$password" format, from which the "type" and "salt" fields will be extracted to produce an encrypted value. (Only the first two fields are needed, the third $ is equivalent to the end of the string.) The return value is an encrypted password in /etc/passwd format, with all three $ separated fields. It's stored in a static buffer, 128 bytes long.
So when checking an existing password, if pw_encrypt(text, old_encrypted_password) returns a string that compares identical to old_encrypted_password, you've got the right password. When setting a new password, generate a random 8 character salt string, put it in the right format with sprintf(buffer, "$%c$%s", type, salt), and feed buffer as the second argument to pw_encrypt(text,buffer).
Fork and vfork
On systems that haven't got a Memory Management Unit, fork() is unreasonably expensive to implement (and sometimes even impossible), so a less capable function called vfork() is used instead.
Busybox hides the difference between fork() and vfork() in libbb. If you ever want to fork and exec, use [x]spawn(argv) (which returns a pid) + waid4pid(pid) (which returns exitcode), or even spawn_and_wait(argv).
Making program to daemonize is trickier. Usually, it's done by forking, and exiting in parent, leaving child to continue to run. This is not possible with vfork, because with vfork, while child is running, parent does not return from vfork, and therefore it can't exit. This can be worked around by execing the same program (with parameters set up so that it knows that it doesn't need to daemonize anymore) after vfork in child. This unblocks parent, which can then exit. bb_daemonize_or_rexec(flags, argv) hides all NOMMU machinery of this process. Consult comments in libbb.h for details.
This description is here in case you want to know gory details.
Implementing fork() depends on having a Memory Management Unit. With a MMU you can simply set up a second set of page tables and share the physical memory via copy-on-write. So a fork() followed quickly by exec() only copies a few pages of the parent's memory, just the ones it changes before freeing them.
With a very primitive MMU (using a base pointer plus length instead of page tables, which can provide virtual addresses and protect processes from each other, but no copy on write) you can still implement fork. But it's unreasonably expensive, because you have to copy all the parent process' memory into the new process (which could easily be several megabytes per fork). And you have to do this even though that memory gets freed again as soon as the exec happens. (This is not just slow and a waste of space but causes memory usage spikes that can easily cause the system to run out of memory.)
Without even a primitive MMU, you have no virtual addresses. Every process can reach out and touch any other process' memory, because all pointers are to physical addresses with no protection. Even if you copy a process' memory to new physical addresses, all of its pointers point to the old objects in the old process. (Searching through the new copy's memory for pointers and redirect them to the new locations is not an easy problem.)
So with a primitive or missing MMU, fork() is just not a good idea.
In theory, vfork() is just a fork() that writeably shares process memory rather than copying it (so what one process writes the other one sees). In practice, vfork() has to suspend the parent process until the child does exec, at which point the parent wakes up and resumes by returning from the call to vfork(). All modern kernel/libc combinations implement vfork() to put the parent to sleep until the child does its exec. There's just no other way to make it work: the parent has to know the child has done its exec() or exit() before it's safe to return from the function it's in, so it has to block until that happens. In fact without suspending the parent there's no way to even store separate copies of the return value (the pid) from the vfork() call itself: both assignments write into the same memory location.
One way to understand (and in fact implement) vfork() is this: imagine the parent does a setjmp and then continues on (pretending to be the child) until the exec() comes around, then the _exec_ does the actual fork, and the parent does a longjmp back to the original vfork call and continues on from there. (It thus becomes obvious why the child should not return (this would destroy data on stack needed by parent), or modify any memory variables it doesn't want the parent to see changed when it resumes.)
Note a common mistake: the need for vfork doesn't mean you can't have two processes running at the same time. It means you can't have two processes sharing the same memory without stomping all over each other. As soon as the child calls exec(), the parent resumes.
If the child's attempt to call exec() fails, the child usually calls _exit() rather than a normal exit(). This avoids any atexit() code that might confuse the parent. In most circumstances, fflush(stdout) and _exit() is good enough.
Another thing to keep in mind is that if vforked child allocates any memory and does not free it before exec or exit, parent will also have this memory allocated. Unless child takes care to record the address of these memory areas and parent frees them, they will be leaked. This should be avoided if parent process is long lived. The prime example is the shell.
Short reads and writes
Busybox has special functions, full_read() and full_write(), to check that all the data we asked for got read or written. Is this a real world consideration? Try the following:
while true; do echo hello; sleep 1; done | tee out.txt
If tee is implemented with full_read(), tee doesn't display output in real time but blocks until its entire input buffer (generally a couple kilobytes) is read, then displays it all at once. In that case, we _want_ the short read, for user interface reasons. (Note that read() should never return 0 unless it has hit the end of input, and an attempt to write 0 bytes should be ignored by the OS.)
As for short writes, play around with two processes piping data to each other on the command line (cat bigfile | gzip > out.gz) and suspend and resume a few times (ctrl-z to suspend, "fg" to resume). The writer can experience short writes, which are especially dangerous because if you don't notice them you'll discard data. They can also happen when a system is under load and a fast process is piping to a slower one. (Such as an xterm waiting on x11 when the scheduler decides X is being a CPU hog with all that text console scrolling...)
So will data always be read from the far end of a pipe at the same chunk sizes it was written in? Nope. Don't rely on that. For one counterexample, see rfc 896 for Nagle's algorithm, which waits a fraction of a second or so before sending out small amounts of data through a TCP/IP connection in case more data comes in that can be merged into the same packet. (In case you were wondering why action games that use TCP/IP set TCP_NODELAY to lower the latency on their their sockets, now you know.)
Memory used by relocatable code, PIC, and static linking
The downside of standard dynamic linking is that it results in self-modifying code. Although each executable's pages are mmaped() into a process' address space from the executable file and are thus naturally shared between processes out of the page cache, the library loader (ld-linux.so.2 or ld-uClibc.so.0) writes to these pages to supply addresses for relocatable symbols. This dirties the pages, triggering copy-on-write allocation of new memory for each processes' dirtied pages.
One solution to this is Position Independent Code (PIC), a way of linking a file so all the relocations are grouped together. This dirties fewer pages (often just a single page) for each process' relocations. The down side is this results in larger executables, which take up more space on disk (and a correspondingly larger space in memory). But when many copies of the same program are running, PIC dynamic linking trades a larger disk footprint for a smaller memory footprint, by sharing more pages.
A third solution is static linking. A statically linked program has no relocations, and thus the entire executable is shared between all running instances. This tends to have a significantly larger disk footprint, but on a system with only one or two executables, shared libraries aren't much of a win anyway.
You can tell the glibc linker to display debugging information about its relocations with the environment variable "LD_DEBUG". Try "LD_DEBUG=help /bin/true" for a list of commands. Learning to interpret "LD_DEBUG=statistics cat /proc/self/statm" could be interesting.
For more on this topic, here's Rich Felker:
Dynamic linking (without fixed load addresses) fundamentally requires at least one dirty page per dso that uses symbols. Making calls (but never taking the address explicitly) to functions within the same dso does not require a dirty page by itself, but will with ELF unless you use -Bsymbolic or hidden symbols when linking.
ELF uses significant additional stack space for the kernel to pass all the ELF data structures to the newly created process image. These are located above the argument list and environment. This normally adds 1 dirty page to the process size.
The ELF dynamic linker has its own data segment, adding one or more dirty pages. I believe it also performs relocations on itself.
The ELF dynamic linker makes significant dynamic allocations to manage the global symbol table and the loaded dso's. This data is never freed. It will be needed again if libdl is used, so unconditionally freeing it is not possible, but normal programs do not use libdl. Of course with glibc all programs use libdl (due to nsswitch) so the issue was never addressed.
ELF also has the issue that segments are not page-aligned on disk. This saves up to 4k on disk, but at the expense of using an additional dirty page in most cases, due to a large portion of the first data page being filled with a duplicate copy of the last text page.
The above is just a partial list of the tiny memory penalties of ELF dynamic linking, which eventually add up to quite a bit. The smallest I've been able to get a process down to is 8 dirty pages, and the above factors seem to mostly account for it (but some were difficult to measure).
Including kernel headers
The "linux" or "asm" directories of /usr/include contain Linux kernel headers, so that the C library can talk directly to the Linux kernel. In a perfect world, applications shouldn't include these headers directly, but we don't live in a perfect world.
For example, Busybox's losetup code wants linux/loop.c because nothing else #defines the structures to call the kernel's loopback device setup ioctls. Attempts to cut and paste the information into a local Busybox header file proved incredibly painful, because portions of the loop_info structure vary by architecture, namely the type __kernel_dev_t has different sizes on alpha, arm, x86, and so on. Meaning we either #include
This is aside from the fact that the relevant type defined in posix_types.h was renamed to __kernel_old_dev_t during the 2.5 series, so to cut and paste the structure into our header we have to #include
The Busybox developers spent two years trying to figure out a clean way to do all this. There isn't one. The losetup in the util-linux package from kernel.org isn't doing it cleanly either, they just hide the ugliness by nesting #include files. Their mount/loop.h #includes "my_dev_t.h", which #includes
Just because directly #including kernel headers is sometimes unavoidable doesn't me we should include them when there's a better way to do it. However, block copying information out of the kernel headers is not a better way.
Who are the Busybox developers?
The following login accounts currently exist on busybox.net. (I.E. these people can commit patches into the Busybox, uClibc, and buildroot projects.)
aldot :Bernhard Reutner-Fischer andersen :Erik Andersen - uClibc and BuildRoot maintainer bug1 :Glenn McGrath davidm :David McCullough gkajmowi :Garrett Kajmowicz - uClibc++ maintainer jbglaw :Jan-Benedict Glaw jocke :Joakim Tjernlund landley :Rob Landley lethal :Paul Mundt mjn3 :Manuel Novoa III osuadmin :osuadmin pgf :Paul Fox pkj :Peter Kjellerstedt prpplague :David Anders psm :Peter S. Mazinger russ :Russ Dill sandman :Robert Griebl sjhill :Steven J. Hill solar :Ned Ludd timr :Tim Riker tobiasa :Tobias Anderberg vapier :Mike Frysinger vda :Denys Vlasenko - Busybox maintainer
The following accounts used to exist on busybox.net, but don't anymore so I can't ask /etc/passwd for their names. Rob Wentworth
aaronl :Aaron Lehmann beppu :John Beppu dwhedon :David Whedon erik :Erik Andersen gfeldman :Gennady Feldman jimg :Jim Gleason kraai :Matt Kraai markw :Mark Whitley miles :Miles Bader proski :Pavel Roskin rjune :Richard June tausq :Randolph Chung vodz :Vladimir N. Oleynik