1 <html><head><title>toybox source code walkthrough</title></head>
2 <!--#include file="header.html" -->
4 <p><h1><a name="style" /><a href="#style">Code style</a></h1></p>
6 <p>The primary goal of toybox is _simple_ code. Keeping the code small is
7 second, with speed and lots of features coming in somewhere after that.
8 (For more on that, see the <a href=design.html>design</a> page.)</p>
10 <p>A simple implementation usually takes up fewer lines of source code,
11 meaning more code can fit on the screen at once, meaning the programmer can
12 see more of it on the screen and thus keep more if in their head at once.
13 This helps code auditing and thus reduces bugs. That said, sometimes being
14 more explicit is preferable to being clever enough to outsmart yourself:
15 don't be so terse your code is unreadable.</p>
17 <p>Toybox has an actual coding style guide over on
18 <a href=design.html#codestyle>the design page</a>, but in general we just
19 want the code to be consistent.</p>
21 <p><h1><a name="building" /><a href="#building">Building Toybox</a></h1></p>
23 <p>Toybox is configured using the Kconfig language pioneered by the Linux
24 kernel, and adopted by many other projects (uClibc, OpenEmbedded, etc).
25 This generates a ".config" file containing the selected options, which
26 controls which features are included when compiling toybox.</p>
28 <p>Each configuration option has a default value. The defaults indicate the
29 "maximum sane configuration", I.E. if the feature defaults to "n" then it
30 either isn't complete or is a special-purpose option (such as debugging
31 code) that isn't intended for general purpose use.</p>
33 <p>The standard build invocation is:</p>
36 <li>make defconfig #(or menuconfig)</li>
41 <p>Type "make help" to see all available build options.</p>
43 <p>The file "configure" contains a number of environment variable definitions
44 which influence the build, such as specifying which compiler to use or where
45 to install the resulting binaries. This file is included by the build, but
46 accepts existing definitions of the environment variables, so it may be sourced
47 or modified by the developer before building and the definitions exported
48 to the environment will take precedence.</p>
50 <p>(To clarify: "configure" describes the build and installation environment,
51 ".config" lists the features selected by defconfig/menuconfig.)</p>
53 <p><h1><a name="running"><a href="#running">Running a command</a></h1></p>
57 <p>The toybox main() function is at the end of main.c at the top level. It has
58 two possible codepaths, only one of which is configured into any given build
61 <p>If CONFIG_SINGLE is selected, toybox is configured to contain only a single
62 command, so most of the normal setup can be skipped. In this case the
63 multiplexer isn't used, instead main() calls toy_singleinit() (also in main.c)
64 to set up global state and parse command line arguments, calls the command's
65 main function out of toy_list (in the CONFIG_SINGLE case the array has a single entry, no need to search), and if the function returns instead of exiting
66 it flushes stdout (detecting error) and returns toys.exitval.</p>
68 <p>When CONFIG_SINGLE is not selected, main() uses basename() to find the
69 name it was run as, shifts its argument list one to the right so it lines up
70 with where the multiplexer function expects it, and calls toybox_main(). This
71 leverages the multiplexer command's infrastructure to find and run the
72 appropriate command. (A command name starting with "toybox" will
73 recursively call toybox_main(); you can go "./toybox toybox toybox toybox ls"
74 if you want to...)</p>
78 <p>The toybox_main() function is also in main,c. It handles a possible
79 --help option ("toybox --help ls"), prints the list of available commands if no
80 arguments were provided to the multiplexer (or with full path names if any
81 other option is provided before a command name, ala "toybox --list").
82 Otherwise it calls toy_exec() on its argument list.</p>
84 <p>Note that the multiplexer is the first entry in toy_list (the rest of the
85 list is sorted alphabetically to allow binary search), so toybox_main can
86 cheat and just grab the first entry to quickly set up its context without
87 searching. Since all command names go through the multiplexer at least once
88 in the non-TOYBOX_SINGLE case, this avoids a redundant search of
91 <p>The toy_exec() function is also in main.c. It performs toy_find() to
92 perform a binary search on the toy_list array to look up the command's
93 entry by name and saves it in the global variable which, calls toy_init()
94 to parse command line arguments and set up global state (using which->options),
95 and calls the appropriate command's main() function (which->toy_main). On
96 return it flushes all pending ansi FILE * I/O, detects if stdout had an
97 error, and then calls xexit() (which uses toys.exitval).</p>
99 <p><h1><a name="infrastructure" /><a href="#infrastructure">Infrastructure</a></h1></p>
101 <p>The toybox source code is in following directories:</p>
103 <li>The <a href="#top">top level directory</a> contains the file main.c (were
104 execution starts), the header file toys.h (included by every command), and
105 other global infrastructure.</li>
106 <li>The <a href="#lib">lib directory</a> contains common functions shared by
107 multiple commands:</li>
109 <li><a href="#lib_lib">lib/lib.c</a></li>
110 <li><a href="#lib_xwrap">lib/xwrap.c</a></li>
111 <li><a href="#lib_llist">lib/llist.c</a></li>
112 <li><a href="#lib_args">lib/args.c</a></li>
113 <li><a href="#lib_dirtree">lib/dirtree.c</a></li>
115 <li>The <a href="#toys">toys directory</a> contains the C files implementating
116 each command. Currently it contains five subdirectories categorizing the
117 commands: posix, lsb, other, example, and pending.</li>
118 <li>The <a href="#scripts">scripts directory</a> contains the build and
119 test infrastructure.</li>
120 <li>The <a href="#kconfig">kconfig directory</a> contains the configuration
121 infrastructure implementing menuconfig (copied from the Linux kernel).</li>
122 <li>The <a href="#generated">generated directory</a> contains intermediate
123 files generated from other parts of the source code.</li>
127 <p><h1><a href="#adding">Adding a new command</a></h1></p>
128 <p>To add a new command to toybox, add a C file implementing that command to
129 one of the subdirectories under the toys directory. No other files need to
130 be modified; the build extracts all the information it needs (such as command
131 line arguments) from specially formatted comments and macros in the C file.
132 (See the description of the <a href="#generated">"generated" directory</a>
135 <p>Currently there are five subdirectories under "toys", one for commands
136 defined by the POSIX standard, one for commands defined by the Linux Standard
137 Base, an "other" directory for commands not covered by an obvious standard,
138 a directory of example commands (templates to use when starting new commands),
139 and a "pending" directory of commands that need further review/cleanup
140 before moving to one of the other directories (run these at your own risk,
141 cleanup patches welcome).
142 These directories are just for developer convenience sorting the commands,
143 the directories are otherwise functionally identical. To add a new category,
144 create the appropriate directory with a README file in it whose first line
145 is the description menuconfig should use for the directory.)</p>
147 <p>An easy way to start a new command is copy the file "toys/example/hello.c"
148 to the name of the new command, and modify this copy to implement the new
149 command (more or less by turning every instance of "hello" into the
150 name of your command, updating the command line arguments, globals, and
151 help data, and then filling out its "main" function with code that does
152 something interesting).</p>
154 <p>You could also start with "toys/example/skeleton.c", which provides a lot
155 more example code (showing several variants of command line option
156 parsing, how to implement multiple commands in the same file, and so on).
157 But usually it's just more stuff to delete.</p>
159 <p>Here's a checklist of steps to turn hello.c into another command:</p>
162 <li><p>First "cp toys/example/hello.c toys/other/yourcommand.c" and open
163 the new file in your preferred text editor.</p>
164 <ul><li><p>Note that the
165 name of the new file is significant: it's the name of the new command you're
166 adding to toybox. The build includes all *.c files under toys/*/ whose
167 names are a case insensitive match for an enabled config symbol. So
168 toys/posix/cat.c only gets included if you have "CAT=y" in ".config".</p></li>
171 <li><p>Change the one line comment at the top of the file (currently
172 "hello.c - A hello world program") to describe your new file.</p></li>
174 <li><p>Change the copyright notice to your name, email, and the current
177 <li><p>Give a URL to the relevant standards document, where applicable.
178 (Sample links to SUSv4 and LSB are provided, feel free to link to other
179 documentation or standards as appropriate.)</p></li>
181 <li><p>Update the USE_YOURCOMMAND(NEWTOY(yourcommand,"blah",0)) line.
182 The NEWTOY macro fills out this command's <a href="#toy_list">toy_list</a>
183 structure. The arguments to the NEWTOY macro are:</p>
186 <li><p>the name used to run your command</p></li>
187 <li><p>the command line argument <a href="#lib_args">option parsing string</a> (0 if none)</p></li>
188 <li><p>a bitfield of TOYFLAG values
189 (defined in toys.h) providing additional information such as where your
190 command should be installed on a running system, whether to blank umask
191 before running, whether or not the command must run as root (and thus should
192 retain root access if installed SUID), and so on.</p></li>
196 <li><p>Change the kconfig data (from "config YOURCOMMAND" to the end of the
197 comment block) to supply your command's configuration and help
198 information. The uppper case config symbols are used by menuconfig, and are
199 also what the CFG_ and USE_() macros are generated from (see [TODO]). The
200 help information here is used by menuconfig, and also by the "help" command to
201 describe your new command. (See [TODO] for details.) By convention,
202 unfinished commands default to "n" and finished commands default to "y",
203 so "make defconfig" selects all finished commands. (Note, "finished" means
204 "ready to be used", not that it'll never change again.)<p>
206 <p>Each help block should start with a "usage: yourcommand" line explaining
207 any command line arguments added by this config option. The "help" command
208 outputs this text, and scripts/config2help.c in the build infrastructure
209 collates these usage lines for commands with multiple configuration
210 options when producing generated/help.h.</p>
213 <li><p>Change the "#define FOR_hello" line to "#define FOR_yourcommand" right
214 before the "#include <toys.h>". (This selects the appropriate FLAG_ macros and
215 does a "#define TT this.yourcommand" so you can access the global variables
216 out of the space-saving union of structures. If you aren't using any command
217 flag bits and aren't defining a GLOBAL block, you can delete this line.)</p></li>
219 <li><p>Update the GLOBALS() macro to contain your command's global
220 variables. If your command has no global variables, delete this macro.</p>
222 <p>Variables in the GLOBALS() block are are stored in a space saving
223 <a href="#toy_union">union of structures</a> format, which may be accessed
224 using the TT macro as if TT were a global structure (so TT.membername).
225 If you specified two-character command line arguments in
226 NEWTOY(), the first few global variables will be initialized by the automatic
227 argument parsing logic, and the type and order of these variables must
228 correspond to the arguments specified in NEWTOY().
229 (See <a href="#lib_args">lib/args.c</a> for details.)</p></li>
231 <li><p>Rename hello_main() to yourcommand_main(). This is the main() function
232 where execution of your command starts. Your command line options are
233 already sorted into this.optflags, this.optargs, this.optc, and the GLOBALS()
234 as appropriate by the time this function is called. (See
235 <a href="#lib_args">get_optflags()</a> for details.)</p></li>
237 <li><p>Switch on TOYBOX_DEBUG in menuconfig (toybox global settings menu)
238 the first time you build and run your new command. If anything is wrong
239 with your option string, that will give you error messages.</p>
241 <p>Otherwise it'll just segfault without
242 explanation when it falls off the end because it didn't find a matching
243 end parantheses for a longopt, or you put a nonexistent option in a square
244 bracket grouping... Since these kind of errors can only be caused by a
245 developer, not by end users, we don't normally want runtime checks for
246 them. Once you're happy with your option string, you can switch TOYBOX_DEBUG
250 <a name="headers" /><h2><a href="#headers">Headers.</a></h2>
252 <p>Commands generally don't have their own headers. If it's common code
253 it can live in lib/, if it isn't put it in the command's .c file. (The line
254 between implementing multiple commands in a C file via OLDTOY() to share
255 infrastructure and moving that shared infrastructure to lib/ is a judgement
256 call. Try to figure out which is simplest.)</p>
258 <p>The top level toys.h should #include all the standard (posix) headers
259 that any command uses. (Partly this is friendly to ccache and partly this
260 makes the command implementations shorter.) Individual commands should only
261 need to include nonstandard headers that might prevent that command from
262 building in some context we'd care about (and thus requiring that command to
263 be disabled to avoid a build break).</p>
265 <p>Target-specific stuff (differences between compiler versions, libc versions,
266 or operating systems) should be confined to lib/portability.h and
267 lib/portability.c. (There's even some minimal compile-time environment probing
268 that writes data to generated/portability.h, see scripts/genconfig.sh.)</p>
270 <p>Only include linux/*.h headers from individual commands (not from other
271 headers), and only if you really need to. Data that varies per architecture
272 is a good reason to include a header. If you just need a couple constants
273 that haven't changed since the 1990's, it's ok to #define them yourself or
274 just use the constant inline with a comment explaining what it is. (A
275 #define that's only used once isn't really helping.)</p>
277 <p><a name="top" /><h1><a href="#top">Top level directory.</a></h1></p>
279 <p>This directory contains global infrastructure.</p>
282 <p>Each command #includes "toys.h" as part of its standard prolog. It
283 may "#define FOR_commandname" before doing so to get some extra entries
284 specific to this command.</p>
286 <p>This file sucks in most of the commonly used standard #includes, so
287 individual files can just #include "toys.h" and not have to worry about
288 stdargs.h and so on. Individual commands still need to #include
289 special-purpose headers that may not be present on all systems (and thus would
290 prevent toybox from building that command on such a system with that command
291 enabled). Examples include regex support, any "linux/" or "asm/" headers, mtab
292 support (mntent.h and sys/mount.h), and so on.</p>
294 <p>The toys.h header also defines structures for most of the global variables
295 provided to each command by toybox_main(). These are described in
296 detail in the description for main.c, where they are initialized.</p>
298 <p>The global variables are grouped into structures (and a union) for space
299 savings, to more easily track the amount of memory consumed by them,
300 so that they may be automatically cleared/initialized as needed, and so
301 that access to global variables is more easily distinguished from access to
305 <p>Contains the main() function where execution starts, plus
306 common infrastructure to initialize global variables and select which command
307 to run. The "toybox" multiplexer command also lives here. (This is the
308 only command defined outside of the toys directory.)</p>
310 <p>Execution starts in main() which trims any path off of the first command
311 name and calls toybox_main(), which calls toy_exec(), which calls toy_find()
312 and toy_init() before calling the appropriate command's function from
313 toy_list[] (via toys.which->toy_main()).
314 If the command is "toybox", execution recurses into toybox_main(), otherwise
315 the call goes to the appropriate commandname_main() from a C file in the toys
318 <p>The following global variables are defined in main.c:</p>
320 <a name="toy_list" />
321 <li><p><b>struct toy_list toy_list[]</b> - array describing all the
322 commands currently configured into toybox. The first entry (toy_list[0]) is
323 for the "toybox" multiplexer command, which runs all the other built-in commands
324 without symlinks by using its first argument as the name of the command to
325 run and the rest as that command's argument list (ala "./toybox echo hello").
326 The remaining entries are the commands in alphabetical order (for efficient
329 <p>This is a read-only array initialized at compile time by
330 defining macros and #including generated/newtoys.h.</p>
332 <p>Members of struct toy_list (defined in "toys.h") include:</p>
334 <li><p>char *<b>name</b> - the name of this command.</p></li>
335 <li><p>void (*<b>toy_main</b>)(void) - function pointer to run this
337 <li><p>char *<b>options</b> - command line option string (used by
338 get_optflags() in lib/args.c to intialize toys.optflags, toys.optargs, and
339 entries in the toy's GLOBALS struct). When this is NULL, no option
340 parsing is done before calling toy_main().</p></li>
341 <li><p>int <b>flags</b> - Behavior flags for this command. The following flags are currently understood:</p>
344 <li><b>TOYFLAG_USR</b> - Install this command under /usr</li>
345 <li><b>TOYFLAG_BIN</b> - Install this command under /bin</li>
346 <li><b>TOYFLAG_SBIN</b> - Install this command under /sbin</li>
347 <li><b>TOYFLAG_NOFORK</b> - This command can be used as a shell builtin.</li>
348 <li><b>TOYFLAG_UMASK</b> - Call umask(0) before running this command.</li>
349 <li><b>TOYFLAG_STAYROOT</b> - Don't drop permissions for this command if toybox is installed SUID root.</li>
350 <li><b>TOYFLAG_NEEDROOT</b> - This command cannot function unless run with root access.</li>
354 <p>These flags are combined with | (or). For example, to install a command
355 in /usr/bin, or together TOYFLAG_USR|TOYFLAG_BIN.</p>
359 <li><p><b>struct toy_context toys</b> - global structure containing information
360 common to all commands, initializd by toy_init() and defined in "toys.h".
361 Members of this structure include:</p>
363 <li><p>struct toy_list *<b>which</b> - a pointer to this command's toy_list
364 structure. Mostly used to grab the name of the running command
365 (toys->which.name).</p>
367 <li><p>int <b>exitval</b> - Exit value of this command. Defaults to zero. The
368 error_exit() functions will return 1 if this is zero, otherwise they'll
369 return this value.</p></li>
370 <li><p>char **<b>argv</b> - "raw" command line options, I.E. the original
371 unmodified string array passed in to main(). Note that modifying this changes
372 "ps" output, and is not recommended. This array is null terminated; a NULL
373 entry indicates the end of the array.</p>
374 <p>Most commands don't use this field, instead the use optargs, optflags,
375 and the fields in the GLOBALS struct initialized by get_optflags().</p>
377 <li><p>unsigned <b>optflags</b> - Command line option flags, set by
378 <a href="#lib_args">get_optflags()</a>. Indicates which of the command line options listed in
379 toys->which.options occurred this time.</p>
381 <p>The rightmost command line argument listed in toys->which.options sets bit
382 1, the next one sets bit 2, and so on. This means the bits are set in the same
383 order the binary digits would be listed if typed out as a string. For example,
384 the option string "abcd" would parse the command line "-c" to set optflags to 2,
385 "-a" would set optflags to 8, and "-bd" would set optflags to 6 (4|2).</p>
387 <p>Only letters are relevant to optflags. In the string "a*b:c#d", d=1, c=2,
388 b=4, a=8. Punctuation after a letter initializes global variables at the
389 start of the GLOBALS() block (see <a href="#toy_union">union toy_union this</a>
392 <p>The build infrastructure creates FLAG_ macros for each option letter,
393 corresponding to the bit position, so you can check (toys.optflags & FLAG_x)
394 to see if a flag was specified. (The correct set of FLAG_ macros is selected
395 by defining FOR_mycommand before #including toys.h. The macros live in
396 toys/globals.h which is generated by scripts/make.sh.)</p>
398 <p>For more information on option parsing, see <a href="#lib_args">get_optflags()</a>.</p>
401 <li><p>char **<b>optargs</b> - Null terminated array of arguments left over
402 after get_optflags() removed all the ones it understood. Note: optarg[0] is
403 the first argument, not the command name. Use toys.which->name for the command
405 <li><p>int <b>optc</b> - Optarg count, equivalent to argc but for
407 <li><p>int <b>exithelp</b> - Whether error_exit() should print a usage message
408 via help_main() before exiting. (True during option parsing, defaults to
409 false afterwards.)</p></li>
412 <a name="toy_union" />
413 <li><p><b>union toy_union this</b> - Union of structures containing each
414 command's global variables.</p>
416 <p>Global variables are useful: they reduce the overhead of passing extra
417 command line arguments between functions, they conveniently start prezeroed to
418 save initialization costs, and the command line argument parsing infrastructure
419 can also initialize global variables with its results.</p>
421 <p>But since each toybox process can only run one command at a time, allocating
422 space for global variables belonging to other commands you aren't currently
423 running would be wasteful.</p>
425 <p>Toybox handles this by encapsulating each command's global variables in
426 a structure, and declaring a union of those structures with a single global
427 instance (called "this"). The GLOBALS() macro contains the global
428 variables that should go in the current command's global structure. Each
429 variable can then be accessed as "this.commandname.varname".
430 If you #defined FOR_commandname before including toys.h, the macro TT is
431 #defined to this.commandname so the variable can then be accessed as
432 "TT.variable". See toys/hello.c for an example.</p>
434 <p>A command that needs global variables should declare a structure to
435 contain them all, and add that structure to this union. A command should never
436 declare global variables outside of this, because such global variables would
437 allocate memory when running other commands that don't use those global
440 <p>The first few fields of this structure can be intialized by <a href="#lib_args">get_optargs()</a>,
441 as specified by the options field off this command's toy_list entry. See
442 the get_optargs() description in lib/args.c for details.</p>
445 <li><b>char toybuf[4096]</b> - a common scratch space buffer so
446 commands don't need to allocate their own. Any command is free to use this,
447 and it should never be directly referenced by functions in lib/ (although
448 commands are free to pass toybuf in to a library function as an argument).</li>
451 <p>The following functions are defined in main.c:</p>
453 <li><p>struct toy_list *<b>toy_find</b>(char *name) - Return the toy_list
454 structure for this command name, or NULL if not found.</p></li>
455 <li><p>void <b>toy_init</b>(struct toy_list *which, char *argv[]) - fill out
456 the global toys structure, calling get_optargs() if necessary.</p></li>
457 <li><p>void <b>toy_exec</b>(char *argv[]) - Run a built-in command with
459 <p>Calls toy_find() on argv[0] (which must be just a command name
460 without path). Returns if it can't find this command, otherwise calls
461 toy_init(), toys->which.toy_main(), and exit() instead of returning.</p>
463 <p>Use the library function xexec() to fall back to external executables
464 in $PATH if toy_exec() can't find a built-in command. Note that toy_exec()
465 does not strip paths before searching for a command, so "./command" will
466 never match an internal command.</li>
468 <li><p>void <b>toybox_main</b>(void) - the main function for the multiplexer
469 command (I.E. "toybox"). Given a command name as its first argument, calls
470 toy_exec() on its arguments. With no arguments, it lists available commands.
471 If the first argument starts with "-" it lists each command with its default
472 install path prepended.</p></li>
478 <p>Top level configuration file in a stylized variant of
479 <a href=http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt>kconfig</a> format. Includes generated/Config.in.</p>
481 <p>These files are directly used by "make menuconfig" to select which commands
482 to build into toybox (thus generating a .config file), and by
483 scripts/config2help.py to create generated/help.h.</p>
485 <a name="generated" />
486 <h1><a href="#generated">Temporary files:</a></h1>
488 <p>There is one temporary file in the top level source directory:</p>
490 <li><p><b>.config</b> - Configuration file generated by kconfig, indicating
491 which commands (and options to commands) are currently enabled. Used
492 to make generated/config.h and determine which toys/*/*.c files to build.</p>
494 <p>You can create a human readable "miniconfig" version of this file using
495 <a href=http://landley.net/aboriginal/new_platform.html#miniconfig>these
496 instructions</a>.</p>
500 <p><h2>Directory generated/</h2></p>
502 <p>The remaining temporary files live in the "generated/" directory,
503 which is for files generated at build time from other source files.</p>
506 <li><p><b>generated/Config.in</b> - Kconfig entries for each command, included
507 from the top level Config.in. The help text here is used to generate
510 <p>Each command has a configuration entry with an upper case version of
511 the command name. Options to commands start with the command
512 name followed by an underscore and the option name. Global options are attached
513 to the "toybox" command, and thus use the prefix "TOYBOX_". This organization
514 is used by scripts/cfg2files to select which toys/*/*.c files to compile for a
518 <li><p><b>generated/config.h</b> - list of CFG_SYMBOL and USE_SYMBOL() macros,
519 generated from .config by a sed invocation in scripts/make.sh.</p>
521 <p>CFG_SYMBOL is a comple time constant set to 1 for enabled symbols and 0 for
522 disabled symbols. This allows the use of normal if() statements to remove
523 code at compile time via the optimizer's dead code elimination (which removes
524 from the binary any code that cannot be reached). This saves space without
525 cluttering the code with #ifdefs or leading to configuration dependent build
526 breaks. (See the 1992 Usenix paper
527 <a href=http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>#ifdef
528 Considered Harmful</a> for more information.)</p>
530 <p>When you can't entirely avoid an #ifdef, the USE_SYMBOL(code) macro
531 provides a less intrusive alternative, evaluating to the code in parentheses
532 when the symbol is enabled, and nothing when the symbol is disabled. This
533 is most commonly used around NEWTOY() declarations (so only the enabled
534 commands show up in toy_list), and in option strings. This can also be used
535 for things like varargs or structure members which can't always be
536 eliminated by a simple test on CFG_SYMBOL. Remember, unlike CFG_SYMBOL
537 this is really just a variant of #ifdef, and can still result in configuration
538 dependent build breaks. Use with caution.</p>
541 <li><p><b>generated/flags.h</b> - FLAG_? macros indicating which command
542 line options were seen. The option parsing in lib/args.c sets bits in
543 toys.optflags, which can be tested by anding with the appropriate FLAG_
544 macro. (Bare longopts, which have no corresponding short option, will
545 have the longopt name after FLAG_. All others use the single letter short
548 <p>To get the appropriate macros for your command, #define FOR_commandname
549 before #including toys.h. To switch macro sets (because you have an OLDTOY()
550 with different options in the same .c file), #define CLEANUP_oldcommand
551 and also #define FOR_newcommand, then #include "generated/flags.h" to switch.
555 <li><p><b>generated/globals.h</b> -
556 Declares structures to hold the contents of each command's GLOBALS(),
557 and combines them into "global_union this". (Yes, the name was
558 chosen to piss off C++ developers who think that C
559 is merely a subset of C++, not a language in its own right.)</p>
561 <p>The union reuses the same memory for each command's global struct:
562 since only one command's globals are in use at any given time, collapsing
563 them together saves space. The headers #define TT to the appropriate
564 "this.commandname", so you can refer to the current command's global
565 variables out of "this" as TT.variablename.</p>
567 <p>The globals start zeroed, and the first few are filled out by the
568 lib/args.c argument parsing code called from main.c.</p>
571 <li><p><b>toys/help.h</b> - Help strings for use by the "help" command and
572 --help options. This file #defines a help_symbolname string for each
573 symbolname, but only the symbolnames matching command names get used
574 by show_help() in lib/help.c to display help for commands.</p>
576 <p>This file is created by scripts/make.sh, which compiles scripts/config2help.c
577 into the binary generated/config2help, and then runs it against the top
578 level .config and Config.in files to extract the help text from each config
579 entry and collate together dependent options.</p>
581 <p>This file contains help text for all commands, regardless of current
582 configuration, but only the ones currently enabled in the .config file
583 wind up in the help_data[] array, and only the enabled dependent options
584 have their help text added to the command they depend on.</p>
587 <li><p><b>generated/newtoys.h</b> -
588 All the NEWTOY() and OLDTOY() macros from toys/*/*.c. The "toybox" multiplexer
589 is the first entry, the rest are in alphabetical order. Each line should be
590 inside an appropriate USE_ macro, so code that #includes this file only sees
591 the currently enabled commands.</p>
593 <p>By #definining NEWTOY() to various things before #including this file,
594 it may be used to create function prototypes (in toys.h), initialize the
595 help_data array (in lib/help.c), initialize the toy_list array (in main.c,
596 the alphabetical order lets toy_find() do a binary search, the exception to
597 the alphabetical order lets it use the multiplexer without searching), and so
598 on. (It's even used to initialize the NEED_OPTIONS macro, which produces a 1
599 or 0 for each command using command line option parsing, which is ORed together
600 to allow compile-time dead code elimination to remove the whole of
601 lib/args.c if nothing currently enabled is using it.)<p>
603 <p>Each NEWTOY and OLDTOY macro contains the command name, command line
604 option string (telling lib/args.c how to parse command line options for
605 this command), recommended install location, and miscelaneous data such
606 as whether this command should retain root permissions if installed suid.</p>
609 <li><p><b>toys/oldtoys.h</b> - Macros with the command line option parsing
610 string for each NEWTOY. This allows an OLDTOY that's just an alias for an
611 existing command to refer to the existing option string instead of
612 having to repeat it.</p>
617 <h2>Directory lib/</h2>
619 <p>TODO: document lots more here.</p>
621 <p>lib: getmountlist(), error_msg/error_exit, xmalloc(),
622 strlcpy(), xexec(), xopen()/xread(), xgetcwd(), xabspath(), find_in_path(),
627 <a name="lib_xwrap"><h3>lib/xwrap.c</h3>
629 <p>Functions prefixed with the letter x call perror_exit() when they hit
630 errors, to eliminate common error checking. This prints an error message
631 and the strerror() string for the errno encountered.</p>
633 <p>You can intercept this exit by assigning a setjmp/longjmp buffer to
634 toys.rebound (set it back to zero to restore the default behavior).
635 If you do this, cleaning up resource leaks is your problem.</p>
638 <li><b>void xstrncpy(char *dest, char *src, size_t size)</b></li>
639 <li><b>void xexit(void)</b></li>
640 <li><b>void *xmalloc(size_t size)</b></li>
641 <li><b>void *xzalloc(size_t size)</b></li>
642 <li><b>void *xrealloc(void *ptr, size_t size)</b></li>
643 <li><b>char *xstrndup(char *s, size_t n)</b></li>
644 <li><b>char *xstrdup(char *s)</b></li>
645 <li><b>char *xmprintf(char *format, ...)</b></li>
646 <li><b>void xprintf(char *format, ...)</b></li>
647 <li><b>void xputs(char *s)</b></li>
648 <li><b>void xputc(char c)</b></li>
649 <li><b>void xflush(void)</b></li>
650 <li><b>pid_t xfork(void)</b></li>
651 <li><b>void xexec_optargs(int skip)</b></li>
652 <li><b>void xexec(char **argv)</b></li>
653 <li><b>pid_t xpopen(char **argv, int *pipes)</b></li>
654 <li><b>int xpclose(pid_t pid, int *pipes)</b></li>
655 <li><b>void xaccess(char *path, int flags)</b></li>
656 <li><b>void xunlink(char *path)</b></li>
657 <li><p><b>int xcreate(char *path, int flags, int mode)<br />
658 int xopen(char *path, int flags)</b></p>
660 <p>The xopen() and xcreate() functions open an existing file (exiting if
661 it's not there) and create a new file (exiting if it can't).</p>
663 <p>They default to O_CLOEXEC so the filehandles aren't passed on to child
664 processes. Feed in O_CLOEXEC to disable this.</p>
666 <li><p><b>void xclose(int fd)</b></p>
668 <p>Because NFS is broken, and won't necessarily perform the requested
669 operation (and report the error) until you close the file. Of course, this
670 being NFS, it's not guaranteed to report the error there either, but it
673 <p>Nothing else ever reports an error on close, everywhere else it's just a
674 VFS operation freeing some resources. NFS is _special_, in a way that
675 other network filesystems like smbfs and v9fs aren't..</p>
677 <li><b>int xdup(int fd)</b></li>
678 <li><p><b>size_t xread(int fd, void *buf, size_t len)</b></p>
680 <p>Can return 0, but not -1.</p>
682 <li><p><b>void xreadall(int fd, void *buf, size_t len)</b></p>
684 <p>Reads the entire len-sized buffer, retrying to complete short
685 reads. Exits if it can't get enough data.</p></li>
687 <li><p><b>void xwrite(int fd, void *buf, size_t len)</b></p>
689 <p>Retries short writes, exits if can't write the entire buffer.</p></li>
691 <li><b>off_t xlseek(int fd, off_t offset, int whence)</b></li>
692 <li><b>char *xgetcwd(void)</b></li>
693 <li><b>void xstat(char *path, struct stat *st)</b></li>
694 <li><p><b>char *xabspath(char *path, int exact) </b></p>
696 <p>After several years of
697 <a href=http://landley.net/notes-2007.html#18-06-2007>wrestling</a>
698 <a href=http://landley.net/notes-2008.html#19-01-2008>with</a> realpath(),
699 I broke down and <a href=http://landley.net/notes-2012.html#20-11-2012>wrote
700 my own</a> implementation that doesn't use the one in libc. As I explained:
702 <blockquote><p>If the path ends with a broken link,
703 readlink -f should show where the link points to, not where the broken link
704 lives. (The point of readlink -f is "if I write here, where would it attempt
705 to create a file".) The problem is, realpath() returns NULL for a path ending
706 with a broken link, and I can't beat different behavior out of code locked
707 away in libc.</p></blockquote>
711 <li><b>void xchdir(char *path)</b></li>
712 <li><b>void xchroot(char *path)</b></li>
714 <li><p><b>struct passwd *xgetpwuid(uid_t uid)<br />
715 struct group *xgetgrgid(gid_t gid)<br />
716 struct passwd *xgetpwnam(char *name)</b></p>
723 <li><b>void xsetuser(struct passwd *pwd)</b></li>
724 <li><b>char *xreadlink(char *name)</b></li>
725 <li><b>char *xreadfile(char *name, char *buf, off_t len)</b></li>
726 <li><b>int xioctl(int fd, int request, void *data)</b></li>
727 <li><b>void xpidfile(char *name)</b></li>
728 <li><b>void xsendfile(int in, int out)</b></li>
729 <li><b>long xparsetime(char *arg, long units, long *fraction)</b></li>
730 <li><b>void xregcomp(regex_t *preg, char *regex, int cflags)</b></li>
733 <a name="lib_lib"><h3>lib/lib.c</h3>
734 <p>Eight gazillion common functions:</p>
737 <li><b>void verror_msg(char *msg, int err, va_list va)</b></li>
738 <li><b>void error_msg(char *msg, ...)</b></li>
739 <li><b>void perror_msg(char *msg, ...)</b></li>
740 <li><b>void error_exit(char *msg, ...)</b></li>
741 <li><b>void perror_exit(char *msg, ...)</b></li>
742 <li><b>ssize_t readall(int fd, void *buf, size_t len)</b></li>
743 <li><b>ssize_t writeall(int fd, void *buf, size_t len)</b></li>
744 <li><b>off_t lskip(int fd, off_t offset)</b></li>
745 <li><b>int mkpathat(int atfd, char *dir, mode_t lastmode, int flags)</b></li>
746 <li><b>struct string_list **splitpath(char *path, struct string_list **list)</b></li>
747 <li><b>struct string_list *find_in_path(char *path, char *filename)</b></li>
748 <li><b>long atolx(char *numstr)</b></li>
749 <li><b>long atolx_range(char *numstr, long low, long high)</b></li>
750 <li><b>int numlen(long l)</b></li>
751 <li><b>int stridx(char *haystack, char needle)</b></li>
752 <li><b>int strstart(char **a, char *b)</b></li>
753 <li><b>off_t fdlength(int fd)</b></li>
754 <li><b>char *readfile(char *name, char *ibuf, off_t len)</b></li>
755 <li><b>void msleep(long miliseconds)</b></li>
756 <li><b>int64_t peek_le(void *ptr, unsigned size)</b></li>
757 <li><b>int64_t peek_be(void *ptr, unsigned size)</b></li>
758 <li><b>int64_t peek(void *ptr, unsigned size)</b></li>
759 <li><b>void poke(void *ptr, uint64_t val, int size)</b></li>
760 <li><b>void loopfiles_rw(char **argv, int flags, int permissions, int failok,</b></li>
761 <li><b>void loopfiles(char **argv, void (*function)(int fd, char *name))</b></li>
762 <li><b>char *get_rawline(int fd, long *plen, char end)</b></li>
763 <li><b>char *get_line(int fd)</b></li>
764 <li><b>int wfchmodat(int fd, char *name, mode_t mode)</b></li>
765 <li><b>static void tempfile_handler(int i)</b></li>
766 <li><b>int copy_tempfile(int fdin, char *name, char **tempname)</b></li>
767 <li><b>void delete_tempfile(int fdin, int fdout, char **tempname)</b></li>
768 <li><b>void replace_tempfile(int fdin, int fdout, char **tempname)</b></li>
769 <li><b>void crc_init(unsigned int *crc_table, int little_endian)</b></li>
770 <li><b>int terminal_size(unsigned *xx, unsigned *yy)</b></li>
771 <li><b>int yesno(char *prompt, int def)</b></li>
772 <li><b>void generic_signal(int sig)</b></li>
773 <li><b>void sigatexit(void *handler)</b></li>
774 <li><b>int sig_to_num(char *pidstr)</b></li>
775 <li><b>char *num_to_sig(int sig)</b></li>
776 <li><b>mode_t string_to_mode(char *modestr, mode_t mode)</b></li>
777 <li><b>void mode_to_string(mode_t mode, char *buf)</b></li>
778 <li><b>void names_to_pid(char **names, int (*callback)(pid_t pid, char *name))</b></li>
779 <li><b>int human_readable(char *buf, unsigned long long num)</b></li>
782 <h3>lib/portability.h</h3>
784 <p>This file is automatically included from the top of toys.h, and smooths
785 over differences between platforms (hardware targets, compilers, C libraries,
786 operating systems, etc).</p>
788 <p>This file provides SWAP macros (SWAP_BE16(x) and SWAP_LE32(x) and so on).</p>
790 <p>A macro like SWAP_LE32(x) means "The value in x is stored as a little
791 endian 32 bit value, so perform the translation to/from whatever the native
792 32-bit format is". You do the swap once on the way in, and once on the way
793 out. If your target is already little endian, the macro is a NOP.</p>
795 <p>The SWAP macros come in BE and LE each with 16, 32, and 64 bit versions.
796 In each case, the name of the macro refers to the _external_ representation,
797 and converts to/from whatever your native representation happens to be (which
798 can vary depending on what you're currently compiling for).</p>
800 <a name="lib_llist"><h3>lib/llist.c</h3>
802 <p>Some generic single and doubly linked list functions, which take
803 advantage of a couple properties of C:</p>
806 <li><p>Structure elements are laid out in memory in the order listed, and
807 the first element has no padding. This means you can always treat (typecast)
808 a pointer to a structure as a pointer to the first element of the structure,
809 even if you don't know anything about the data following it.</p></li>
811 <li><p>An array of length zero at the end of a structure adds no space
812 to the sizeof() the structure, but if you calculate how much extra space
813 you want when you malloc() the structure it will be available at the end.
814 Since C has no bounds checking, this means each struct can have one variable
815 length array.</p></li>
818 <p>Toybox's list structures always have their <b>next</b> pointer as
819 the first entry of each struct, and singly linked lists end with a NULL pointer.
820 This allows generic code to traverse such lists without knowing anything
821 else about the specific structs composing them: if your pointer isn't NULL
822 typecast it to void ** and dereference once to get the next entry.</p>
824 <p><b>lib/lib.h</b> defines three structure types:</p>
826 <li><p><b>struct string_list</b> - stores a single string (<b>char str[0]</b>),
827 memory for which is allocated as part of the node. (I.E. llist_traverse(list,
828 free); can clean up after this type of list.)</p></li>
830 <li><p><b>struct arg_list</b> - stores a pointer to a single string
831 (<b>char *arg</b>) which is stored in a separate chunk of memory.</p></li>
833 <li><p><b>struct double_list</b> - has a second pointer (<b>struct double_list
834 *prev</b> along with a <b>char *data</b> for payload.</p></li>
837 <b>List Functions</b>
840 <li><p>void *<b>llist_pop</b>(void **list) - advances through a list ala
841 <b>node = llist_pop(&list);</b> This doesn't modify the list contents,
842 but does advance the pointer you feed it (which is why you pass the _address_
843 of that pointer, not the pointer itself).</p></li>
845 <li><p>void <b>llist_traverse</b>(void *list, void (*using)(void *data)) -
846 iterate through a list calling a function on each node.</p></li>
848 <li><p>struct double_list *<b>dlist_add</b>(struct double_list **llist, char *data)
849 - append an entry to a circular linked list.
850 This function allocates a new struct double_list wrapper and returns the
851 pointer to the new entry (which you can usually ignore since it's llist->prev,
852 but if llist was NULL you need it). The argument is the ->data field for the
854 <ul><li><p>void <b>dlist_add_nomalloc</b>(struct double_list **llist,
855 struct double_list *new) - append existing struct double_list to
856 list, does not allocate anything.</p></li></ul>
859 <b>List code trivia questions:</b>
862 <li><p><b>Why do arg_list and double_list contain a char * payload instead of
863 a void *?</b> - Because you always have to typecast a void * to use it, and
864 typecasting a char * does no harm. Since strings are the most common
865 payload, and doing math on the pointer ala
866 "(type *)(ptr+sizeof(thing)+sizeof(otherthing))" requires ptr to be char *
867 anyway (at least according to the C standard), defaulting to char * saves
871 <li><p><b>Why do the names ->str, ->arg, and ->data differ?</b> - To force
872 you to keep track of which one you're using, calling free(node->str) would
873 be bad, and _failing_ to free(node->arg) leaks memory.</p></li>
875 <li><p><b>Why does llist_pop() take a void * instead of void **?</b> -
876 because the stupid compiler complains about "type punned pointers" when
877 you typecast and dereference on the same line,
878 due to insane FSF developers hardwiring limitations of their optimizer
879 into gcc's warning system. Since C automatically typecasts any other
880 pointer type to and from void *, the current code works fine. It's sad that it
881 won't warn you if you forget the &, but the code crashes pretty quickly in
884 <li><p><b>How do I assemble a singly-linked-list in order?</b> - use
885 a double_list, dlist_add() your entries, and then break the circle with
886 <b>list->prev->next = NULL;</b> when done.</li>
889 <a name="lib_args"><h3>lib/args.c</h3>
891 <p>Toybox's main.c automatically parses command line options before calling the
892 command's main function. Option parsing starts in get_optflags(), which stores
893 results in the global structures "toys" (optflags and optargs) and "this".</p>
895 <p>The option parsing infrastructure stores a bitfield in toys.optflags to
896 indicate which options the current command line contained, and defines FLAG
897 macros code can use to check whether each argument's bit is set. Arguments
898 attached to those options are saved into the command's global structure
899 ("this"). Any remaining command line arguments are collected together into
900 the null-terminated array toys.optargs, with the length in toys.optc. (Note
901 that toys.optargs does not contain the current command name at position zero,
902 use "toys.which->name" for that.) The raw command line arguments get_optflags()
903 parsed are retained unmodified in toys.argv[].</p>
905 <p>Toybox's option parsing logic is controlled by an "optflags" string, using
906 a format reminiscent of getopt's optargs but with several important differences.
907 Toybox does not use the getopt()
908 function out of the C library, get_optflags() is an independent implementation
909 which doesn't permute the original arguments (and thus doesn't change how the
910 command is displayed in ps and top), and has many features not present in
911 libc optargs() (such as the ability to describe long options in the same string
912 as normal options).</p>
914 <p>Each command's NEWTOY() macro has an optflags string as its middle argument,
915 which sets toy_list.options for that command to tell get_optflags() what
916 command line arguments to look for, and what to do with them.
917 If a command has no option
918 definition string (I.E. the argument is NULL), option parsing is skipped
919 for that command, which must look at the raw data in toys.argv to parse its
920 own arguments. (If no currently enabled command uses option parsing,
921 get_optflags() is optimized out of the resulting binary by the compiler's
922 --gc-sections option.)</p>
924 <p>You don't have to free the option strings, which point into the environment
925 space (I.E. the string data is not copied). A TOYFLAG_NOFORK command
926 that uses the linked list type "*" should free the list objects but not
927 the data they point to, via "llist_free(TT.mylist, NULL);". (If it's not
928 NOFORK, exit() will free all the malloced data anyway unless you want
929 to implement a CONFIG_TOYBOX_FREE cleanup for it.)</p>
931 <h4>Optflags format string</h4>
933 <p>Note: the optflags option description string format is much more
934 concisely described by a large comment at the top of lib/args.c.</p>
936 <p>The general theory is that letters set optflags, and punctuation describes
937 other actions the option parsing logic should take.</p>
939 <p>For example, suppose the command line <b>command -b fruit -d walrus -a 42</b>
940 is parsed using the optflags string "<b>a#b:c:d</b>". (I.E.
941 toys.which->options="a#b:c:d" and argv = ["command", "-b", "fruit", "-d",
942 "walrus", "-a", "42"]). When get_optflags() returns, the following data is
943 available to command_main():
946 <li><p>In <b>struct toys</b>:
948 <li>toys.optflags = 13; // FLAG_a = 8 | FLAG_b = 4 | FLAG_d = 1</li>
949 <li>toys.optargs[0] = "walrus"; // leftover argument</li>
950 <li>toys.optargs[1] = NULL; // end of list</li>
951 <li>toys.optc = 1; // there was 1 leftover argument</li>
952 <li>toys.argv[] = {"-b", "fruit", "-d", "walrus", "-a", "42"}; // The original command line arguments
956 <li><p>In <b>union this</b> (treated as <b>long this[]</b>):
958 <li>this[0] = NULL; // -c didn't get an argument this time, so get_optflags() didn't change it and toys_init() zeroed "this" during setup.)</li>
959 <li>this[1] = (long)"fruit"; // argument to -b</li>
960 <li>this[2] = 42; // argument to -a</li>
965 <p>If the command's globals are:</p>
975 <p>That would mean TT.c == NULL, TT.b == "fruit", and TT.a == 42. (Remember,
976 each entry that receives an argument must be a long or pointer, to line up
977 with the array position. Right to left in the optflags string corresponds to
978 top to bottom in GLOBALS().</p>
980 <p>Put globals not filled out by the option parsing logic at the end of the
981 GLOBALS block. Common practice is to list the options one per line (to
982 make the ordering explicit, first to last in globals corresponds to right
983 to left in the option string), then leave a blank line before any non-option
986 <p><b>long toys.optflags</b></p>
988 <p>Each option in the optflags string corresponds to a bit position in
989 toys.optflags, with the same value as a corresponding binary digit. The
990 rightmost argument is (1<<0), the next to last is (1<<1) and so on. If
991 the option isn't encountered while parsing argv[], its bit remains 0.</p>
993 <p>Each option -x has a FLAG_x macro for the command letter. Bare --longopts
994 with no corresponding short option have a FLAG_longopt macro for the long
995 optionname. Commands enable these macros by #defining FOR_commandname before
996 #including <toys.h>. When multiple commands are implemented in the same
997 source file, you can switch flag contexts later in the file by
998 #defining CLEANUP_oldcommand and #defining FOR_newcommand, then
999 #including <generated/flags.h>.</p>
1001 <p>Options disabled in the current configuration (wrapped in
1002 a USE_BLAH() macro for a CONFIG_BLAH that's switched off) have their
1003 corresponding FLAG macro set to zero, so code checking them ala
1004 if (toys.optargs & FLAG_x) gets optimized out via dead code elimination.
1005 #defining FORCE_FLAGS when switching flag context disables this
1006 behavior: the flag is never zero even if the config is disabled. This
1007 allows code shared between multiple commands to use the same flag
1008 values, as long as the common flags match up right to left in both option
1012 the optflags string "abcd" would parse the command line argument "-c" to set
1013 optflags to 2, "-a" would set optflags to 8, "-bd" would set optflags to
1014 6 (I.E. 4|2), and "-a -c" would set optflags to 10 (2|8). To check if -c
1015 was encountered, code could test: if (toys.optflags & FLAG_c) printf("yup");
1016 (See the toys/examples directory for more.)</p>
1018 <p>Only letters are relevant to optflags, punctuation is skipped: in the
1019 string "a*b:c#d", d=1, c=2, b=4, a=8. The punctuation after a letter
1020 usually indicate that the option takes an argument.</p>
1022 <p>Since toys.optflags is an unsigned int, it only stores 32 bits. (Which is
1023 the amount a long would have on 32-bit platforms anyway; 64 bit code on
1024 32 bit platforms is too expensive to require in common code used by almost
1025 all commands.) Bit positions beyond the 1<<31 aren't recorded, but
1026 parsing higher options can still set global variables.</p>
1028 <p><b>Automatically setting global variables from arguments (union this)</b></p>
1030 <p>The following punctuation characters may be appended to an optflags
1031 argument letter, indicating the option takes an additional argument:</p>
1034 <li><b>:</b> - plus a string argument, keep most recent if more than one.</li>
1035 <li><b>*</b> - plus a string argument, appended to a linked list.</li>
1036 <li><b>@</b> - plus an occurrence counter (stored in a long)</li>
1037 <li><b>#</b> - plus a signed long argument.
1038 <li><b>-</b> - plus a signed long argument defaulting to negative (start argument with + to force a positive value).</li>
1039 <li><b>.</b> - plus a floating point argument (if CFG_TOYBOX_FLOAT).</li>
1040 <ul>The following can be appended to a float or double:
1041 <li><b><123</b> - error if argument is less than this</li>
1042 <li><b>>123</b> - error if argument is greater than this</li>
1043 <li><b>=123</b> - default value if argument not supplied</li>
1047 <p><b>GLOBALS</b></p>
1049 <p>Options which have an argument fill in the corresponding slot in the global
1050 union "this" (see generated/globals.h), treating it as an array of longs
1051 with the rightmost saved in this[0]. As described above, using "a*b:c#d",
1052 "-c 42" would set this[0] = 42; and "-b 42" would set this[1] = "42"; each
1053 slot is left NULL if the corresponding argument is not encountered.</p>
1055 <p>This behavior is useful because the LP64 standard ensures long and pointer
1056 are the same size. C99 guarantees structure members will occur in memory
1057 in the same order they're declared, and that padding won't be inserted between
1058 consecutive variables of register size. Thus the first few entries can
1059 be longs or pointers corresponding to the saved arguments.</p>
1061 <p>See toys/example/*.c for longer examples of parsing options into the
1064 <p><b>char *toys.optargs[]</b></p>
1066 <p>Command line arguments in argv[] which are not consumed by option parsing
1067 (I.E. not recognized either as -flags or arguments to -flags) will be copied
1068 to toys.optargs[], with the length of that array in toys.optc.
1069 (When toys.optc is 0, no unrecognized command line arguments remain.)
1070 The order of entries is preserved, and as with argv[] this new array is also
1071 terminated by a NULL entry.</p>
1073 <p>Option parsing can require a minimum or maximum number of optargs left
1074 over, by adding "<1" (read "at least one") or ">9" ("at most nine") to the
1075 start of the optflags string.</p>
1077 <p>The special argument "--" terminates option parsing, storing all remaining
1078 arguments in optargs. The "--" itself is consumed.</p>
1080 <p><b>Other optflags control characters</b></p>
1082 <p>The following characters may occur at the start of each command's
1083 optflags string, before any options that would set a bit in toys.optflags:</p>
1086 <li><b>^</b> - stop at first nonoption argument (for nice, xargs...)</li>
1087 <li><b>?</b> - allow unknown arguments (pass non-option arguments starting
1088 with - through to optargs instead of erroring out).</li>
1089 <li><b>&</b> - the first argument has imaginary dash (ala tar/ps. If given twice, all arguments have imaginary dash.)</li>
1090 <li><b><</b> - must be followed by a decimal digit indicating at least this many leftover arguments are needed in optargs (default 0)</li>
1091 <li><b>></b> - must be followed by a decimal digit indicating at most this many leftover arguments allowed (default MAX_INT)</li>
1094 <p>The following characters may be appended to an option character, but do
1095 not by themselves indicate an extra argument should be saved in this[].
1096 (Technically any character not recognized as a control character sets an
1097 optflag, but letters are never control characters.)</p>
1100 <li><b>^</b> - stop parsing options after encountering this option, everything else goes into optargs.</li>
1101 <li><b>|</b> - this option is required. If more than one marked, only one is required.</li>
1104 <p>The following may be appended to a float or double:</p>
1107 <li><b><123</b> - error if argument is less than this</li>
1108 <li><b>>123</b> - error if argument is greater than this</li>
1109 <li><b>=123</b> - default value if argument not supplied</li>
1112 <p>Option parsing only understands <>= after . when CFG_TOYBOX_FLOAT
1113 is enabled. (Otherwise the code to determine where floating point constants
1114 end drops out. When disabled, it can reserve a global data slot for the
1115 argument so offsets won't change, but will never fill it out.) You can handle
1116 this by using the USE_BLAH() macros with C string concatenation, ala:</p>
1118 <blockquote>"abc." USE_TOYBOX_FLOAT("<1.23>4.56=7.89") "def"</blockquote>
1120 <p><b>--longopts</b></p>
1122 <p>The optflags string can contain long options, which are enclosed in
1123 parentheses. They may be appended to an existing option character, in
1124 which case the --longopt is a synonym for that option, ala "a:(--fred)"
1125 which understands "-a blah" or "--fred blah" as synonyms.</p>
1127 <p>Longopts may also appear before any other options in the optflags string,
1128 in which case they have no corresponding short argument, but instead set
1129 their own bit based on position. So for "(walrus)#(blah)xy:z", "command
1130 --walrus 42" would set toys.optflags = 16 (-z = 1, -y = 2, -x = 4, --blah = 8)
1131 and would assign this[1] = 42;</p>
1133 <p>A short option may have multiple longopt synonyms, "a(one)(two)", but
1134 each "bare longopt" (ala "(one)(two)abc" before any option characters)
1135 always sets its own bit (although you can group them with +X).</p>
1137 <p>Only bare longopts have a FLAG_ macro with the longopt name
1138 (ala --fred would #define FLAG_fred). Other longopts use the short
1139 option's FLAG macro to test the toys.optflags bit.</p>
1141 <p>Options with a semicolon ";" after their data type can only set their
1142 corresponding GLOBALS() entry via "--longopt=value". For example, option
1143 string "x(boing): y" would set TT.x if it saw "--boing=value", but would
1144 treat "--boing value" as setting FLAG_x in toys.optargs, leaving TT.x NULL,
1145 and keeping "value" in toys.optargs[]. (This lets "ls --color" and
1146 "ls --color=auto" both work.)</p>
1148 <p><b>[groups]</b></p>
1150 <p>At the end of the option string, square bracket groups can define
1151 relationships between existing options. (This only applies to short
1152 options, bare --longopts can't participate.)</p>
1154 <p>The first character of the group defines the type, the remaining
1155 characters are options it applies to:</p>
1158 <li><b>-</b> - Exclusive, switch off all others in this group.</li>
1159 <li><b>+</b> - Inclusive, switch on all others in this group.</li>
1160 <li><b>!</b> - Error, fail if more than one defined.</li>
1163 <p>So "abc[-abc]" means -ab = -b, -ba = -a, -abc = -c. "abc[+abc]"
1164 means -ab=-abc, -c=-abc, and "abc[!abc] means -ab calls error_exit("no -b
1165 with -a"). Note that [-] groups clear the GLOBALS option slot of
1166 options they're switching back off, but [+] won't set options it didn't see
1167 (just the optflags).</p>
1169 <p><b>whitespace</b></p>
1171 <p>Arguments may occur with or without a space (I.E. "-a 42" or "-a42").
1172 The command line argument "-abc" may be interepreted many different ways:
1173 the optflags string "cba" sets toys.optflags = 7, "c:ba" sets toys.optflags=4
1174 and saves "ba" as the argument to -c, and "cb:a" sets optflags to 6 and saves
1175 "c" as the argument to -b.</p>
1177 <p>Note that & changes whitespace handling, so that the command line
1178 "tar cvfCj outfile.tar.bz2 topdir filename" is parsed the same as
1179 "tar filename -c -v -j -f outfile.tar.bz2 -C topdir". Note that "tar -cvfCj
1180 one two three" would equal "tar -c -v -f Cj one two three". (This matches
1181 historical usage.)</p>
1183 <p>Appending a space to the option in the option string ("a: b") makes it
1184 require a space, I.E. "-ab" is interpreted as "-a" "-b". That way "kill -stop"
1185 differs from "kill -s top".</p>
1187 <p>Appending ; to a longopt in the option string makes its argument optional,
1188 and only settable with =, so in ls "(color):;" can accept "ls --color" and
1189 "ls --color=auto" without complaining that the first has no argument.</p>
1191 <a name="lib_dirtree"><h3>lib/dirtree.c</h3>
1193 <p>The directory tree traversal code should be sufficiently generic
1194 that commands never need to use readdir(), scandir(), or the fts.h family
1197 <p>These functions do not call chdir() or rely on PATH_MAX. Instead they
1198 use openat() and friends, using one filehandle per directory level to
1199 recurseinto subdirectories. (I.E. they can descend 1000 directories deep
1200 if setrlimit(RLIMIT_NOFILE) allows enough open filehandles, and the default
1201 in /proc/self/limits is generally 1024.)</p>
1203 <p>The basic dirtree functions are:</p>
1206 <li><p><b>dirtree_read(char *path, int (*callback)(struct dirtree node))</b> -
1207 recursively read directories, either applying callback() or returning
1208 a tree of struct dirtree if callback is NULL.</p></li>
1210 <li><p><b>dirtree_path(struct dirtree *node, int *plen)</b> - malloc() a
1211 string containing the path from the root of this tree to this node. If
1212 plen isn't NULL then *plen is how many extra bytes to malloc at the end
1215 <li><p><b>dirtree_parentfd(struct dirtree *node)</b> - return fd of
1216 containing directory, for use with openat() and such.</p></li>
1219 <p>The <b>dirtree_read()</b> function takes two arguments, a starting path for
1220 the root of the tree, and a callback function. The callback takes a
1221 <b>struct dirtree *</b> (from lib/lib.h) as its argument. If the callback is
1222 NULL, the traversal uses a default callback (dirtree_notdotdot()) which
1223 recursively assembles a tree of struct dirtree nodes for all files under
1224 this directory and subdirectories (filtering out "." and ".." entries),
1225 after which dirtree_read() returns the pointer to the root node of this
1228 <p>Otherwise the callback() is called on each entry in the directory,
1229 with struct dirtree * as its argument. This includes the initial
1230 node created by dirtree_read() at the top of the tree.</p>
1232 <p><b>struct dirtree</b></p>
1234 <p>Each struct dirtree node contains <b>char name[]</b> and <b>struct stat
1235 st</b> entries describing a file, plus a <b>char *symlink</b>
1236 which is NULL for non-symlinks.</p>
1238 <p>During a callback function, the <b>int data</b> field of directory nodes
1239 contains a dirfd (for use with the openat() family of functions). This is
1240 generally used by calling dirtree_parentfd() on the callback's node argument.
1241 For symlinks, data contains the length of the symlink string. On the second
1242 callback from DIRTREE_COMEAGAIN (depth-first traversal) data = -1 for
1243 all nodes (that's how you can tell it's the second callback).</p>
1245 <p>Users of this code may put anything they like into the <b>long extra</b>
1246 field. For example, "cp" and "mv" use this to store a dirfd for the destination
1247 directory (and use DIRTREE_COMEAGAIN to get the second callback so they can
1248 close(node->extra) to avoid running out of filehandles).
1249 This field is not directly used by the dirtree code, and
1250 thanks to LP64 it's large enough to store a typecast pointer to an
1251 arbitrary struct.</p>
1253 <p>The return value of the callback combines flags (with boolean or) to tell
1254 the traversal infrastructure how to behave:</p>
1257 <li><p><b>DIRTREE_SAVE</b> - Save this node, assembling a tree. (Without
1258 this the struct dirtree is freed after the callback returns. Filtering out
1259 siblings is fine, but discarding a parent while keeping its child leaks
1261 <li><p><b>DIRTREE_ABORT</b> - Do not examine any more entries in this
1262 directory. (Does not propagate up tree: to abort entire traversal,
1263 return DIRTREE_ABORT from parent callbacks too.)</p></li>
1264 <li><p><b>DIRTREE_RECURSE</b> - Examine directory contents. Ignored for
1265 non-directory entries. The remaining flags only take effect when
1266 recursing into the children of a directory.</p></li>
1267 <li><p><b>DIRTREE_COMEAGAIN</b> - Call the callback a second time after
1268 examining all directory contents, allowing depth-first traversal.
1269 On the second call, dirtree->data = -1.</p></li>
1270 <li><p><b>DIRTREE_SYMFOLLOW</b> - follow symlinks when populating children's
1271 <b>struct stat st</b> (by feeding a nonzero value to the symfollow argument of
1272 dirtree_add_node()), which means DIRTREE_RECURSE treats symlinks to
1273 directories as directories. (Avoiding infinite recursion is the callback's
1274 problem: the non-NULL dirtree->symlink can still distinguish between
1278 <p>Each struct dirtree contains three pointers (next, parent, and child)
1279 to other struct dirtree.</p>
1281 <p>The <b>parent</b> pointer indicates the directory
1282 containing this entry; even when not assembling a persistent tree of
1283 nodes the parent entries remain live up to the root of the tree while
1284 child nodes are active. At the top of the tree the parent pointer is
1285 NULL, meaning the node's name[] is either an absolute path or relative
1286 to cwd. The function dirtree_parentfd() gets the directory file descriptor
1287 for use with openat() and friends, returning AT_FDCWD at the top of tree.</p>
1289 <p>The <b>child</b> pointer points to the first node of the list of contents of
1290 this directory. If the directory contains no files, or the entry isn't
1291 a directory, child is NULL.</p>
1293 <p>The <b>next</b> pointer indicates sibling nodes in the same directory as this
1294 node, and since it's the first entry in the struct the llist.c traversal
1295 mechanisms work to iterate over sibling nodes. Each dirtree node is a
1296 single malloc() (even char *symlink points to memory at the end of the node),
1297 so llist_free() works but its callback must descend into child nodes (freeing
1298 a tree, not just a linked list), plus whatever the user stored in extra.</p>
1300 <p>The <b>dirtree_read</b>() function is a simple wrapper, calling <b>dirtree_add_node</b>()
1301 to create a root node relative to the current directory, then calling
1302 <b>handle_callback</b>() on that node (which recurses as instructed by the callback
1303 return flags). Some commands (such as chgrp) bypass this wrapper, for example
1304 to control whether or not to follow symlinks to the root node; symlinks
1305 listed on the command line are often treated differently than symlinks
1306 encountered during recursive directory traversal).
1308 <p>The ls command not only bypasses the wrapper, but never returns
1309 <b>DIRTREE_RECURSE</b> from the callback, instead calling <b>dirtree_recurse</b>() manually
1310 from elsewhere in the program. This gives ls -lR manual control
1311 of traversal order, which is neither depth first nor breadth first but
1312 instead a sort of FIFO order requried by the ls standard.</p>
1315 <h1><a href="#toys">Directory toys/</a></h1>
1317 <p>This directory contains command implementations. Each command is a single
1318 self-contained file. Adding a new command involves adding a single
1319 file, and removing a command involves removing that file. Commands use
1320 shared infrastructure from the lib/ and generated/ directories.</p>
1322 <p>Currently there are five subdirectories under "toys/" containing "posix"
1323 commands described in POSIX-2008, "lsb" commands described in the Linux
1324 Standard Base 4.1, "other" commands not described by either standard,
1325 "pending" commands awaiting cleanup (which default to "n" in menuconfig
1326 because they don't necessarily work right yet), and "example" code showing
1327 how toybox infrastructure works and providing template/skeleton files to
1328 start new commands.</p>
1330 <p>The only difference directory location makes is which menu the command
1331 shows up in during "make menuconfig", the directories are otherwise identical.
1332 Note that the commands exist within a single namespace at runtime, so you can't
1333 have the same command in multiple subdirectories. (The build tries to fail
1334 informatively when you do that.)</p>
1336 <p>There is one more sub-menus in "make menuconfig" containing global
1337 configuration options for toybox. This menu is defined in the top level
1340 <p>See <a href="#adding">adding a new command</a> for details on the
1341 layout of a command file.</p>
1343 <h2>Directory scripts/</h2>
1345 <p>Build infrastructure. The makefile calls scripts/make.sh for "make"
1346 and scripts/install.sh for "make install".</p>
1348 <p>There's also a test suite, "make test" calls make/test.sh, which runs all
1349 the tests in make/test/*. You can run individual tests via
1350 "scripts/test.sh command", or "TEST_HOST=1 scripts/test.sh command" to run
1351 that test against the host implementation instead of the toybox one.</p>
1353 <h3>scripts/cfg2files.sh</h3>
1355 <p>Run .config through this filter to get a list of enabled commands, which
1356 is turned into a list of files in toys via a sed invocation in the top level
1360 <h2>Directory kconfig/</h2>
1362 <p>Menuconfig infrastructure copied from the Linux kernel. See the
1363 Linux kernel's Documentation/kbuild/kconfig-language.txt</p>
1367 Better OLDTOY and multiple command explanation. From Config.in:
1369 <p>A command with multiple names (or multiple similar commands implemented in
1370 the same .c file) should have config symbols prefixed with the name of their
1371 C file. I.E. config symbol prefixes are NEWTOY() names. If OLDTOY() names
1372 have config symbols they must be options (symbols with an underscore and
1373 suffix) to the NEWTOY() name. (See generated/toylist.h)</p>
1376 <!--#include file="footer.html" -->
OSDN Git Service
