Gnulib Basics
=============

While portability across operating systems is not one of GNU's primary goals, it has helped introduce many people to the GNU system, and is worthwhile when it can be achieved at a low cost.  This collection helps lower that cost.

Gnulib is intended to be the canonical source for most of the important ``portability'' and/or common files for GNU projects.  These are files intended to be shared at the source level; Gnulib is not a typical library meant to be installed and linked against.  Thus, unlike most projects, Gnulib does not normally generate a source tarball distribution; instead, developers grab modules directly from the source repository.

The easiest, and recommended, way to do this is to use the gnulib-tool script.  Since there is no installation procedure for Gnulib, gnulib-tool needs to be run directly in the directory that contains the Gnulib source code.  You can do this either by specifying the absolute filename of gnulib-tool, or by using a symbolic link from a place inside your PATH to the gnulib-tool file of your preferred Gnulib checkout.  For example:

     $ ln -s $HOME/gnu/src/gnulib.git/gnulib-tool $HOME/bin/gnulib-tool

Git Checkout
============

Gnulib is available for anonymous checkout.  In any Bourne-shell the following should work:

     $ git clone https://git.savannah.gnu.org/git/gnulib.git

For a read-write checkout you need to have a login on savannah.gnu.org and be a member of the Gnulib project at https://savannah.gnu.org/projects/gnulib.  Then, instead of the URL https://git.savannah.gnu.org/git/gnulib.git, use the URL ssh://USER@git.savannah.gnu.org/srv/git/gnulib where USER is your login name on savannah.gnu.org.

git resources:

Overview:
     https://en.wikipedia.org/wiki/Git_(software)

Homepage:
     https://git-scm.com/

When you use git annotate or git blame with Gnulib, it's recommended that you use the -w option, in order to ignore massive whitespace changes that happened in 2009.

Keeping Up-to-date
==================

The best way to work with Gnulib is to check it out of git. To synchronize, you can use git pull.

Subscribing to the bug-gnulib@gnu.org mailing list will help you to plan when to update your local copy of Gnulib (which you use to maintain your software) from git.  You can review the archives, subscribe, etc., via https://lists.gnu.org/mailman/listinfo/bug-gnulib.

Sometimes, using an updated version of Gnulib will require you to use newer versions of GNU Automake or Autoconf.  You may find it helpful to join the autotools-announce mailing list to be advised of such changes.

Contributing to Gnulib
======================

All software here is copyrighted by the Free Software Foundation---you need to have filled out an assignment form for a project that uses the module for that contribution to be accepted here.

If you have a piece of code that you would like to contribute, please email bug-gnulib@gnu.org.

Generally we are looking for files that fulfill at least one of the following requirements:

  * If your .c and .h files define functions that are broken or missing on some other system, we should be able to include it.

  * If your functions remove arbitrary limits from existing functions (either under the same name, or as a slightly different name), we should be able to include it.

If your functions define completely new but rarely used functionality, you should probably consider packaging it as a separate library.

Gnulib licensing
----------------

Gnulib contains code both under GPL and LGPL.  Because several packages that use Gnulib are GPL, the files state they are licensed under GPL. However, to support LGPL projects as well, you may use some of the files under LGPL.  The ``License:'' information in the files under modules/ clarifies the real license that applies to the module source.

Keep in mind that if you submit patches to files in Gnulib, you should license them under a compatible license, which means that sometimes the contribution will have to be LGPL, if the original file is available under LGPL via a ``License: LGPL'' information in the projects' modules/ file.

Indent with spaces not TABs
---------------------------

We use space-only indentation in nearly all files. This includes all *.h, *.c, *.y files, except for the regex module. Makefile and ChangeLog files are excluded, since TAB characters are part of their format.

In order to tell your editor to produce space-only indentation, you can use these instructions.

  * For Emacs: Add these lines to your Emacs initialization file ($HOME/.emacs or similar):

          ;; In Gnulib, indent with spaces everywhere (not TABs).
          ;; Exceptions: Makefile and ChangeLog modes.
          (add-hook 'find-file-hook '(lambda ()
            (if (and buffer-file-name
                     (string-match "/gnulib\\>" (buffer-file-name))
                     (not (string-equal mode-name "Change Log"))
                     (not (string-equal mode-name "Makefile")))
                (setq indent-tabs-mode nil))))

  * For vi (vim): Add these lines to your $HOME/.vimrc file:

          " Don't use tabs for indentation. Spaces are nicer to work with.
          set expandtab

     For Makefile and ChangeLog files, compensate for this by adding this to your $HOME/.vim/after/indent/make.vim file, and similarly for your $HOME/.vim/after/indent/changelog.vim file:

          " Use tabs for indentation, regardless of the global setting.
          set noexpandtab

  * For Eclipse: In the ``Window|Preferences'' dialog (or ``Eclipse|Preferences'' dialog on Mac OS),

      1. Under ``General|Editors|Text Editors'', select the ``Insert spaces for tabs'' checkbox.

      2. Under ``C/C++|Code Style'', select a code style profile that has the ``Indentation|Tab policy'' combobox set to ``Spaces only'', such as the ``GNU [built-in]'' policy.

     If you use the GNU indent program, pass it the option --no-tabs.

How to add a new module
-----------------------

  * Add the header files and source files to lib/.

  * If the module needs configure-time checks, write an Autoconf macro for it in m4/MODULE.m4. See m4/README for details.

  * Write a module description modules/MODULE, based on modules/TEMPLATE.

  * If the module contributes a section to the end-user documentation, put this documentation in doc/MODULE.texi and add it to the ``Files'' section of modules/MODULE.  Most modules don't do this; they have only documentation for the programmer (= Gnulib user).  Such documentation usually goes into the lib/ source files.  It may also go into doc/; but don't add it to the module description in this case.

  * Add the module to the list in MODULES.html.sh.

You can test that a module builds correctly with:

     $ ./gnulib-tool --create-testdir --dir=/tmp/testdir module1 ... moduleN
     $ cd /tmp/testdir
     $ ./configure && make

Other things:

  * Check the license and copyright year of headers.

  * Check that the source code follows the GNU coding standards; see https://www.gnu.org/prep/standards.

  * Add source files to config/srclist* if they are identical to upstream and should be upgraded in Gnulib whenever the upstream source changes.

  * Include header files in source files to verify the function prototypes.

  * Make sure a replacement function doesn't cause warnings or clashes on systems that have the function.

  * Autoconf functions can use gl_* prefix. The AC_* prefix is for autoconf internal functions.

  * Build files only if they are needed on a platform.  Look at the alloca and fnmatch modules for how to achieve this.  If for some reason you cannot do this, and you have a .c file that leads to an empty .o file on some platforms (through some big #if around all the code), then ensure that the compilation unit is not empty after preprocessing.  One way to do this is to #include <stddef.h> or <stdio.h> before the big #if.

Portability guidelines
======================

Gnulib code is intended to be portable to a wide variety of platforms, not just GNU platforms.  Gnulib typically attempts to support a platform as long as it is still supported by its provider, even if the platform is not the latest version.  See Target Platforms.

Many Gnulib modules exist so that applications need not worry about undesirable variability in implementations.  For example, an application that uses the malloc module need not worry about malloc (0) returning a null pointer on some Standard C platforms; and glob users need not worry about glob silently omitting symbolic links to nonexistent files on some platforms that do not conform to POSIX.

Gnulib code is intended to port without problem to new hosts, e.g., hosts conforming to recent C and POSIX standards.  Hence Gnulib code should avoid using constructs that these newer standards no longer require, without first testing for the presence of these constructs. For example, because C11 made variable length arrays optional, Gnulib code should avoid them unless it first uses the vararrays module to check whether they are supported.

The following subsections discuss some exceptions and caveats to the general Gnulib portability guidelines.

C language versions
-------------------

Currently Gnulib assumes at least a freestanding C99 compiler, possibly operating with a C library that predates C99; with time this assumption will likely be strengthened to later versions of the C standard.  Old platforms currently supported include AIX 6.1, HP-UX 11i v1 and Solaris 10, though these platforms are rarely tested. Gnulib itself is so old that it contains many fixes for obsolete platforms, fixes that may be removed in the future.

Because of the freestanding C99 assumption, Gnulib code can include <float.h>, <limits.h>, <stdarg.h>, <stddef.h>, and <stdint.h> unconditionally; <stdbool.h> is also in the C99 freestanding list but is obsolescent as of C23.  Gnulib code can also assume the existence of <ctype.h>, <errno.h>, <fcntl.h>, <locale.h>, <signal.h>, <stdio.h>, <stdlib.h>, <string.h>, and <time.h>.  Similarly, many modules include <sys/types.h> even though it's not in the C standard; that's OK since <sys/types.h> has been around nearly forever.

Even if the include files exist, they may not conform to the C standard. However, GCC has a fixincludes script that attempts to fix most conformance problems.  Gnulib currently assumes include files largely conform to C99 or better.  People still using ancient hosts should use fixincludes or fix their include files manually.

Even if the include files conform, the library itself may not. For example, strtod and mktime have some bugs on some platforms. You can work around some of these problems by requiring the relevant modules, e.g., the Gnulib mktime module supplies a working and conforming mktime.

C99 features assumed by Gnulib
------------------------------

Although the C99 standard specifies many features, Gnulib code is conservative about using them, partly because Gnulib predates the widespread adoption of C99, and partly because many C99 features are not well-supported in practice.  C99 features that are reasonably portable nowadays include:

  * A declaration after a statement, or as the first clause in a for statement.

  * long long int.

  * <stdbool.h>, although Gnulib code no longer uses it directly, preferring plain bool via the bool module instead. See stdbool.h.

  * <stdint.h>, assuming the stdint-h module is used. See stdint.h.

  * Compound literals and designated initializers.

  * Empty arguments in macro calls.

  * Variadic macros.

     Note: The handling of __VA_ARGS__ in MSVC differs from the one in ISO C 99, see https://stackoverflow.com/questions/5134523/. But usually this matters only for macros that decompose __VA_ARGS__.

  * static inline functions.

  * __func__, assuming the func module is used.  See func.

  * The restrict qualifier, assuming AC_REQUIRE([AC_C_RESTRICT]) is used. This qualifier is sometimes implemented via a macro, so C++ code that uses Gnulib should avoid using restrict as an identifier.

  * Flexible array members (however, see the flexmember module).

C99 features avoided by Gnulib
------------------------------

Gnulib avoids some features even though they are standardized by C99, as they have portability problems in practice.  Here is a partial list of avoided C99 features.  Many other C99 features are portable only if their corresponding modules are used; Gnulib code that uses such a feature should require the corresponding module.

  * Variable length arrays (VLAs) or variably modified types, without checking whether __STDC_NO_VLA__ is defined. See the vararrays and vla modules.

  * Block-scope variable length arrays, without checking whether either GNULIB_NO_VLA or __STDC_NO_VLA__ is defined. This lets you define GNULIB_NO_VLA to pacify GCC when using its -Wvla-larger-than warnings option, and to avoid large stack usage that may have security implications. GNULIB_NO_VLA does not affect Gnulib's other uses of VLAs and variably modified types, such as array declarations in function prototype scope.

  * Converting to pointers via integer types other than intptr_t or uintptr_t.  Although the C standard says that values of these integer types, if they exist, should be convertible to and from intmax_t and uintmax_t without loss of information, on CHERI platforms such conversions result in integers that, if converted back to a pointer, cannot be dereferenced.

  * extern inline functions, without checking whether they are supported.  See extern inline.

  * Type-generic math functions.

  * Universal character names in source code.

  * <iso646.h>, since GNU programs need not worry about deficient source-code encodings.

  * Comments beginning with //.  This is mostly for style reasons.

Other portability assumptions made by Gnulib
--------------------------------------------

Gnulib code makes the following assumptions that go beyond what C and POSIX require:

  * Standard internal types like ptrdiff_t and size_t are no wider than long.  The GNU coding standards allow code to make this assumption, POSIX requires implementations to support at least one programming environment where this is true, and such environments are recommended for Gnulib-using applications.  When it is easy to port to non-POSIX platforms like MinGW where these types are wider than long, new Gnulib code should do so, e.g., by using ptrdiff_t instead of long.  However, it is not always that easy, and no effort has been made to check that all Gnulib modules work on MinGW-like environments.

  * int and unsigned int are at least 32 bits wide.  POSIX and the GNU coding standards both require this.

  * Signed integer arithmetic is two's complement.

     Previously, Gnulib code sometimes also assumed that signed integer arithmetic wraps around, but modern compiler optimizations sometimes do not guarantee this, and Gnulib code with this assumption is now considered to be questionable. See Integer Properties.

     Although some Gnulib modules contain explicit support for ones' complement and signed magnitude integer representations, which are allowed by C17 and earlier, these modules are the exception rather than the rule. All practical Gnulib targets use two's complement, which is required by C23.

  * There are no ``holes'' in integer values: all the bits of an integer contribute to its value in the usual way. In particular, an unsigned type and its signed counterpart have the same number of bits when you count the latter's sign bit. (As an exception, Gnulib code is portable to CHERI platforms even though this assumption is false for CHERI.)

  * Objects with all bits zero are treated as zero or as null pointers. For example, memset (A, 0, sizeof A) initializes an array A of pointers to null pointers.

  * The types intptr_t and uintptr_t exist, and pointers can be converted to and from these types without loss of information.

  * Addresses and sizes behave as if objects reside in a flat address space. In particular:

       * If two nonoverlapping objects have sizes S and T represented as ptrdiff_t or size_t values, then S + T cannot overflow.

       * A pointer P points within an object O if and only if (char *) &O <= (char *) P && (char *) P < (char *) (&O + 1).

       * Arithmetic on a valid pointer is equivalent to the same arithmetic on the pointer converted to uintptr_t, except that offsets are multiplied by the size of the pointed-to objects. For example, if P + I is a valid expression involving a pointer P and an integer I, then (uintptr_t) (P + I) == (uintptr_t) ((uintptr_t) P + I * sizeof *P). Similar arithmetic can be done with intptr_t, although more care must be taken in case of integer overflow or negative integers.

       * A pointer P has alignment A if and only if (uintptr_t) P % A is zero, and similarly for intptr_t. Although C23 says that portable code should instead test whether memalignment (P) % A is zero, memalignment is missing in many non-C23 systems, is not specified to work portably on pointers to atomic or volatile memory, and is likely a bit slower than the cast.

       * If an existing object has size S, and if T is sufficiently small (e.g., 8 KiB), then S + T cannot overflow. Overflow in this case would mean that the rest of your program fits into T bytes, which can't happen in realistic flat-address-space hosts.

       * Adding zero to a null pointer does not change the pointer. For example, 0 + (char *) NULL == (char *) NULL. Similarly, subtracting zero does not change a null pointer, and subtracting two null pointers results in zero. A future C standard is planned to require this behavior; see ``Allow zero length operations on null pointers (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3322.pdf)'', WG14 N3322 (2024-08-28).

Some system platforms violate these assumptions and are therefore not Gnulib porting targets.  See Unsupported Platforms.

High Quality
============

We develop and maintain a testsuite for Gnulib.  The goal is to have a 100% firm interface so that maintainers can feel free to update to the code in git at any time and know that their application will not break.  This means that before any change can be committed to the repository, a test suite program must be produced that exposes the bug for regression testing.

Stable Branches
---------------

In Gnulib, we don't use topic branches for experimental work. Therefore, occasionally a broken commit may be pushed in Gnulib. It does not happen often, but it does happen.

To compensate for this, Gnulib offers ``stable branches''.  These are branches of the Gnulib code that are maintained over some longer period (a year, for example) and include

  * bug fixes,

  * portability enhancements (to existing as well as to new platforms),

  * updates to config.guess and config.sub.

Not included in the stable branches are:

  * new features, such as new modules,

  * optimizations,

  * refactorings,

  * complex or risky changes in general,

  * updates to texinfo.tex,

  * documentation updates.

So far, we have seven stable branches:

stable-202501
     A stable branch that starts at the beginning of January 2025.

stable-202407
     A stable branch that starts at the beginning of July 2024.

stable-202401
     A stable branch that starts at the beginning of January 2024. It is no longer updated.

stable-202307
     A stable branch that starts at the beginning of July 2023. It is no longer updated.

stable-202301
     A stable branch that starts at the beginning of January 2023. It is no longer updated.

stable-202207
     A stable branch that starts at the beginning of July 2022. It is no longer updated.

stable-202201
     A stable branch that starts at the beginning of January 2022. It is no longer updated.

The two use-cases of stable branches are thus:

  * You want to protect yourself from occasional breakage in Gnulib.

  * When making a bug-fix release of your code, you can incorporate bug fixes in Gnulib, by pulling in the newest commits from the same stable branch that you were already using for the previous release.

Writing reliable code
---------------------

When compiling and testing Gnulib and Gnulib-using programs, certain compiler options can help improve reliability.  First of all, make it a habit to use -Wall in all compilation commands. Beyond that, the manywarnings module enables several forms of static checking in GCC and related compilers (see manywarnings).

For dynamic checking, you can run configure with CFLAGS options appropriate for your compiler.  For example:

     ./configure \
      CPPFLAGS='-Wall'\
      CFLAGS='-g3 -O2'\
     ' -D_FORTIFY_SOURCE=2'\
     ' -fsanitize=undefined'\
     ' -fsanitize-undefined-trap-on-error'

Here:

  * -D_FORTIFY_SOURCE=2 enables extra security hardening checks in the GNU C library.

  * -fsanitize=undefined enables GCC's undefined behavior sanitizer (ubsan), and

  * -fsanitize-undefined-trap-on-error causes ubsan to abort the program (through an ``illegal instruction'' signal).  This measure stops exploit attempts and also allows you to debug the issue.

Without the -fsanitize-undefined-trap-on-error option, -fsanitize=undefined causes messages to be printed, and execution continues after an undefined behavior situation. The message printing causes GCC-like compilers to arrange for the program to dynamically link to libraries it might not otherwise need. With GCC, instead of -fsanitize-undefined-trap-on-error you can use the -static-libubsan option to arrange for two of the extra libraries (libstdc++ and libubsan) to be linked statically rather than dynamically, though this typically bloats the executable and the remaining extra libraries are still linked dynamically.

It is also good to occasionally run the programs under valgrind (see Running self-tests under valgrind).

     [file join-gnu.texi not included]