Edit
kc3-lang/automake/HACKING
Karl Berry
- HACKING
-
============================================================================ = This file * This file attempts to describe the conventions to use when hacking automake. * After git checkout from Savannah, you can do an initial build with, e.g.: ./bootstrap && ./configure --prefix=/tmp/amdev && make ============================================================================ = Administrivia * The correct response to most actual bugs is to write a new test case which demonstrates the bug. Then fix the bug, rerun the test suite, and check everything in. Run the test suite in parallel or it takes ages. * If you incorporate a change from somebody on the net: - First, if it is not a tiny change, you must make sure they have signed the appropriate paperwork. - Second, add their name and email address to THANKS. * If a change fixes or adds a test, mention the test in the commit message. If a change fixes a bug registered in the Automake debbugs tracker, mention the bug number in the commit message, using the short url form, like https://bugs.gnu.org/1234. See section below about commit messages. * If a person or people report a new bug, mention their name(s) in the commit message that fixes or exposes the bug, and add a line for them in THANKS. * When documenting a non-trivial idiom or example in the manual, be sure to add a test case for it, and to reference such test case from a proper Texinfo comment. * Some files in the automake package are not owned by automake (many by gnulib, https://gnu.org/s/gnulib); these files are listed in the $(FETCHFILES) variable in maintainer/maint.mk. They should never be edited here. All but two of them can be updated from respective upstreams with "make fetch" (this should be done especially before releases). The only exceptions are help2man (install the current version and copy in by hand) and 'lib/COPYING' (from FSF), which should be updated by hand whenever the GPL gets updated (which shouldn't happen that often anyway :-). Conversely, automake holds the master copy of a number of files that are copied into other projects, such as install-sh, mdate-sh, Channels.pm and ChannelDefs.pm, and plenty more; grep for bug-automake in lib/* for most of the scripts, and see <gnulib>/config/srclist.txt and <autoconf>/build-aux/fetch.pl for lists of such files on two notable receiving ends. Do your best not to break them. * All changes from the last release that are not trivial bug fixes should be mentioned in NEWS. * Changes which are potentially controversial, require a non-trivial plan, or must be implemented gradually with a roadmap spanning several releases (either minor or major) should be discussed on the list, and have a proper entry in the PLANS directory. This entry should be always committed in the "maint" branch, even if the change it deals with is only for the master branch, or a topic branch. Usually, in addition to this, it is useful to open a "wishlist" report on the Automake debbugs tracker, to keep the idea more visible, and have the discussions surrounding it easily archived in a central place. =========================================================================== = Setting the development environment * In development, ./GNUmakefile is used, not (the generated) ./Makefile. Run make V=1 to see the commands that are run. * The required and optional dependencies used by Automake and its test suite can be automatically fetched using the GNU Guix package manager with the following command: guix environment automake --ad-hoc \ gettext help2man texinfo libtool flex bison dejagnu zip icedtea \ python gcc-toolchain gfortran pkg-config vala For other environments, you'll need to install the equivalent. * To run the bin/automake or bin/aclocal scripts in a checkout, it is necessary to set PERL5LIB, as in: am=/path/to/automake/checkout env PERL5LIB=$am/lib $am/bin/automake --help env PERL5LIB=$am/lib $am/bin/aclocal --help * If you need to test different Python versions, pyenv is probably the most convenient way at this writing. https://github.com/pyenv ============================================================================ = Naming * Automake's convention is that internal AC_SUBSTs and make variables should be named with a leading 'am__', and internally generated targets should be named with a leading 'am--'. This convention, although in place from at least February 2001, isn't yet universally used. But all new code should use it. We used to use '_am_' as the prefix for an internal AC_SUBSTs. However, it turns out that NEWS-OS 4.2R complains if a Makefile variable begins with the underscore character. Yay for them. We changed the target naming convention just to be safe. * If you'd like to read some general background on automake and the other GNU autotools, you might try this not-too-long web page (not affiliated with GNU): https://www.linux.com/news/best-practices-autotools/ ============================================================================ = Editing '.am' files * For variables, always use $(...) and not ${...}. * Prefer ':' over 'true', mostly for consistency with existing code. * Use '##' comments liberally. Comment anything even remotely unusual, and even usual things, to help future readers of the code understand. * Never use basename or dirname. Instead, use sed. * Do not use 'cd' within backquotes, use '$(am__cd)' instead. Otherwise the directory name may be printed, depending on CDPATH. More generally, do not ever use plain 'cd' together with a relative directory that does not start with a dot, or you might end up in one computed with CDPATH. * For install and uninstall rules, if a loop is required, it should be silent. Then the body of the loop itself should print each "important" command it runs. The printed commands should be preceded by a single space. * Ensure install rules do not create any installation directory where nothing is to be actually installed. See automake bug#11030. ============================================================================ = Editing conventions * Indent using GNU style. For historical reasons, the perl code contains portions indented using Larry Wall's style (perl-mode's default), and other portions using the GNU style (cperl-mode's default). Write new code using GNU style. * Don't use & for function calls, unless really required. The use of & prevents prototypes from being checked; it's also uncommon in modern Perl code. (Perl prototypes are unlike function prototypes in other languages, so understand what they do. They are not required.) * For automake.texi: - After and during editing, run make info for error checking, and make pdf to catch TeX-only errors. - After adding a new node, you can update the @detailmenu with M-x texinfo-master-menu (not tested with current Emacs releases, but hopefully it still works). - You'll probably have already added it to the higher-level menu under which the new node was added, since that's necessary to run "make info". ============================================================================ = Automake versioning and compatibility scheme * There are three kinds of automake releases: - new major releases (e.g., 2.0, 5.0) - new minor releases (e.g., 1.14, 2.1) - micro a.k.a. "bug-fixing" releases (e.g., 1.13.2, 2.0.1, 3.5.17). A new major release should have the major version number bumped, and the minor and micro version numbers reset to zero. A new minor release should have the major version number unchanged, the minor version number bumped, and the micro version number reset to zero. Finally, a new micro version should have the major and minor version numbers unchanged, and the micro version number bumped by one. For example, the first minor version after 1.13.2 will be 1.14; the first bug-fixing version after 1.14 that will be 1.14.1; the first new major version after all such releases will be 2.0; the first bug-fixing version after 2.0 will be 2.0.1; and a further bug-fixing version after 2.0.1 will be 2.0.2. * Micro releases should be just bug-fixing releases; no new features should be added, and ideally, only trivial bugs, recent regressions, or documentation issues should be addressed by them. On the other hand, it's OK to include testsuite work and even testsuite refactoring in a micro version, since a regression there is not going to annoy or inconvenience Automake users, but only the Automake developers. * Minor releases can introduce new "safe" features, do non-trivial but mostly safe code clean-ups, and even add new runtime warnings (rigorously non-fatal). But they shouldn't include any backward incompatible change, nor contain any potentially destabilizing refactoring or sweeping change, nor introduce new features whose implementation might be liable to cause bugs or regressions in existing code. However, it might be acceptable to introduce very limited and localized backward-incompatibility, *only* if that is necessary to fix non-trivial bugs, address serious performance issues, or greatly enhance usability. But please, do this sparsely and rarely! * Major releases can introduce backward incompatibility (albeit such incompatibility should be announced well in advance, and a smooth transition plan prepared for them), and try more risky and daring refactorings and code cleanups. Still, backward incompatibility is extremely undesirable and should be avoided at all costs. * For more information, refer to the extensive discussion associated with automake bug#13578. ============================================================================ = Working with git * To regenerate dependent files created by aclocal and automake, use the 'bootstrap' script. It uses the code from the source tree, so the resulting files (aclocal.m4 and Makefile.in) should be the same as you would get if you install this version of automake and use it to generate those files. * To get a faithful and correct rebuild, run: ./bootstrap && ./config.status --recheck && make clean all * Usually, it is best to run against the latest stable versions of Autoconf, Libtool, etc., since that is what most users will do. However, sometimes it may be necessary to use the development versions due to bugs fixed, etc. Whatever is first in PATH wins. (Some background: https://bugs.gnu.org/11347) * The Automake git tree currently carries three basic branches: 'master', 'next' and 'maint'. In practice, for quite a few years now, as of this writing in 2023, we have been using only the master branch, due to lack of time and desire to deal with anything but fixing bugs. The branch information in the items below and above should thus be taken with a large dose of salt. * The 'master' branch is where the development of the next release takes place. It should be kept in a stable, almost-releasable state, to simplify testing and deploying of new minor version. Note that this is not a hard rule, and such "stability" is not expected to be absolute (emergency releases are cut from the 'maint' branch anyway). * When planning a release a dedicated branch should be created and after the release is done, the release branch is to be merged both into the 'master' branch and the 'maint' branch. * Besides merges from release branches, the 'maint' branch can contain fixes for regressions, trivial bugs, or documentation issues, that will be part of an emergency regression-fixing or security releases. As a consequence it should always be kept in a releasable state and no "active" development should be done whatsoever. * The 'next' branch is reserved for the development of the next major release. Experimenting a little is OK here, but don't let the branch grow too unstable; if you need to do exploratory programming or over-arching change, you should use a dedicated topic branch, and only merge that back once it is reasonably stable. * The 'master' branch should be kept regularly merged into the 'next' branch. It is advisable to merge only after a set of related commits have been applied, to avoid introducing too much noise in the history. * There may be a number of longer-lived feature branches for new developments. They should be based off of a common ancestor of all active branches to which the feature should or might be merged later. * When merging, prefer 'git merge --log' over plain 'git merge', so that a later 'git log' gives an indication of which actual patches were merged even when they don't appear early in the list. * The 'master', 'maint' and 'next' branches should not be rewound, i.e., should always fast-forward, except maybe for privacy issues. For feature branches, the announcement for the branch should document the rewinding policy. If a topic branch is expected to be rewound, it is good practice to put it in the 'experimental/*' namespace; for example, a rewindable branch dealing with Vala support could be named like "experimental/vala-work". * If you need to trivially fix a change after a commit, e.g., a typo in the NEWS entry, edit the file and then: git commit --amend --no-edit NEWS git commit --amend --date="$(date -R)" # update date of commit * If you want to just completely lose your local changes: git fetch && git reset --hard origin && git checkout && git submodule update (the submodule stuff is for gnulib). * The preferred way to create a patch to mail around: git format-patch --stdout -1 >/tmp/some-patch.diff which can then be applied with: git am mboxfile ============================================================================ = Writing a good commit message * Here is the general format that Automake's commit messages are expected to follow. See the further points below for clarifications and minor corrections. topic: brief description (this is the "summary line"). <reference to relevant bugs, if any> Here goes a more detailed explanation of why the commit is needed, and a general overview of what it does, and how. This section should almost always be provided, possibly only with the exception of obvious fixes or very trivial changes. And if the detailed explanation is quite long or detailed, you can want to break it in more paragraphs. Then you can add references to relevant mailing list discussions (if any), with proper links. But don't take this as an excuse for writing incomplete commit messages! The "distilled" conclusions reached in such discussions should have been placed in the paragraphs above. Finally, here you can thank people that motivated or helped the change. So, thanks to John Doe for bringing up the issue, and to J. Random Hacker for providing suggestions and testing the patch. <detailed list of touched files> * To choose the topic label, please review the previous commits in the git log or the ChangeLog file from a release tarball. There are no hard-and-fast rules; the aim is a usable description. So script names, Makefile targets, and more are all viable. Some examples: aclocal automake build cosmetics doc maint python release test ylwrap * The <detailed list of touched files> should usually be provided (but for short or trivial changes), and should follow the GNU guidelines for ChangeLog entries (described explicitly in the GNU Coding Standards); it might be something of this sort: * some/file (func1): Improved frobnication. (func2): Adjusted accordingly. * another/file (foo, bar): Likewise. * t/foo.sh: New test. * t/list-of-tests.mk (handwritten_TESTS): Add it. * doc/automake.texi (Some Node): Document it. * NEWS: Mention it. * If your commit fixes an automake bug registered in the tracker (say numbered 1234), you should put the following line after the summary line: Fixes automake bug https://bugs.gnu.org/1234. * If your commit is just related to the given bug report, but does not fix it, you might want to add a line like this instead: Related to automake bug https://bugs.gnu.org/1234. * When referring to older commits, use 'git describe' output as pointer. But also try to identify the given commit by date and/or summary line if possible. Examples: Since yesterday's commit, v1.11-2019-g4d2bf42, ... ... removed in commit 'v1.11-1674-g02e9072' of 2012-01-01, "dist: ditch support for lzma"... * If the commit is a tiny change that is exempt from copyright paperwork, the commit message should contain a separate line after the detailed list of touched files like the following: Copyright-paperwork-exempt: yes * Generally write the commit message in present tense. ============================================================================ = Test suite * See file 't/README' for more information. * Use "make check" and "make maintainer-check" liberally. It is often useful to set VERBOSE=1 for more reporting on what is being done. * Export the 'keep_testdirs' environment variable to "yes" to keep *.dir test directories for successful tests also. * Use Perl coverage information to ensure your new code is thoroughly tested by your new tests. * When there is a new Perl release, and at Automake releases, run the tests with PERL5OPT=-Mwarnings=FATAL,all in the environment. This makes Perl warnings into fatal errors, which will (presumably) cause tests to fail when Perl gives a new warning for something in our code. * To run the tests, you should install expect, shar, language compilers, gettext macros. Anything you don't install won't be tested. The test suite will report on tests skipped due to software not available. Look at the list and install anything feasible. * Run the test suite in parallel (e.g., "make -j12 check"), both so it doesn't take forever and because that is what most users will do. You can also parallelize the makes that run inside each test with, e.g.: make check AM_TESTSUITE_MAKE="make -j$(( 1*$(nproc) + 1 ))" If you like, try different levels of parallelization to see what runs the fastest on your machine. We'll use -j12 in our examples; adjust as needed. * To summarize, here is a typical make check invocation: make -j12 VERBOSE=1 check keep_testdirs=yes * Then check t/test-suite.log for the overall results. The directory t/TESTNAME.dir is where the work will be left, if the test fails. * You can run a single test, with, e.g., make check TESTS='t/aclocal-acdir.sh' where t/aclocal-acdir.sh can be any t/*.sh test, including a new one you are writing. You may want to add --no-print-dir to silence GNU make about the many cd commands, and/or env VERBOSE=1 to get more information about what make is running. * Sometimes you may want to see when each test starts running, not only when they complete. This can be done by setting TESTS_ENVIRONMENT: make -j12 VERBOSE=1 TESTS_ENVIRONMENT='echo RUNNING: "$$f";' check * Run "make maintainer-check" before commit. Watch out for trailing spaces. Probably useful to run it more verbosely: make AM_V_GEN= AM_V_at= VERBOSE=1 maintainer-check * After "make -j12 check" succeeds. Run "make -j12 distcheck" before pushing a commit, since that exercises yet more of the code. * To unsilence Automake's "pretty" output for debugging, see the "Unsilencing Automake" node in the manual; in short, run make --debug=p. For the Automake test suite in particular, add VERBOSE=1. * To set up a new test, first write the test file in t/good-name.sh. Choose a name that fits with existing ones, as best you can devise. - You'll likely want to copy material from an existing test, which is fine and good; depending on how much is copied, it may be useful to mention the other test(s) you used. - Nevertheless, start the copyright year list in the new file fresh, with the current year. - You can run the new test on its own with make check TESTS='t/good-name.sh' - During development of a new test, it can be useful to end it with "exit 33" (or any random value) to force it to fail even when it would otherwise succeed, so you can inspect the test.dir/test.log and other test.dir/* files to be sure things worked as expected. - At some point before releasing, add the test to the appropriate variable in t/list-of-tests.mk, most likely the (alphabetical) handwritten_TESTS. * Some hints for debugging test failures or trying to understand the (complex) test infrastructure. - The t/ax/ dir contains most of the test infrastructure files. - You may want to try a simple test just to exercise the setup; t/amassign.sh is such a test. For a simple test that runs automake twice (sometimes useful in finding concurrency failures), try t/nodef.sh. - To trace the (complex) test initialization code, change the set -e to set -ex (or whatever you wish) in t/ax/test-init.sh. Automake itself enables -x for the *.log and test-suite.log files, in am_test_setup in t/ax/test-lib.sh. You may want to change to -vx. - If you want to run the individual commands of a test one by one in a shell, you have to reproduce the test environment. Start a subshell and: PATH=$am/bin:$am/t/ax:$PATH # where $am is your automake source checkout set +u # if needed; automake cannot handle this "nounset" option . test-init.sh # beginning of test Then each command you type will be executed in the test environment, including -x being enabled. Each test prints the PATH value at the beginning. - If you need to debug a test which fails under installcheck, you need to invoke make with am_running_installcheck=yes, as is done by the installcheck-testsuite target in t/local.mk. - If you want to see the (complex shell) commands that make is running, despite all of Automake's attempt at silence, run (GNU) make --debug=p ... other make args ... ============================================================================ = Bug tracker * Automake uses the debbugs instance at https://bugs.gnu.org. Email from the tracker is sent to bug-automake@gnu.org, and vice versa. Ditto automake-patches@gnu.org. (See https://gnu.org/s/automake for all the mailing lists; if you're working on Automake, please subscribe.) * To see all open bugs resp. patches (and recently closed ones): https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=automake https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=automake-patches * For a full search form, initialized to show only bugs tagged confirmed: https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake;include=tags%3Aconfirmed * We use the "confirmed" tag to mean bugs that have been reviewed, are evidently valid, but no one is currently working or plans to work on them. Thus they are good candidates for new volunteers to get involved. * To download the so-called maintainer mbox for a bug, say 12345: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=12345;mbox=yes;mboxmaint=yes Only the maintainer mbox consistently has the bug# prefix on Subject: lines, so it is the most usable, as far as we know. * To close a bug, you can mail (or bcc) to 12345-done@debbugs.gnu.org. It's best to do this with a note saying that the bug is being closed because the fix was pushed, or whatever the reason. * To add tags "help" and "confirmed" to bug 12345, mail to control@debbugs.gnu.org with a one-line body: tags 12345 + help confirmed Available tags: patch wontfix moreinfo unreproducible notabug fixed * To merge bugs 12345 and 56789, mail to control@debbugs.gnu.org with a one-line body: merge 12345 56789 * In general, all bug operations are done by mail. Reference information: https://debbugs.gnu.org/server-control.html # dev info ============================================================================ = Release procedure * The steps outlined here are meant to be followed for all releases, whether stable, beta, alpha, or other. Where differences are expected, they will be explicitly mentioned. * Fetch new versions of the files that are maintained elsewhere by running "make fetch". help2man has to be updated separately. If any files in the automake repository get updated, commit them and rerun the testsuite. * Ensure that the copyright notices of the distributed files are up to date. The maintainer-only target "update-copyright" can help with this. * Check NEWS; specifically, ensure that all the substantive differences from the last release are included. Include bug numbers or mailing list references wherever relevant. Update the version number and include the release date. * Update the AC_INIT version number in configure.ac. Leading up to a minor release (x.y), say 1.17, pretests should be numbered from 1.16.90. Even numbers (1.16.90, 1.16.92, ...) should be pretest releases; odd numbers (1.16.91, 1.16.93, ...) should be only development versions. Leading up to a micro release (x.y.z), say 1.17.1, we would have 1.17.0.91 for the initial post-release development version, 1.17.0.92 for the first pretest, etc. Before 1.17 (July 2024), we used suffixed letters for pretests, as is traditional, but deficient sorting algorithms did not like that. The code in lib/Automake/Version.pm, along with the regexp in lib/Automake/Options.pm, has to be updated to understand any change in version number schemes. The test t/pm/Version[.pl] comprehensively checks valid and invalid version strings. * If making a minor or major release (1.x), also update APIVERSION in configure.ac. But don't change APIVERSION for micro releases (1.x.y) or pretests. E.g., APIVERSION=1.17 for the pretest 1.17.90, even though it's leading up to 1.18. * grep $newversion NEWS # ensure that the NEWS file has the new version. * Create an announcement message with "make announcement". It's useful to do this early, because carefully reading the list of NEWS can easily bring to light things that still need to be worked on. But don't worry about every detail, because the below invocation of announce-gen will create another announcement to be merged with this. * Run these commands, in this order (as mentioned, adjust -j as desired): make bootstrap make -j12 check keep_testdirs=yes make maintainer-check make -j12 check-no-trailing-backslash-in-recipes make -j12 check-cc-no-c-o make -j12 distcheck # regular distcheck make -j12 distcheck AM_TESTSUITE_MAKE="make -j12" # parallelize makes The resulting automake-$VERSION.tar.* files are what will be the release. You can run "git clean -fdx" before invoking the bootstrap, to ensure a completely clean rebuild. However, it must be done carefully, because that command will remove *all* the files that are not tracked by git! Run git status first to ensure all removals are ok. * git push all changed files. * Run "make git-tag-release". This will run the maintainer checks, verify that the local git repository and working tree are clean and up-to-date, and create a proper signed git tag for the release, based on the contents of $(VERSION). It's highly undesirable (at least) to change anything after the tag is made, so don't run this until everything is committed and pushed and you're ready to release. (If you need to delete the tag locally because you ran it too soon: git tag --delete v$VERSION.) * Create additional info for the announcement with the announce-gen script that is part of gnulib. It requires the new release tarball, and the already-tagged repository. pkg=automake prever=1.16 # 1.17 newver=1.16.94 # 1.18 reltype=alpha # stable (possibly also beta) host=`if test $reltype = stable; then echo ftp; else echo alpha; fi` gpgkey=0x... # gpg --fingerprint $gnulib/build-aux/announce-gen \ --release-type=$reltype \ --package-name=$pkg \ --previous-version=$prever \ --current-version=$newver \ --gpg-key-id=$gpgkey \ --news=NEWS \ --url-directory=https://$host.gnu.org/gnu/$pkg and merge this with the just-written announcement file. * Run "make git-upload-release". For pretest releases: "DEVEL_SNAPSHOT=1 make git-upload-release". This will first verify that you are releasing from a tagged version and that the local git repository and working tree are clean and up-to-date, and will then run "make dist" to create the tarballs, and invoke the 'gnupload' script sign and upload them to the correct locations. If you need to sign with a non-default key, you can use "make GNUPLOADFLAGS='--user KEY' git-upload-release". There can be a delay of up to several minutes before the new files appear on the server. * For stable releases you also need to update the manuals on www.gnu.org. - Check links: make html make checklinkx # various perl prerequisites, see contrib/checklinkx or use: <https://validator.w3.org/checklink> - Generate manuals (with the help of the standard gendocs.sh script): make web-manual The ready-to-be-uploaded manuals (in several formats) will be left in the 'doc/web-manual' directory. - Commit the updated manuals to web CVS. This requires the "cvsu" utility from the "cvsutils" package. make web-manual-update If your local username is different from your username at Savannah, you'll have to override the 'CVS_USER' make variable accordingly; for example: make web-manual-update CVS_USER=slattarini * Update version number in configure.ac to next alpha number. Rerun ./bootstrap and commit and push. * Send the announcement generated in the earlier steps at least to <autotools-announce@gnu.org> and <automake@gnu.org>. If the release is a stable one, the announcement also goes to <info-gnu@gnu.org>; if it is an alpha or beta release, announcement should be sent also to <platform-testers@gnu.org>, to maximize the possibility of early testing on the widest variety of systems. Finally, copy an abridged version of the announcement into the NEWS feed at: <https://savannah.gnu.org/projects/automake>. Be sure to link a version to the complete announcement (from the version you sent to the automake list, as archived at <https://lists.gnu.org/archive/html/automake/>). --------- Copyright (C) 2003-2025 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>. Local Variables: mode: text End: