|
95f329b4
|
2020-03-10T21:07:34
|
|
azure: upgrade to newer hosted VM images
Azure is phasing out old images on March 23rd 2020, but we're currently
still using them. So let's upgrade images as following:
- Ubuntu 16.04 -> ubuntu-18.04
- macOS 10.13 -> macOS-10.15
- Hosted Windows machines -> vs2017-win2016
Each of them is currently the latest version. As the new Microsoft
Windows machine has upgraded to MSVS2017, we need to also adjust our
CMake generators to "Visual Studio 15 2017". As this CMake generator
doesn't accept the target platform name anymore, we instead need to set
it up via either "-A Win32" or "-A x64".
[1]: https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/
|
|
|
5ac33ced
|
2020-03-10T21:39:39
|
|
azure: docurium: fix build failure due to bumped CMake requirements
Our Docurium builds currently depend on Debian Jessie, which has CMake
v3.0 available. As rugged has bumped its CMake requirements to need at
least v3.5 now, the documentation build is thus failing.
Fix this by converting our Docurium Docker image to be based on Ubuntu
Bionic. We already do base all of our images on Ubuntu, so I don't see
any sense in using Debian here. If this was only to speed up builds, we
should just go all the way and use some minimal container like Alpine
anyway.
Also remove cache busters. As we're rebuilding the image every time,
it's we really don't need them at all.
|
|
|
c76c1e87
|
2020-03-10T20:39:09
|
|
azure: docker: consistently silence curl but show errors
We currently pass the "--silent" flag to most invocations of curl, but
in fact this does not only suppress the progress meter, but also any
errors. So let's also pass "--show-error", too.
|
|
dc55d0e8
|
2020-03-02T10:22:54
|
|
ci: provide globalsign certs for bionic
tls.mbed.org has neglected to send their full certificate chain. Add
their intermediate cert manually. 🙄
|
|
|
0f316d59
|
2020-03-01T14:42:03
|
|
ci: provide globalsign certs
tls.mbed.org has neglected to send their full certificate chain. Add
their intermediate cert manually. 🙄
|
|
|
6efe3d35
|
2020-02-19T11:34:55
|
|
azure: docker: set up HOME variable to fix Coverity builds
In commit 01a834066 (azure: docker: fix ARM builds by replacing gosu(1),
2020-02-18), we've switched our entrypoint from gosu(1) to use sudo(1)
instead to fix our ARM builds. The switch introduced an incompatibility
that now causes our Coverity builds to fail, as the "--preserve-env"
switch will also keep HOME at its current value. As a result, Coverity
now tries to set up its configuration directory in root's home
directory, which it naturally can't write to.
Fix the issue by adding the "--set-home" flag to sudo(1).
|
|
|
01a83406
|
2020-02-18T15:20:43
|
|
azure: docker: fix ARM builds by replacing gosu(1)
Our nightly builds are currently failing due to our ARM-based jobs.
These jobs crash immediately when entering the Docker container with a
exception thrown by Go's language runtime. As we're able to successfully
builds the Docker images in previous steps, it's unlikely to be a bug in
Docker itself. Instead, this exception is thrown by gosu(1), which is a
Go-based utility to drop privileges and run by our entrypoint.
Fix the issue by dropping gosu(1) in favor of sudo(1).
|
|
|
76b49caf
|
2020-02-18T15:20:08
|
|
azure: docker: synchronize Xenial/Bionic build instructions
Our two Docker build instructions for Xenial and Bionic have diverged a
bit. Let's re-synchronize them with each other to make them as similar
as possible.
|
|
|
f9985688
|
2020-02-18T15:17:45
|
|
azure: docker: detect errors when building images
The build step for our Docker images currently succeeds even if building
the Docker image fails due to missing && chains in the build script. Fix
this by adding them in.
|
|
|
68bfacb1
|
2020-02-18T15:17:17
|
|
azure: remove unused Linux setup script
Since migrating to Docker containings for our build and test
infrastructure, we do not use the "setup-linux.sh" script anymore.
Remove it to avoid any confusion.
|
|
|
49bb4237
|
2020-02-07T14:04:07
|
|
azure: test: silence termination message when killing git-daemon(1)
In order to properly tear down the test environment, we will kill
git-daemon(1) if we've exercised it. As git-daemon(1) is spawned as a
background process, it is still owned by the shell and thus killing it
later on will print a termination message to the shell's stderr, causing
Azure to report it as an error.
Fix this by disowning the background process.
|
|
|
fb03f02a
|
2020-02-07T13:56:36
|
|
azure: docker: avoid re-creating libgit2 home directory
The Docker entrypoint currently creates the libgit2 user with "useradd
--create-home". As we start the Docker container with two volumes
pointing into "/home/libgit2/", the home directory will already exist.
While useradd(1) copes with this just fine, it will print error messages
to stderr which end up as failures in our Azure pipelines.
Fix this by simply removing the "--create-home" parameter.
|
|
|
52cb4040
|
2020-02-07T13:52:07
|
|
azure: test: silence curl to not cause Azure to trop
Without the "--silent" parameter, curl will print a progress meter to
stderr. Azure has the nice feature of interpreting any output to stderr
as errors with a big red warning towards the end of the build. Let's
thus silence curl to not generate any misleading messages.
|
|
|
a3ec07d7
|
2020-02-07T13:34:18
|
|
azure: docker: pipe downloaded archives into tar(1) directly
When building dependencies for our Docker images, we first download the
sources to disk first, unpack them and finally remove the archive again.
This can be sped up by piping the downloading archive into tar(1)
directly to parallelize both tasks. Furthermore, let's silence curl(1)
to not print to status information to stderr, which tends to be
interpreted as errors by Azure Pipelines.
|
|
|
86c54cc8
|
2020-02-04T11:51:56
|
|
azure: coverity: fix Coverity builds due to various issues
There's several issues with our Coverity builds, like e.g. missing wget
in our containers. Simplify our Coverity pipeline and fix these issues.
|
|
|
ccffea6b
|
2020-02-04T11:47:16
|
|
azure: coverity: convert to use self-built containers
Back in commit 5a6740e7f (azure: build Docker images as part of the
pipeline, 2019-08-02), we have converted our pipelines to use self-built
Docker images to ease making changes to our Dockerfiles. The commit
didn't adjust our Coverity pipeline, though, so let's do this now.
|
|
|
b4eb0282
|
2020-02-04T11:46:05
|
|
azure: coverity: fix invalid syntax for Docker image
In commit bbc0b20bd (azure: fix Coverity's build due to wrong container
name, 2019-08-02), Coverity builds were fixed to use the correct
container names. Unfortunately, the "fix" completely broke our Coverity
builds due to using wrong syntax for the Docker task. Let's fix this by
using "imageName" instead of the Docker dict.
|
|
|
c51bd2f2
|
2020-02-04T12:15:56
|
|
azure: tests: reset FAILED status if flaky re-run succeeds
While we already do have logic to re-run flaky tests, the FAILED
variable currently does not get reset to "0". As a result, successful
reruns will still cause the test to be registered as failed.
Fix this by resetting the variable accordingly.
|
|
|
b33ad764
|
2020-02-04T11:26:57
|
|
azure: tests: re-run flaky proxy tests
The proxy tests regularly fail in our CI environment. Unfortunately,
this is expected due to the network layer. Thus, let's re-try the proxy
tests up to five times in case they fail.
|
|
|
7fd9b3f5
|
2020-01-01T20:48:15
|
|
ci: add NTLM tests
Download poxygit, a debugging git server, and clone from it using NTLM,
both IIS-style (with connection affinity) and Apache-style ("broken",
requiring constant reauthentication).
|
|
|
bf55facf
|
2019-10-25T12:24:34
|
|
tests: allow users to use expect/continue
|
|
|
9b96adf3
|
2019-11-22T18:03:50
|
|
docker build: add krb5 libraries and command-line tools
|
|
|
e5fb5fe5
|
2019-10-20T17:19:01
|
|
ci: perform SPNEGO tests
Attempt to obtain a Kerberos ticket from LIBGIT2.ORG and then clone the
Negotiate-protected site at test.libgit2.org with that ticket.
|
|
|
b8e00b98
|
2019-11-23T21:17:15
|
|
ci: cache docker layers
Our docker builds are getting expensive, let's cache some of this.
|
|
|
c64b7aaa
|
2019-11-23T20:38:30
|
|
ci: build our own valgrind
The valgrind in the PPA is broken and ignores `--exit-errorcode`.
Build and install our own.
|
|
|
fd831275
|
2019-11-23T12:40:46
|
|
ci: build shared libssh2
|
|
|
84807884
|
2019-11-23T12:40:02
|
|
ci: break dockerfile into stages
Use a multi-stage docker build so that we can cache early stages and not
need to download the apt-provided dependencies during every build (when
only later stages change).
|
|
|
7a3d04dc
|
2019-11-23T12:14:23
|
|
ci: don't delete the apt cache
Deleting the apt cache can be helpful for reducing the size of a
container, but since we don't push it anywhere, it only hinders our
ability to debug problems while working on the container. Keep it.
|
|
|
f592c737
|
2019-11-23T11:55:50
|
|
ci: don't install libssh2 since we build it
|
|
|
767990e9
|
2019-11-23T11:25:38
|
|
ci: show distribution information
The lsb-release command is missing on our images; just show the
information from the file instead of relying on it.
|
|
|
91ba65af
|
2019-11-23T10:58:38
|
|
ci: provide a default for xcode generator
Provide a sane default for `CMAKE_GENERATOR` in the build script so that
it can be invoked without having to set that in the environment.
|
|
3e862514
|
2019-09-21T22:50:27
|
|
azure: enable GSS on macOS builds
|
|
|
3c884cc3
|
2019-09-21T15:05:36
|
|
azure: avoid building and testing in Docker as root
Right now, all tests in libgit2's CI are being executed as root
user. As libgit2 will usually not run as a root user in "normal"
usecases and furthermore as there are tests that rely on the
ability to _not_ be able to create certain paths, let's instead
create an unprivileged user "libgit2" and use that across all
docker images.
|
|
|
48d23a8c
|
2019-08-02T12:36:19
|
|
azure: convert to use Ninja as build tool
While we were still supporting Trusty, using Ninja as a build tool would
have required us to first setup pip and then use it to install Ninja.
As a result, the speedups from using Ninja were drowned out by the
time required to install Ninja. But as we have deprecated Trusty now,
both Xenial and Bionic have recent versions of Ninja in their
repositories and thus we can now use Ninja.
|
|
|
4e07a205
|
2019-08-02T13:29:54
|
|
docker: fix Valgrind errors on Xenial by updating to v3.12.0
The Valgrind version shipped with Xenial has some bugs that keep our
tests from working due to bad interactions with openssl [1]. Fix this by
using the "hola-launchpad/valgrind" PPA that provides a newer version
for which the bug has been fixed.
[1]: https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1574437
|
|
|
3c59d451
|
2019-08-02T12:34:10
|
|
docker: use "--no-install-recommends" to reduce build time
Pass the flag "--no-install-recommends" to apt-get in order to trim down
the number of packages installed, both reducing build time and image
size. As this also causes some required packages to not be installed
anymore, add these explicitly to the set of packages installed.
|
|
|
39b7e8b0
|
2019-08-02T12:26:30
|
|
docker: convert apt-get to use best practices
Reformat both Xenial and Bionic's Dockerfiles to use best practices.
Most importantly, we now run `apt-get update` and `apt-get install` in
one step followed up by removing the package lists to speed up
installation and keep down the image size.
|
|
|
9f91d57e
|
2019-08-02T14:25:02
|
|
docker: install libssh2 1.8.2 on Xenial
While Xenial provides libssh2 in its repositories, it only has version
1.5.0 available. This version will unfortunately not be able to connect
to GitHub due to their removal of weak cryptographic standards [1]. To
still enable our CI to execute tests against GitHub, we thus have to
update the provided libssh2 version to a newer one.
Manually install libssh2 1.8.2 on Xenial. There's no need to do the same
for Bionic, as it already provides libssh2 1.8.0.
[1]: https://github.blog/2018-02-01-crypto-removal-notice/
|
|
|
253dbea2
|
2019-08-02T10:21:32
|
|
docker: install mbedTLS on both Bionic and Xenial
We're about to phase out support for Trusty, but neither Bionic nor
Xenial images provide the mbedTLS library that's available in Trusty.
Build them for both to pull them in line with Trusty.
|
|
|
bbc0b20b
|
2019-08-02T10:27:24
|
|
azure: fix Coverity's build due to wrong container name
The Coverity build is still referencing an old "trusty-openssl"
container that is not provided by either our own now-inlined images nor
by the libgit2/libgit2-docker repository.
Convert it to build and use Xenial images instead.
|
|
|
76327381
|
2019-08-02T10:50:11
|
|
azure: deprecate Trusty in favor of Xenial
Support for the LTS release Ubuntu 14.04 Trusty has been dropped in
April 2019, but Azure is still using Trusty as its primary platform to
build and test against. Let's deprecate it in favor of Xenial.
|
|
|
5a6740e7
|
2019-08-02T09:58:55
|
|
azure: build Docker images as part of the pipeline
The Docker images used for our continuous integration builds currently
live in the libgit2/libgit2-docker repository. To make any changes in
them, one has to make a PR there, get it reviewed, re-build the images
and publish them to Docker Hub. This process is slow and tedious, making
it harder than necessary to perform any updates to our Docker-based
build pipeline.
To fix this, we include all Dockerfiles used by Azure from the mentioned
repository and inline them into our own repo. Instead of having to
manually push them to the CI, it will now build the required containers
on each pull request, allowing much greater flexibility.
|
|
|
415ee616
|
2019-07-12T09:40:13
|
|
azure-pipelines: make gitdaemon tests work on Win32
On Win32 builds, the PID file created by git-daemon contained in invalid
PID that we were not able to kill afterwards. Somehow, it seems like the
contained PID was wrapped in braces. Consequentially, kill(1) failed and
thus caused the build to error.
Fix this by directly grabbing the PID of the spawned git-daemon process.
|
|
|
f867bfa8
|
2019-07-20T18:35:46
|
|
azure: skip SSH tests on Win32 platforms
On Win32 build hosts, we do not have an SSH daemon readily available and
thus cannot perform the SSH tests. Let's skip the tests to not let Azure
Pipelines fail.
|
|
|
0cda5252
|
2019-07-19T12:09:58
|
|
azure: use bash scripts across all platforms
Right now, we maintain semantically equivalent build scripts in
both Bash and Powershell to support both Windows and non-Windows
hosts. Azure Pipelines supports Bash on Windows, too, via Git for
Windows, and as such it's not really required to maintain the
Powershell scripts at all.
Remove them to reduce our own maintenance burden.
|
|
|
1be4f896
|
2019-07-19T12:07:59
|
|
azure: avoid executing compiler if there is none
Until now, we always had the CC variable defined in the build.sh
pipeline. But as we're about to migrate the Windows jobs to Bash, as
well, those will not have CC defined and thus we cannot use "$CC" to
determine the compiler version.
Fix this by only executing "$CC" if the variable was set.
|
|
|
8e356f48
|
2019-07-20T18:35:20
|
|
azure: explicitly specify CMake generator
We currently specify the CMake generator as part of the CMAKE_OPTIONS
variable. This is fine in the current setup, but during the conversion
to drop PowerShell scripts this will prove problematic for all
generators that have spaces in their names due to quoting issues.
Convert to use an explicit CMAKE_GENERATOR variable that makes it easier
to get quoting right.
|
|
|
443df2df
|
2019-06-24T16:27:00
|
|
azure: replace mingw setup with bash
We're about to phase out our Powershell scripts as Azure
Pipelines does in fact support Bash scripts on all platforms. As
a preparatory step, let's replace our MinGW setup script with a
Bash script.
|
|
|
d8e85d57
|
2019-06-27T15:01:24
|
|
azure: fix building in MinGW via Bash
Azure Pipelines supports bash tasks on Windows hosts due to it always
having Git for Windows included. To support this, the Git for Window
directory is added to the PATH environment to make the bash shell
available for execution. Unfortunately, this breaks CMake with the MinGW
generator, as it has sanity checks to verify that no bash executable is
in the PATH. So we can either remove Git for Windows from the path, but
then we're unable to execute bash jobs. Or we can add it to the path,
but then we're unable to execute CMake with the MinGW generator.
Let's re-model how we set the PATH environment. Instead of setting up
PATH for the complete build job, we now set a variable "BUILD_PATH" for
the job. This variable is only being used when executing CMake so that
it encounters a sanitizied PATH environment without GfW's bash shell.
|
|
|
ffac520e
|
2019-06-24T16:19:35
|
|
azure: move build scripts into "azure-pipelines" directory
Since we have migrated to Azure Pipelines, we have deprecated and
subsequentally removed all infrastructure for AppVeyor and
Travis. Thus it doesn't make a lot of sense to have the split
between "ci/" and "azure-pipelines/" directories anymoer, as
"azure-pipelines/" is essentially our only CI.
Move all CI scripts into the "azure-pipelines/" directory to have
everything centrally located and to remove clutter in the
top-level directory.
|
|
|
d827b11b
|
2019-06-28T13:20:54
|
|
tests: execute leak checker via CTest directly
Right now, we have an awful hack in our test CI setup that extracts the
test command from CTest's output and then prepends the leak checker.
This is dependent on non-machine-parseable output from CMake and also
breaks on various ocassions, like for example when we have spaces in the
current path or when the path contains backslashes. Both conditions may
easily be triggered on Win32 systems, and in fact they do break our
Azure Pipelines builds.
Remove the awful hack in favour of a new CMake build option
"USE_LEAK_CHECKER". If specifying e.g. "-DUSE_LEAK_CHECKER=valgrind",
then we will set up all tests to be run under valgrind. Like this, we
can again simply execute ctest without needing to rely on evil sourcery.
|
|
|
270fd807
|
2019-07-18T13:44:10
|
|
azure: compile one Windows platform with the WinHTTP SHA1 backend
We currently have no job that compiles libgit2 with the WinHTTP backend
for SHA1. Due to this, a compile error has been introduced and not
noticed for several months. Change the x86 MSVC job to use the HTTPS
backend for SHA1. The x86 job was chosen with no particular reason.
|
|
|
94fc83b6
|
2019-06-13T16:48:35
|
|
cmake: Modulize our TLS & hash detection
The interactions between `USE_HTTPS` and `SHA1_BACKEND` have been
streamlined. Previously we would have accepted not quite working
configurations (like, `-DUSE_HTTPS=OFF -DSHA1_BACKEND=OpenSSL`) and, as
the OpenSSL detection only ran with `USE_HTTPS`, the link would fail.
The detection was moved to a new `USE_SHA1`, modeled after `USE_HTTPS`,
which takes the values "CollisionDetection/Backend/Generic", to better
match how the "hashing backend" is selected, the default (ON) being
"CollisionDetection".
Note that, as `SHA1_BACKEND` is still used internally, you might need to
check what customization you're using it for.
|
|
|
afb04a95
|
2019-05-21T14:03:04
|
|
ci: use a mix of regex backends
Explicitly enable the `builtin` regex backend and the PCRE backend for
some Linux builds.
|
|
|
c7e5eca6
|
2019-02-28T10:46:26
|
|
Revert "foo"
This reverts commit 1fe3fa5e59818c851d50efc6563db5f8a5d7ae9b.
|
|
|
1fe3fa5e
|
2019-02-28T10:44:34
|
|
foo
|
|
|
3f823c2b
|
2019-02-14T00:00:06
|
|
ci: enable hard deprecation
Enable hard deprecation in our builds to ensure that we do not call
deprecated functions internally.
|
|
|
ef91917f
|
2019-02-14T09:19:32
|
|
ci: skip ssh tests on macOS nightly
Like 811c1c0f8f80521dccc746a7bff180cd77a783ff, disable the SSH tests on
macOS until we can resolve the newly introduced infrastructure issues.
|
|
|
0cf5b6b1
|
2019-01-28T10:48:49
|
|
ci: ignore coverity failures in nightly runs
Coverity is back but it's only read-only! Agh. Just allow it to fail
and not impact the overall job run.
|
|
|
08d71f72
|
2019-01-27T22:46:07
|
|
ci: return coverity to the nightlies
|
|
|
4a798a91
|
2018-10-28T17:57:53
|
|
nightly: use latest images, not test images
|
|
|
1ebf3a7d
|
2019-01-19T00:34:55
|
|
ci: only run invasive tests during nightly runs
|
|
|
3c6d1979
|
2019-01-11T02:06:41
|
|
ci: move coverity in its own pipeline
Since Coverity is down for a unspecified timeframe, isolate it from the
"hosted" nightlies.
|
|
|
f195c385
|
2018-10-26T14:10:13
|
|
nightly: the path to yaml templates is relative
Don't prefix the path to the yaml templates - the nightly template
itself is already in the `azure-pipelines` directory. Instead, just
use the relative path.
|
|
|
be5a2ae2
|
2018-10-25T23:19:42
|
|
ci: run all the jobs during nightly builds
Instead of running the oddball builds, run all the builds (the ones
that we always run during PR validation and CI) during a nightly
build for increased coverage.
|
|
|
d82800e8
|
2018-10-21T09:31:42
|
|
ci: use bionic for non-amd64 builds
Use Bionic so that we have a modern libssh2 (for communicating with
GitHub). We've ported fixes to our Trusty-based amd64 images, but
maintaining patches for multiple platforms is heinous.
|
|
|
b244ea79
|
2018-10-21T09:24:08
|
|
ci: introduce nightly x86 linux builds
|
|
|
0e521abd
|
2018-10-21T09:15:24
|
|
ci: introduce nightly arm docker builds
Use multiarch arm32 and arm64 docker images to run Xenial-based images
for those platforms. We can support all the tests on ARM32 and 64
_except_ the proxy-based tests. Our proxy on ARM seems regrettably
unstable, either due to some shoddy dependencies (with native code?)
or the JREs themselves.
Run these platforms as part of our nightly builds; do not run them
during pull request or CI validation.
|
|
|
4ec597dc
|
2018-10-21T09:12:43
|
|
ci: move configuration yaml to its own directory
As the number of each grow, separate the CI build scripts from
the YAML definitions.
|