kmx git

Commit	Date	Message
6a917c04	2020-06-28T15:51:43	Add CI support for Memory and UndefinedBehavior Sanitizers This change adds two new build targets: MSan and UBSan. This is because even though OSS-Fuzz is great and adds a lot of coverage, it only does that for the fuzz targets, so the rest of the codebase is not necessarily run with the Sanitizers ever :( So this change makes sure that MSan/UBSan warnings don't make it into the codebase. As part of this change, the Ubuntu focal container is introduced. It builds mbedTLS and libssh2 as debug libraries into /usr/local and as MSan-enabled libraries into /usr/local/msan. This latter part is needed because MSan requires the binary and all its dependent libraries to be built with MSan support so that memory allocations and deallocations are tracked correctly to avoid false positives.
4852d8da	2020-06-10T22:18:13	docker: don't take BASE as an argument The xenial image depends on ubuntu:xenial; the bionic one on ubuntu:bionic. No need for this to be a variable, that's just additional (unnecessary) state to manage in the CI setup(s).
c16ba496	2020-06-07T00:37:55	yo
80c6b0a2	2020-06-07T00:16:57	Update bionic
4a95ee8d	2020-06-07T00:16:39	Update xenial
c2ac139c	2020-06-07T00:12:47	Update xenial
0d82e49f	2020-06-07T00:12:17	Update bionic
153199ae	2020-03-17T09:42:41	ci: don't use --insecure mbedTLS has fixed their certificate. 🎉
8621bdda	2020-03-13T22:42:51	azure: docker: use insecure flag to fix curl downloads We currently hve some problems with our curl downloads when building Docker images. It's not quite obvious what the problem is and they seem to occur semi-randomly. To unblock our CI, let's add the "--insecure" flag to curl to ignore any certificate errors. This is intended as a temporary solution only.
5ac33ced	2020-03-10T21:39:39	azure: docurium: fix build failure due to bumped CMake requirements Our Docurium builds currently depend on Debian Jessie, which has CMake v3.0 available. As rugged has bumped its CMake requirements to need at least v3.5 now, the documentation build is thus failing. Fix this by converting our Docurium Docker image to be based on Ubuntu Bionic. We already do base all of our images on Ubuntu, so I don't see any sense in using Debian here. If this was only to speed up builds, we should just go all the way and use some minimal container like Alpine anyway. Also remove cache busters. As we're rebuilding the image every time, it's we really don't need them at all.
c76c1e87	2020-03-10T20:39:09	azure: docker: consistently silence curl but show errors We currently pass the "--silent" flag to most invocations of curl, but in fact this does not only suppress the progress meter, but also any errors. So let's also pass "--show-error", too.
dc55d0e8	2020-03-02T10:22:54	ci: provide globalsign certs for bionic tls.mbed.org has neglected to send their full certificate chain. Add their intermediate cert manually. 🙄
0f316d59	2020-03-01T14:42:03	ci: provide globalsign certs tls.mbed.org has neglected to send their full certificate chain. Add their intermediate cert manually. 🙄
6efe3d35	2020-02-19T11:34:55	azure: docker: set up HOME variable to fix Coverity builds In commit 01a834066 (azure: docker: fix ARM builds by replacing gosu(1), 2020-02-18), we've switched our entrypoint from gosu(1) to use sudo(1) instead to fix our ARM builds. The switch introduced an incompatibility that now causes our Coverity builds to fail, as the "--preserve-env" switch will also keep HOME at its current value. As a result, Coverity now tries to set up its configuration directory in root's home directory, which it naturally can't write to. Fix the issue by adding the "--set-home" flag to sudo(1).
01a83406	2020-02-18T15:20:43	azure: docker: fix ARM builds by replacing gosu(1) Our nightly builds are currently failing due to our ARM-based jobs. These jobs crash immediately when entering the Docker container with a exception thrown by Go's language runtime. As we're able to successfully builds the Docker images in previous steps, it's unlikely to be a bug in Docker itself. Instead, this exception is thrown by gosu(1), which is a Go-based utility to drop privileges and run by our entrypoint. Fix the issue by dropping gosu(1) in favor of sudo(1).
76b49caf	2020-02-18T15:20:08	azure: docker: synchronize Xenial/Bionic build instructions Our two Docker build instructions for Xenial and Bionic have diverged a bit. Let's re-synchronize them with each other to make them as similar as possible.
fb03f02a	2020-02-07T13:56:36	azure: docker: avoid re-creating libgit2 home directory The Docker entrypoint currently creates the libgit2 user with "useradd --create-home". As we start the Docker container with two volumes pointing into "/home/libgit2/", the home directory will already exist. While useradd(1) copes with this just fine, it will print error messages to stderr which end up as failures in our Azure pipelines. Fix this by simply removing the "--create-home" parameter.
a3ec07d7	2020-02-07T13:34:18	azure: docker: pipe downloaded archives into tar(1) directly When building dependencies for our Docker images, we first download the sources to disk first, unpack them and finally remove the archive again. This can be sped up by piping the downloading archive into tar(1) directly to parallelize both tasks. Furthermore, let's silence curl(1) to not print to status information to stderr, which tends to be interpreted as errors by Azure Pipelines.
9b96adf3	2019-11-22T18:03:50	docker build: add krb5 libraries and command-line tools
c64b7aaa	2019-11-23T20:38:30	ci: build our own valgrind The valgrind in the PPA is broken and ignores `--exit-errorcode`. Build and install our own.
fd831275	2019-11-23T12:40:46	ci: build shared libssh2
84807884	2019-11-23T12:40:02	ci: break dockerfile into stages Use a multi-stage docker build so that we can cache early stages and not need to download the apt-provided dependencies during every build (when only later stages change).
7a3d04dc	2019-11-23T12:14:23	ci: don't delete the apt cache Deleting the apt cache can be helpful for reducing the size of a container, but since we don't push it anywhere, it only hinders our ability to debug problems while working on the container. Keep it.
f592c737	2019-11-23T11:55:50	ci: don't install libssh2 since we build it
3c884cc3	2019-09-21T15:05:36	azure: avoid building and testing in Docker as root Right now, all tests in libgit2's CI are being executed as root user. As libgit2 will usually not run as a root user in "normal" usecases and furthermore as there are tests that rely on the ability to _not_ be able to create certain paths, let's instead create an unprivileged user "libgit2" and use that across all docker images.
48d23a8c	2019-08-02T12:36:19	azure: convert to use Ninja as build tool While we were still supporting Trusty, using Ninja as a build tool would have required us to first setup pip and then use it to install Ninja. As a result, the speedups from using Ninja were drowned out by the time required to install Ninja. But as we have deprecated Trusty now, both Xenial and Bionic have recent versions of Ninja in their repositories and thus we can now use Ninja.
4e07a205	2019-08-02T13:29:54	docker: fix Valgrind errors on Xenial by updating to v3.12.0 The Valgrind version shipped with Xenial has some bugs that keep our tests from working due to bad interactions with openssl [1]. Fix this by using the "hola-launchpad/valgrind" PPA that provides a newer version for which the bug has been fixed. [1]: https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1574437
3c59d451	2019-08-02T12:34:10	docker: use "--no-install-recommends" to reduce build time Pass the flag "--no-install-recommends" to apt-get in order to trim down the number of packages installed, both reducing build time and image size. As this also causes some required packages to not be installed anymore, add these explicitly to the set of packages installed.
39b7e8b0	2019-08-02T12:26:30	docker: convert apt-get to use best practices Reformat both Xenial and Bionic's Dockerfiles to use best practices. Most importantly, we now run `apt-get update` and `apt-get install` in one step followed up by removing the package lists to speed up installation and keep down the image size.
9f91d57e	2019-08-02T14:25:02	docker: install libssh2 1.8.2 on Xenial While Xenial provides libssh2 in its repositories, it only has version 1.5.0 available. This version will unfortunately not be able to connect to GitHub due to their removal of weak cryptographic standards [1]. To still enable our CI to execute tests against GitHub, we thus have to update the provided libssh2 version to a newer one. Manually install libssh2 1.8.2 on Xenial. There's no need to do the same for Bionic, as it already provides libssh2 1.8.0. [1]: https://github.blog/2018-02-01-crypto-removal-notice/
253dbea2	2019-08-02T10:21:32	docker: install mbedTLS on both Bionic and Xenial We're about to phase out support for Trusty, but neither Bionic nor Xenial images provide the mbedTLS library that's available in Trusty. Build them for both to pull them in line with Trusty.
76327381	2019-08-02T10:50:11	azure: deprecate Trusty in favor of Xenial Support for the LTS release Ubuntu 14.04 Trusty has been dropped in April 2019, but Azure is still using Trusty as its primary platform to build and test against. Let's deprecate it in favor of Xenial.
5a6740e7	2019-08-02T09:58:55	azure: build Docker images as part of the pipeline The Docker images used for our continuous integration builds currently live in the libgit2/libgit2-docker repository. To make any changes in them, one has to make a PR there, get it reviewed, re-build the images and publish them to Docker Hub. This process is slow and tedious, making it harder than necessary to perform any updates to our Docker-based build pipeline. To fix this, we include all Dockerfiles used by Azure from the mentioned repository and inline them into our own repo. Instead of having to manually push them to the CI, it will now build the required containers on each pull request, allowing much greater flexibility.

6a917c04

2020-06-28T15:51:43

Add CI support for Memory and UndefinedBehavior Sanitizers This change adds two new build targets: MSan and UBSan. This is because even though OSS-Fuzz is great and adds a lot of coverage, it only does that for the fuzz targets, so the rest of the codebase is not necessarily run with the Sanitizers ever :( So this change makes sure that MSan/UBSan warnings don't make it into the codebase. As part of this change, the Ubuntu focal container is introduced. It builds mbedTLS and libssh2 as debug libraries into /usr/local and as MSan-enabled libraries into /usr/local/msan. This latter part is needed because MSan requires the binary and all its dependent libraries to be built with MSan support so that memory allocations and deallocations are tracked correctly to avoid false positives.

4852d8da

2020-06-10T22:18:13

docker: don't take BASE as an argument The xenial image depends on ubuntu:xenial; the bionic one on ubuntu:bionic. No need for this to be a variable, that's just additional (unnecessary) state to manage in the CI setup(s).

c16ba496

2020-06-07T00:37:55

yo

80c6b0a2

2020-06-07T00:16:57

Update bionic

4a95ee8d

2020-06-07T00:16:39

Update xenial

c2ac139c

2020-06-07T00:12:47

Update xenial

0d82e49f

2020-06-07T00:12:17

Update bionic

153199ae

2020-03-17T09:42:41

ci: don't use --insecure mbedTLS has fixed their certificate. 🎉

8621bdda

2020-03-13T22:42:51

azure: docker: use insecure flag to fix curl downloads We currently hve some problems with our curl downloads when building Docker images. It's not quite obvious what the problem is and they seem to occur semi-randomly. To unblock our CI, let's add the "--insecure" flag to curl to ignore any certificate errors. This is intended as a temporary solution only.

5ac33ced

2020-03-10T21:39:39

azure: docurium: fix build failure due to bumped CMake requirements Our Docurium builds currently depend on Debian Jessie, which has CMake v3.0 available. As rugged has bumped its CMake requirements to need at least v3.5 now, the documentation build is thus failing. Fix this by converting our Docurium Docker image to be based on Ubuntu Bionic. We already do base all of our images on Ubuntu, so I don't see any sense in using Debian here. If this was only to speed up builds, we should just go all the way and use some minimal container like Alpine anyway. Also remove cache busters. As we're rebuilding the image every time, it's we really don't need them at all.

c76c1e87

2020-03-10T20:39:09

azure: docker: consistently silence curl but show errors We currently pass the "--silent" flag to most invocations of curl, but in fact this does not only suppress the progress meter, but also any errors. So let's also pass "--show-error", too.

dc55d0e8

2020-03-02T10:22:54

ci: provide globalsign certs for bionic tls.mbed.org has neglected to send their full certificate chain. Add their intermediate cert manually. 🙄

0f316d59

2020-03-01T14:42:03

ci: provide globalsign certs tls.mbed.org has neglected to send their full certificate chain. Add their intermediate cert manually. 🙄

6efe3d35

2020-02-19T11:34:55

azure: docker: set up HOME variable to fix Coverity builds In commit 01a834066 (azure: docker: fix ARM builds by replacing gosu(1), 2020-02-18), we've switched our entrypoint from gosu(1) to use sudo(1) instead to fix our ARM builds. The switch introduced an incompatibility that now causes our Coverity builds to fail, as the "--preserve-env" switch will also keep HOME at its current value. As a result, Coverity now tries to set up its configuration directory in root's home directory, which it naturally can't write to. Fix the issue by adding the "--set-home" flag to sudo(1).

01a83406

2020-02-18T15:20:43

azure: docker: fix ARM builds by replacing gosu(1) Our nightly builds are currently failing due to our ARM-based jobs. These jobs crash immediately when entering the Docker container with a exception thrown by Go's language runtime. As we're able to successfully builds the Docker images in previous steps, it's unlikely to be a bug in Docker itself. Instead, this exception is thrown by gosu(1), which is a Go-based utility to drop privileges and run by our entrypoint. Fix the issue by dropping gosu(1) in favor of sudo(1).

76b49caf

2020-02-18T15:20:08

azure: docker: synchronize Xenial/Bionic build instructions Our two Docker build instructions for Xenial and Bionic have diverged a bit. Let's re-synchronize them with each other to make them as similar as possible.

fb03f02a

2020-02-07T13:56:36

azure: docker: avoid re-creating libgit2 home directory The Docker entrypoint currently creates the libgit2 user with "useradd --create-home". As we start the Docker container with two volumes pointing into "/home/libgit2/", the home directory will already exist. While useradd(1) copes with this just fine, it will print error messages to stderr which end up as failures in our Azure pipelines. Fix this by simply removing the "--create-home" parameter.

a3ec07d7

2020-02-07T13:34:18

azure: docker: pipe downloaded archives into tar(1) directly When building dependencies for our Docker images, we first download the sources to disk first, unpack them and finally remove the archive again. This can be sped up by piping the downloading archive into tar(1) directly to parallelize both tasks. Furthermore, let's silence curl(1) to not print to status information to stderr, which tends to be interpreted as errors by Azure Pipelines.

9b96adf3

2019-11-22T18:03:50

docker build: add krb5 libraries and command-line tools

c64b7aaa

2019-11-23T20:38:30

ci: build our own valgrind The valgrind in the PPA is broken and ignores `--exit-errorcode`. Build and install our own.

fd831275

2019-11-23T12:40:46

ci: build shared libssh2

84807884

2019-11-23T12:40:02

ci: break dockerfile into stages Use a multi-stage docker build so that we can cache early stages and not need to download the apt-provided dependencies during every build (when only later stages change).

7a3d04dc

2019-11-23T12:14:23

ci: don't delete the apt cache Deleting the apt cache can be helpful for reducing the size of a container, but since we don't push it anywhere, it only hinders our ability to debug problems while working on the container. Keep it.

f592c737

2019-11-23T11:55:50

ci: don't install libssh2 since we build it

3c884cc3

2019-09-21T15:05:36

azure: avoid building and testing in Docker as root Right now, all tests in libgit2's CI are being executed as root user. As libgit2 will usually not run as a root user in "normal" usecases and furthermore as there are tests that rely on the ability to _not_ be able to create certain paths, let's instead create an unprivileged user "libgit2" and use that across all docker images.

48d23a8c

2019-08-02T12:36:19

azure: convert to use Ninja as build tool While we were still supporting Trusty, using Ninja as a build tool would have required us to first setup pip and then use it to install Ninja. As a result, the speedups from using Ninja were drowned out by the time required to install Ninja. But as we have deprecated Trusty now, both Xenial and Bionic have recent versions of Ninja in their repositories and thus we can now use Ninja.

4e07a205

2019-08-02T13:29:54

docker: fix Valgrind errors on Xenial by updating to v3.12.0 The Valgrind version shipped with Xenial has some bugs that keep our tests from working due to bad interactions with openssl [1]. Fix this by using the "hola-launchpad/valgrind" PPA that provides a newer version for which the bug has been fixed. [1]: https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1574437

3c59d451

2019-08-02T12:34:10

docker: use "--no-install-recommends" to reduce build time Pass the flag "--no-install-recommends" to apt-get in order to trim down the number of packages installed, both reducing build time and image size. As this also causes some required packages to not be installed anymore, add these explicitly to the set of packages installed.

39b7e8b0

2019-08-02T12:26:30

docker: convert apt-get to use best practices Reformat both Xenial and Bionic's Dockerfiles to use best practices. Most importantly, we now run `apt-get update` and `apt-get install` in one step followed up by removing the package lists to speed up installation and keep down the image size.

9f91d57e

2019-08-02T14:25:02

docker: install libssh2 1.8.2 on Xenial While Xenial provides libssh2 in its repositories, it only has version 1.5.0 available. This version will unfortunately not be able to connect to GitHub due to their removal of weak cryptographic standards [1]. To still enable our CI to execute tests against GitHub, we thus have to update the provided libssh2 version to a newer one. Manually install libssh2 1.8.2 on Xenial. There's no need to do the same for Bionic, as it already provides libssh2 1.8.0. [1]: https://github.blog/2018-02-01-crypto-removal-notice/

253dbea2

2019-08-02T10:21:32

docker: install mbedTLS on both Bionic and Xenial We're about to phase out support for Trusty, but neither Bionic nor Xenial images provide the mbedTLS library that's available in Trusty. Build them for both to pull them in line with Trusty.

76327381

2019-08-02T10:50:11

azure: deprecate Trusty in favor of Xenial Support for the LTS release Ubuntu 14.04 Trusty has been dropped in April 2019, but Azure is still using Trusty as its primary platform to build and test against. Let's deprecate it in favor of Xenial.

5a6740e7

2019-08-02T09:58:55

azure: build Docker images as part of the pipeline The Docker images used for our continuous integration builds currently live in the libgit2/libgit2-docker repository. To make any changes in them, one has to make a PR there, get it reviewed, re-build the images and publish them to Docker Hub. This process is slow and tedious, making it harder than necessary to perform any updates to our Docker-based build pipeline. To fix this, we include all Dockerfiles used by Azure from the mentioned repository and inline them into our own repo. Instead of having to manually push them to the CI, it will now build the required containers on each pull request, allowing much greater flexibility.

thodg/libgit2/azure-pipelines/docker

azure-pipelines/docker

Log