Much of the initial text of the README file for Docker image use no
longer applies, as we do not publish Docker images to the Docker Hub
or any other registry, and various new options have been added over time
to the docker/run_dockers.bsh script. Therefore we can simplify the
README considerably, and update the remaining portions to reflect our
current practices and utilities, project locations, and terminology.
The --prune command-line option was added to the docker/run_dockers.bsh
script in commit 4a2e96b74337b91012b95314633db1d5f4181a52 of PR #4104,
and the --arch command-line options was added in commit
569b5ce4e56dcfda2fb26d6277047182d8ec4d11 of PR #4728. Earlier, the
REPO_HOSTNAME environment variable was removed from all the corresponding
Dockerfiles in the git-lfs/build-dockers repository in commit
git-lfs/build-dockers@d4c2fe6f79.
We therefore update the docker/run_dockers.bsh script now to remove
the unused REPO_HOSTNAME environment variable and document the new
--prune and --arch command-line options.
When the scripts to build Docker images of Git LFS were first introduced
in PRs #511 and #555, an optional capabililty was included to sign the RPM
or Debian packages after building them in the containers. This option
was triggered by the presence of a git-lfs_*.key file alongside the
Dockerfile for a given OS and version.
However, we have not published Docker containers to the Docker Hub or
any other registry for at least five years, and we do not use or maintain
this signing capability, so we can remove the associated script code.
If we ever decide to begin publishing containers again, we can restore
this functionality from our Git history.
The docker/test_dockers.bsh script is also out of date and unused, so we
remove it as well.
Now that we've centralized our distro information in one place, call our
distro tool to acquire the list of images to build. To make it easier
to handle this case by reading the output of a command and splitting on
whitespace, let's switch from a bash array to a standard shell string.
We indicate that there are two different places we keep a list of
distros, but in fact, there are four. Let's make a note at each place
indicating these and refer to the others to make sure they're updated.
A good future direction would be to store this data in some sort of data
file and read it in each case, but we don't currently do that.
We no longer support either CentOS 5 or 6, per commit
b560b853a6e557195c0c68875218f45c89dce56b of PR #1298 and commit
git-lfs/build-dockers@64a3a9fc4d
of PR git-lfs/build-dockers#3, and commit
git-lfs/build-dockers@898d9b045e
of PR git-lfs/build-dockers#33.
We also now build the Asciidoctor Ruby gem in order to generate
our man pages, rather than using ronn and several other gems,
per commit db9a82132a2bb066876d8ddf06c5255da2f199a4 of PR #5054.
We therefore update the documentation for our RPM package build
utilities and for our Docker container image build utilities to
reflect these changes.
Debian 9 (stretch) is now beyond LTS support and on to ELTS support,
which means that the typical Debian infrastructure is no longer involved
and updates are no longer available from Debian. As a result, it's not
really appropriate for us to continue to build packages on stretch since
we can't be certain our dependencies or build environment are secure.
It is the case that some of the OSes that use Debian 9 packages, such as
some versions of Ubuntu, do still receive security updates, but since we
don't build specifically for those OSes, we'll need to drop support for
them as well. Of the OSes receiving mainline (non-extended) security
support, only Ubuntu bionic is affected.
Since CentOS is now no longer a plain rebuild of RHEL, use Rocky Linux
as the RHEL 9 equivalent. Add entries in all the required files for
this to build properly.
We never upload Docker images anymore, so it's no longer useful to have
DOCKER_AUTOPULL set to true. Let's remove this flag and simply assume
the user has built the images.
Allow users to build for a different architecture if they'd like. Right
now, the Docker images only support amd64 and arm64, but it's possible
that we could support more images in the future.
There are two types of pattern matching with wildmatch patterns in Git:
gitignore and gitattributes patterns. We'd like to support both, but
without the lazy matching that was the cause of so many problems before.
Add a type of pattern to use when creating a new filepathfilter and use
it to instantiate the wildmatch pattern with the proper flags. Remove
the dirs flag, which is unused and has been for sometime, and also the
stripping of trailing slashes, which we want to use to indicate a
directory. Drop the non-strict mode since our pattern matching is now
much improved and compatible with Git.
Debian has recently deprecated the which utility and will remove it in a
future version, since POSIX specifies "command -v" for finding the
location of a path and has for some time. In addition, which has
started complaining loudly to standard error, so it causes test and
makefile output to be hard to read.
Let's switch to the preferred invocation, command -v, in the Makefile,
build tooling, and test suite.
Debian 8 went out of support on June 30, 2020. Ubuntu 14.04, which uses
the same packages, went out of support in 2019.
Since there's no longer any free security support and we cannot
reasonably be expected to run insecure code in CI, since that's
dangerous to others on the Internet, let's drop support for Debian 8.
CentOS 6 went out of support on November 30, 2020. Moreover, building
packages for it requires that we build Ruby in addition to all the
gems, plus a version of Git, meaning we spend significant CI resources
and time on this platform.
Since there's no longer any free security support and we cannot
reasonably be expected to run insecure code in CI, since that's
dangerous to others on the Internet, let's drop support for CentOS 6.
At the moment, we're running out of disk space when building packages in
Docker containers. This is because our CI VMs have only 14 GB of
storage. To help deal with this problem, let's add an option to the
run_dockers.bsh script to prune images after using them so we can free
some of that disk space and let the remaining jobs complete.
Build packages for CentOS 8. Add it to the list of containers we build
for and the list of packages we have downloads available for.
In the gem spec files, use a different tooling on CentOS 8, since the
old method no longer works there. Build using a more standard gem build
process so that RPM does the abstraction for us now that it's capable of
it. Stop deleting the root in the install step since we may have
installed things there by that point, and RPM has handled this properly
for many versions.
Since the syntax "%{?el8}" expands to "1" on CentOS 8 and nothing on
earlier versions, add a 0 in front of it any time we use the macro so
that we don't end up with an empty conditional on earlier versions.
"01" is still true and "0" is false, so the logic works out.
It's been verified that the packages still build on CentOS 7 as well as
CentOS 8.
When building packages in containers, we pass options to docker.
However, when running in a noninteractive environment such as GitHub
Actions, we can't use the -i option, since standard input will not be a
TTY and docker will consequently fail. Check for a TTY and pass the -i
option only if one is present.
Debian 10 is due to be released on July 6, 2019. In order to ensure that
we can build packages for the release planned in early July, add support
for building on and uploading for Debian 10.
Debian 7 is definitively end of life and we can no longer use apt to
fetch packages for it. Consequently, we cannot build Docker images in
order to build packages, so remove it from the list of available builds.
Currently, the process of uploading release assets to GitHub is manual
and tedious. Add a script, with appropriate --help output, that helps
maintainers create a draft release and upload release assets to GitHub
automatically.
On Linux, lookup of user information such as the home directory is
handled by the name service switch in libc. This allows administrators
to use the mechanism of their choice (such as LDAP) for enumerating
users. Go normally doesn't use libc on Linux, but it does when looking
up information for users, so that programs written in Go handle these
operations like other (non-Go) programs on the system do.
When our build containers run the integration testsuite on i386 Linux,
we don't actually have a 32-bit copy of libc available. Therefore, we
must be sure not to rely on behavior that needs it, such as the name
service switch. Our tests want to look up the user's home directory so
that we can expand the core.hooksPath variable, but we also need the
user's name in order to do that. We have HOME set for the home
directory, but lack USER. Consequently, we try to call libc to find the
current user, but cannot, and the resulting test failure causes the
build to abort.
Set the USER environment variable in the containers to "root" so that we
can enumerate the current user without the need to call libc and ensure
our tests pass during release builds.
Removed all the files that were duplicated on lfs_dockers repo
Re-routed the transport of pgp keys using the newer more correct
gpg_agent docker image
Added some logic to work with mingw32 too
Moving the the docker image generation to a separately maintained repo
and uploading the image to docker hub instead. Should be a far friendlier
experience for the developer.
Almost all of the files in the docker folder should be redundant except
the run_dockers.bsh file at this point.
When files are copied from windows, they have 755 permissions.
For Centos, this is not a problem... however with Debian, at
some point, the package building fails. Reseting the
permissions according to git makes all the problems go away.
Started README update
Simplified gpg to a single file per docker
Auto dump public key in ./repo/os/version
Cleaned up test dockerfiles
Simplified gpg daemon scripts
Fixed bug not cleaning up old containers
Removed hack for #548
