Debian 9 (stretch) is now beyond LTS support and on to ELTS support,
which means that the typical Debian infrastructure is no longer involved
and updates are no longer available from Debian. As a result, it's not
really appropriate for us to continue to build packages on stretch since
we can't be certain our dependencies or build environment are secure.
It is the case that some of the OSes that use Debian 9 packages, such as
some versions of Ubuntu, do still receive security updates, but since we
don't build specifically for those OSes, we'll need to drop support for
them as well. Of the OSes receiving mainline (non-extended) security
support, only Ubuntu bionic is affected.
Since CentOS is now no longer a plain rebuild of RHEL, use Rocky Linux
as the RHEL 9 equivalent. Add entries in all the required files for
this to build properly.
For CentOS, we've traditionally included the el7 component, so let's
include the same component for Rocky 8. While we're at it, let's be
honest about what we're shipping and say Rocky Linux and not CentOS,
since CentOS 8 is dead.
On macOS, we can match our Windows BMP and ICO icons as having trailing
whitespace. Since we don't really care about the whitespace content of
binary files, exclude them from the list of files we want to consider.
It was reported in https://github.com/git-lfs/git-lfs/discussions/5031
that Git LFS' installer aborts because it does not find the `git`
executable when Git for Windows was installed with the "Bash Only" PATH
option, i.e. in the mode where `PATH` is not modified at all.
Detect this situation, and fall back to implicitly extend the `PATH` to
find Git for Windows' `git.exe`.
Co-authored-by: Chris Darroch <chrisd@apache.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we're verifying with multiple programs, it can be hard to read the
output between GnuPG and the hashes. Produce nicer output by printing
intermediate messages to break up the text.
Remove several distro versions which are now EOL. In addition, add a
new version of Linux Mint and OpenSUSE, and update the EOL date for some
distros.
Note that we have not added Fedora 36 or Ubuntu kinetic because they are
not yet available on packagecloud.io.
Right now, we provide signed SHA-256 hashes for our releases. This is
fine and sufficient, and also cryptographically secure. However, many
distributors use other algorithms, and it would be convenient if we
could provide easy access to those hashes as well. For example, NetBSD
uses SHA-512 and BLAKE2s.
Let's add an additional file, hashes.asc, which contains a general set
of hashes in the BSD format. The advantage of the BSD format over the
traditional GNU format is that it includes the hash algorithm, which
allows us to distinguish between hashes of the same length, such as
SHA-256, SHA-512/256, and SHA3-256. It is generated by shasum, sha*sum,
sha3sum, and b2sum with the --tag format, and all of these programs
accept it for verification with no problems.
Using the BSD format means that we need only provide one additional file
with all the additional algorithms. There is therefore no need to add
multiple new files, and if we desire to add additional algorithms in the
future, that's easily done without modification.
If the user has sha3sum (which comes from Perl's Digest::SHA3) or b2sum
(part of GNU coreutils), then we use them to verify our hashes. There
are no known commands available on a typical Linux system to verify
BLAKE2s, but we assume that if OpenSSL and our Ruby script correctly
generated the SHA-2 entries, then it will also have properly generated
the other hashes as well.
Since we must now run programs inside the repository, we need to know
where that file is located, and therefore we use git to find the root of
the repository and now must run within the repository. Since this
script is only run by Git LFS core team members or the CI system when
doing releases, this is not expected to be an issue.
Right now, we provide signed SHA-256 hashes for our releases. This is
fine and sufficient, and also cryptographically secure. However, many
distributors use other algorithms, and it would be convenient if we
could provide easy access to those hashes as well. For example, NetBSD
uses SHA-512 and BLAKE2s.
Let's add a script to hash files with various algorithms and output them
in the BSD format. The advantage of the BSD format over the traditional
GNU format is that it includes the hash algorithm, which allows us to
distinguish between hashes of the same length, such as SHA-256,
SHA-512/256, and SHA3-256. It is generated by shasum, sha*sum, sha3sum,
and b2sum with the --tag format, and all of these programs accept it for
verification with no problems.
Using the BSD format means that we need only provide one additional file
with all the additional algorithms. There is therefore no need to add
multiple new files, and if we desire to add additional algorithms in the
future, that's easily done without modification.
For aesthetics, we sort first by hash name and then by filename in the
output. Unlike sorting with `sort`, this keeps the SHA-2 and SHA-3
algorithms separate instead of interspersing them, which aids in
reading. Add some comments because the algorithm, while logical,
is somewhat subtle.
After upgrading to Go version 1.18 the "go get" command no longer
builds or installs packages by default; it only manages dependencies
in go.mod and go.sum.
We therefore switch to using "go install" to build and install
the goimports and goversioninfo. We then also no longer need to
reset our working tree because "go install" does not modify
go.mod or go.sum.
Because CentOS 8 is now EOL, we've switched to Rocky Linux for building
RHEL 8-compatible packages. However, those packages don't end up
containing the ".el8" segment, so the link is broken. Fix this by
removing that segment from the URL.
In commit a5b751f541abb52e30b60aebe530f9dbd72838df we added
a GitFoundInPath() function which we check before executing
a full installation; if it returns False, we do not install
the Git LFS binary.
However, we still install the uninstaller and add the
installation path to the PATH and GIT_LFS_PATH environment
variables, which leads to a partial and incomplete setup;
also, the Inno Setup installer still displays a final dialog
box with the message "Setup has finished installing Git LFS
on your computer", which is confusing.
However, if we call our GitFoundInPath() check function in
the InitializeSetup() event function and return its value,
then if it returns False the entire setup process is halted
and no actual installation occurs at all. This leads to a
cleaner and more intuitive experience for users.
This also aligns the installation and uninstallation steps,
since the uninstaller also checks GitFoundInPath() in its
initialization event function and if the return value is
False then it terminates immediately without making any
changes.
When uninstalling Git LFS on Windows, as of commit
cbd52be073a340808253208ee4430b6f6451c271 we now use the
term "deregister" in part of the message we display when
a Git exectuable is found in an unconventional location.
Since we also use the term "registered" in another part
of that message, we change it to switch to the term
"deregistered" in the uninstallation case.
When the Git executable on the PATH is in an unexpected location, we
want to warn the user about that. But we only want to do that once, not
four times. Let's cache the result once we have it, and ask only once.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When Git cannot be found, we still proceed with installing Git LFS,
we're just not registering it with Git via the system config. Let's
clarify that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It would confuse users to see "Do you want to register Git LFS?" when
uninstalling. Let's show the message "Do you want to deregister Git
LFS?" in that case instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When installing and uninstalling Git LFS, the Git LFS
program configures its "clean" and "smudge" Git filters,
and to do this it executes Git so as to change the Git
global configuration (or, optionally, the system-wide,
local user, or worktree-specific configuration).
The Git program executed is the first one found using the
PATH environment variable (and, on Windows, the PATHEXT
environment variable).
Therefore, when installing Git LFS as an administrator with
elevated privileges, final responsibility lies with the
administrator to ensure there are no compromised executables
in their system PATH.
For instance, on Linux the "secure_path" configuration
value might be set in /etc/sudoers before running the command
"sudo git lfs install --system".
We can, however, attempt to assist the administrator on
Windows where we provide a dedicated installer and also
anticipate that Git will be installed under a common set
of directories.
For that reason we update our Inno Setup installer script so
that if it detects that the Git program found with the relevant
PATH and PATHEXT environment variables (either the user or
system ones, depending on the user's role) is not within
either of the "C:\Program Files" or "C:\Program Files (x86)"
directories, then a warning is displayed and the user prompted
to decide whether to continue.
And for convenience, we now report a failure message if no
Git program is found, which avoids subsequent errors during
the installation or uninstallation steps for any user.
Note, though, that if a Windows administrator runs the
"git-lfs.exe install" command manually, the checks we are
adding to the Inno Setup script will not be performed, and
the situation then is no different than a macOS or Linux
user running "sudo git-lfs install" without confidence that
the system PATH and installed Git binary are already secure.
Co-authored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
In PR #875 in 2016 the Windows installer was changed from NSIS
to Inno Setup, and so logic was added in commit
1651e55ce5cc37c69bf5cb3bf92e626b83ad5e0a to silently try to run
the old NSIS git-lfs-uninstaller.exe binary if it existed in
the same directory as git.exe, as found using the normal PATH
environment variables.
As all users of Git LFS should have upgraded to a recent version
installed with the Inno Setup installer, we can simply drop this
additional logic along with the GetExistingGitInstallation()
function of which it was the sole caller.
Co-authored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Sometimes we'd like to test to see how much of or whether our code is
localized. However, it's very burdensome to create a full localization
in order to do so. To help us, let's create a special, autogenerated
locale, i-reverse, which is generated by simply reversing all the
English words. Add a script to do this automatically for us.
However, don't generate this locale on typical builds, since users might
not have xgotext installed, and adding this locale would bloat the
binary needlessly in most cases.
Remove Fedora 32 and 33, which are EOL. Add Fedora 35, which is the
latest version. Add the latest versions of Ubuntu and Linux Mint, and
remove EOL versions of Ubuntu. Fix the alignment of the comments.
When the upload script is used in finalize mode, which is a manual part
of the release process, it currently requires the bin/releases directory
to be empty. This is because it uses the upload_assets function, which
uploads everything in that directory, and we obviously don't want to
upload whatever the developer may have in that directory as part of our
release, only the signed hashes.
Since the person doing the release is usually the person who has built
the pre-release assets for the pull request, this means that we
invariably bother this person, which, while not the end of the world, is
a minor annoyance. Let's make this experience better for the core team
member doing the release and lift this restriction.
Make the upload_assets function take a location of assets to upload, and
when we're uploading assets in non-finalize mode (which is now
effectively restricted to our release workflow) use the bin/releases
directory, but use a temporary directory for the finalize step. Remove
the check an empty bin/releases directory, since we no longer need it.
Remove Ubuntu impish and AlmaLinux because they are not supported by
packagecloud.io. Add Debian bookworm, which is currently testing, and
SLES 15.3, which is the latest version.
Scutiger is our reference git-lfs-transfer implementation, so let's
install it during CI, clean up after tests, ignore our temporary
directory, and add code to appropriately make use of the binary in
tests.
If Scutiger is not available, let the tests pass unless we're in CI,
where we should always have the binary available. Also, skip the tests
when the repository is a SHA-256 repository, since Scutiger uses
libgit2, which doesn't yet support SHA-256 repositories.
With macOS Big Sur, ARM64 is now a supported architecture for Darwin.
Building binaries for this case is supported since Go 1.16. Since users
would like to have native signed binaries for macOS, let's go ahead and
build Darwin packages as part of the make release command.
Note that this will necessarily require users to have Go 1.16 available
to run that command, since otherwise the attempt to build for ARM64
Darwin will fail. Since this is mostly run in CI and on developer
machines, that's okay.
We also fix an issue in the release-darwin target that would try to
unzip all files at once, which doesn't do the right thing when we have
multiple files, and make sure that we upload the ARM64 zip file as well.
Debian 8 went out of support on June 30, 2020. Ubuntu 14.04, which uses
the same packages, went out of support in 2019.
Since there's no longer any free security support and we cannot
reasonably be expected to run insecure code in CI, since that's
dangerous to others on the Internet, let's drop support for Debian 8.
CentOS 6 went out of support on November 30, 2020. Moreover, building
packages for it requires that we build Ruby in addition to all the
gems, plus a version of Git, meaning we spend significant CI resources
and time on this platform.
Since there's no longer any free security support and we cannot
reasonably be expected to run insecure code in CI, since that's
dangerous to others on the Internet, let's drop support for CentOS 6.
Ruby's URI.escape is now deprecated because different components of a
path require different escaping rules. However, we still want to
perform a similar escaping style, so let's do it ourselves with a
String#gsub call. We could use CGI.escape, but that encodes spaces as
plus signs and it's unclear whether that's a desirable change to make
here. Note that there are no negative consequences for encoding more
data than we want, since the server will decode it for us.
GitHub Actions ships its own version of the openssl Homebrew package
that fails to reinstall with an EPERM error. Since the reason we did
these contortions was to relink gettext and we finally gave up and
disabled gettext, let's just go back to a brew link --force and stop
trying to reinstall.
While we don't currently have access to a version of Git that supports
SHA-256, one will soon be available, and we'll need to support those
versions. To make sure our test suite works properly, we'll need to run
a test with SHA-256, so let's allow passing the GIT_DEFAULT_HASH
environment variable into our CI job script.
We currently intend for our CI check for trailing whitespace to be
fatal; that is, if any trailing whitespace is present, our CI script
should fail.
However, currently that doesn't work, since apparently set -e is not
concerned with a negated pipeline having a nonzero exit code. To make
things more robust, let's make this into an if statement and exit 1 if
we match any files (that is, if grep returns 0).
Note that this used to work because it was the last command in the file,
but it no longer does for the above reason.
On macOS, Gatekeeper requires binaries that are signed with a trusted
code-signing certificate and notarized by Apple in order for them to
run. To ease the burden for Mac users, let's start providing signed
binaries.
The macOS codesign tool can only read certificates from a keychain.
However, setting keychains up to work in a non-interactive way is
complex and error prone. We create a target to import the certificates
from a PKCS #12 file and pull them into a temporary keychain which has
been specially set up to work in CI. This requires multiple complex and
poorly documented incantations to work correctly, but it does currently
work. These incantations are not to be meant run on a user system
because they modify various keychain properties, such as the default
keychain, so add a comment to that effect.
We sign both the binary and the zip file, since we cannot notarize the
binary alone but would like to have a signed binary. Only zip files,
pkg files, and disk images can be notarized; this is why we have
switched to a zip file for macOS.
Note that the notarization process requires a particular developer to
submit the binary for notarization using their Apple account. That
developer's ID and their app password are specified from the environment
and can be read from the secret store. This is so that this can easily
be rotated to reflect a new user without needing to involve code
changes. Similarly, the cert ID, although not secret, is passed in in a
similar way.
When we perform the notarization, we do it in a loop, since Apple's
servers can sometimes "forget" the fact that we submitted a request and
therefore cause gon, the notarization tool we use, to spuriously fail
when it checks on the status of our request. We don't use seq to count
in our loop because it is not portable to non-Linux systems.
Finally, we use "darwin" in the Makefile because everything else in the
Makefile already uses that, but we use "MACOS" for secrets for
consistency with the GitHub Actions workflow, which uses that. We
translate in the workflow file.
Remove distros which have reached end of life, and remove some
commented-out entries for versions which have already been listed
elsewhere. Promptly trimming EOL versions makes our releases faster
since we need not upload packages for those versions.
Note that we remove CentOS 5 because we're no longer building packages
for it. Since RHEL is not freely available, we must rely on building
packages for CentOS, which means we must use CentOS's EOL dates, not
RHEL's.
