For an unknown reason, sometimes libintl.h is missing on macOS when it
should not be. Try harder to fix this by reinstalling the packages
which are already allegedly installed and forcing an unlink and re-link.
Build packages for CentOS 8. Add it to the list of containers we build
for and the list of packages we have downloads available for.
In the gem spec files, use a different tooling on CentOS 8, since the
old method no longer works there. Build using a more standard gem build
process so that RPM does the abstraction for us now that it's capable of
it. Stop deleting the root in the install step since we may have
installed things there by that point, and RPM has handled this properly
for many versions.
Since the syntax "%{?el8}" expands to "1" on CentOS 8 and nothing on
earlier versions, add a 0 in front of it any time we use the macro so
that we don't end up with an empty conditional on earlier versions.
"01" is still true and "0" is false, so the logic works out.
It's been verified that the packages still build on CentOS 7 as well as
CentOS 8.
Sometimes we get contributions from folks who aren't very familiar with
Go. This is great, but since they aren't familiar with Go, they don't
know to use go fmt or goimports to tidy files. This leads to files
that are misformatted, which then causes problems when users who do have
these tools are doing development.
Let's help people check formatting automatically by running it as part
of the CI job. We already check for trailing whitespace, which is
complementary (since it looks at non-Go files as well), and that check
has been effective, so likely this one will be as well.
There is one potential downside: sometimes output changes between
versions of Go. This is unfortunate, but we'll need to just pin to
whatever version GitHub Actions uses.
When the build and upload process is automated, the only thing left for
the maintainer to do will be to download the assets, sign the hashes,
and upload the signed manifest. To make this easy and mostly automated,
add an option to the upload script, --finalize, that downloads assets,
optionally allows the maintainer to inspect them, and then uploads the
signed manifest to the GitHub release.
We already have the changelog in CHANGELOG.md, so there's no point in
requiring the user to provide it. Let's extract it automatically from
the existing file.
In the future, we'll want to use this script solely to download assets
and stage them for future processes. In preparation for that, split the
download function into a separate location.
While the ability to verify assets is important to ensuring our releases
are intact, we also want to be able to skip this step if we're running
in GitHub Actions. We may upload assets in multiple steps and not want
to verify immediately, or even have a signed manifest until the final,
manual stages of the process. Add a --skip-verify flag that allows
skipping the verification process.
When we automate the release process using GitHub Actions, we'll want to
upload an unsigned list of hashes that the maintainer can then sign and
re-upload. In order to do so, we'll need to handle the case where we're
uploading an unsigned list of hashes, so teach our upload script about
plain files named "sha256sums".
Currently, we have three different CI systems that handle our CI: Travis
for Linux, CircleCI for macOS, and AppVeyor for Windows. This results
in widely varying performance across systems and the need to maintain
code that works differently across different CI systems.
In addition, we'd like to use GitHub Actions to automate the release
process, so it makes sense to use it for CI as well. Switch over by
adding a CI workflow that runs our existing jobs. Ensure that we filter
out the environment variables that GitHub Actions provides, since they
will cause tests that run "git lfs env" to fail.
Add a script for those jobs where we build a custom Git and install the
appropriate dependencies. In the cibuild script, hoist the Windows
handling to the top, set a specific environment variable for us to
remember that we're on Windows, and then disable locking, which fails on
Windows and causes the testsuite to abort. These same environment
variables were set for AppVeyor and are also needed on Windows
development systems.
Debian 10 is due to be released on July 6, 2019. In order to ensure that
we can build packages for the release planned in early July, add support
for building on and uploading for Debian 10.
Debian 7 is definitively end of life and we can no longer use apt to
fetch packages for it. Consequently, we cannot build Docker images in
order to build packages, so remove it from the list of available builds.
Every time we build a release, we build new CentOS packages of Ruby and
ronn, among others. These packages have the same names as the packages
built during previous releases. However, attempting to upload a package
with an identical name fails because PackageCloud doesn't allow us to
overwrite packages in that way.
To prevent the maintainer from having to delete these packages by hand,
and to allow the maintainer to retry package uploads if they fail
partway through, look for the error message we get if a package already
exists and simply ignore it, continuing on as if no error occurred.
We'd like to ensure we have a complete release when uploading files, in
case a network error occurs or something else unfortunate happens during
the upload process. Moreover, since we publish hashes and signatures, it
makes sense to verify that our assets are correct according to those
signatures, since users will verify them (we hope).
To do so, ensure all the files are uploaded, then download all the files
again using the API URL and verify that the signature is valid on
sha256sums.asc and that the hashes match. If this succeeds, we can, with
high probability, be confident that our release is intact, containing
all the files we expect.
When uploading binary assets, such as tarballs and zip files, we would
like to retain all of the contents, including newlines and such. To
ensure we do so, use the --data-binary option to curl instead of the -d
option.
Currently, the process of uploading release assets to GitHub is manual
and tedious. Add a script, with appropriate --help output, that helps
maintainers create a draft release and upload release assets to GitHub
automatically.
While the current changelog generator is useful, it is interactive,
which makes it hard to extract the changelog output from the standard
output of the script. Provide a noninteractive mode, which specifies
sections but doesn't categorize any of the items. This can be used
to store the output in a file and allow the user to edit it with the
editor of their choice. This file can then be programmatically inserted
into the various places the changelog entry is required.
The current release process is mostly manual. To improve automation,
add a script that updates all the various places we store the version
number in preparation for release. Make the script idempotent.
In order to generate a changelog entry for the Debian changelog, use
the current user's committer identity and a timestamp similar to those
already used. Specify a time zone of -0000, as that means that the
timestamp is in UTC, but unlike +0000, doesn't imply that we ourselves
are in that timezone.
Since we'd like to avoid trailing whitespace in our codebase, help
developers catch issues by checking for trailing whitespace during CI.
Exclude the vendor directory, as we don't want to change vendored code,
and exclude any git objects or indices we store in the repository, since
they're binary zlib-compressed data. Exclude batch files as well, since
they contain carriage returns, which are considered non-linefeed
whitespace on Unix.
Note that we use git grep -E, since not all versions of git grep are
compiled with PCRE for the -P option. Fortunately, POSIX provides a
character class that matches all whitespace.
Since we are now building on Go 1.11 (as of 074a2d4f (all: use Go 1.11
in CI, 2018-08-28)) and Go 1.11 supports Go Modules [1], let's stop
using Glide, and begin using Go Modules.
This involves a few things:
* Teach the Makefile how to build go.sum files instead of glide.lock
files.
* Teach continuous integration services to build Git LFS in a
non-$GOPATH environment, since (without setting GO111MODULE=on
explicitly, which we choose not to do), this will break compiling
Git LFS, because Go 1.11 will ignore modules present in a Go
checkout beneath $GOPATH.
* In order to do the above, let's also make sure that we are
un-setting $GOCACHE in the environment, as this causes Go to work
without modules support [2].
* Because we're no longer building in a `$GOPATH`-based location,
let's instruct the CircleCI base image to archive the new location,
too.
* Similarly, teach the RPM spec to build in a non-$GOPATH location.
* By contrast, since we use dh_golang to build git-lfs binaries on
Debian, let's wait until the upstream dh_golang package is released
with support for Go 1.11 module support explicitly. Therefore, force
GO111MODULE to be on so that we can build a copy of Git LFS whose
checkout is within a $GOPATH.
Although the go.mod versions match the glide.yaml ones, the diff
attached is large because Go Modules do not vendor `_test.go` files,
whereas Glide does.
[1]: https://golang.org/doc/go1.11#modules
[2]: `GOCACHE=on` will be deprecated in Go 1.12, so this change makes
sense for that reason, too.
Co-authored-by: brian m. carlson <bk2204@github.com>
In [1], it is noted that script/install.sh is no longer executable when
run from the tarball'd and zip'd release artifacts that we distribute as
a part of the normal Git LFS release process.
This is due to 22351986 (Makefile: replace script/bootstrap with
'make', 'make all', 2018-07-19), where this file lost permission
100755 and gained permission 100644.
The later is no longer executable, but needs to be. So, let's restore it
as such.
[1]: https://github.com/git-lfs/git-lfs/issues/3154
