mirror of
https://gitlab.kitware.com/vtk/vtk-m
synced 2024-09-08 13:23:51 +00:00
Add 3rd-party harness.
Add docs and scripts for 3rd party modules in VTK-m. This is an import of scripts from VTK.
This commit is contained in:
Utkarsh Ayachit
parent
fc2f9f3391
commit
2aab6ba47e
105
vtkm/thirdparty/UPDATING.md
vendored
Normal file
105
vtkm/thirdparty/UPDATING.md
vendored
Normal file
@ -0,0 +1,105 @@
|
||||
# Updating Third Party Projects
|
||||
|
||||
When updating a third party project, any changes to the imported project
|
||||
itself (e.g., the `diy/vtkmdiy` directory for diy), should go through the
|
||||
`update.sh` framework. This framework ensures that all patches to the third
|
||||
party projects are tracked externally and available for (preferably) upstream
|
||||
or other projects also embedding the library.
|
||||
|
||||
# Updating a Project
|
||||
|
||||
Once converted, a project should be updated by applying patches to the
|
||||
repository specified in its `update.sh` script. Once the changes are merged,
|
||||
pulling the changes involves running the `update.sh` script. This will update
|
||||
the local copy of the project to the version specified in `update.sh` (usually
|
||||
a `for/foo` branch, like `for/vtk-m` for example, but may be `master` or any
|
||||
other Git reference) and merge it into the main tree.
|
||||
|
||||
This requires a Git 2.5 or higher due the `worktree` tool being used to
|
||||
simplify the availability of the commits to the main checkout.
|
||||
|
||||
Here's an example of updating the `diy` project from tag v2.0 to v2.1,
|
||||
starting with updating the third-party repo
|
||||
|
||||
```sh
|
||||
$ cd diy
|
||||
$ git checkout for/vtk-m
|
||||
$ git fetch origin
|
||||
$ git rebase --onto v2.1 v2.0
|
||||
$ git push
|
||||
```
|
||||
|
||||
Now import into VTK-m
|
||||
|
||||
```sh
|
||||
$ cd vtkm/ThirdParty/diy
|
||||
$ git checkout -b update_diy
|
||||
$ ./update.sh
|
||||
```
|
||||
|
||||
Now you can review the change and make a merge request from the branch as normal.
|
||||
|
||||
# Porting a Project
|
||||
|
||||
When converting a project, if there are any local patches, a project should be
|
||||
created [on GitLab](https://gitlab.kitware.com/third-party) to track it. If
|
||||
the upstream project does not use Git, it should be imported into Git (there
|
||||
may be existing conversions available on Github already). The project's
|
||||
description should indicate where the source repository lives.
|
||||
|
||||
Once a mirror of the project is created, a branch named `for/foo` should be
|
||||
created where patches for the `foo` project will be applied (i.e., `for/vtk-m`
|
||||
for VTK-m's patches to the project). Usually, changes to the build system, the
|
||||
source code for mangling, the addition of `.gitattributes` files, and other
|
||||
changes belong here. Functional changes should be submitted upstream (but may
|
||||
still be tracked so that they may be used).
|
||||
|
||||
The basic steps to import a project `diy` based on the tag
|
||||
`v2.0` looks like this:
|
||||
|
||||
```sh
|
||||
$ git clone https://github.com/diatomic/diy.git
|
||||
$ cd diy/
|
||||
$ git remote add kitware git@gitlab.kitware.com:third-party/diy.git
|
||||
$ git push -u kitware
|
||||
$ git push -u kitware --tags
|
||||
$ git checkout v2.0
|
||||
$ git checkout -b for/vtk-m
|
||||
$ git push --set-upstream kitware for/vtk-m
|
||||
```
|
||||
|
||||
Making the initial import involves filling out the project's `update.sh`
|
||||
script in its directory. The [update-common.sh](update-common.sh) script
|
||||
describes what is necessary, but in a nutshell, it is basically metadata such
|
||||
as the name of the project and where it goes in the importing project.
|
||||
|
||||
The most important bit is the `extract_source` function which should subset
|
||||
the repository. If all that needs to be done is to extract the files given in
|
||||
the `paths` variable (described in the `update-common.sh` script), the
|
||||
`git_archive` function may be used if the `git archive` tool generates a
|
||||
suitable subset.
|
||||
|
||||
Make sure `update.sh` is executable before commit. On Unix, run:
|
||||
|
||||
```sh
|
||||
$ chmod u+x update.sh && git add -u update.sh
|
||||
```
|
||||
|
||||
On Windows, run:
|
||||
|
||||
```sh
|
||||
$ git update-index --chmod=+x update.sh
|
||||
```
|
||||
|
||||
# Process
|
||||
|
||||
The basic process involves a second branch where the third party project's
|
||||
changes are tracked. This branch has a commit for each time it has been
|
||||
updated and is stripped to only contain the relevant parts (no unit tests,
|
||||
documentation, etc.). This branch is then merged into the main branch as a
|
||||
subdirectory using the `subtree` merge strategy.
|
||||
|
||||
Initial conversions will require a manual push by the maintainers since the
|
||||
conversion involves a root commit which is not allowed under normal
|
||||
circumstances. Please send an email to the mailing list asking for assistance
|
||||
if necessary.
|
||||
181
vtkm/thirdparty/update-common.sh
vendored
Normal file
181
vtkm/thirdparty/update-common.sh
vendored
Normal file
@ -0,0 +1,181 @@
|
||||
#=============================================================================
|
||||
# Copyright 2015-2016 Kitware, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#=============================================================================
|
||||
|
||||
########################################################################
|
||||
# Script for updating third party packages.
|
||||
#
|
||||
# This script should be sourced in a project-specific script which sets
|
||||
# the following variables:
|
||||
#
|
||||
# name
|
||||
# The name of the project.
|
||||
# ownership
|
||||
# A git author name/email for the commits.
|
||||
# subtree
|
||||
# The location of the thirdparty package within the main source
|
||||
# tree.
|
||||
# repo
|
||||
# The git repository to use as upstream.
|
||||
# tag
|
||||
# The tag, branch or commit hash to use for upstream.
|
||||
# shortlog
|
||||
# Optional. Set to 'true' to get a shortlog in the commit message.
|
||||
#
|
||||
# Additionally, an "extract_source" function must be defined. It will be
|
||||
# run within the checkout of the project on the requested tag. It should
|
||||
# should place the desired tree into $extractdir/$name-reduced. This
|
||||
# directory will be used as the newest commit for the project.
|
||||
#
|
||||
# For convenience, the function may use the "git_archive" function which
|
||||
# does a standard "git archive" extraction using the (optional) "paths"
|
||||
# variable to only extract a subset of the source tree.
|
||||
########################################################################
|
||||
|
||||
########################################################################
|
||||
# Utility functions
|
||||
########################################################################
|
||||
git_archive () {
|
||||
git archive --worktree-attributes --prefix="$name-reduced/" HEAD -- $paths | \
|
||||
tar -C "$extractdir" -x
|
||||
}
|
||||
|
||||
disable_custom_gitattributes() {
|
||||
pushd "${extractdir}/${name}-reduced"
|
||||
# Git does not allow custom attributes in a subdirectory where we
|
||||
# are about to merge the `.gitattributes` file, so disable them.
|
||||
sed -i '/^\[attr\]/ {s/^/#/}' .gitattributes
|
||||
popd
|
||||
}
|
||||
|
||||
die () {
|
||||
echo >&2 "$@"
|
||||
exit 1
|
||||
}
|
||||
|
||||
warn () {
|
||||
echo >&2 "warning: $@"
|
||||
}
|
||||
|
||||
readonly regex_date='20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
|
||||
readonly basehash_regex="$name $regex_date ([0-9a-f]*)"
|
||||
readonly basehash="$( git rev-list --author="$ownership" --grep="$basehash_regex" -n 1 HEAD )"
|
||||
readonly upstream_old_short="$( git cat-file commit "$basehash" | sed -n '/'"$basehash_regex"'/ {s/.*(//;s/)//;p}' | egrep '^[0-9a-f]+$' )"
|
||||
|
||||
########################################################################
|
||||
# Sanity checking
|
||||
########################################################################
|
||||
[ -n "$name" ] || \
|
||||
die "'name' is empty"
|
||||
[ -n "$ownership" ] || \
|
||||
die "'ownership' is empty"
|
||||
[ -n "$subtree" ] || \
|
||||
die "'subtree' is empty"
|
||||
[ -n "$repo" ] || \
|
||||
die "'repo' is empty"
|
||||
[ -n "$tag" ] || \
|
||||
die "'tag' is empty"
|
||||
[ -n "$basehash" ] || \
|
||||
warn "'basehash' is empty; performing initial import"
|
||||
readonly do_shortlog="${shortlog-false}"
|
||||
|
||||
readonly workdir="$PWD/work"
|
||||
readonly upstreamdir="$workdir/upstream"
|
||||
readonly extractdir="$workdir/extract"
|
||||
|
||||
[ -d "$workdir" ] && \
|
||||
die "error: workdir '$workdir' already exists"
|
||||
|
||||
trap "rm -rf '$workdir'" EXIT
|
||||
|
||||
# Get upstream
|
||||
git clone "$repo" "$upstreamdir"
|
||||
|
||||
if [ -n "$basehash" ]; then
|
||||
# Use the existing package's history
|
||||
git worktree add "$extractdir" "$basehash"
|
||||
# Clear out the working tree
|
||||
pushd "$extractdir"
|
||||
git ls-files | xargs rm -v
|
||||
find . -type d -empty -delete
|
||||
popd
|
||||
else
|
||||
# Create a repo to hold this package's history
|
||||
mkdir -p "$extractdir"
|
||||
git -C "$extractdir" init
|
||||
fi
|
||||
|
||||
# Extract the subset of upstream we care about
|
||||
pushd "$upstreamdir"
|
||||
git checkout "$tag"
|
||||
readonly upstream_hash="$( git rev-parse HEAD )"
|
||||
readonly upstream_hash_short="$( git rev-parse --short=8 "$upstream_hash" )"
|
||||
readonly upstream_datetime="$( git rev-list "$upstream_hash" --format='%ci' -n 1 | grep -e "^$regex_date" )"
|
||||
readonly upstream_date="$( echo "$upstream_datetime" | grep -o -e "$regex_date" )"
|
||||
if $do_shortlog && [ -n "$basehash" ]; then
|
||||
readonly commit_shortlog="
|
||||
|
||||
Upstream Shortlog
|
||||
-----------------
|
||||
|
||||
$( git shortlog --no-merges --abbrev=8 --format='%h %s' "$upstream_old_short".."$upstream_hash" )"
|
||||
else
|
||||
readonly commit_shortlog=""
|
||||
fi
|
||||
extract_source || \
|
||||
die "failed to extract source"
|
||||
popd
|
||||
|
||||
[ -d "$extractdir/$name-reduced" ] || \
|
||||
die "expected directory to extract does not exist"
|
||||
readonly commit_summary="$name $upstream_date ($upstream_hash_short)"
|
||||
|
||||
# Commit the subset
|
||||
pushd "$extractdir"
|
||||
mv -v "$name-reduced/"* .
|
||||
rmdir "$name-reduced/"
|
||||
git add -A .
|
||||
git commit -n --author="$ownership" --date="$upstream_datetime" -F - <<-EOF
|
||||
$commit_summary
|
||||
|
||||
Code extracted from:
|
||||
|
||||
$repo
|
||||
|
||||
at commit $upstream_hash ($tag).$commit_shortlog
|
||||
EOF
|
||||
git branch -f "upstream-$name"
|
||||
popd
|
||||
|
||||
# Merge the subset into this repository
|
||||
if [ -n "$basehash" ]; then
|
||||
git merge --log -s recursive "-Xsubtree=$subtree/" --no-commit "upstream-$name"
|
||||
else
|
||||
# Note: on Windows 'git merge' will open a browser, and the check will fail,
|
||||
# so use the flag by default.
|
||||
unrelated_histories_flag=""
|
||||
if git --version | grep -q windows; then
|
||||
unrelated_histories_flag="--allow-unrelated-histories "
|
||||
elif git merge --help | grep -q -e allow-unrelated-histories; then
|
||||
unrelated_histories_flag="--allow-unrelated-histories "
|
||||
fi
|
||||
readonly unrelated_histories_flag
|
||||
|
||||
git fetch "$extractdir" "+upstream-$name:upstream-$name"
|
||||
git merge --log -s ours --no-commit $unrelated_histories_flag "upstream-$name"
|
||||
git read-tree -u --prefix="$subtree/" "upstream-$name"
|
||||
fi
|
||||
git commit --no-edit
|
||||
git branch -d "upstream-$name"
|
||||
Loading…
Reference in New Issue
Block a user