2017-06-07 17:47:58 +00:00
|
|
|
package commands
|
|
|
|
|
|
|
|
import (
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
"bufio"
|
2017-11-16 23:44:20 +00:00
|
|
|
"fmt"
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
"io"
|
2017-06-09 23:37:15 +00:00
|
|
|
"path/filepath"
|
2017-06-09 23:38:59 +00:00
|
|
|
"strings"
|
2017-06-09 23:37:15 +00:00
|
|
|
|
2017-06-09 23:38:16 +00:00
|
|
|
"github.com/git-lfs/git-lfs/errors"
|
2017-06-09 23:37:15 +00:00
|
|
|
"github.com/git-lfs/git-lfs/git"
|
2017-06-09 23:40:42 +00:00
|
|
|
"github.com/git-lfs/git-lfs/git/githistory"
|
2017-11-22 22:07:24 +00:00
|
|
|
"github.com/git-lfs/git-lfs/tasklog"
|
2018-07-05 16:49:10 +00:00
|
|
|
"github.com/git-lfs/gitobj"
|
2017-06-07 17:47:58 +00:00
|
|
|
"github.com/spf13/cobra"
|
|
|
|
)
|
|
|
|
|
2017-06-09 23:35:11 +00:00
|
|
|
var (
|
|
|
|
// migrateIncludeRefs is a set of Git references to explicitly include
|
|
|
|
// in the migration.
|
|
|
|
migrateIncludeRefs []string
|
|
|
|
// migrateExcludeRefs is a set of Git references to explicitly exclude
|
|
|
|
// in the migration.
|
|
|
|
migrateExcludeRefs []string
|
2017-09-07 17:47:17 +00:00
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
// migrateYes indicates that an answer of 'yes' should be presumed
|
|
|
|
// whenever 'git lfs migrate' asks for user input.
|
|
|
|
migrateYes bool
|
|
|
|
|
2017-11-20 17:48:50 +00:00
|
|
|
// migrateSkipFetch assumes that the client has the latest copy of
|
|
|
|
// remote references, and thus should not contact the remote for a set
|
|
|
|
// of updated references.
|
|
|
|
migrateSkipFetch bool
|
2017-11-17 16:45:30 +00:00
|
|
|
|
2017-09-07 17:47:17 +00:00
|
|
|
// migrateEverything indicates the presence of the --everything flag,
|
|
|
|
// and instructs 'git lfs migrate' to migrate all local references.
|
|
|
|
migrateEverything bool
|
2017-09-22 18:43:04 +00:00
|
|
|
|
|
|
|
// migrateVerbose enables verbose logging
|
|
|
|
migrateVerbose bool
|
2018-03-19 13:41:35 +00:00
|
|
|
|
|
|
|
// objectMapFile is the path to the map of old sha1 to new sha1
|
|
|
|
// commits
|
|
|
|
objectMapFilePath string
|
2018-05-22 18:12:40 +00:00
|
|
|
|
|
|
|
// migrateNoRewrite is the flag indicating whether or not the
|
|
|
|
// command should rewrite git history
|
|
|
|
migrateNoRewrite bool
|
2018-05-23 16:36:36 +00:00
|
|
|
// migrateCommitMessage is the message to use with the commit generated
|
|
|
|
// by the migrate command
|
|
|
|
migrateCommitMessage string
|
2018-06-27 18:03:23 +00:00
|
|
|
|
|
|
|
// exportRemote is the remote from which to download objects when
|
|
|
|
// performing an export
|
|
|
|
exportRemote string
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
|
|
|
|
// migrateFixup is the flag indicating whether or not to infer the
|
|
|
|
// included and excluded filepath patterns.
|
|
|
|
migrateFixup bool
|
2017-06-09 23:35:11 +00:00
|
|
|
)
|
2017-06-07 17:47:58 +00:00
|
|
|
|
2018-07-05 16:49:10 +00:00
|
|
|
// migrate takes the given command and arguments, *gitobj.ObjectDatabase, as well
|
2017-06-15 19:52:55 +00:00
|
|
|
// as a BlobRewriteFn to apply, and performs a migration.
|
2017-11-22 22:07:24 +00:00
|
|
|
func migrate(args []string, r *githistory.Rewriter, l *tasklog.Logger, opts *githistory.RewriteOptions) {
|
2017-06-09 23:41:41 +00:00
|
|
|
requireInRepo()
|
|
|
|
|
2017-08-29 21:06:35 +00:00
|
|
|
opts, err := rewriteOptions(args, opts, l)
|
2017-06-09 23:41:41 +00:00
|
|
|
if err != nil {
|
|
|
|
ExitWithError(err)
|
|
|
|
}
|
|
|
|
|
2017-06-21 16:42:36 +00:00
|
|
|
_, err = r.Rewrite(opts)
|
2017-06-09 23:41:41 +00:00
|
|
|
if err != nil {
|
|
|
|
ExitWithError(err)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:37:15 +00:00
|
|
|
// getObjectDatabase creates a *git.ObjectDatabase from the filesystem pointed
|
|
|
|
// at the .git directory of the currently checked-out repository.
|
2018-07-05 16:49:10 +00:00
|
|
|
func getObjectDatabase() (*gitobj.ObjectDatabase, error) {
|
2018-09-05 15:50:46 +00:00
|
|
|
dir, err := git.GitCommonDir()
|
2017-06-09 23:37:15 +00:00
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrap(err, "cannot open root")
|
|
|
|
}
|
2018-07-05 16:49:10 +00:00
|
|
|
return gitobj.FromFilesystem(filepath.Join(dir, "objects"), cfg.TempDir())
|
2017-06-09 23:37:15 +00:00
|
|
|
}
|
|
|
|
|
2017-06-09 23:40:42 +00:00
|
|
|
// rewriteOptions returns *githistory.RewriteOptions able to be passed to a
|
|
|
|
// *githistory.Rewriter that reflect the current arguments and flags passed to
|
2017-06-21 16:54:55 +00:00
|
|
|
// an invocation of git-lfs-migrate(1).
|
2017-06-09 23:40:42 +00:00
|
|
|
//
|
2017-06-21 16:54:55 +00:00
|
|
|
// It is merged with the given "opts". In other words, an identical "opts" is
|
|
|
|
// returned, where the Include and Exclude fields have been filled based on the
|
|
|
|
// following rules:
|
2017-06-09 23:40:42 +00:00
|
|
|
//
|
|
|
|
// The included and excluded references are determined based on the output of
|
|
|
|
// includeExcludeRefs (see below for documentation and detail).
|
|
|
|
//
|
|
|
|
// If any of the above could not be determined without error, that error will be
|
|
|
|
// returned immediately.
|
2017-11-22 22:07:24 +00:00
|
|
|
func rewriteOptions(args []string, opts *githistory.RewriteOptions, l *tasklog.Logger) (*githistory.RewriteOptions, error) {
|
2017-08-29 21:06:35 +00:00
|
|
|
include, exclude, err := includeExcludeRefs(l, args)
|
2017-06-09 23:40:42 +00:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
return &githistory.RewriteOptions{
|
|
|
|
Include: include,
|
|
|
|
Exclude: exclude,
|
|
|
|
|
2018-03-19 13:41:35 +00:00
|
|
|
UpdateRefs: opts.UpdateRefs,
|
|
|
|
Verbose: opts.Verbose,
|
|
|
|
ObjectMapFilePath: opts.ObjectMapFilePath,
|
2017-06-22 00:26:24 +00:00
|
|
|
|
2018-07-06 19:14:31 +00:00
|
|
|
BlobFn: opts.BlobFn,
|
|
|
|
TreePreCallbackFn: opts.TreePreCallbackFn,
|
|
|
|
TreeCallbackFn: opts.TreeCallbackFn,
|
2017-06-09 23:40:42 +00:00
|
|
|
}, nil
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:40:08 +00:00
|
|
|
// includeExcludeRefs returns fully-qualified sets of references to include, and
|
|
|
|
// exclude, or an error if those could not be determined.
|
|
|
|
//
|
|
|
|
// They are determined based on the following rules:
|
|
|
|
//
|
|
|
|
// - Include all local refs/heads/<branch> references for each branch
|
|
|
|
// specified as an argument.
|
|
|
|
// - Include the currently checked out branch if no branches are given as
|
|
|
|
// arguments and the --include-ref= or --exclude-ref= flag(s) aren't given.
|
|
|
|
// - Include all references given in --include-ref=<ref>.
|
|
|
|
// - Exclude all references given in --exclude-ref=<ref>.
|
2017-11-22 22:07:24 +00:00
|
|
|
func includeExcludeRefs(l *tasklog.Logger, args []string) (include, exclude []string, err error) {
|
2017-06-09 23:40:08 +00:00
|
|
|
hardcore := len(migrateIncludeRefs) > 0 || len(migrateExcludeRefs) > 0
|
|
|
|
|
2017-09-07 17:47:17 +00:00
|
|
|
if len(args) == 0 && !hardcore && !migrateEverything {
|
2017-06-09 23:40:08 +00:00
|
|
|
// If no branches were given explicitly AND neither
|
|
|
|
// --include-ref or --exclude-ref flags were given, then add the
|
|
|
|
// currently checked out reference.
|
|
|
|
current, err := currentRefToMigrate()
|
|
|
|
if err != nil {
|
|
|
|
return nil, nil, err
|
|
|
|
}
|
|
|
|
args = append(args, current.Name)
|
|
|
|
}
|
|
|
|
|
2017-09-07 17:47:17 +00:00
|
|
|
if migrateEverything && len(args) > 0 {
|
|
|
|
return nil, nil, errors.New("fatal: cannot use --everything with explicit reference arguments")
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:40:08 +00:00
|
|
|
for _, name := range args {
|
2017-12-13 19:13:11 +00:00
|
|
|
var excluded bool
|
|
|
|
if strings.HasPrefix("^", name) {
|
|
|
|
name = name[1:]
|
|
|
|
excluded = true
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:40:08 +00:00
|
|
|
// Then, loop through each branch given, resolve that reference,
|
|
|
|
// and include it.
|
|
|
|
ref, err := git.ResolveRef(name)
|
|
|
|
if err != nil {
|
|
|
|
return nil, nil, err
|
|
|
|
}
|
|
|
|
|
2017-12-13 19:13:11 +00:00
|
|
|
if excluded {
|
|
|
|
exclude = append(exclude, ref.Refspec())
|
|
|
|
} else {
|
|
|
|
include = append(include, ref.Refspec())
|
|
|
|
}
|
2017-06-09 23:40:08 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if hardcore {
|
2017-09-07 17:47:17 +00:00
|
|
|
if migrateEverything {
|
|
|
|
return nil, nil, errors.New("fatal: cannot use --everything with --include-ref or --exclude-ref")
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:40:08 +00:00
|
|
|
// If either --include-ref=<ref> or --exclude-ref=<ref> were
|
|
|
|
// given, append those to the include and excluded reference
|
|
|
|
// set, respectively.
|
|
|
|
include = append(include, migrateIncludeRefs...)
|
|
|
|
exclude = append(exclude, migrateExcludeRefs...)
|
2017-09-07 17:47:17 +00:00
|
|
|
} else if migrateEverything {
|
2018-07-09 19:43:26 +00:00
|
|
|
refs, err := git.AllRefsIn("")
|
2017-09-07 17:47:17 +00:00
|
|
|
if err != nil {
|
|
|
|
return nil, nil, err
|
|
|
|
}
|
|
|
|
|
2018-07-09 19:43:26 +00:00
|
|
|
for _, ref := range refs {
|
|
|
|
switch ref.Type {
|
|
|
|
case git.RefTypeLocalBranch, git.RefTypeLocalTag,
|
2018-09-25 21:15:04 +00:00
|
|
|
git.RefTypeRemoteBranch:
|
2018-07-09 19:43:26 +00:00
|
|
|
|
|
|
|
include = append(include, ref.Refspec())
|
|
|
|
case git.RefTypeOther:
|
|
|
|
parts := strings.SplitN(ref.Refspec(), "/", 3)
|
|
|
|
if len(parts) < 2 {
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
|
|
|
|
switch parts[1] {
|
|
|
|
// The following are GitLab-, GitHub-, VSTS-,
|
|
|
|
// and BitBucket-specific reference naming
|
|
|
|
// conventions.
|
|
|
|
case "merge-requests", "pull", "pull-requests":
|
|
|
|
include = append(include, ref.Refspec())
|
|
|
|
}
|
|
|
|
}
|
2017-09-07 17:47:17 +00:00
|
|
|
}
|
2017-06-09 23:40:08 +00:00
|
|
|
} else {
|
2017-12-05 02:53:08 +00:00
|
|
|
bare, err := git.IsBare()
|
2017-06-09 23:40:08 +00:00
|
|
|
if err != nil {
|
2017-12-05 02:53:08 +00:00
|
|
|
return nil, nil, errors.Wrap(err, "fatal: unable to determine bareness")
|
2017-06-09 23:40:08 +00:00
|
|
|
}
|
|
|
|
|
2017-12-05 02:53:08 +00:00
|
|
|
if !bare {
|
|
|
|
// Otherwise, if neither --include-ref=<ref> or
|
|
|
|
// --exclude-ref=<ref> were given, include no additional
|
|
|
|
// references, and exclude all remote references that
|
|
|
|
// are remote branches or remote tags.
|
|
|
|
remoteRefs, err := getRemoteRefs(l)
|
|
|
|
if err != nil {
|
|
|
|
return nil, nil, err
|
|
|
|
}
|
|
|
|
|
2018-09-25 21:36:14 +00:00
|
|
|
for remote, refs := range remoteRefs {
|
2018-09-25 21:15:04 +00:00
|
|
|
for _, ref := range refs {
|
2018-09-25 21:36:14 +00:00
|
|
|
exclude = append(exclude,
|
|
|
|
formatRefName(ref, remote))
|
commands/command_migrate.go: return all references by remote
We have used the getRemoteRefs since [1] to return a slice of all remote
references, without distinguishing which remote they came from.
The reason for doing so was that we had had modified each reference's
"name" (i.e., the last element of splitting the full reference name by
'/').
This was OK to do, since we modified the reference's name (i.e., the
last component of the fully qualified reference name as read from Git
when splitting on the '/' character) to include the remote name. This
allowed us to call '(*git.Ref).Refspec()', which would theoretically
format us a correctly printed reference name, for all kinds of
references.
This is broken in practice, so to prepare for a future commit that will
fix it, let's return a map of remote name to all references present on
that remote, such that we can format each based on its remote and full
name individually.
[1]: ce89eb35 (commands/command_migrate: teach how to find all remote
refs, 2017-06-09)
2018-09-25 03:47:35 +00:00
|
|
|
}
|
2017-12-05 02:53:08 +00:00
|
|
|
}
|
2017-11-16 23:44:20 +00:00
|
|
|
}
|
2017-06-09 23:40:08 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return include, exclude, nil
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:39:34 +00:00
|
|
|
// getRemoteRefs returns a fully qualified set of references belonging to all
|
|
|
|
// remotes known by the currently checked-out repository, or an error if those
|
|
|
|
// references could not be determined.
|
commands/command_migrate.go: return all references by remote
We have used the getRemoteRefs since [1] to return a slice of all remote
references, without distinguishing which remote they came from.
The reason for doing so was that we had had modified each reference's
"name" (i.e., the last element of splitting the full reference name by
'/').
This was OK to do, since we modified the reference's name (i.e., the
last component of the fully qualified reference name as read from Git
when splitting on the '/' character) to include the remote name. This
allowed us to call '(*git.Ref).Refspec()', which would theoretically
format us a correctly printed reference name, for all kinds of
references.
This is broken in practice, so to prepare for a future commit that will
fix it, let's return a map of remote name to all references present on
that remote, such that we can format each based on its remote and full
name individually.
[1]: ce89eb35 (commands/command_migrate: teach how to find all remote
refs, 2017-06-09)
2018-09-25 03:47:35 +00:00
|
|
|
func getRemoteRefs(l *tasklog.Logger) (map[string][]*git.Ref, error) {
|
|
|
|
refs := make(map[string][]*git.Ref)
|
2017-06-09 23:39:34 +00:00
|
|
|
|
|
|
|
remotes, err := git.RemoteList()
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2017-11-20 17:48:50 +00:00
|
|
|
if !migrateSkipFetch {
|
2017-11-17 16:45:30 +00:00
|
|
|
w := l.Waiter("migrate: Fetching remote refs")
|
|
|
|
if err := git.Fetch(remotes...); err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
w.Complete()
|
2017-08-29 21:06:20 +00:00
|
|
|
}
|
|
|
|
|
2017-06-09 23:39:34 +00:00
|
|
|
for _, remote := range remotes {
|
2017-11-17 16:45:30 +00:00
|
|
|
var refsForRemote []*git.Ref
|
2017-11-20 17:48:50 +00:00
|
|
|
if migrateSkipFetch {
|
2017-11-17 16:45:30 +00:00
|
|
|
refsForRemote, err = git.CachedRemoteRefs(remote)
|
|
|
|
} else {
|
|
|
|
refsForRemote, err = git.RemoteRefs(remote)
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:39:34 +00:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
commands/command_migrate.go: return all references by remote
We have used the getRemoteRefs since [1] to return a slice of all remote
references, without distinguishing which remote they came from.
The reason for doing so was that we had had modified each reference's
"name" (i.e., the last element of splitting the full reference name by
'/').
This was OK to do, since we modified the reference's name (i.e., the
last component of the fully qualified reference name as read from Git
when splitting on the '/' character) to include the remote name. This
allowed us to call '(*git.Ref).Refspec()', which would theoretically
format us a correctly printed reference name, for all kinds of
references.
This is broken in practice, so to prepare for a future commit that will
fix it, let's return a map of remote name to all references present on
that remote, such that we can format each based on its remote and full
name individually.
[1]: ce89eb35 (commands/command_migrate: teach how to find all remote
refs, 2017-06-09)
2018-09-25 03:47:35 +00:00
|
|
|
refs[remote] = refsForRemote
|
2017-06-09 23:39:34 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return refs, nil
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:38:59 +00:00
|
|
|
// formatRefName returns the fully-qualified name for the given Git reference
|
|
|
|
// "ref".
|
|
|
|
func formatRefName(ref *git.Ref, remote string) string {
|
2018-09-25 21:31:36 +00:00
|
|
|
if ref.Type == git.RefTypeRemoteBranch {
|
|
|
|
return strings.Join([]string{
|
|
|
|
"refs", "remotes", remote, ref.Name}, "/")
|
2017-06-09 23:38:59 +00:00
|
|
|
}
|
2018-09-25 21:31:36 +00:00
|
|
|
return ref.Refspec()
|
2017-06-09 23:38:59 +00:00
|
|
|
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:38:16 +00:00
|
|
|
// currentRefToMigrate returns the fully-qualified name of the currently
|
|
|
|
// checked-out reference, or an error if the reference's type was not a local
|
|
|
|
// branch.
|
|
|
|
func currentRefToMigrate() (*git.Ref, error) {
|
|
|
|
current, err := git.CurrentRef()
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
if current.Type == git.RefTypeOther ||
|
2018-09-25 21:15:04 +00:00
|
|
|
current.Type == git.RefTypeRemoteBranch {
|
2017-06-09 23:38:16 +00:00
|
|
|
|
|
|
|
return nil, errors.Errorf("fatal: cannot migrate non-local ref: %s", current.Name)
|
|
|
|
}
|
|
|
|
return current, nil
|
|
|
|
}
|
|
|
|
|
2017-06-09 23:41:15 +00:00
|
|
|
// getHistoryRewriter returns a history rewriter that includes the filepath
|
2018-07-09 17:20:43 +00:00
|
|
|
// filter given by the --include and --exclude arguments.
|
2018-07-05 16:49:10 +00:00
|
|
|
func getHistoryRewriter(cmd *cobra.Command, db *gitobj.ObjectDatabase, l *tasklog.Logger) *githistory.Rewriter {
|
2017-06-09 23:41:15 +00:00
|
|
|
include, exclude := getIncludeExcludeArgs(cmd)
|
|
|
|
filter := buildFilepathFilter(cfg, include, exclude)
|
|
|
|
|
2017-06-14 22:10:42 +00:00
|
|
|
return githistory.NewRewriter(db,
|
2017-08-29 21:05:11 +00:00
|
|
|
githistory.WithFilter(filter), githistory.WithLogger(l))
|
2017-06-09 23:41:15 +00:00
|
|
|
}
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
func ensureWorkingCopyClean(in io.Reader, out io.Writer) {
|
|
|
|
dirty, err := git.IsWorkingCopyDirty()
|
|
|
|
if err != nil {
|
|
|
|
ExitWithError(errors.Wrap(err,
|
|
|
|
"fatal: could not determine if working copy is dirty"))
|
|
|
|
}
|
|
|
|
|
|
|
|
if !dirty {
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
var proceed bool
|
|
|
|
if migrateYes {
|
|
|
|
proceed = true
|
|
|
|
} else {
|
|
|
|
answer := bufio.NewReader(in)
|
|
|
|
L:
|
|
|
|
for {
|
|
|
|
fmt.Fprintf(out, "migrate: override changes in your working copy? [Y/n] ")
|
commands/command_migrate.go: print newline conditionally
Sometimes when invoking 'git lfs migrate import', mysterious output like
the following can be seen:
$ git lfs migrate import
migrate: override changes in your working copy? [Y/n] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done
Where an extra newline is printed between the answer, 'y', and the next
line of output from the migrator.
Instead, let's only print that secondary newline when one isn't given in
the answer. This should never be the case (c.f., ReadString()), but will
harden the code to changes like opening /dev/tty in raw mode and reading
character-by-character.
To do so, ensure that the answer doesn't satisfy:
strings.HasSuffix(answer, '\n')
...and print a newline iff it doesn't.
In a related sense, ignore io.EOF errors.
2018-07-16 18:14:20 +00:00
|
|
|
s, err := answer.ReadString('\n')
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
if err != nil {
|
commands/command_migrate.go: print newline conditionally
Sometimes when invoking 'git lfs migrate import', mysterious output like
the following can be seen:
$ git lfs migrate import
migrate: override changes in your working copy? [Y/n] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done
Where an extra newline is printed between the answer, 'y', and the next
line of output from the migrator.
Instead, let's only print that secondary newline when one isn't given in
the answer. This should never be the case (c.f., ReadString()), but will
harden the code to changes like opening /dev/tty in raw mode and reading
character-by-character.
To do so, ensure that the answer doesn't satisfy:
strings.HasSuffix(answer, '\n')
...and print a newline iff it doesn't.
In a related sense, ignore io.EOF errors.
2018-07-16 18:14:20 +00:00
|
|
|
if err == io.EOF {
|
|
|
|
break L
|
|
|
|
}
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
ExitWithError(errors.Wrap(err,
|
|
|
|
"fatal: could not read answer"))
|
|
|
|
}
|
|
|
|
|
commands/command_migrate.go: print newline conditionally
Sometimes when invoking 'git lfs migrate import', mysterious output like
the following can be seen:
$ git lfs migrate import
migrate: override changes in your working copy? [Y/n] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done
Where an extra newline is printed between the answer, 'y', and the next
line of output from the migrator.
Instead, let's only print that secondary newline when one isn't given in
the answer. This should never be the case (c.f., ReadString()), but will
harden the code to changes like opening /dev/tty in raw mode and reading
character-by-character.
To do so, ensure that the answer doesn't satisfy:
strings.HasSuffix(answer, '\n')
...and print a newline iff it doesn't.
In a related sense, ignore io.EOF errors.
2018-07-16 18:14:20 +00:00
|
|
|
switch strings.TrimSpace(s) {
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
case "n", "N":
|
|
|
|
proceed = false
|
|
|
|
break L
|
|
|
|
case "y", "Y":
|
|
|
|
proceed = true
|
|
|
|
break L
|
|
|
|
}
|
commands/command_migrate.go: print newline conditionally
Sometimes when invoking 'git lfs migrate import', mysterious output like
the following can be seen:
$ git lfs migrate import
migrate: override changes in your working copy? [Y/n] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done
Where an extra newline is printed between the answer, 'y', and the next
line of output from the migrator.
Instead, let's only print that secondary newline when one isn't given in
the answer. This should never be the case (c.f., ReadString()), but will
harden the code to changes like opening /dev/tty in raw mode and reading
character-by-character.
To do so, ensure that the answer doesn't satisfy:
strings.HasSuffix(answer, '\n')
...and print a newline iff it doesn't.
In a related sense, ignore io.EOF errors.
2018-07-16 18:14:20 +00:00
|
|
|
|
|
|
|
if !strings.HasSuffix(s, "\n") {
|
|
|
|
fmt.Fprintf(out, "\n")
|
|
|
|
}
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if proceed {
|
|
|
|
fmt.Fprintf(out, "migrate: changes in your working copy will be overridden ...\n")
|
|
|
|
} else {
|
|
|
|
Exit("migrate: working copy must not be dirty")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-06-07 17:47:58 +00:00
|
|
|
func init() {
|
2017-06-12 21:22:49 +00:00
|
|
|
info := NewCommand("info", migrateInfoCommand)
|
|
|
|
info.Flags().IntVar(&migrateInfoTopN, "top", 5, "--top=<n>")
|
2017-07-31 17:47:13 +00:00
|
|
|
info.Flags().StringVar(&migrateInfoAboveFmt, "above", "", "--above=<n>")
|
2017-06-15 21:08:15 +00:00
|
|
|
info.Flags().StringVar(&migrateInfoUnitFmt, "unit", "", "--unit=<unit>")
|
2017-06-12 21:22:49 +00:00
|
|
|
|
2017-06-21 16:43:56 +00:00
|
|
|
importCmd := NewCommand("import", migrateImportCommand)
|
2017-09-22 18:43:04 +00:00
|
|
|
importCmd.Flags().BoolVar(&migrateVerbose, "verbose", false, "Verbose logging")
|
2018-03-19 13:41:35 +00:00
|
|
|
importCmd.Flags().StringVar(&objectMapFilePath, "object-map", "", "Object map file")
|
2018-05-22 18:12:40 +00:00
|
|
|
importCmd.Flags().BoolVar(&migrateNoRewrite, "no-rewrite", false, "Add new history without rewriting previous")
|
2018-05-23 16:36:36 +00:00
|
|
|
importCmd.Flags().StringVarP(&migrateCommitMessage, "message", "m", "", "With --no-rewrite, an optional commit message")
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
importCmd.Flags().BoolVar(&migrateFixup, "fixup", false, "Infer filepaths based on .gitattributes")
|
2017-06-21 16:43:56 +00:00
|
|
|
|
2018-06-13 21:10:32 +00:00
|
|
|
exportCmd := NewCommand("export", migrateExportCommand)
|
2018-06-25 20:06:43 +00:00
|
|
|
exportCmd.Flags().BoolVar(&migrateVerbose, "verbose", false, "Verbose logging")
|
|
|
|
exportCmd.Flags().StringVar(&objectMapFilePath, "object-map", "", "Object map file")
|
2018-06-27 18:03:23 +00:00
|
|
|
exportCmd.Flags().StringVar(&exportRemote, "remote", "", "Remote from which to download objects")
|
2018-06-13 21:10:32 +00:00
|
|
|
|
2017-06-09 23:35:11 +00:00
|
|
|
RegisterCommand("migrate", nil, func(cmd *cobra.Command) {
|
2017-09-08 17:57:52 +00:00
|
|
|
cmd.PersistentFlags().StringVarP(&includeArg, "include", "I", "", "Include a list of paths")
|
|
|
|
cmd.PersistentFlags().StringVarP(&excludeArg, "exclude", "X", "", "Exclude a list of paths")
|
|
|
|
|
|
|
|
cmd.PersistentFlags().StringSliceVar(&migrateIncludeRefs, "include-ref", nil, "An explicit list of refs to include")
|
|
|
|
cmd.PersistentFlags().StringSliceVar(&migrateExcludeRefs, "exclude-ref", nil, "An explicit list of refs to exclude")
|
|
|
|
cmd.PersistentFlags().BoolVar(&migrateEverything, "everything", false, "Migrate all local references")
|
2017-11-20 17:48:50 +00:00
|
|
|
cmd.PersistentFlags().BoolVar(&migrateSkipFetch, "skip-fetch", false, "Assume up-to-date remote references.")
|
2017-06-09 23:35:11 +00:00
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
cmd.PersistentFlags().BoolVarP(&migrateYes, "yes", "y", false, "Don't prompt for answers.")
|
|
|
|
|
2018-06-25 20:06:43 +00:00
|
|
|
cmd.AddCommand(exportCmd, importCmd, info)
|
2017-06-09 23:35:11 +00:00
|
|
|
})
|
2017-06-07 17:47:58 +00:00
|
|
|
}
|