diff options
| -rw-r--r-- | Documentation/git-pack-refs.adoc | 53 | ||||
| -rw-r--r-- | Documentation/git-refs.adoc | 10 | ||||
| -rw-r--r-- | Documentation/pack-refs-options.adoc | 52 | ||||
| -rw-r--r-- | Makefile | 1 | ||||
| -rw-r--r-- | builtin/pack-refs.c | 54 | ||||
| -rw-r--r-- | builtin/refs.c | 17 | ||||
| -rw-r--r-- | meson.build | 1 | ||||
| -rw-r--r-- | pack-refs.c | 56 | ||||
| -rw-r--r-- | pack-refs.h | 23 | ||||
| -rw-r--r-- | refs.c | 5 | ||||
| -rw-r--r-- | refs.h | 6 | ||||
| -rw-r--r-- | refs/files-backend.c | 10 | ||||
| -rw-r--r-- | refs/refs-internal.h | 3 | ||||
| -rw-r--r-- | refs/reftable-backend.c | 7 | ||||
| -rw-r--r-- | t/meson.build | 1 | ||||
| -rw-r--r-- | t/pack-refs-tests.sh | 431 | ||||
| -rwxr-xr-x | t/t0601-reffiles-pack-refs.sh | 430 | ||||
| -rwxr-xr-x | t/t1463-refs-optimize.sh | 17 |
18 files changed, 647 insertions, 530 deletions
diff --git a/Documentation/git-pack-refs.adoc b/Documentation/git-pack-refs.adoc index 42b90051e6..fde9f2f294 100644 --- a/Documentation/git-pack-refs.adoc +++ b/Documentation/git-pack-refs.adoc @@ -45,58 +45,7 @@ unpacked. OPTIONS ------- ---all:: - -The command by default packs all tags and refs that are already -packed, and leaves other refs -alone. This is because branches are expected to be actively -developed and packing their tips does not help performance. -This option causes all refs to be packed as well, with the exception -of hidden refs, broken refs, and symbolic refs. Useful for a repository -with many branches of historical interests. - ---no-prune:: - -The command usually removes loose refs under `$GIT_DIR/refs` -hierarchy after packing them. This option tells it not to. - ---auto:: - -Pack refs as needed depending on the current state of the ref database. The -behavior depends on the ref format used by the repository and may change in the -future. -+ - - "files": Loose references are packed into the `packed-refs` file - based on the ratio of loose references to the size of the - `packed-refs` file. The bigger the `packed-refs` file, the more loose - references need to exist before we repack. -+ - - "reftable": Tables are compacted such that they form a geometric - sequence. For two tables N and N+1, where N+1 is newer, this - maintains the property that N is at least twice as big as N+1. Only - tables that violate this property are compacted. - ---include <pattern>:: - -Pack refs based on a `glob(7)` pattern. Repetitions of this option -accumulate inclusion patterns. If a ref is both included in `--include` and -`--exclude`, `--exclude` takes precedence. Using `--include` will preclude all -tags from being included by default. Symbolic refs and broken refs will never -be packed. When used with `--all`, it will be a noop. Use `--no-include` to clear -and reset the list of patterns. - ---exclude <pattern>:: - -Do not pack refs matching the given `glob(7)` pattern. Repetitions of this option -accumulate exclusion patterns. Use `--no-exclude` to clear and reset the list of -patterns. If a ref is already packed, including it with `--exclude` will not -unpack it. -+ -When used with `--all`, pack only loose refs which do not match any of -the provided `--exclude` patterns. -+ -When used with `--include`, refs provided to `--include`, minus refs that are -provided to `--exclude` will be packed. +include::pack-refs-options.adoc[] BUGS diff --git a/Documentation/git-refs.adoc b/Documentation/git-refs.adoc index bfa9b3ea2d..fa33680cc7 100644 --- a/Documentation/git-refs.adoc +++ b/Documentation/git-refs.adoc @@ -19,6 +19,7 @@ git refs list [--count=<count>] [--shell|--perl|--python|--tcl] [(--exclude=<pattern>)...] [--start-after=<marker>] [ --stdin | (<pattern>...)] git refs exists <ref> +git refs optimize [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>] DESCRIPTION ----------- @@ -45,6 +46,11 @@ exists:: failed with an error other than the reference being missing. This does not verify whether the reference resolves to an actual object. +optimize:: + Optimizes references to improve repository performance and reduce disk + usage. This subcommand is an alias for linkgit:git-pack-refs[1] and + offers identical functionality. + OPTIONS ------- @@ -80,6 +86,10 @@ The following options are specific to 'git refs list': include::for-each-ref-options.adoc[] +The following options are specific to 'git refs optimize': + +include::pack-refs-options.adoc[] + KNOWN LIMITATIONS ----------------- diff --git a/Documentation/pack-refs-options.adoc b/Documentation/pack-refs-options.adoc new file mode 100644 index 0000000000..0b11282941 --- /dev/null +++ b/Documentation/pack-refs-options.adoc @@ -0,0 +1,52 @@ +--all:: + +The command by default packs all tags and refs that are already +packed, and leaves other refs +alone. This is because branches are expected to be actively +developed and packing their tips does not help performance. +This option causes all refs to be packed as well, with the exception +of hidden refs, broken refs, and symbolic refs. Useful for a repository +with many branches of historical interests. + +--no-prune:: + +The command usually removes loose refs under `$GIT_DIR/refs` +hierarchy after packing them. This option tells it not to. + +--auto:: + +Pack refs as needed depending on the current state of the ref database. The +behavior depends on the ref format used by the repository and may change in the +future. ++ + - "files": Loose references are packed into the `packed-refs` file + based on the ratio of loose references to the size of the + `packed-refs` file. The bigger the `packed-refs` file, the more loose + references need to exist before we repack. ++ + - "reftable": Tables are compacted such that they form a geometric + sequence. For two tables N and N+1, where N+1 is newer, this + maintains the property that N is at least twice as big as N+1. Only + tables that violate this property are compacted. + +--include <pattern>:: + +Pack refs based on a `glob(7)` pattern. Repetitions of this option +accumulate inclusion patterns. If a ref is both included in `--include` and +`--exclude`, `--exclude` takes precedence. Using `--include` will preclude all +tags from being included by default. Symbolic refs and broken refs will never +be packed. When used with `--all`, it will be a noop. Use `--no-include` to clear +and reset the list of patterns. + +--exclude <pattern>:: + +Do not pack refs matching the given `glob(7)` pattern. Repetitions of this option +accumulate exclusion patterns. Use `--no-exclude` to clear and reset the list of +patterns. If a ref is already packed, including it with `--exclude` will not +unpack it. ++ +When used with `--all`, pack only loose refs which do not match any of +the provided `--exclude` patterns. ++ +When used with `--include`, refs provided to `--include`, minus refs that are +provided to `--exclude` will be packed. @@ -1093,6 +1093,7 @@ LIB_OBJS += pack-bitmap.o LIB_OBJS += pack-check.o LIB_OBJS += pack-mtimes.o LIB_OBJS += pack-objects.o +LIB_OBJS += pack-refs.o LIB_OBJS += pack-revindex.o LIB_OBJS += pack-write.o LIB_OBJS += packfile.o diff --git a/builtin/pack-refs.c b/builtin/pack-refs.c index 5e28d0f9e8..3446b84cda 100644 --- a/builtin/pack-refs.c +++ b/builtin/pack-refs.c @@ -1,60 +1,16 @@ #include "builtin.h" -#include "config.h" -#include "environment.h" #include "gettext.h" -#include "parse-options.h" -#include "refs.h" -#include "revision.h" - -static char const * const pack_refs_usage[] = { - N_("git pack-refs [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>]"), - NULL -}; +#include "pack-refs.h" int cmd_pack_refs(int argc, const char **argv, const char *prefix, struct repository *repo) { - struct ref_exclusions excludes = REF_EXCLUSIONS_INIT; - struct string_list included_refs = STRING_LIST_INIT_NODUP; - struct pack_refs_opts pack_refs_opts = { - .exclusions = &excludes, - .includes = &included_refs, - .flags = PACK_REFS_PRUNE, - }; - struct string_list option_excluded_refs = STRING_LIST_INIT_NODUP; - struct string_list_item *item; - int pack_all = 0; - int ret; - - struct option opts[] = { - OPT_BOOL(0, "all", &pack_all, N_("pack everything")), - OPT_BIT(0, "prune", &pack_refs_opts.flags, N_("prune loose refs (default)"), PACK_REFS_PRUNE), - OPT_BIT(0, "auto", &pack_refs_opts.flags, N_("auto-pack refs as needed"), PACK_REFS_AUTO), - OPT_STRING_LIST(0, "include", pack_refs_opts.includes, N_("pattern"), - N_("references to include")), - OPT_STRING_LIST(0, "exclude", &option_excluded_refs, N_("pattern"), - N_("references to exclude")), - OPT_END(), + static char const * const pack_refs_usage[] = { + N_("git pack-refs " PACK_REFS_OPTS), + NULL }; - repo_config(repo, git_default_config, NULL); - if (parse_options(argc, argv, prefix, opts, pack_refs_usage, 0)) - usage_with_options(pack_refs_usage, opts); - - for_each_string_list_item(item, &option_excluded_refs) - add_ref_exclusion(pack_refs_opts.exclusions, item->string); - - if (pack_all) - string_list_append(pack_refs_opts.includes, "*"); - - if (!pack_refs_opts.includes->nr) - string_list_append(pack_refs_opts.includes, "refs/tags/*"); - - ret = refs_pack_refs(get_main_ref_store(repo), &pack_refs_opts); - clear_ref_exclusions(&excludes); - string_list_clear(&included_refs, 0); - string_list_clear(&option_excluded_refs, 0); - return ret; + return pack_refs_core(argc, argv, prefix, repo, pack_refs_usage); } diff --git a/builtin/refs.c b/builtin/refs.c index 91548783b7..3064f888b2 100644 --- a/builtin/refs.c +++ b/builtin/refs.c @@ -2,6 +2,7 @@ #include "builtin.h" #include "config.h" #include "fsck.h" +#include "pack-refs.h" #include "parse-options.h" #include "refs.h" #include "strbuf.h" @@ -18,6 +19,9 @@ #define REFS_EXISTS_USAGE \ N_("git refs exists <ref>") +#define REFS_OPTIMIZE_USAGE \ + N_("git refs optimize " PACK_REFS_OPTS) + static int cmd_refs_migrate(int argc, const char **argv, const char *prefix, struct repository *repo UNUSED) { @@ -159,6 +163,17 @@ out: return ret; } +static int cmd_refs_optimize(int argc, const char **argv, const char *prefix, + struct repository *repo) +{ + static char const * const refs_optimize_usage[] = { + REFS_OPTIMIZE_USAGE, + NULL + }; + + return pack_refs_core(argc, argv, prefix, repo, refs_optimize_usage); +} + int cmd_refs(int argc, const char **argv, const char *prefix, @@ -169,6 +184,7 @@ int cmd_refs(int argc, REFS_VERIFY_USAGE, "git refs list " COMMON_USAGE_FOR_EACH_REF, REFS_EXISTS_USAGE, + REFS_OPTIMIZE_USAGE, NULL, }; parse_opt_subcommand_fn *fn = NULL; @@ -177,6 +193,7 @@ int cmd_refs(int argc, OPT_SUBCOMMAND("verify", &fn, cmd_refs_verify), OPT_SUBCOMMAND("list", &fn, cmd_refs_list), OPT_SUBCOMMAND("exists", &fn, cmd_refs_exists), + OPT_SUBCOMMAND("optimize", &fn, cmd_refs_optimize), OPT_END(), }; diff --git a/meson.build b/meson.build index fccb6d2eec..eae4b56c9d 100644 --- a/meson.build +++ b/meson.build @@ -406,6 +406,7 @@ libgit_sources = [ 'pack-check.c', 'pack-mtimes.c', 'pack-objects.c', + 'pack-refs.c', 'pack-revindex.c', 'pack-write.c', 'packfile.c', diff --git a/pack-refs.c b/pack-refs.c new file mode 100644 index 0000000000..1a5e07d8b8 --- /dev/null +++ b/pack-refs.c @@ -0,0 +1,56 @@ +#include "builtin.h" +#include "config.h" +#include "environment.h" +#include "pack-refs.h" +#include "parse-options.h" +#include "refs.h" +#include "revision.h" + +int pack_refs_core(int argc, + const char **argv, + const char *prefix, + struct repository *repo, + const char * const *usage_opts) +{ + struct ref_exclusions excludes = REF_EXCLUSIONS_INIT; + struct string_list included_refs = STRING_LIST_INIT_NODUP; + struct pack_refs_opts pack_refs_opts = { + .exclusions = &excludes, + .includes = &included_refs, + .flags = PACK_REFS_PRUNE, + }; + struct string_list option_excluded_refs = STRING_LIST_INIT_NODUP; + struct string_list_item *item; + int pack_all = 0; + int ret; + + struct option opts[] = { + OPT_BOOL(0, "all", &pack_all, N_("pack everything")), + OPT_BIT(0, "prune", &pack_refs_opts.flags, N_("prune loose refs (default)"), PACK_REFS_PRUNE), + OPT_BIT(0, "auto", &pack_refs_opts.flags, N_("auto-pack refs as needed"), PACK_REFS_AUTO), + OPT_STRING_LIST(0, "include", pack_refs_opts.includes, N_("pattern"), + N_("references to include")), + OPT_STRING_LIST(0, "exclude", &option_excluded_refs, N_("pattern"), + N_("references to exclude")), + OPT_END(), + }; + repo_config(repo, git_default_config, NULL); + if (parse_options(argc, argv, prefix, opts, usage_opts, 0)) + usage_with_options(usage_opts, opts); + + for_each_string_list_item(item, &option_excluded_refs) + add_ref_exclusion(pack_refs_opts.exclusions, item->string); + + if (pack_all) + string_list_append(pack_refs_opts.includes, "*"); + + if (!pack_refs_opts.includes->nr) + string_list_append(pack_refs_opts.includes, "refs/tags/*"); + + ret = refs_optimize(get_main_ref_store(repo), &pack_refs_opts); + + clear_ref_exclusions(&excludes); + string_list_clear(&included_refs, 0); + string_list_clear(&option_excluded_refs, 0); + return ret; +} diff --git a/pack-refs.h b/pack-refs.h new file mode 100644 index 0000000000..5de27e7da8 --- /dev/null +++ b/pack-refs.h @@ -0,0 +1,23 @@ +#ifndef PACK_REFS_H +#define PACK_REFS_H + +struct repository; + +/* + * Shared usage string for options common to git-pack-refs(1) + * and git-refs-optimize(1). The command-specific part (e.g., "git refs optimize ") + * must be prepended by the caller. + */ +#define PACK_REFS_OPTS \ + "[--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>]" + +/* + * The core logic for pack-refs and its clones. + */ +int pack_refs_core(int argc, + const char **argv, + const char *prefix, + struct repository *repo, + const char * const *usage_opts); + +#endif /* PACK_REFS_H */ @@ -2304,6 +2304,11 @@ int refs_pack_refs(struct ref_store *refs, struct pack_refs_opts *opts) return refs->be->pack_refs(refs, opts); } +int refs_optimize(struct ref_store *refs, struct pack_refs_opts *opts) +{ + return refs->be->optimize(refs, opts); +} + int peel_iterated_oid(struct repository *r, const struct object_id *base, struct object_id *peeled) { if (current_ref_iter && @@ -483,6 +483,12 @@ struct pack_refs_opts { int refs_pack_refs(struct ref_store *refs, struct pack_refs_opts *opts); /* + * Optimize the ref store. The exact behavior is up to the backend. + * For the files backend, this is equivalent to packing refs. + */ +int refs_optimize(struct ref_store *refs, struct pack_refs_opts *opts); + +/* * Setup reflog before using. Fill in err and return -1 on failure. */ int refs_create_reflog(struct ref_store *refs, const char *refname, diff --git a/refs/files-backend.c b/refs/files-backend.c index bc3347d18c..bb2bec3807 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1528,6 +1528,15 @@ static int files_pack_refs(struct ref_store *ref_store, return 0; } +static int files_optimize(struct ref_store *ref_store, struct pack_refs_opts *opts) +{ + /* + * For the "files" backend, "optimizing" is the same as "packing". + * So, we just call the existing worker function for packing. + */ + return files_pack_refs(ref_store, opts); +} + /* * People using contrib's git-new-workdir have .git/logs/refs -> * /some/other/path/.git/logs/refs, and that may live on another device. @@ -3989,6 +3998,7 @@ struct ref_storage_be refs_be_files = { .transaction_abort = files_transaction_abort, .pack_refs = files_pack_refs, + .optimize = files_optimize, .rename_ref = files_rename_ref, .copy_ref = files_copy_ref, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 54c2079c12..4ef3bd75c6 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -447,6 +447,8 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs, typedef int pack_refs_fn(struct ref_store *ref_store, struct pack_refs_opts *opts); +typedef int optimize_fn(struct ref_store *ref_store, + struct pack_refs_opts *opts); typedef int rename_ref_fn(struct ref_store *ref_store, const char *oldref, const char *newref, const char *logmsg); @@ -572,6 +574,7 @@ struct ref_storage_be { ref_transaction_abort_fn *transaction_abort; pack_refs_fn *pack_refs; + optimize_fn *optimize; rename_ref_fn *rename_ref; copy_ref_fn *copy_ref; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 9e889da2ff..9884b876c1 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1741,6 +1741,12 @@ out: return ret; } +static int reftable_be_optimize(struct ref_store *ref_store, + struct pack_refs_opts *opts) +{ + return reftable_be_pack_refs(ref_store, opts); +} + struct write_create_symref_arg { struct reftable_ref_store *refs; struct reftable_stack *stack; @@ -2727,6 +2733,7 @@ struct ref_storage_be refs_be_reftable = { .transaction_abort = reftable_be_transaction_abort, .pack_refs = reftable_be_pack_refs, + .optimize = reftable_be_optimize, .rename_ref = reftable_be_rename_ref, .copy_ref = reftable_be_copy_ref, diff --git a/t/meson.build b/t/meson.build index 7974795fe4..d87166c0db 100644 --- a/t/meson.build +++ b/t/meson.build @@ -213,6 +213,7 @@ integration_tests = [ 't1460-refs-migrate.sh', 't1461-refs-list.sh', 't1462-refs-exists.sh', + 't1463-refs-optimize.sh', 't1500-rev-parse.sh', 't1501-work-tree.sh', 't1502-rev-parse-parseopt.sh', diff --git a/t/pack-refs-tests.sh b/t/pack-refs-tests.sh new file mode 100644 index 0000000000..3dbcc01718 --- /dev/null +++ b/t/pack-refs-tests.sh @@ -0,0 +1,431 @@ +pack_refs=${pack_refs:-pack-refs} + +test_expect_success 'enable reflogs' ' + git config core.logallrefupdates true +' + +test_expect_success 'prepare a trivial repository' ' + echo Hello > A && + git update-index --add A && + git commit -m "Initial commit." && + HEAD=$(git rev-parse --verify HEAD) +' + +test_expect_success '${pack_refs} --prune --all' ' + test_path_is_missing .git/packed-refs && + git ${pack_refs} --no-prune --all && + test_path_is_file .git/packed-refs && + N=$(find .git/refs -type f | wc -l) && + test "$N" != 0 && + + git ${pack_refs} --prune --all && + test_path_is_file .git/packed-refs && + N=$(find .git/refs -type f) && + test -z "$N" +' + +SHA1= + +test_expect_success 'see if git show-ref works as expected' ' + git branch a && + SHA1=$(cat .git/refs/heads/a) && + echo "$SHA1 refs/heads/a" >expect && + git show-ref a >result && + test_cmp expect result +' + +test_expect_success 'see if a branch still exists when packed' ' + git branch b && + git ${pack_refs} --all && + rm -f .git/refs/heads/b && + echo "$SHA1 refs/heads/b" >expect && + git show-ref b >result && + test_cmp expect result +' + +test_expect_success 'git branch c/d should barf if branch c exists' ' + git branch c && + git ${pack_refs} --all && + rm -f .git/refs/heads/c && + test_must_fail git branch c/d +' + +test_expect_success 'see if a branch still exists after git ${pack_refs} --prune' ' + git branch e && + git ${pack_refs} --all --prune && + echo "$SHA1 refs/heads/e" >expect && + git show-ref e >result && + test_cmp expect result +' + +test_expect_success 'see if git ${pack_refs} --prune remove ref files' ' + git branch f && + git ${pack_refs} --all --prune && + ! test -f .git/refs/heads/f +' + +test_expect_success 'see if git ${pack_refs} --prune removes empty dirs' ' + git branch r/s/t && + git ${pack_refs} --all --prune && + ! test -e .git/refs/heads/r +' + +test_expect_success 'git branch g should work when git branch g/h has been deleted' ' + git branch g/h && + git ${pack_refs} --all --prune && + git branch -d g/h && + git branch g && + git ${pack_refs} --all && + git branch -d g +' + +test_expect_success 'git branch i/j/k should barf if branch i exists' ' + git branch i && + git ${pack_refs} --all --prune && + test_must_fail git branch i/j/k +' + +test_expect_success 'test git branch k after branch k/l/m and k/lm have been deleted' ' + git branch k/l && + git branch k/lm && + git branch -d k/l && + git branch k/l/m && + git branch -d k/l/m && + git branch -d k/lm && + git branch k +' + +test_expect_success 'test git branch n after some branch deletion and pruning' ' + git branch n/o && + git branch n/op && + git branch -d n/o && + git branch n/o/p && + git branch -d n/op && + git ${pack_refs} --all --prune && + git branch -d n/o/p && + git branch n +' + +test_expect_success 'test excluded refs are not packed' ' + git branch dont_pack1 && + git branch dont_pack2 && + git branch pack_this && + git ${pack_refs} --all --exclude "refs/heads/dont_pack*" && + test -f .git/refs/heads/dont_pack1 && + test -f .git/refs/heads/dont_pack2 && + ! test -f .git/refs/heads/pack_this' + +test_expect_success 'test --no-exclude refs clears excluded refs' ' + git branch dont_pack3 && + git branch dont_pack4 && + git ${pack_refs} --all --exclude "refs/heads/dont_pack*" --no-exclude && + ! test -f .git/refs/heads/dont_pack3 && + ! test -f .git/refs/heads/dont_pack4' + +test_expect_success 'test only included refs are packed' ' + git branch pack_this1 && + git branch pack_this2 && + git tag dont_pack5 && + git ${pack_refs} --include "refs/heads/pack_this*" && + test -f .git/refs/tags/dont_pack5 && + ! test -f .git/refs/heads/pack_this1 && + ! test -f .git/refs/heads/pack_this2' + +test_expect_success 'test --no-include refs clears included refs' ' + git branch pack1 && + git branch pack2 && + git ${pack_refs} --include "refs/heads/pack*" --no-include && + test -f .git/refs/heads/pack1 && + test -f .git/refs/heads/pack2' + +test_expect_success 'test --exclude takes precedence over --include' ' + git branch dont_pack5 && + git ${pack_refs} --include "refs/heads/pack*" --exclude "refs/heads/pack*" && + test -f .git/refs/heads/dont_pack5' + +test_expect_success 'see if up-to-date packed refs are preserved' ' + git branch q && + git ${pack_refs} --all --prune && + git update-ref refs/heads/q refs/heads/q && + ! test -f .git/refs/heads/q +' + +test_expect_success 'pack, prune and repack' ' + git tag foo && + git ${pack_refs} --all --prune && + git show-ref >all-of-them && + git ${pack_refs} && + git show-ref >again && + test_cmp all-of-them again +' + +test_expect_success 'explicit ${pack_refs} with dangling packed reference' ' + git commit --allow-empty -m "soon to be garbage-collected" && + git ${pack_refs} --all && + git reset --hard HEAD^ && + git reflog expire --expire=all --all && + git prune --expire=all && + git ${pack_refs} --all 2>result && + test_must_be_empty result +' + +test_expect_success 'delete ref with dangling packed version' ' + git checkout -b lamb && + git commit --allow-empty -m "future garbage" && + git ${pack_refs} --all && + git reset --hard HEAD^ && + git checkout main && + git reflog expire --expire=all --all && + git prune --expire=all && + git branch -d lamb 2>result && + test_must_be_empty result +' + +test_expect_success 'delete ref while another dangling packed ref' ' + git branch lamb && + git commit --allow-empty -m "future garbage" && + git ${pack_refs} --all && + git reset --hard HEAD^ && + git reflog expire --expire=all --all && + git prune --expire=all && + git branch -d lamb 2>result && + test_must_be_empty result +' + +test_expect_success 'pack ref directly below refs/' ' + git update-ref refs/top HEAD && + git ${pack_refs} --all --prune && + grep refs/top .git/packed-refs && + test_path_is_missing .git/refs/top +' + +test_expect_success 'do not pack ref in refs/bisect' ' + git update-ref refs/bisect/local HEAD && + git ${pack_refs} --all --prune && + ! grep refs/bisect/local .git/packed-refs >/dev/null && + test_path_is_file .git/refs/bisect/local +' + +test_expect_success 'disable reflogs' ' + git config core.logallrefupdates false && + rm -rf .git/logs +' + +test_expect_success 'create packed foo/bar/baz branch' ' + git branch foo/bar/baz && + git ${pack_refs} --all --prune && + test_path_is_missing .git/refs/heads/foo/bar/baz && + test_must_fail git reflog exists refs/heads/foo/bar/baz +' + +test_expect_success 'notice d/f conflict with existing directory' ' + test_must_fail git branch foo && + test_must_fail git branch foo/bar +' + +test_expect_success 'existing directory reports concrete ref' ' + test_must_fail git branch foo 2>stderr && + test_grep refs/heads/foo/bar/baz stderr +' + +test_expect_success 'notice d/f conflict with existing ref' ' + test_must_fail git branch foo/bar/baz/extra && + test_must_fail git branch foo/bar/baz/lots/of/extra/components +' + +test_expect_success 'reject packed-refs with unterminated line' ' + cp .git/packed-refs .git/packed-refs.bak && + test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && + printf "%s" "$HEAD refs/zzzzz" >>.git/packed-refs && + echo "fatal: unterminated line in .git/packed-refs: $HEAD refs/zzzzz" >expected_err && + test_must_fail git for-each-ref >out 2>err && + test_cmp expected_err err +' + +test_expect_success 'reject packed-refs containing junk' ' + cp .git/packed-refs .git/packed-refs.bak && + test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && + printf "%s\n" "bogus content" >>.git/packed-refs && + echo "fatal: unexpected line in .git/packed-refs: bogus content" >expected_err && + test_must_fail git for-each-ref >out 2>err && + test_cmp expected_err err +' + +test_expect_success 'reject packed-refs with a short SHA-1' ' + cp .git/packed-refs .git/packed-refs.bak && + test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && + printf "%.7s %s\n" $HEAD refs/zzzzz >>.git/packed-refs && + printf "fatal: unexpected line in .git/packed-refs: %.7s %s\n" $HEAD refs/zzzzz >expected_err && + test_must_fail git for-each-ref >out 2>err && + test_cmp expected_err err +' + +test_expect_success 'timeout if packed-refs.lock exists' ' + LOCK=.git/packed-refs.lock && + >"$LOCK" && + test_when_finished "rm -f $LOCK" && + test_must_fail git ${pack_refs} --all --prune +' + +test_expect_success 'retry acquiring packed-refs.lock' ' + LOCK=.git/packed-refs.lock && + >"$LOCK" && + test_when_finished "wait && rm -f $LOCK" && + { + ( sleep 1 && rm -f $LOCK ) & + } && + git -c core.packedrefstimeout=3000 ${pack_refs} --all --prune +' + +test_expect_success SYMLINKS 'pack symlinked packed-refs' ' + # First make sure that symlinking works when reading: + git update-ref refs/heads/lossy refs/heads/main && + git for-each-ref >all-refs-before && + mv .git/packed-refs .git/my-deviant-packed-refs && + ln -s my-deviant-packed-refs .git/packed-refs && + git for-each-ref >all-refs-linked && + test_cmp all-refs-before all-refs-linked && + git ${pack_refs} --all --prune && + git for-each-ref >all-refs-packed && + test_cmp all-refs-before all-refs-packed && + test -h .git/packed-refs && + test "$(test_readlink .git/packed-refs)" = "my-deviant-packed-refs" +' + +# The 'packed-refs' file is stored directly in .git/. This means it is global +# to the repository, and can only contain refs that are shared across all +# worktrees. +test_expect_success 'refs/worktree must not be packed' ' + test_commit initial && + test_commit wt1 && + test_commit wt2 && + git worktree add wt1 wt1 && + git worktree add wt2 wt2 && + git checkout initial && + git update-ref refs/worktree/foo HEAD && + git -C wt1 update-ref refs/worktree/foo HEAD && + git -C wt2 update-ref refs/worktree/foo HEAD && + git ${pack_refs} --all && + test_path_is_missing .git/refs/tags/wt1 && + test_path_is_file .git/refs/worktree/foo && + test_path_is_file .git/worktrees/wt1/refs/worktree/foo && + test_path_is_file .git/worktrees/wt2/refs/worktree/foo +' + +# we do not want to count on running ${pack_refs} to +# actually pack it, as it is perfectly reasonable to +# skip processing a broken ref +test_expect_success 'create packed-refs file with broken ref' ' + test_tick && git commit --allow-empty -m one && + recoverable=$(git rev-parse HEAD) && + test_tick && git commit --allow-empty -m two && + missing=$(git rev-parse HEAD) && + rm -f .git/refs/heads/main && + cat >.git/packed-refs <<-EOF && + $missing refs/heads/main + $recoverable refs/heads/other + EOF + echo $missing >expect && + git rev-parse refs/heads/main >actual && + test_cmp expect actual +' + +test_expect_success '${pack_refs} does not silently delete broken packed ref' ' + git ${pack_refs} --all --prune && + git rev-parse refs/heads/main >actual && + test_cmp expect actual +' + +test_expect_success '${pack_refs} does not drop broken refs during deletion' ' + git update-ref -d refs/heads/other && + git rev-parse refs/heads/main >actual && + test_cmp expect actual +' + +for command in "git ${pack_refs} --all --auto" "git maintenance run --task=${pack_refs} --auto" +do + test_expect_success "$command does not repack below 16 refs without packed-refs" ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + git config set maintenance.auto false && + git commit --allow-empty --message "initial" && + + # Create 14 additional references, which brings us to + # 15 together with the default branch. + printf "create refs/heads/loose-%d HEAD\n" $(test_seq 14) >stdin && + git update-ref --stdin <stdin && + test_path_is_missing .git/packed-refs && + git ${pack_refs} --auto --all && + test_path_is_missing .git/packed-refs && + + # Create the 16th reference, which should cause us to repack. + git update-ref refs/heads/loose-15 HEAD && + git ${pack_refs} --auto --all && + test_path_is_file .git/packed-refs + ) + ' + + test_expect_success "$command does not repack below 16 refs with small packed-refs" ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + git config set maintenance.auto false && + git commit --allow-empty --message "initial" && + + git ${pack_refs} --all && + test_line_count = 2 .git/packed-refs && + + # Create 15 loose references. + printf "create refs/heads/loose-%d HEAD\n" $(test_seq 15) >stdin && + git update-ref --stdin <stdin && + git ${pack_refs} --auto --all && + test_line_count = 2 .git/packed-refs && + + # Create the 16th loose reference, which should cause us to repack. + git update-ref refs/heads/loose-17 HEAD && + git ${pack_refs} --auto --all && + test_line_count = 18 .git/packed-refs + ) + ' + + test_expect_success "$command scales with size of packed-refs" ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + git config set maintenance.auto false && + git commit --allow-empty --message "initial" && + + # Create 99 packed refs. This should cause the heuristic + # to require more than the minimum amount of loose refs. + test_seq 99 | + while read i + do + printf "create refs/heads/packed-%d HEAD\n" $i || return 1 + done >stdin && + git update-ref --stdin <stdin && + git ${pack_refs} --all && + test_line_count = 101 .git/packed-refs && + + # Create 24 loose refs, which should not yet cause us to repack. + printf "create refs/heads/loose-%d HEAD\n" $(test_seq 24) >stdin && + git update-ref --stdin <stdin && + git ${pack_refs} --auto --all && + test_line_count = 101 .git/packed-refs && + + # Create another handful of refs to cross the border. + # Note that we explicitly do not check for strict + # boundaries here, as this also depends on the size of + # the object hash. + printf "create refs/heads/addn-%d HEAD\n" $(test_seq 10) >stdin && + git update-ref --stdin <stdin && + git ${pack_refs} --auto --all && + test_line_count = 135 .git/packed-refs + ) + ' +done + +test_done diff --git a/t/t0601-reffiles-pack-refs.sh b/t/t0601-reffiles-pack-refs.sh index aa7f6ecd81..12cf5d1dcb 100755 --- a/t/t0601-reffiles-pack-refs.sh +++ b/t/t0601-reffiles-pack-refs.sh @@ -17,432 +17,4 @@ export GIT_TEST_DEFAULT_REF_FORMAT . ./test-lib.sh -test_expect_success 'enable reflogs' ' - git config core.logallrefupdates true -' - -test_expect_success 'prepare a trivial repository' ' - echo Hello > A && - git update-index --add A && - git commit -m "Initial commit." && - HEAD=$(git rev-parse --verify HEAD) -' - -test_expect_success 'pack-refs --prune --all' ' - test_path_is_missing .git/packed-refs && - git pack-refs --no-prune --all && - test_path_is_file .git/packed-refs && - N=$(find .git/refs -type f | wc -l) && - test "$N" != 0 && - - git pack-refs --prune --all && - test_path_is_file .git/packed-refs && - N=$(find .git/refs -type f) && - test -z "$N" -' - -SHA1= - -test_expect_success 'see if git show-ref works as expected' ' - git branch a && - SHA1=$(cat .git/refs/heads/a) && - echo "$SHA1 refs/heads/a" >expect && - git show-ref a >result && - test_cmp expect result -' - -test_expect_success 'see if a branch still exists when packed' ' - git branch b && - git pack-refs --all && - rm -f .git/refs/heads/b && - echo "$SHA1 refs/heads/b" >expect && - git show-ref b >result && - test_cmp expect result -' - -test_expect_success 'git branch c/d should barf if branch c exists' ' - git branch c && - git pack-refs --all && - rm -f .git/refs/heads/c && - test_must_fail git branch c/d -' - -test_expect_success 'see if a branch still exists after git pack-refs --prune' ' - git branch e && - git pack-refs --all --prune && - echo "$SHA1 refs/heads/e" >expect && - git show-ref e >result && - test_cmp expect result -' - -test_expect_success 'see if git pack-refs --prune remove ref files' ' - git branch f && - git pack-refs --all --prune && - ! test -f .git/refs/heads/f -' - -test_expect_success 'see if git pack-refs --prune removes empty dirs' ' - git branch r/s/t && - git pack-refs --all --prune && - ! test -e .git/refs/heads/r -' - -test_expect_success 'git branch g should work when git branch g/h has been deleted' ' - git branch g/h && - git pack-refs --all --prune && - git branch -d g/h && - git branch g && - git pack-refs --all && - git branch -d g -' - -test_expect_success 'git branch i/j/k should barf if branch i exists' ' - git branch i && - git pack-refs --all --prune && - test_must_fail git branch i/j/k -' - -test_expect_success 'test git branch k after branch k/l/m and k/lm have been deleted' ' - git branch k/l && - git branch k/lm && - git branch -d k/l && - git branch k/l/m && - git branch -d k/l/m && - git branch -d k/lm && - git branch k -' - -test_expect_success 'test git branch n after some branch deletion and pruning' ' - git branch n/o && - git branch n/op && - git branch -d n/o && - git branch n/o/p && - git branch -d n/op && - git pack-refs --all --prune && - git branch -d n/o/p && - git branch n -' - -test_expect_success 'test excluded refs are not packed' ' - git branch dont_pack1 && - git branch dont_pack2 && - git branch pack_this && - git pack-refs --all --exclude "refs/heads/dont_pack*" && - test -f .git/refs/heads/dont_pack1 && - test -f .git/refs/heads/dont_pack2 && - ! test -f .git/refs/heads/pack_this' - -test_expect_success 'test --no-exclude refs clears excluded refs' ' - git branch dont_pack3 && - git branch dont_pack4 && - git pack-refs --all --exclude "refs/heads/dont_pack*" --no-exclude && - ! test -f .git/refs/heads/dont_pack3 && - ! test -f .git/refs/heads/dont_pack4' - -test_expect_success 'test only included refs are packed' ' - git branch pack_this1 && - git branch pack_this2 && - git tag dont_pack5 && - git pack-refs --include "refs/heads/pack_this*" && - test -f .git/refs/tags/dont_pack5 && - ! test -f .git/refs/heads/pack_this1 && - ! test -f .git/refs/heads/pack_this2' - -test_expect_success 'test --no-include refs clears included refs' ' - git branch pack1 && - git branch pack2 && - git pack-refs --include "refs/heads/pack*" --no-include && - test -f .git/refs/heads/pack1 && - test -f .git/refs/heads/pack2' - -test_expect_success 'test --exclude takes precedence over --include' ' - git branch dont_pack5 && - git pack-refs --include "refs/heads/pack*" --exclude "refs/heads/pack*" && - test -f .git/refs/heads/dont_pack5' - -test_expect_success 'see if up-to-date packed refs are preserved' ' - git branch q && - git pack-refs --all --prune && - git update-ref refs/heads/q refs/heads/q && - ! test -f .git/refs/heads/q -' - -test_expect_success 'pack, prune and repack' ' - git tag foo && - git pack-refs --all --prune && - git show-ref >all-of-them && - git pack-refs && - git show-ref >again && - test_cmp all-of-them again -' - -test_expect_success 'explicit pack-refs with dangling packed reference' ' - git commit --allow-empty -m "soon to be garbage-collected" && - git pack-refs --all && - git reset --hard HEAD^ && - git reflog expire --expire=all --all && - git prune --expire=all && - git pack-refs --all 2>result && - test_must_be_empty result -' - -test_expect_success 'delete ref with dangling packed version' ' - git checkout -b lamb && - git commit --allow-empty -m "future garbage" && - git pack-refs --all && - git reset --hard HEAD^ && - git checkout main && - git reflog expire --expire=all --all && - git prune --expire=all && - git branch -d lamb 2>result && - test_must_be_empty result -' - -test_expect_success 'delete ref while another dangling packed ref' ' - git branch lamb && - git commit --allow-empty -m "future garbage" && - git pack-refs --all && - git reset --hard HEAD^ && - git reflog expire --expire=all --all && - git prune --expire=all && - git branch -d lamb 2>result && - test_must_be_empty result -' - -test_expect_success 'pack ref directly below refs/' ' - git update-ref refs/top HEAD && - git pack-refs --all --prune && - grep refs/top .git/packed-refs && - test_path_is_missing .git/refs/top -' - -test_expect_success 'do not pack ref in refs/bisect' ' - git update-ref refs/bisect/local HEAD && - git pack-refs --all --prune && - ! grep refs/bisect/local .git/packed-refs >/dev/null && - test_path_is_file .git/refs/bisect/local -' - -test_expect_success 'disable reflogs' ' - git config core.logallrefupdates false && - rm -rf .git/logs -' - -test_expect_success 'create packed foo/bar/baz branch' ' - git branch foo/bar/baz && - git pack-refs --all --prune && - test_path_is_missing .git/refs/heads/foo/bar/baz && - test_must_fail git reflog exists refs/heads/foo/bar/baz -' - -test_expect_success 'notice d/f conflict with existing directory' ' - test_must_fail git branch foo && - test_must_fail git branch foo/bar -' - -test_expect_success 'existing directory reports concrete ref' ' - test_must_fail git branch foo 2>stderr && - test_grep refs/heads/foo/bar/baz stderr -' - -test_expect_success 'notice d/f conflict with existing ref' ' - test_must_fail git branch foo/bar/baz/extra && - test_must_fail git branch foo/bar/baz/lots/of/extra/components -' - -test_expect_success 'reject packed-refs with unterminated line' ' - cp .git/packed-refs .git/packed-refs.bak && - test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && - printf "%s" "$HEAD refs/zzzzz" >>.git/packed-refs && - echo "fatal: unterminated line in .git/packed-refs: $HEAD refs/zzzzz" >expected_err && - test_must_fail git for-each-ref >out 2>err && - test_cmp expected_err err -' - -test_expect_success 'reject packed-refs containing junk' ' - cp .git/packed-refs .git/packed-refs.bak && - test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && - printf "%s\n" "bogus content" >>.git/packed-refs && - echo "fatal: unexpected line in .git/packed-refs: bogus content" >expected_err && - test_must_fail git for-each-ref >out 2>err && - test_cmp expected_err err -' - -test_expect_success 'reject packed-refs with a short SHA-1' ' - cp .git/packed-refs .git/packed-refs.bak && - test_when_finished "mv .git/packed-refs.bak .git/packed-refs" && - printf "%.7s %s\n" $HEAD refs/zzzzz >>.git/packed-refs && - printf "fatal: unexpected line in .git/packed-refs: %.7s %s\n" $HEAD refs/zzzzz >expected_err && - test_must_fail git for-each-ref >out 2>err && - test_cmp expected_err err -' - -test_expect_success 'timeout if packed-refs.lock exists' ' - LOCK=.git/packed-refs.lock && - >"$LOCK" && - test_when_finished "rm -f $LOCK" && - test_must_fail git pack-refs --all --prune -' - -test_expect_success 'retry acquiring packed-refs.lock' ' - LOCK=.git/packed-refs.lock && - >"$LOCK" && - test_when_finished "wait && rm -f $LOCK" && - { - ( sleep 1 && rm -f $LOCK ) & - } && - git -c core.packedrefstimeout=3000 pack-refs --all --prune -' - -test_expect_success SYMLINKS 'pack symlinked packed-refs' ' - # First make sure that symlinking works when reading: - git update-ref refs/heads/lossy refs/heads/main && - git for-each-ref >all-refs-before && - mv .git/packed-refs .git/my-deviant-packed-refs && - ln -s my-deviant-packed-refs .git/packed-refs && - git for-each-ref >all-refs-linked && - test_cmp all-refs-before all-refs-linked && - git pack-refs --all --prune && - git for-each-ref >all-refs-packed && - test_cmp all-refs-before all-refs-packed && - test -h .git/packed-refs && - test "$(test_readlink .git/packed-refs)" = "my-deviant-packed-refs" -' - -# The 'packed-refs' file is stored directly in .git/. This means it is global -# to the repository, and can only contain refs that are shared across all -# worktrees. -test_expect_success 'refs/worktree must not be packed' ' - test_commit initial && - test_commit wt1 && - test_commit wt2 && - git worktree add wt1 wt1 && - git worktree add wt2 wt2 && - git checkout initial && - git update-ref refs/worktree/foo HEAD && - git -C wt1 update-ref refs/worktree/foo HEAD && - git -C wt2 update-ref refs/worktree/foo HEAD && - git pack-refs --all && - test_path_is_missing .git/refs/tags/wt1 && - test_path_is_file .git/refs/worktree/foo && - test_path_is_file .git/worktrees/wt1/refs/worktree/foo && - test_path_is_file .git/worktrees/wt2/refs/worktree/foo -' - -# we do not want to count on running pack-refs to -# actually pack it, as it is perfectly reasonable to -# skip processing a broken ref -test_expect_success 'create packed-refs file with broken ref' ' - test_tick && git commit --allow-empty -m one && - recoverable=$(git rev-parse HEAD) && - test_tick && git commit --allow-empty -m two && - missing=$(git rev-parse HEAD) && - rm -f .git/refs/heads/main && - cat >.git/packed-refs <<-EOF && - $missing refs/heads/main - $recoverable refs/heads/other - EOF - echo $missing >expect && - git rev-parse refs/heads/main >actual && - test_cmp expect actual -' - -test_expect_success 'pack-refs does not silently delete broken packed ref' ' - git pack-refs --all --prune && - git rev-parse refs/heads/main >actual && - test_cmp expect actual -' - -test_expect_success 'pack-refs does not drop broken refs during deletion' ' - git update-ref -d refs/heads/other && - git rev-parse refs/heads/main >actual && - test_cmp expect actual -' - -for command in "git pack-refs --all --auto" "git maintenance run --task=pack-refs --auto" -do - test_expect_success "$command does not repack below 16 refs without packed-refs" ' - test_when_finished "rm -rf repo" && - git init repo && - ( - cd repo && - git config set maintenance.auto false && - git commit --allow-empty --message "initial" && - - # Create 14 additional references, which brings us to - # 15 together with the default branch. - printf "create refs/heads/loose-%d HEAD\n" $(test_seq 14) >stdin && - git update-ref --stdin <stdin && - test_path_is_missing .git/packed-refs && - git pack-refs --auto --all && - test_path_is_missing .git/packed-refs && - - # Create the 16th reference, which should cause us to repack. - git update-ref refs/heads/loose-15 HEAD && - git pack-refs --auto --all && - test_path_is_file .git/packed-refs - ) - ' - - test_expect_success "$command does not repack below 16 refs with small packed-refs" ' - test_when_finished "rm -rf repo" && - git init repo && - ( - cd repo && - git config set maintenance.auto false && - git commit --allow-empty --message "initial" && - - git pack-refs --all && - test_line_count = 2 .git/packed-refs && - - # Create 15 loose references. - printf "create refs/heads/loose-%d HEAD\n" $(test_seq 15) >stdin && - git update-ref --stdin <stdin && - git pack-refs --auto --all && - test_line_count = 2 .git/packed-refs && - - # Create the 16th loose reference, which should cause us to repack. - git update-ref refs/heads/loose-17 HEAD && - git pack-refs --auto --all && - test_line_count = 18 .git/packed-refs - ) - ' - - test_expect_success "$command scales with size of packed-refs" ' - test_when_finished "rm -rf repo" && - git init repo && - ( - cd repo && - git config set maintenance.auto false && - git commit --allow-empty --message "initial" && - - # Create 99 packed refs. This should cause the heuristic - # to require more than the minimum amount of loose refs. - test_seq 99 | - while read i - do - printf "create refs/heads/packed-%d HEAD\n" $i || return 1 - done >stdin && - git update-ref --stdin <stdin && - git pack-refs --all && - test_line_count = 101 .git/packed-refs && - - # Create 24 loose refs, which should not yet cause us to repack. - printf "create refs/heads/loose-%d HEAD\n" $(test_seq 24) >stdin && - git update-ref --stdin <stdin && - git pack-refs --auto --all && - test_line_count = 101 .git/packed-refs && - - # Create another handful of refs to cross the border. - # Note that we explicitly do not check for strict - # boundaries here, as this also depends on the size of - # the object hash. - printf "create refs/heads/addn-%d HEAD\n" $(test_seq 10) >stdin && - git update-ref --stdin <stdin && - git pack-refs --auto --all && - test_line_count = 135 .git/packed-refs - ) - ' -done - -test_done +. "$TEST_DIRECTORY"/pack-refs-tests.sh diff --git a/t/t1463-refs-optimize.sh b/t/t1463-refs-optimize.sh new file mode 100755 index 0000000000..c11c905d79 --- /dev/null +++ b/t/t1463-refs-optimize.sh @@ -0,0 +1,17 @@ +#!/bin/sh + +test_description='git refs optimize should not change the branch semantic + +This test runs git refs optimize and git show-ref and checks that the branch +semantic is still the same. +' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT + +. ./test-lib.sh + +pack_refs='refs optimize' +. "$TEST_DIRECTORY"/pack-refs-tests.sh |