OSDN Git Service
Junio C Hamano [Tue, 19 Sep 2017 01:47:56 +0000 (10:47 +0900)]
Merge branch 'kw/merge-recursive-cleanup'
A leakfix and code clean-up.
* kw/merge-recursive-cleanup:
merge-recursive: change current file dir string_lists to hashmap
merge-recursive: remove return value from get_files_dirs
merge-recursive: fix memory leak
Junio C Hamano [Tue, 19 Sep 2017 01:47:56 +0000 (10:47 +0900)]
Merge branch 'sb/merge-commit-msg-hook'
As "git commit" to conclude a conflicted "git merge" honors the
commit-msg hook, "git merge" that recoreds a merge commit that
cleanly auto-merges should, but it didn't.
* sb/merge-commit-msg-hook:
builtin/merge: honor commit-msg hook for merges
Junio C Hamano [Tue, 19 Sep 2017 01:47:55 +0000 (10:47 +0900)]
Merge branch 'jk/leak-checkers'
Many of our programs consider that it is OK to release dynamic
storage that is used throughout the life of the program by simply
exiting, but this makes it harder to leak detection tools to avoid
reporting false positives. Plug many existing leaks and introduce
a mechanism for developers to mark that the region of memory
pointed by a pointer is not lost/leaking to help these tools.
* jk/leak-checkers:
add UNLEAK annotation for reducing leak false positives
set_git_dir: handle feeding gitdir to itself
repository: free fields before overwriting them
reset: free allocated tree buffers
reset: make tree counting less confusing
config: plug user_config leak
update-index: fix cache entry leak in add_one_file()
add: free leaked pathspec after add_files_to_cache()
test-lib: set LSAN_OPTIONS to abort by default
test-lib: --valgrind should not override --verbose-log
Junio C Hamano [Tue, 19 Sep 2017 01:47:55 +0000 (10:47 +0900)]
Merge branch 'nm/pull-submodule-recurse-config'
"git -c submodule.recurse=yes pull" did not work as if the
"--recurse-submodules" option was given from the command line.
This has been corrected.
* nm/pull-submodule-recurse-config:
pull: honor submodule.recurse config option
pull: fix cli and config option parsing order
Junio C Hamano [Tue, 19 Sep 2017 01:47:55 +0000 (10:47 +0900)]
Merge branch 'mh/packed-ref-store-prep'
Fix regression to "gitk --bisect" by a recent update.
* mh/packed-ref-store-prep:
rev-parse: don't trim bisect refnames
Junio C Hamano [Tue, 19 Sep 2017 01:47:55 +0000 (10:47 +0900)]
Merge branch 'ma/remove-config-maybe-bool'
Finishing touches to a recent topic.
* ma/remove-config-maybe-bool:
config: remove git_config_maybe_bool
Junio C Hamano [Tue, 19 Sep 2017 01:47:55 +0000 (10:47 +0900)]
Merge branch 'jk/system-path-cleanup'
Code clean-up.
* jk/system-path-cleanup:
git_extract_argv0_path: do nothing without RUNTIME_PREFIX
system_path: move RUNTIME_PREFIX to a sub-function
Junio C Hamano [Tue, 19 Sep 2017 01:47:54 +0000 (10:47 +0900)]
Merge branch 'jh/hashmap-disable-counting'
Our hashmap implementation in hashmap.[ch] is not thread-safe when
adding a new item needs to expand the hashtable by rehashing; add
an API to disable the automatic rehashing to work it around.
* jh/hashmap-disable-counting:
hashmap: add API to disable item counting when threaded
Junio C Hamano [Tue, 19 Sep 2017 01:47:54 +0000 (10:47 +0900)]
Merge branch 'bb/doc-eol-dirty'
Doc update.
* bb/doc-eol-dirty:
Documentation: mention that `eol` can change the dirty status of paths
Junio C Hamano [Tue, 19 Sep 2017 01:47:53 +0000 (10:47 +0900)]
Merge branch 'jt/packmigrate'
Remove unneeded file added by a topic already in 'master'.
* jt/packmigrate:
Remove inadvertently added outgoing/packfile.h
Junio C Hamano [Tue, 19 Sep 2017 01:47:53 +0000 (10:47 +0900)]
Merge branch 'jk/incore-lockfile-removal'
The long-standing rule that an in-core lockfile instance, once it
is used, must not be freed, has been lifted and the lockfile and
tempfile APIs have been updated to reduce the chance of programming
errors.
* jk/incore-lockfile-removal:
stop leaking lock structs in some simple cases
ref_lock: stop leaking lock_files
lockfile: update lifetime requirements in documentation
tempfile: auto-allocate tempfiles on heap
tempfile: remove deactivated list entries
tempfile: use list.h for linked list
tempfile: release deactivated strbufs instead of resetting
tempfile: robustify cleanup handler
tempfile: factor out deactivation
tempfile: factor out activation
tempfile: replace die("BUG") with BUG()
tempfile: handle NULL tempfile pointers gracefully
tempfile: prefer is_tempfile_active to bare access
lockfile: do not rollback lock on failed close
tempfile: do not delete tempfile on failed close
always check return value of close_tempfile
verify_signed_buffer: prefer close_tempfile() to close()
setup_temporary_shallow: move tempfile struct into function
setup_temporary_shallow: avoid using inactive tempfile
write_index_as_tree: cleanup tempfile on error
Junio C Hamano [Tue, 19 Sep 2017 01:47:53 +0000 (10:47 +0900)]
Merge branch 'nd/prune-in-worktree'
"git gc" and friends when multiple worktrees are used off of a
single repository did not consider the index and per-worktree refs
of other worktrees as the root for reachability traversal, making
objects that are in use only in other worktrees to be subject to
garbage collection.
* nd/prune-in-worktree:
refs.c: reindent get_submodule_ref_store()
refs.c: remove fallback-to-main-store code get_submodule_ref_store()
rev-list: expose and document --single-worktree
revision.c: --reflog add HEAD reflog from all worktrees
files-backend: make reflog iterator go through per-worktree reflog
revision.c: --all adds HEAD from all worktrees
refs: remove dead for_each_*_submodule()
refs.c: move for_each_remote_ref_submodule() to submodule.c
revision.c: use refs_for_each*() instead of for_each_*_submodule()
refs: add refs_head_ref()
refs: move submodule slash stripping code to get_submodule_ref_store
refs.c: refactor get_submodule_ref_store(), share common free block
revision.c: --indexed-objects add objects from all worktrees
revision.c: refactor add_index_objects_to_pending()
refs.c: use is_dir_sep() in resolve_gitlink_ref()
revision.h: new flag in struct rev_info wrt. worktree-related refs
Junio C Hamano [Tue, 19 Sep 2017 01:47:53 +0000 (10:47 +0900)]
Merge branch 'ma/split-symref-update-fix'
A leakfix.
* ma/split-symref-update-fix:
refs/files-backend: add `refname`, not "HEAD", to list
refs/files-backend: correct return value in lock_ref_for_update
refs/files-backend: fix memory leak in lock_ref_for_update
refs/files-backend: add longer-scoped copy of string to list
Junio C Hamano [Tue, 19 Sep 2017 01:47:52 +0000 (10:47 +0900)]
Merge branch 'mh/notes-cleanup'
Code clean-up.
* mh/notes-cleanup:
load_subtree(): check that `prefix_len` is in the expected range
load_subtree(): declare some variables to be `size_t`
hex_to_bytes(): simpler replacement for `get_oid_hex_segment()`
get_oid_hex_segment(): don't pad the rest of `oid`
load_subtree(): combine some common code
get_oid_hex_segment(): return 0 on success
load_subtree(): only consider blobs to be potential notes
load_subtree(): check earlier whether an internal node is a tree entry
load_subtree(): separate logic for internal vs. terminal entries
load_subtree(): fix incorrect comment
load_subtree(): reduce the scope of some local variables
load_subtree(): remove unnecessary conditional
notes: make GET_NIBBLE macro more robust
Junio C Hamano [Tue, 19 Sep 2017 01:47:52 +0000 (10:47 +0900)]
Merge branch 'mg/timestamp-t-fix'
A mismerge fix.
* mg/timestamp-t-fix:
name-rev: change ULONG_MAX to TIME_MAX
Junio C Hamano [Tue, 19 Sep 2017 01:47:52 +0000 (10:47 +0900)]
Merge branch 'ma/pkt-line-leakfix'
A leakfix.
* ma/pkt-line-leakfix:
pkt-line: re-'static'-ify buffer in packet_write_fmt_1()
Junio C Hamano [Tue, 19 Sep 2017 01:47:51 +0000 (10:47 +0900)]
Merge branch 'jk/config-lockfile-leak-fix'
A leakfix.
* jk/config-lockfile-leak-fix:
config: use a static lock_file struct
Junio C Hamano [Tue, 19 Sep 2017 01:47:50 +0000 (10:47 +0900)]
Merge branch 'dw/diff-highlight-makefile-fix'
Build clean-up.
* dw/diff-highlight-makefile-fix:
diff-highlight: add clean target to Makefile
Junio C Hamano [Tue, 19 Sep 2017 01:47:50 +0000 (10:47 +0900)]
Merge branch 'ti/external-sha1dc'
Platforms that ship with a separate sha1 with collision detection
library can link to it instead of using the copy we ship as part of
our source tree.
* ti/external-sha1dc:
sha1dc: allow building with the external sha1dc library
sha1dc: build git plumbing code more explicitly
Junio C Hamano [Sun, 10 Sep 2017 08:15:43 +0000 (17:15 +0900)]
Sync with maint
* maint:
RelNotes: further fixes for 2.14.2 from the master front
Junio C Hamano [Sun, 10 Sep 2017 08:15:09 +0000 (17:15 +0900)]
The seventh batch post 2.14
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Sun, 10 Sep 2017 08:08:25 +0000 (17:08 +0900)]
Merge branch 'rs/apply-epoch'
Code simplification.
* rs/apply-epoch:
apply: remove epoch date from regex
apply: check date of potential epoch timestamps first
Junio C Hamano [Sun, 10 Sep 2017 08:08:25 +0000 (17:08 +0900)]
Merge branch 'jk/drop-sha1-entry-pos'
Code clean-up.
* jk/drop-sha1-entry-pos:
sha1-lookup: remove sha1_entry_pos() from header file
Junio C Hamano [Sun, 10 Sep 2017 08:08:23 +0000 (17:08 +0900)]
Merge branch 'nd/worktree-kill-parse-ref'
"git branch -M a b" while on a branch that is completely unrelated
to either branch a or branch b misbehaved when multiple worktree
was in use. This has been fixed.
* nd/worktree-kill-parse-ref:
branch: fix branch renaming not updating HEADs correctly
Junio C Hamano [Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)]
Merge branch 'mm/send-email-cc-cruft'
In addition to "cc: <a@dd.re.ss> # cruft", "cc: a@dd.re.ss # cruft"
was taught to "git send-email" as a valid way to tell it that it
needs to also send a carbon copy to <a@dd.re.ss> in the trailer
section.
* mm/send-email-cc-cruft:
send-email: don't use Mail::Address, even if available
send-email: fix garbage removal after address
Junio C Hamano [Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)]
Merge branch 'ls/convert-filter-progress'
The codepath to call external process filter for smudge/clean
operation learned to show the progress meter.
* ls/convert-filter-progress:
convert: display progress for filtered objects that have been delayed
Junio C Hamano [Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)]
Merge branch 'ma/up-to-date'
Message and doc updates.
* ma/up-to-date:
treewide: correct several "up-to-date" to "up to date"
Documentation/user-manual: update outdated example output
Junio C Hamano [Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)]
Merge branch 'ma/ts-cleanups'
Assorted bugfixes and clean-ups.
* ma/ts-cleanups:
ThreadSanitizer: add suppressions
strbuf_setlen: don't write to strbuf_slopbuf
pack-objects: take lock before accessing `remaining`
convert: always initialize attr_action in convert_attrs
Junio C Hamano [Sun, 10 Sep 2017 08:06:09 +0000 (17:06 +0900)]
RelNotes: further fixes for 2.14.2 from the master front
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Sun, 10 Sep 2017 08:03:10 +0000 (17:03 +0900)]
Merge branch 'jt/doc-pack-objects-fix' into maint
Doc updates.
* jt/doc-pack-objects-fix:
Doc: clarify that pack-objects makes packs, plural
Junio C Hamano [Sun, 10 Sep 2017 08:03:09 +0000 (17:03 +0900)]
Merge branch 'jn/vcs-svn-cleanup' into maint
Code clean-up.
* jn/vcs-svn-cleanup:
vcs-svn: move remaining repo_tree functions to fast_export.h
vcs-svn: remove repo_delete wrapper function
vcs-svn: remove custom mode constants
vcs-svn: remove more unused prototypes and declarations
Junio C Hamano [Sun, 10 Sep 2017 08:03:08 +0000 (17:03 +0900)]
Merge branch 'bc/vcs-svn-cleanup' into maint
Code clean-up.
* bc/vcs-svn-cleanup:
vcs-svn: rename repo functions to "svn_repo"
vcs-svn: remove unused prototypes
Junio C Hamano [Sun, 10 Sep 2017 08:03:07 +0000 (17:03 +0900)]
Merge branch 'jk/doc-the-this' into maint
Doc clean-up.
* jk/doc-the-this:
doc: fix typo in sendemail.identity
Junio C Hamano [Sun, 10 Sep 2017 08:03:07 +0000 (17:03 +0900)]
Merge branch 'rs/commit-h-single-parent-cleanup' into maint
Code clean-up.
* rs/commit-h-single-parent-cleanup:
commit: remove unused inline function single_parent()
Junio C Hamano [Sun, 10 Sep 2017 08:03:06 +0000 (17:03 +0900)]
Merge branch 'mg/format-ref-doc-fix' into maint
Doc fix.
* mg/format-ref-doc-fix:
Documentation/git-for-each-ref: clarify peeling of tags for --format
Documentation: use proper wording for ref format strings
Junio C Hamano [Sun, 10 Sep 2017 08:03:06 +0000 (17:03 +0900)]
Merge branch 'sb/submodule-parallel-update' into maint
Code clean-up.
* sb/submodule-parallel-update:
submodule.sh: remove unused variable
Junio C Hamano [Sun, 10 Sep 2017 08:03:05 +0000 (17:03 +0900)]
Merge branch 'hv/t5526-andand-chain-fix' into maint
Test fix.
* hv/t5526-andand-chain-fix:
t5526: fix some broken && chains
Junio C Hamano [Sun, 10 Sep 2017 08:03:04 +0000 (17:03 +0900)]
Merge branch 'sb/sha1-file-cleanup' into maint
Code clean-up.
* sb/sha1-file-cleanup:
sha1_file: make read_info_alternates static
Junio C Hamano [Sun, 10 Sep 2017 08:03:04 +0000 (17:03 +0900)]
Merge branch 'rs/t1002-do-not-use-sum' into maint
Test simplification.
* rs/t1002-do-not-use-sum:
t1002: stop using sum(1)
Junio C Hamano [Sun, 10 Sep 2017 08:03:03 +0000 (17:03 +0900)]
Merge branch 'ah/doc-empty-string-is-false' into maint
Doc update.
* ah/doc-empty-string-is-false:
doc: clarify "config --bool" behaviour with empty string
Junio C Hamano [Sun, 10 Sep 2017 08:03:02 +0000 (17:03 +0900)]
Merge branch 'rs/merge-microcleanup' into maint
Code clean-up.
* rs/merge-microcleanup:
merge: use skip_prefix()
Junio C Hamano [Sun, 10 Sep 2017 08:03:02 +0000 (17:03 +0900)]
Merge branch 'rs/find-pack-entry-bisection' into maint
Code clean-up.
* rs/find-pack-entry-bisection:
sha1_file: avoid comparison if no packed hash matches the first byte
Junio C Hamano [Sun, 10 Sep 2017 08:03:01 +0000 (17:03 +0900)]
Merge branch 'rs/apply-lose-prefix-length' into maint
Code clean-up.
* rs/apply-lose-prefix-length:
apply: remove prefix_length member from apply_state
Junio C Hamano [Sun, 10 Sep 2017 08:03:00 +0000 (17:03 +0900)]
Merge branch 'rj/add-chmod-error-message' into maint
Message fix.
* rj/add-chmod-error-message:
builtin/add: add detail to a 'cannot chmod' error message
Junio C Hamano [Sun, 10 Sep 2017 08:02:59 +0000 (17:02 +0900)]
Merge branch 'jk/hashcmp-memcmp' into maint
Code clean-up.
* jk/hashcmp-memcmp:
hashcmp: use memcmp instead of open-coded loop
Junio C Hamano [Sun, 10 Sep 2017 08:02:58 +0000 (17:02 +0900)]
Merge branch 'rs/t3700-clean-leftover' into maint
A test fix.
* rs/t3700-clean-leftover:
t3700: fix broken test under !POSIXPERM
Junio C Hamano [Sun, 10 Sep 2017 08:02:57 +0000 (17:02 +0900)]
Merge branch 'jc/perl-git-comment-typofix' into maint
A comment fix.
* jc/perl-git-comment-typofix:
perl/Git.pm: typofix in a comment
Junio C Hamano [Sun, 10 Sep 2017 08:02:56 +0000 (17:02 +0900)]
Merge branch 'mf/no-dashed-subcommands' into maint
Code clean-up.
* mf/no-dashed-subcommands:
scripts: use "git foo" not "git-foo"
Junio C Hamano [Sun, 10 Sep 2017 08:02:56 +0000 (17:02 +0900)]
Merge branch 'ab/ref-filter-no-contains' into maint
A test fix.
* ab/ref-filter-no-contains:
tests: don't give unportable ">" to "test" built-in, use -gt
Junio C Hamano [Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)]
Merge branch 'rs/archive-excluded-directory' into maint
"git archive" did not work well with pathspecs and the
export-ignore attribute.
We may want to resurrect the "we don't archive an empty directory"
bonus patch, but I do not mind merging the above early to 'next'
and leave it as a separate follow-up enhancement.
cf. <
20170820090629.tumvqwzkromcykjf@sigill.intra.peff.net>
* rs/archive-excluded-directory:
archive: don't queue excluded directories
archive: factor out helper functions for handling attributes
t5001: add tests for export-ignore attributes and exclude pathspecs
Junio C Hamano [Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)]
Merge branch 'mg/killed-merge' into maint
Killing "git merge --edit" before the editor returns control left
the repository in a state with MERGE_MSG but without MERGE_HEAD,
which incorrectly tells the subsequent "git commit" that there was
a squash merge in progress. This has been fixed.
* mg/killed-merge:
merge: save merge state earlier
merge: split write_merge_state in two
merge: clarify call chain
Documentation/git-merge: explain --continue
Junio C Hamano [Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)]
Merge branch 'tb/apply-with-crlf' into maint
"git apply" that is used as a better "patch -p1" failed to apply a
taken from a file with CRLF line endings to a file with CRLF line
endings. The root cause was because it misused convert_to_git()
that tried to do "safe-crlf" processing by looking at the index
entry at the same path, which is a nonsense---in that mode, "apply"
is not working on the data in (or derived from) the index at all.
This has been fixed.
* tb/apply-with-crlf:
apply: file commited with CRLF should roundtrip diff and apply
convert: add SAFE_CRLF_KEEP_CRLF
Junio C Hamano [Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)]
Merge branch 'cc/subprocess-handshake-missing-capabilities' into maint
When handshake with a subprocess filter notices that the process
asked for an unknown capability, Git did not report what program
the offending subprocess was running. This has been corrected.
We may want a follow-up fix to tighten the error checking, though.
* cc/subprocess-handshake-missing-capabilities:
sub-process: print the cmd when a capability is unsupported
Junio C Hamano [Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)]
Merge branch 'as/grep-quiet-no-match-exit-code-fix' into maint
"git grep -L" and "git grep --quiet -L" reported different exit
codes; this has been corrected.
* as/grep-quiet-no-match-exit-code-fix:
git-grep: correct exit code with --quiet and -L
Junio C Hamano [Sun, 10 Sep 2017 08:02:54 +0000 (17:02 +0900)]
Merge branch 'kd/stash-with-bash-4.4' into maint
bash 4.4 or newer gave a warning on NUL byte in command
substitution done in "git stash"; this has been squelched.
* kd/stash-with-bash-4.4:
stash: prevent warning about null bytes in input
Junio C Hamano [Sun, 10 Sep 2017 08:02:54 +0000 (17:02 +0900)]
Merge branch 'rs/win32-syslog-leakfix' into maint
Memory leak in an error codepath has been plugged.
* rs/win32-syslog-leakfix:
win32: plug memory leak on realloc() failure in syslog()
Junio C Hamano [Sun, 10 Sep 2017 08:02:53 +0000 (17:02 +0900)]
Merge branch 'rs/unpack-entry-leakfix' into maint
Memory leak in an error codepath has been plugged.
* rs/unpack-entry-leakfix:
sha1_file: release delta_stack on error in unpack_entry()
Junio C Hamano [Sun, 10 Sep 2017 08:02:52 +0000 (17:02 +0900)]
Merge branch 'rs/fsck-obj-leakfix' into maint
Memory leak in an error codepath has been plugged.
* rs/fsck-obj-leakfix:
fsck: free buffers on error in fsck_obj()
Junio C Hamano [Sun, 10 Sep 2017 08:02:52 +0000 (17:02 +0900)]
Merge branch 'ur/svn-local-zone' into maint
"git svn" used with "--localtime" option did not compute the tz
offset for the timestamp in question and instead always used the
current time, which has been corrected.
* ur/svn-local-zone:
git svn fetch: Create correct commit timestamp when using --localtime
Junio C Hamano [Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)]
Merge branch 'pw/am-signoff' into maint
"git am -s" has been taught that some input may end with a trailer
block that is not Signed-off-by: and it should refrain from adding
an extra blank line before adding a new sign-off in such a case.
* pw/am-signoff:
am: fix signoff when other trailers are present
Junio C Hamano [Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)]
Merge branch 'rs/in-obsd-basename-dirname-take-const' into maint
Portability fix.
* rs/in-obsd-basename-dirname-take-const:
test-path-utils: handle const parameter of basename and dirname
Junio C Hamano [Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)]
Merge branch 'rs/t4062-obsd' into maint
Test portability fix.
* rs/t4062-obsd:
t4062: use less than 256 repetitions in regex
Junio C Hamano [Sun, 10 Sep 2017 08:02:50 +0000 (17:02 +0900)]
Merge branch 'rs/obsd-getcwd-workaround' into maint
Test portability fix for BSDs.
* rs/obsd-getcwd-workaround:
t0001: skip test with restrictive permissions if getpwd(3) respects them
Junio C Hamano [Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)]
Merge branch 'bw/clone-recursive-quiet' into maint
"git clone --recurse-submodules --quiet" did not pass the quiet
option down to submodules.
* bw/clone-recursive-quiet:
clone: teach recursive clones to respect -q
Junio C Hamano [Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)]
Merge branch 'pw/sequence-rerere-autoupdate' into maint
Commands like "git rebase" accepted the --rerere-autoupdate option
from the command line, but did not always use it. This has been
fixed.
* pw/sequence-rerere-autoupdate:
cherry-pick/revert: reject --rerere-autoupdate when continuing
cherry-pick/revert: remember --rerere-autoupdate
t3504: use test_commit
rebase -i: honor --rerere-autoupdate
rebase: honor --rerere-autoupdate
am: remember --rerere-autoupdate setting
Junio C Hamano [Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)]
Merge branch 'bw/push-options-recursively-to-submodules' into maint
"git push --recurse-submodules $there HEAD:$target" was not
propagated down to the submodules, but now it is.
* bw/push-options-recursively-to-submodules:
submodule--helper: teach push-check to handle HEAD
Junio C Hamano [Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)]
Merge branch 'ma/pager-per-subcommand-action' into maint
The "tag.pager" configuration variable was useless for those who
actually create tag objects, as it interfered with the use of an
editor. A new mechanism has been introduced for commands to enable
pager depending on what operation is being carried out to fix this,
and then "git tag -l" is made to run pager by default.
If this works out OK, I think there are low-hanging fruits in
other commands like "git branch" that outputs long list in one mode
while taking input in another.
* ma/pager-per-subcommand-action:
git.c: ignore pager.* when launching builtin as dashed external
tag: change default of `pager.tag` to "on"
tag: respect `pager.tag` in list-mode only
t7006: add tests for how git tag paginates
git.c: provide setup_auto_pager()
git.c: let builtins opt for handling `pager.foo` themselves
builtin.h: take over documentation from api-builtin.txt
Junio C Hamano [Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)]
Merge branch 'jk/rev-list-empty-input' into maint
"git log --tag=no-such-tag" showed log starting from HEAD, which
has been fixed---it now shows nothing.
* jk/rev-list-empty-input:
revision: do not fallback to default when rev_input_given is set
rev-list: don't show usage when we see empty ref patterns
revision: add rev_input_given flag
t6018: flesh out empty input/output rev-list tests
Junio C Hamano [Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)]
Merge branch 'st/lib-gpg-kill-stray-agent' into maint
Some versions of GnuPG fails to kill gpg-agent it auto-spawned
and such a left-over agent can interfere with a test. Work it
around by attempting to kill one before starting a new test.
* st/lib-gpg-kill-stray-agent:
t: lib-gpg: flush gpg agent on startup
Martin Ågren [Sat, 9 Sep 2017 06:57:18 +0000 (08:57 +0200)]
refs/files-backend: add `refname`, not "HEAD", to list
An earlier patch rewrote `split_symref_update()` to add a copy of a
string to a string list instead of adding the original string. That was
so that the original string could be freed in a later patch, but it is
also conceptually cleaner, since now all calls to `string_list_insert()`
and `string_list_append()` add `update->refname`. --- Except a literal
"HEAD" is added in `split_head_update()`.
Restructure `split_head_update()` in the same way as the earlier patch
did for `split_symref_update()`. This does not correct any practical
problem, but makes things conceptually cleaner. The downside is a call
to `string_list_has_string()`, which should be relatively cheap.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Martin Ågren [Sat, 9 Sep 2017 06:57:17 +0000 (08:57 +0200)]
refs/files-backend: correct return value in lock_ref_for_update
In one code path we return a literal -1 and not a symbolic constant. The
value -1 would be interpreted as TRANSACTION_NAME_CONFLICT, which is
wrong. Use TRANSACTION_GENERIC_ERROR instead (that is the only other
return value we have to choose from).
Noticed-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Martin Ågren [Sat, 9 Sep 2017 06:57:16 +0000 (08:57 +0200)]
refs/files-backend: fix memory leak in lock_ref_for_update
After the previous patch, none of the functions we call hold on to
`referent.buf`, so we can safely release the string buffer before
returning.
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Martin Ågren [Sat, 9 Sep 2017 06:57:15 +0000 (08:57 +0200)]
refs/files-backend: add longer-scoped copy of string to list
split_symref_update() receives a string-pointer `referent` and adds it
to the list of `affected_refnames`. The list simply holds on to the
pointers it is given, it does not copy the strings and it does not ever
free them. The `referent` string in split_symref_update() belongs to a
string buffer in the caller. After we return, the string will be leaked.
In the next patch, we want to properly release the string buffer in the
caller, but we can't safely do so until we've made sure that
`affected_refnames` will not be holding on to a pointer to the string.
We could configure the list to handle its own resources, but it would
mean some alloc/free-churning. The list is already handling other
strings (through other code paths) which we do not need to worry about,
and we'd be memory-churning those strings too, completely unnecessary.
Observe that split_symref_update() creates a `new_update`-object through
ref_transaction_add_update(), after which `new_update->refname` is a
copy of `referent`. The difference is, this copy will be freed, and it
will be freed *after* `affected_refnames` has been cleared.
Rearrange the handling of `referent`, so that we don't add it directly
to `affected_refnames`. Instead, first just check whether `referent`
exists in the string list, and later add `new_update->refname`.
Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Michael Haggerty [Fri, 8 Sep 2017 16:10:10 +0000 (18:10 +0200)]
load_subtree(): check that `prefix_len` is in the expected range
This value, which is stashed in the last byte of an object_id hash,
gets handed around a lot. So add a sanity check before using it in
`load_subtree()`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Fri, 8 Sep 2017 06:38:41 +0000 (02:38 -0400)]
add UNLEAK annotation for reducing leak false positives
It's a common pattern in git commands to allocate some
memory that should last for the lifetime of the program and
then not bother to free it, relying on the OS to throw it
away.
This keeps the code simple, and it's fast (we don't waste
time traversing structures or calling free at the end of the
program). But it also triggers warnings from memory-leak
checkers like valgrind or LSAN. They know that the memory
was still allocated at program exit, but they don't know
_when_ the leaked memory stopped being useful. If it was
early in the program, then it's probably a real and
important leak. But if it was used right up until program
exit, it's not an interesting leak and we'd like to suppress
it so that we can see the real leaks.
This patch introduces an UNLEAK() macro that lets us do so.
To understand its design, let's first look at some of the
alternatives.
Unfortunately the suppression systems offered by
leak-checking tools don't quite do what we want. A
leak-checker basically knows two things:
1. Which blocks were allocated via malloc, and the
callstack during the allocation.
2. Which blocks were left un-freed at the end of the
program (and which are unreachable, but more on that
later).
Their suppressions work by mentioning the function or
callstack of a particular allocation, and marking it as OK
to leak. So imagine you have code like this:
int cmd_foo(...)
{
/* this allocates some memory */
char *p = some_function();
printf("%s", p);
return 0;
}
You can say "ignore allocations from some_function(),
they're not leaks". But that's not right. That function may
be called elsewhere, too, and we would potentially want to
know about those leaks.
So you can say "ignore the callstack when main calls
some_function". That works, but your annotations are
brittle. In this case it's only two functions, but you can
imagine that the actual allocation is much deeper. If any of
the intermediate code changes, you have to update the
suppression.
What we _really_ want to say is that "the value assigned to
p at the end of the function is not a real leak". But
leak-checkers can't understand that; they don't know about
"p" in the first place.
However, we can do something a little bit tricky if we make
some assumptions about how leak-checkers work. They
generally don't just report all un-freed blocks. That would
report even globals which are still accessible when the
leak-check is run. Instead they take some set of memory
(like BSS) as a root and mark it as "reachable". Then they
scan the reachable blocks for anything that looks like a
pointer to a malloc'd block, and consider that block
reachable. And then they scan those blocks, and so on,
transitively marking anything reachable from a global as
"not leaked" (or at least leaked in a different category).
So we can mark the value of "p" as reachable by putting it
into a variable with program lifetime. One way to do that is
to just mark "p" as static. But that actually affects the
run-time behavior if the function is called twice (you
aren't likely to call main() twice, but some of our cmd_*()
functions are called from other commands).
Instead, we can trick the leak-checker by putting the value
into _any_ reachable bytes. This patch keeps a global
linked-list of bytes copied from "unleaked" variables. That
list is reachable even at program exit, which confers
recursive reachability on whatever values we unleak.
In other words, you can do:
int cmd_foo(...)
{
char *p = some_function();
printf("%s", p);
UNLEAK(p);
return 0;
}
to annotate "p" and suppress the leak report.
But wait, couldn't we just say "free(p)"? In this toy
example, yes. But UNLEAK()'s byte-copying strategy has
several advantages over actually freeing the memory:
1. It's recursive across structures. In many cases our "p"
is not just a pointer, but a complex struct whose
fields may have been allocated by a sub-function. And
in some cases (e.g., dir_struct) we don't even have a
function which knows how to free all of the struct
members.
By marking the struct itself as reachable, that confers
reachability on any pointers it contains (including those
found in embedded structs, or reachable by walking
heap blocks recursively.
2. It works on cases where we're not sure if the value is
allocated or not. For example:
char *p = argc > 1 ? argv[1] : some_function();
It's safe to use UNLEAK(p) here, because it's not
freeing any memory. In the case that we're pointing to
argv here, the reachability checker will just ignore
our bytes.
3. Likewise, it works even if the variable has _already_
been freed. We're just copying the pointer bytes. If
the block has been freed, the leak-checker will skip
over those bytes as uninteresting.
4. Because it's not actually freeing memory, you can
UNLEAK() before we are finished accessing the variable.
This is helpful in cases like this:
char *p = some_function();
return another_function(p);
Writing this with free() requires:
int ret;
char *p = some_function();
ret = another_function(p);
free(p);
return ret;
But with unleak we can just write:
char *p = some_function();
UNLEAK(p);
return another_function(p);
This patch adds the UNLEAK() macro and enables it
automatically when Git is compiled with SANITIZE=leak. In
normal builds it's a noop, so we pay no runtime cost.
It also adds some UNLEAK() annotations to show off how the
feature works. On top of other recent leak fixes, these are
enough to get t0000 and t0001 to pass when compiled with
LSAN.
Note the case in commit.c which actually converts a
strbuf_release() into an UNLEAK. This code was already
non-leaky, but the free didn't do anything useful, since
we're exiting. Converting it to an annotation means that
non-leak-checking builds pay no runtime cost. The cost is
minimal enough that it's probably not worth going on a
crusade to convert these kinds of frees to UNLEAKS. I did it
here for consistency with the "sb" leak (though it would
have been equally correct to go the other way, and turn them
both into strbuf_release() calls).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Kevin Willford [Thu, 7 Sep 2017 16:25:56 +0000 (10:25 -0600)]
merge-recursive: change current file dir string_lists to hashmap
The code was using two string_lists, one for the directories and
one for the files. The code never checks the lists independently
so we should be able to only use one list. The string_list also
is a O(log n) for lookup and insertion. Switching this to use a
hashmap will give O(1) which will save some time when there are
millions of paths that will be checked.
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stefan Beller [Thu, 7 Sep 2017 22:04:29 +0000 (15:04 -0700)]
builtin/merge: honor commit-msg hook for merges
Similar to
65969d43d1 (merge: honor prepare-commit-msg hook, 2011-02-14)
merge should also honor the commit-msg hook: When a merge is stopped due
to conflicts or --no-commit, the subsequent commit calls the commit-msg
hook. However, it is not called after a clean merge. Fix this
inconsistency by invoking the hook after clean merges as well.
This change is motivated by Gerrit's commit-msg hook to install a ChangeId
trailer into the commit message. Without such a ChangeId, Gerrit refuses
to accept any commit by default, such that the inconsistency of (not)
running the commit-msg hook between commit and merge leads to confusion
and might block people from getting their work done.
As the githooks man page is very vocal about the possibility of skipping
the commit-msg hook via the --no-verify option, implement the option
in merge, too.
'git merge --continue' is currently implemented as calling cmd_commit
with no further arguments. This works for most other merge related options,
such as demonstrated via the --allow-unrelated-histories flag in the
test. The --no-verify option however is not remembered across invocations
of git-merge. Originally the author assumed an alternative in which the
'git merge --continue' command accepts the --no-verify flag, but that
opens up the discussion which flags are allows to the continued merge
command and which must be given in the first invocation.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Nicolas Morey-Chaisemartin [Wed, 6 Sep 2017 06:48:09 +0000 (08:48 +0200)]
pull: honor submodule.recurse config option
"git pull" supports a --recurse-submodules option but does not parse the
submodule.recurse configuration item to set the default for that option.
Meanwhile "git fetch" does support submodule.recurse, producing
confusing behavior: when submodule.recurse is enabled, "git pull"
recursively fetches submodules but does not update them after fetch.
Handle submodule.recurse in "git pull" to fix this.
Reported-by: Magnus Homann <magnus@homann.se>
Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Nicolas Morey-Chaisemartin [Wed, 6 Sep 2017 06:48:06 +0000 (08:48 +0200)]
pull: fix cli and config option parsing order
pull parses first the cli options and then the config option.
The expected behavior is the other way around, so that config
options can not override the cli ones.
This patch changes the parsing order so config options are
parsed first.
Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Martin Ågren [Tue, 5 Sep 2017 18:39:59 +0000 (20:39 +0200)]
config: remove git_config_maybe_bool
The function was deprecated in commit
89576613 ("treewide: deprecate
git_config_maybe_bool, use git_parse_maybe_bool", 2017-08-07) and has no
users.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff Hostetler [Wed, 6 Sep 2017 15:43:48 +0000 (15:43 +0000)]
hashmap: add API to disable item counting when threaded
This is to address concerns raised by ThreadSanitizer on the mailing list
about threaded unprotected R/W access to map.size with my previous "disallow
rehash" change (
0607e10009ee4e37cb49b4cec8d28a9dda1656a4).
See:
https://public-inbox.org/git/
adb37b70139fd1e2bac18bfd22c8b96683ae18eb.
1502780344.git.martin.agren@gmail.com/
Add API to hashmap to disable item counting and thus automatic rehashing.
Also include API to later re-enable them.
When item counting is disabled, the map.size field is invalid. So to
prevent accidents, the field has been renamed and an accessor function
hashmap_get_size() has been added. All direct references to this
field have been been updated. And the name of the field changed
to map.private_size to communicate this.
Here is the relevant output from ThreadSanitizer showing the problem:
WARNING: ThreadSanitizer: data race (pid=10554)
Read of size 4 at 0x00000082d488 by thread T2 (mutexes: write M16):
#0 hashmap_add hashmap.c:209
#1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
#2 handle_range_dir name-hash.c:347
#3 handle_range_1 name-hash.c:415
#4 lazy_dir_thread_proc name-hash.c:471
#5 <null> <null>
Previous write of size 4 at 0x00000082d488 by thread T1 (mutexes: write M31):
#0 hashmap_add hashmap.c:209
#1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
#2 handle_range_dir name-hash.c:347
#3 handle_range_1 name-hash.c:415
#4 handle_range_dir name-hash.c:380
#5 handle_range_1 name-hash.c:415
#6 lazy_dir_thread_proc name-hash.c:471
#7 <null> <null>
Martin gives instructions for running TSan on test t3008 in this post:
https://public-inbox.org/git/CAN0heSoJDL9pWELD6ciLTmWf-a=oyxe4EXXOmCKvsG5MSuzxsA@mail.gmail.com/
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 6 Sep 2017 12:32:10 +0000 (08:32 -0400)]
git_extract_argv0_path: do nothing without RUNTIME_PREFIX
When the RUNTIME_PREFIX compile-time knob isn't set, we
never look at the argv0_path we extract. We can push its
declaration inside the #ifdef to make it more clear that the
extract code is effectively a noop.
This also un-confuses leak-checking of the argv0_path
variable when RUNTIME_PREFIX isn't set. The compiler is free
to drop this static variable that we set but never look at
(and "gcc -O2" does so). But the compiler still must call
strbuf_detach(), since it doesn't know whether that function
has side effects; it just throws away the result rather than
putting it into the global.
Leak-checkers which work by scanning the data segment for
pointers to heap blocks would normally consider the block
as reachable at program end. But if the compiler removes the
variable entirely, there's nothing to find.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 6 Sep 2017 12:30:28 +0000 (08:30 -0400)]
system_path: move RUNTIME_PREFIX to a sub-function
The system_path() function has an #ifdef in the middle of
it. Let's move the conditional logic into a sub-function.
This isolates it more, which will make it easier to change
and add to.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ben Boeckel [Thu, 31 Aug 2017 13:19:36 +0000 (09:19 -0400)]
Documentation: mention that `eol` can change the dirty status of paths
When setting the `eol` attribute, paths can change their dirty status
without any change in the working directory. This can cause confusion
and should at least be mentioned with a remedy.
Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
Reviewed-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Wed, 6 Sep 2017 11:53:10 +0000 (07:53 -0400)]
rev-parse: don't trim bisect refnames
Using for_each_ref_in() with a full refname has always been
a questionable practice, but it became an error with
b9c8e7f2fb (prefix_ref_iterator: don't trim too much,
2017-05-22), making "git rev-parse --bisect" pretty reliably
show a BUG.
Commit
03df567fbf (for_each_bisect_ref(): don't trim
refnames, 2017-06-18) fixed this case for revision.c, but
rev-parse handles this option on its own. We can use the
same solution here (and piggy-back on its test).
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:05:01 +0000 (09:05 -0400)]
set_git_dir: handle feeding gitdir to itself
Ideally we'd free the existing gitdir field before assigning
the new one, to avoid a memory leak. But we can't do so
safely because some callers do the equivalent of:
set_git_dir(get_git_dir());
We can detect that case as a noop, but there are even more
complicated cases like:
set_git_dir(remove_leading_path(worktree, get_git_dir());
where we really do need to do some work, but the original
string must remain valid.
Rather than put the burden on callers to make a copy of the
string (only to free it later, since we'll make a copy of it
ourselves), let's solve the problem inside set_git_dir(). We
can make a copy of the pointer for the old gitdir, and then
avoid freeing it until after we've made our new copy.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:57 +0000 (09:04 -0400)]
repository: free fields before overwriting them
It's possible that the repository data may be initialized
twice (e.g., after doing a chdir() to the top of the
worktree we may have to adjust a relative git_dir path). We
should free() any existing fields before assigning to them
to avoid leaks.
This should be safe, as the fields are set based on the
environment or on other strings like the gitdir or
commondir. That makes it impossible that we are feeding an
alias to the just-freed string.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:51 +0000 (09:04 -0400)]
reset: free allocated tree buffers
We read the tree objects with fill_tree_descriptor(), but
never actually free the resulting buffers, causing a memory
leak. This isn't a huge deal because we call this code at
most twice per program invocation. But it does potentially
double our heap usage if you have large root trees. Let's
free the trees before returning.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:28 +0000 (09:04 -0400)]
reset: make tree counting less confusing
Depending on whether we're in --keep mode, git-reset may
feed one or two trees to unpack_trees(). We start a counter
at "1" and then increment it to "2" only for the two-tree
case. But that means we must always subtract one to find the
correct array slot to fill with each descriptor.
Instead, let's start at "0" and just increment our counter
after adding each tree. This skips the extra subtraction,
and will make things much easier when we start to actually
free our tree buffers.
While we're at it, let's make the first allocation use the
slot at "desc + nr", too, even though we know "nr" is 0 at
that point. It makes the two fill_tree_descriptor() calls
consistent (always "desc + nr", followed by always
incrementing "nr").
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:20 +0000 (09:04 -0400)]
config: plug user_config leak
We generate filenames for the user_config ("~/.gitconfig")
and the xdg config ("$XDG_CONFIG_HOME/git/config") and then
decide which to use by looking at the filesystem. But after
selecting one, the unused string is just leaked.
This is a tiny leak, but it creates noise in leak-checker
output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:14 +0000 (09:04 -0400)]
update-index: fix cache entry leak in add_one_file()
When we fail to add the cache entry to the index, we end up
just leaking the struct. We should follow the pattern of the
early-return above and free it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:10 +0000 (09:04 -0400)]
add: free leaked pathspec after add_files_to_cache()
After run_diff_files, we throw away the rev_info struct,
including the pathspec that we copied into it, leaking the
memory. this is probably not a big deal in practice. We
usually only run this once per process, and the leak is
proportional to the pathspec list we're already holding in
memory.
But it's still a leak, and it pollutes leak-checker output,
making it harder to find important leaks.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:04:04 +0000 (09:04 -0400)]
test-lib: set LSAN_OPTIONS to abort by default
We already set ASAN_OPTIONS to abort if it finds any errors.
As we start to experiment with LSAN, the leak sanitizer,
it's convenient if we give it the same treatment.
Note that ASAN is actually a superset of LSAN and can do the
leak detection itself. So this only has an effect if you
specifically build with "make SANITIZE=leak" (leak detection
but not the rest of ASAN). Building with just LSAN results
in a build that runs much faster. That makes the
build-test-fix cycle more pleasant.
In the long run, once we've fixed or suppressed all the
leaks, it will probably be worth turning leak-detection on
for ASAN and just using that (to check both leaks _and_
memory errors in a single test run). But there's still a lot
of work before we get there.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 13:03:54 +0000 (09:03 -0400)]
test-lib: --valgrind should not override --verbose-log
The --verbose test option cannot be used with test harnesses
like "prove". Instead, you must use --verbose-log.
Since the --valgrind option implies --verbose, that means
that it cannot be used with prove. I.e., this does not work:
prove t0000-basic.sh :: --valgrind
You'd think it could be fixed by doing:
prove t0000-basic.sh :: --valgrind --verbose-log
but that doesn't work either, because the implied --verbose
takes precedence over --verbose-log. If the user has given
us a specific option, we should prefer that.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:21 +0000 (08:15 -0400)]
stop leaking lock structs in some simple cases
Now that it's safe to declare a "struct lock_file" on the
stack, we can do so (and avoid an intentional leak). These
leaks were found by running t0000 and t0001 under valgrind
(though certainly other similar leaks exist and just don't
happen to be exercised by those tests).
Initializing the lock_file's inner tempfile with NULL is not
strictly necessary in these cases, but it's a good practice
to model. It means that if we were to call a function like
rollback_lock_file() on a lock that was never taken in the
first place, it becomes a quiet noop (rather than undefined
behavior).
Likewise, it's always safe to rollback_lock_file() on a file
that has already been committed or deleted, since that
operation is a noop on an inactive lockfile (and that's why
the case in config.c can drop the "if (lock)" check as we
move away from using a pointer).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:15 +0000 (08:15 -0400)]
ref_lock: stop leaking lock_files
Since the tempfile code recently relaxed the rule that
tempfile structs (and thus locks) need to hang around
forever, we no longer have to leak our lock_file structs.
In fact, we don't even need to heap-allocate them anymore,
since their lifetime can just match that of the surrounding
ref_lock (and if we forget to delete a lock, the effect is
the same as before: it will eventually go away at program
exit).
Note that there is a check in unlock_ref() to only rollback
a lock file if it has been allocated. We don't need that
check anymore; we zero the ref_lock (and thus the
lock_file), so at worst we pass a NULL pointer to
delete_tempfile(), which considers that a noop.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:12 +0000 (08:15 -0400)]
lockfile: update lifetime requirements in documentation
Now that the tempfile system we rely on has loosened the
lifetime requirements for storage, we can adjust our
documentation to match.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:08 +0000 (08:15 -0400)]
tempfile: auto-allocate tempfiles on heap
The previous commit taught the tempfile code to give up
ownership over tempfiles that have been renamed or deleted.
That makes it possible to use a stack variable like this:
struct tempfile t;
create_tempfile(&t, ...);
...
if (!err)
rename_tempfile(&t, ...);
else
delete_tempfile(&t);
But doing it this way has a high potential for creating
memory errors. The tempfile we pass to create_tempfile()
ends up on a global linked list, and it's not safe for it to
go out of scope until we've called one of those two
deactivation functions.
Imagine that we add an early return from the function that
forgets to call delete_tempfile(). With a static or heap
tempfile variable, the worst case is that the tempfile hangs
around until the program exits (and some functions like
setup_shallow_temporary rely on this intentionally, creating
a tempfile and then leaving it for later cleanup).
But with a stack variable as above, this is a serious memory
error: the variable goes out of scope and may be filled with
garbage by the time the tempfile code looks at it. Let's
see if we can make it harder to get this wrong.
Since many callers need to allocate arbitrary numbers of
tempfiles, we can't rely on static storage as a general
solution. So we need to turn to the heap. We could just ask
all callers to pass us a heap variable, but that puts the
burden on them to call free() at the right time.
Instead, let's have the tempfile code handle the heap
allocation _and_ the deallocation (when the tempfile is
deactivated and removed from the list).
This changes the return value of all of the creation
functions. For the cleanup functions (delete and rename),
we'll add one extra bit of safety: instead of taking a
tempfile pointer, we'll take a pointer-to-pointer and set it
to NULL after freeing the object. This makes it safe to
double-call functions like delete_tempfile(), as the second
call treats the NULL input as a noop. Several callsites
follow this pattern.
The resulting patch does have a fair bit of noise, as each
caller needs to be converted to handle:
1. Storing a pointer instead of the struct itself.
2. Passing the pointer instead of taking the struct
address.
3. Handling a "struct tempfile *" return instead of a file
descriptor.
We could play games to make this less noisy. For example, by
defining the tempfile like this:
struct tempfile {
struct heap_allocated_part_of_tempfile {
int fd;
...etc
} *actual_data;
}
Callers would continue to have a "struct tempfile", and it
would be "active" only when the inner pointer was non-NULL.
But that just makes things more awkward in the long run.
There aren't that many callers, so we can simply bite
the bullet and adjust all of them. And the compiler makes it
easy for us to find them all.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:04 +0000 (08:15 -0400)]
tempfile: remove deactivated list entries
Once a "struct tempfile" is added to the global cleanup
list, it is never removed. This means that its storage must
remain valid for the lifetime of the program. For single-use
tempfiles and locks, this isn't a big deal: we just declare
the struct static. But for library code which may take
multiple simultaneous locks (like the ref code), they're
forced to allocate a struct on the heap and leak it.
This is mostly OK in practice. The size of the leak is
bounded by the number of refs, and most programs exit after
operating on a fixed number of refs (and allocate
simultaneous memory proportional to the number of ref
updates in the first place). But:
1. It isn't hard to imagine a real leak: a program which
runs for a long time taking a series of ref update
instructions and fulfilling them one by one. I don't
think we have such a program now, but it's certainly
plausible.
2. The leaked entries appear as false positives to
tools like valgrind.
Let's relax this rule by keeping only "active" tempfiles on
the list. We can do this easily by moving the list-add
operation from prepare_tempfile_object to activate_tempfile,
and adding a deletion in deactivate_tempfile.
Existing callers do not need to be updated immediately.
They'll continue to leak any tempfile objects they may have
allocated, but that's no different than the status quo. We
can clean them up individually.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Tue, 5 Sep 2017 12:15:00 +0000 (08:15 -0400)]
tempfile: use list.h for linked list
The tempfile API keeps to-be-cleaned tempfiles in a
singly-linked list and never removes items from the list. A
future patch would like to start removing items, but removal
from a singly linked list is O(n), as we have to walk the
list to find the predecessor element. This means that a
process which takes "n" simultaneous lockfiles (for example,
an atomic transaction on "n" refs) may end up quadratic in
"n".
Before we start allowing items to be removed, it would be
nice to have a way to cover this case in linear time.
The simplest solution is to make an assumption about the
order in which tempfiles are added and removed from the
list. If both operations iterate over the tempfiles in the
same order, then by putting new items at the end of the list
our removal search will always find its items at the
beginning of the list. And indeed, that would work for the
case of refs. But it creates a hidden dependency between
unrelated parts of the code. If anybody changes the ref code
(or if we add a new caller that opens multiple simultaneous
tempfiles) they may unknowingly introduce a performance
regression.
Another solution is to use a better data structure. A
doubly-linked list works fine, and we already have an
implementation in list.h. But there's one snag: the elements
of "struct tempfile" are all marked as "volatile", since a
signal handler may interrupt us and iterate over the list at
any moment (even if we were in the middle of adding a new
entry).
We can declare a "volatile struct list_head", but we can't
actually use it with the normal list functions. The compiler
complains about passing a pointer-to-volatile via a regular
pointer argument. And rightfully so, as the sub-function
would potentially need different code to deal with the
volatile case.
That leaves us with a few options:
1. Drop the "volatile" modifier for the list items.
This is probably a bad idea. I checked the assembly
output from "gcc -O2", and the "volatile" really does
impact the order in which it updates memory.
2. Use macros instead of inline functions. The irony here
is that list.h is entirely implemented as trivial
inline functions. So we basically are already
generating custom code for each call. But sadly there's no
way in C to declare the inline function to take a more
generic type.
We could do so by switching the inline functions to
macros, but it does make the end result harder to read.
And it doesn't fully solve the problem (for instance,
the declaration of list_head needs to change so that
its "prev" and "next" pointers point to other volatile
structs).
3. Don't use list.h, and just make our own ad-hoc
doubly-linked list. It's not that much code to
implement the basics that we need here. But if we're
going to do so, why not add the few extra lines
required to model it after the actual list.h interface?
We can even reuse a few of the macro helpers.
So this patch takes option 3, but actually implements a
parallel "volatile list" interface in list.h, where it could
potentially be reused by other code. This implements just
enough for tempfile.c's use, though we could easily port
other functions later if need be.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>