wrapper: NetBSD gives EFTYPE and FreeBSD gives EMFILE where POSIX uses ELOOP

As documented on NetBSD's man page, open with the O_NOFOLLOW flag and a
symlink returns -1 and sets errno to EFTYPE which differs from POSIX.

This patch fixes the following test failure:

    $ sh t0602-reffiles-fsck.sh --verbose
    --- expect	2025-05-02 23:05:23.920890147 +0000
    +++ err	2025-05-02 23:05:23.916794959 +0000
    @@ -1 +1 @@
    -error: packed-refs: badRefFiletype: not a regular file but a symlink
    +error: unable to open '.git/packed-refs': Inappropriate file type or format
    not ok 12 - the filetype of packed-refs should be checked

FreeBSD has the same issue for EMLINK instead of EFTYPE.

This portability issue was introduced in cfea2f2da8 (packed-backend:
check whether the "packed-refs" is regular file, 2025-02-28)

Signed-off-by: Collin Funk <collin.funk1@gmail.com>
Acked-by: brian m. carlson <sandals@crustytoothpaste.net>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

1 files changed, 20 insertions, 1 deletions

diff --git a/wrapper.c b/wrapper.c
index 8b98593149..38fce5327a 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -737,7 +737,26 @@ int is_empty_or_missing_file(const char *filename)
 int open_nofollow(const char *path, int flags)
 {
 #ifdef O_NOFOLLOW
-	return open(path, flags | O_NOFOLLOW);
+	int ret = open(path, flags | O_NOFOLLOW);
+	/*
+	 * NetBSD sets errno to EFTYPE when path is a symlink. The only other
+	 * time this errno occurs when O_REGULAR is used. Since we don't use
+	 * it anywhere we can avoid an lstat here. FreeBSD does the same with
+	 * EMLINK.
+	 */
+# ifdef __NetBSD__
+#  define SYMLINK_ERRNO EFTYPE
+# elif defined(__FreeBSD__)
+#  define SYMLINK_ERRNO EMLINK
+# endif
+# if SYMLINK_ERRNO
+	if (ret < 0 && errno == SYMLINK_ERRNO) {
+		errno = ELOOP;
+		return -1;
+	}
+#  undef SYMLINK_ERRNO
+# endif
+	return ret;
 #else
 	struct stat st;
 	if (lstat(path, &st) < 0)