Home
My Posts

Archive
Login
Signup

Recent Posts

Free subdomains

Want your own xyz.kpaste.net sub-domain for your community? Just type the address into your browser address bar (foo.xyz.kpaste.net is also supported).

Suggestions/Bugs?

Email us at . Try to include as much info as possible.

About

Pastebin is a tool for collaborative debugging or editing.

 

msnfs41client: Patches to add new winfileclone utility, lssparse cstyle, error codes+misc, 2025-04-23
Posted by Anonymous on Wed 23rd Apr 2025 18:09
raw | new post

Submit a correction or amendment below (click here to make a fresh posting)
After submitting an amendment, you'll be able to view the differences between the old and new posts easily.

Syntax highlighting: None Bash C C++ CSS Diff HTML mIRC Script Perl PHP Python Smarty XML ---------------------------- ABAP Actionscript ADA Apache Log AppleScript APT sources.list ASM ASP AutoIT Bash C C# C++ C++ (with QT) CSS Diff HTML INI (Config Files) Java Javascript Lua Make mIRC Script MySQL NSIS Perl PHP Python Q(uick)BASIC robots Ruby Ruby on Rails SDLBasic Smarty SQL TCL Text VB.NET Visual BASIC Visual Fox Pro Visual Prolog Windows Registry Files XML Xorg.conf

To highlight particular lines, prefix each line with {%HIGHLIGHT}
From f56681e82be7b69ae13503abe4431647872fff6c Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Tue, 22 Apr 2025 10:43:24 +0200 Subject: [PATCH 1/6] sys: Increase |ReadAheadGranularity| to 64k to match |CcSetReadAheadGranularity()| spec+fast networks Increase |ReadAheadGranularity| to 64k to match |CcSetReadAheadGranularity()| spec and better match fast networks (>= 1000baseT). |CcSetReadAheadGranularity()| says |ReadAheadGranularity| must be an even power of two and must be greater than or equal to |PAGE_SIZE| But mrxfcb.h defines |DEFAULT_READ_AHEAD_GRANULARITY| as |(0x08000)| (=32768), but |log2(32768)==15|, and |15| is an odd number, violating that rule. We now use 2**16==64k here, per |CcSetReadAheadGranularity()| spec, and to better match much faster networks (|DEFAULT_READ_AHEAD_GRANULARITY| was defined when 10baseT was "normal", today in 2025 1000baseT is the new "normal"). Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- sys/nfs41sys_mount.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/sys/nfs41sys_mount.c b/sys/nfs41sys_mount.c index 93a6d34..e672c0a 100644 --- a/sys/nfs41sys_mount.c +++ b/sys/nfs41sys_mount.c @@ -1,6 +1,6 @@ /* NFSv4.1 client for Windows * Copyright (C) 2012 The Regents of the University of Michigan - * Copyright (C) 2023-2024 Roland Mainz <roland.mainz@nrubsig.org> + * Copyright (C) 2023-2025 Roland Mainz <roland.mainz@nrubsig.org> * * Olga Kornievskaia <aglo@umich.edu> * Casey Bodley <cbodley@umich.edu> @@ -1197,6 +1197,31 @@ NTSTATUS nfs41_CreateVNetRoot( } } pNetRootContext->nfs41d_version = nfs41d_version; + + DbgP("default pNetRoot->DiskParameters=(" + "ClusterSize=%lu, " + "ReadAheadGranularity=%lu, " + "LockThrottlingParameters=(Increment=%lu MaximumDelay=%lu)" + ")\n", + pNetRoot->DiskParameters.ClusterSize, + pNetRoot->DiskParameters.ReadAheadGranularity, + pNetRoot->DiskParameters.LockThrottlingParameters.Increment, + pNetRoot->DiskParameters.LockThrottlingParameters.MaximumDelay); + + /* + * Windows bug(?!): + * |CcSetReadAheadGranularity()| says |ReadAheadGranularity| + * must be an even power of two and must be greater than + * or equal to PAGE_SIZE + * But mrxfcb.h defines |DEFAULT_READ_AHEAD_GRANULARITY| as + * |(0x08000)| (=32768), but |log2(32768)==15|, and |15| is an + * odd number, violating that rule. + * + * We now use 2**16==64k here, per |CcSetReadAheadGranularity()| + * spec + */ + pNetRoot->DiskParameters.ReadAheadGranularity = 64*1024*1024; + #ifdef DEBUG_MOUNT DbgP("Saving new session 0x%p\n", pVNetRootContext->session); #endif -- 2.45.1 From 0d09d27dd2729233d5e6fcd3121ca88f7ceaeba1 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Wed, 23 Apr 2025 10:10:00 +0200 Subject: [PATCH 2/6] daemon,sys: Fix return codes for opening file as dir and opening dir as file Fix return codes for opening file as dir and opening dir as file, based on the |NTSTATUS| values returned by NTFS+SMB. Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- daemon/open.c | 17 +++++++++++++++-- sys/nfs41sys_debug.c | 7 +++++-- sys/nfs41sys_openclose.c | 8 ++++++-- 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/daemon/open.c b/daemon/open.c index 1996e98..905ed5d 100644 --- a/daemon/open.c +++ b/daemon/open.c @@ -903,7 +903,13 @@ static int handle_open(void *daemon_context, nfs41_upcall *upcall) if (args->create_opts & FILE_NON_DIRECTORY_FILE) { DPRINTF(1, ("trying to open directory '%s' as a file\n", state->path.path)); - status = ERROR_DIRECTORY; + /* + * Notes: + * - NTFS+SMB returns |STATUS_FILE_IS_A_DIRECTORY| + * - See |map_open_errors()| for the mapping to + * |STATUS_*| + */ + status = ERROR_DIRECTORY_NOT_SUPPORTED; goto out_free_state; } } else if (info.type == NF4REG) { @@ -911,7 +917,14 @@ static int handle_open(void *daemon_context, nfs41_upcall *upcall) if (args->create_opts & FILE_DIRECTORY_FILE) { DPRINTF(1, ("trying to open file '%s' as a directory\n", state->path.path)); - status = ERROR_BAD_FILE_TYPE; + /* + * Notes: + * - SMB returns |STATUS_OBJECT_TYPE_MISMATCH| + * while NTFS returns |STATUS_NOT_A_DIRECTORY| + * - See |map_open_errors()| for the mapping to + * |STATUS_*| + */ + status = ERROR_DIRECTORY; goto out_free_state; } } else if (info.type == NF4LNK) { diff --git a/sys/nfs41sys_debug.c b/sys/nfs41sys_debug.c index 4e0dac7..ed88ca7 100644 --- a/sys/nfs41sys_debug.c +++ b/sys/nfs41sys_debug.c @@ -734,12 +734,15 @@ void print_open_error(int on, int status) case ERROR_TOO_MANY_LINKS: DbgP("[ERROR] nfs41_Create: STATUS_TOO_MANY_LINKS\n"); break; - case ERROR_DIRECTORY: + case ERROR_DIRECTORY_NOT_SUPPORTED: DbgP("[ERROR] nfs41_Create: STATUS_FILE_IS_A_DIRECTORY\n"); break; - case ERROR_BAD_FILE_TYPE: + case ERROR_DIRECTORY: DbgP("[ERROR] nfs41_Create: STATUS_NOT_A_DIRECTORY\n"); break; + case ERROR_BAD_FILE_TYPE: + DbgP("[ERROR] nfs41_Create: STATUS_BAD_FILE_TYPE\n"); + break; default: DbgP("[ERROR] nfs41_Create: STATUS_INSUFFICIENT_RESOURCES\n"); break; diff --git a/sys/nfs41sys_openclose.c b/sys/nfs41sys_openclose.c index d0cc36f..ad1bd53 100644 --- a/sys/nfs41sys_openclose.c +++ b/sys/nfs41sys_openclose.c @@ -408,8 +408,12 @@ NTSTATUS map_open_errors( case ERROR_SHARING_VIOLATION: return STATUS_SHARING_VIOLATION; case ERROR_REPARSE: return STATUS_REPARSE; case ERROR_TOO_MANY_LINKS: return STATUS_TOO_MANY_LINKS; - case ERROR_DIRECTORY: return STATUS_FILE_IS_A_DIRECTORY; - case ERROR_BAD_FILE_TYPE: return STATUS_NOT_A_DIRECTORY; + case ERROR_INVALID_PARAMETER: return STATUS_INVALID_PARAMETER; + /* See |handle_open()| for |ERROR_DIRECTORY| */ + case ERROR_DIRECTORY: return STATUS_NOT_A_DIRECTORY; + /* See |handle_open()| for |ERROR_DIRECTORY_NOT_SUPPORTED| */ + case ERROR_DIRECTORY_NOT_SUPPORTED: return STATUS_FILE_IS_A_DIRECTORY; + case ERROR_BAD_FILE_TYPE: return STATUS_BAD_FILE_TYPE; case ERROR_DISK_FULL: return STATUS_DISK_FULL; case ERROR_DISK_QUOTA_EXCEEDED: return STATUS_DISK_QUOTA_EXCEEDED; case ERROR_FILE_TOO_LARGE: return STATUS_FILE_TOO_LARGE; -- 2.45.1 From a5164373f56f62ed7d3c6bdd64a40441c2ef9ea1 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Wed, 23 Apr 2025 10:12:14 +0200 Subject: [PATCH 3/6] daemon: |nfs_to_windows_error()| should map |NFS4ERR_NOTDIR|+|NFS4ERR_ISDIR| to independent error codes |nfs_to_windows_error()| should map |NFS4ERR_NOTDIR|+|NFS4ERR_ISDIR| to independent error codes. Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- daemon/util.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/daemon/util.c b/daemon/util.c index bb5bc88..10bdce6 100644 --- a/daemon/util.c +++ b/daemon/util.c @@ -1,5 +1,6 @@ /* NFSv4.1 client for Windows - * Copyright (C) 2012 The Regents of the University of Michigan + * Copyright (C) 2012 The Regents of the University of Michigan + * Copyright (C) 2023-2025 Roland Mainz <roland.mainz@nrubsig.org> * * Olga Kornievskaia <aglo@umich.edu> * Casey Bodley <cbodley@umich.edu> @@ -213,9 +214,22 @@ int nfs_to_windows_error(int status, int default_error) case NFS4ERR_BADCHAR: case NFS4ERR_BADNAME: return ERROR_INVALID_NAME; - case NFS4ERR_NOTDIR: - case NFS4ERR_ISDIR: - case NFS4ERR_SYMLINK: + /* + * |NFS4ERR_NOTDIR| - The current (or saved) filehandle + * designates an object that is not a directory for an operation + * in which a directory is required. + * |ERROR_DIRECTORY| - The directory name is invalid. + */ + case NFS4ERR_NOTDIR: return ERROR_DIRECTORY; + /* + * |NFS4ERR_ISDIR| - The current or saved filehandle designates + * a directory when the current operation does not allow a + * directory to be accepted as the target of this operation. + * |ERROR_DIRECTORY_NOT_SUPPORTED| - An operation is not supported + * on a directory. + */ + case NFS4ERR_ISDIR: return ERROR_DIRECTORY_NOT_SUPPORTED; + case NFS4ERR_SYMLINK: return ERROR_INVALID_PARAMETER; case NFS4ERR_WRONG_TYPE: return ERROR_INVALID_PARAMETER; case NFS4ERR_EXPIRED: -- 2.45.1 From 55d75f3acb287309a0b0a72e6b2e30e6b6c9f6a5 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Wed, 23 Apr 2025 10:26:49 +0200 Subject: [PATCH 4/6] daemon,sys: Error codes for non-empty dirs should use |ERROR_DIR_NOT_EMPTY| Error codes for non-empty dirs should use |ERROR_DIR_NOT_EMPTY|, not |ERROR_NOT_EMPTY|. Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- daemon/util.c | 4 ++-- sys/nfs41sys_fileinfo.c | 2 +- sys/nfs41sys_openclose.c | 2 +- sys/nfs41sys_symlink.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/util.c b/daemon/util.c index 10bdce6..a9ddbd4 100644 --- a/daemon/util.c +++ b/daemon/util.c @@ -199,7 +199,7 @@ int nfs_to_windows_error(int status, int default_error) case NFS4ERR_MLINK: return ERROR_TOO_MANY_LINKS; case NFS4ERR_NAMETOOLONG: return ERROR_FILENAME_EXCED_RANGE; case NFS4ERR_STALE: return ERROR_NETNAME_DELETED; - case NFS4ERR_NOTEMPTY: return ERROR_NOT_EMPTY; + case NFS4ERR_NOTEMPTY: return ERROR_DIR_NOT_EMPTY; case NFS4ERR_DENIED: return ERROR_LOCK_FAILED; case NFS4ERR_NOTSUPP: return ERROR_NOT_SUPPORTED; case NFS4ERR_TOOSMALL: return ERROR_BUFFER_OVERFLOW; @@ -255,7 +255,7 @@ int map_symlink_errors(int status) case NFS4ERR_BADNAME: return ERROR_INVALID_REPARSE_DATA; case NFS4ERR_WRONG_TYPE: return ERROR_NOT_A_REPARSE_POINT; case NFS4ERR_ACCESS: return ERROR_ACCESS_DENIED; - case NFS4ERR_NOTEMPTY: return ERROR_NOT_EMPTY; + case NFS4ERR_NOTEMPTY: return ERROR_DIR_NOT_EMPTY; default: return nfs_to_windows_error(status, ERROR_BAD_NET_RESP); } } diff --git a/sys/nfs41sys_fileinfo.c b/sys/nfs41sys_fileinfo.c index 4c7dceb..e5bd1a0 100644 --- a/sys/nfs41sys_fileinfo.c +++ b/sys/nfs41sys_fileinfo.c @@ -511,7 +511,7 @@ NTSTATUS map_setfile_error( { switch (error) { case NO_ERROR: return STATUS_SUCCESS; - case ERROR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; + case ERROR_DIR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; case ERROR_FILE_EXISTS: return STATUS_OBJECT_NAME_COLLISION; case ERROR_FILE_NOT_FOUND: return STATUS_OBJECT_NAME_NOT_FOUND; case ERROR_PATH_NOT_FOUND: return STATUS_OBJECT_PATH_NOT_FOUND; diff --git a/sys/nfs41sys_openclose.c b/sys/nfs41sys_openclose.c index ad1bd53..9334701 100644 --- a/sys/nfs41sys_openclose.c +++ b/sys/nfs41sys_openclose.c @@ -1104,7 +1104,7 @@ NTSTATUS map_close_errors( case NO_ERROR: return STATUS_SUCCESS; case ERROR_FILE_NOT_FOUND: return STATUS_NO_SUCH_FILE; case ERROR_NETNAME_DELETED: return STATUS_NETWORK_NAME_DELETED; - case ERROR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; + case ERROR_DIR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; case ERROR_FILE_INVALID: return STATUS_FILE_INVALID; case ERROR_DISK_FULL: return STATUS_DISK_FULL; case ERROR_DISK_QUOTA_EXCEEDED: return STATUS_DISK_QUOTA_EXCEEDED; diff --git a/sys/nfs41sys_symlink.c b/sys/nfs41sys_symlink.c index ead2e96..fffcde7 100644 --- a/sys/nfs41sys_symlink.c +++ b/sys/nfs41sys_symlink.c @@ -148,7 +148,7 @@ NTSTATUS map_symlink_errors( case ERROR_INVALID_REPARSE_DATA: return STATUS_IO_REPARSE_DATA_INVALID; case ERROR_NOT_A_REPARSE_POINT: return STATUS_NOT_A_REPARSE_POINT; case ERROR_ACCESS_DENIED: return STATUS_ACCESS_DENIED; - case ERROR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; + case ERROR_DIR_NOT_EMPTY: return STATUS_DIRECTORY_NOT_EMPTY; case ERROR_OUTOFMEMORY: return STATUS_INSUFFICIENT_RESOURCES; case ERROR_INSUFFICIENT_BUFFER: return STATUS_BUFFER_TOO_SMALL; case STATUS_BUFFER_TOO_SMALL: -- 2.45.1 From a438c1dc21cec85f1f93556006be6c009ffb8436 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Wed, 23 Apr 2025 12:03:54 +0200 Subject: [PATCH 5/6] tests: Make lssparse cstyle-clean Make lssparse cstyle-clean. See https://github.com/illumos/illumos-gate/blob/master/usr/src/tools/scripts/cstyle.pl for the cstyle utility used by UNIX/Solaris/Illumos Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- tests/lssparse/lssparse.c | 392 ++++++++++++++++++++------------------ 1 file changed, 203 insertions(+), 189 deletions(-) diff --git a/tests/lssparse/lssparse.c b/tests/lssparse/lssparse.c index 399ac75..ae1c73e 100644 --- a/tests/lssparse/lssparse.c +++ b/tests/lssparse/lssparse.c @@ -11,8 +11,8 @@ * copies of the Software, and to permit persons to whom the Software is * furnished to do so, subject to the following conditions: * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, @@ -28,6 +28,10 @@ * * Written by Roland Mainz <roland.mainz@nrubsig.org> */ +/* + * All changes must pass + * illumos-gate/blob/master/usr/src/tools/scripts/cstyle.pl + */ #include <stdio.h> #include <stdlib.h> @@ -37,17 +41,18 @@ #include <unistd.h> #include <errno.h> -#define EXIT_USAGE (2) +#define EXIT_USAGE (2) static -void usage(const char *progname) +void +usage(const char *progname) { - (void)fprintf(stderr, "Usage: %s [-h] [-xdH] <sparse_file>\n" - " -h: Display this help message\n" - " -x: Print offsets in hexadecimal (base 16)\n" - " -d: Print offsets in decimal (base 10)\n" - " -H: Print hole information\n", - progname); + (void) fprintf(stderr, "Usage: %s [-h] [-xdH] <sparse_file>\n" + " -h: Display this help message\n" + " -x: Print offsets in hexadecimal (base 16)\n" + " -d: Print offsets in decimal (base 10)\n" + " -H: Print hole information\n", + progname); } typedef enum _printbase { @@ -55,186 +60,195 @@ typedef enum _printbase { pb_dec = 2 } printbase; -int main(int argc, char *argv[]) +int +main(int argc, char *argv[]) { - /* Arguments */ - const char *progname = argv[0]; - printbase pb = pb_hex; - bool print_holes = false; - const char *filename; - - int retval; - int opt; - int fd; - - size_t i; - off_t offset = 0; - off_t data_start; - off_t hole_start; - off_t hole_end; - off_t file_size; - off_t data_len; - off_t hole_len; - - while ((opt = getopt(argc, argv, "hxdH")) != -1) { - switch (opt) { - case '?': - case 'h': - usage(progname); - return EXIT_USAGE; - case 'x': - pb = pb_hex; - break; - case 'd': - pb = pb_dec; - break; - case 'H': - print_holes = true; - break; - default: - break; - } - } - - if (optind >= argc) { - usage(progname); - return EXIT_USAGE; - } - - filename = argv[optind]; - - fd = open(filename, O_RDONLY); - if (fd == -1) { - (void)fprintf(stderr, "%s: Open file failed with [%s]\n", - progname, - strerror(errno)); - return EXIT_FAILURE; - } - - /* Get file size */ - file_size = lseek(fd, 0, SEEK_END); - if (file_size == -1) { - (void)fprintf(stderr, "%s: Cannot seek [%s]\n", - progname, - strerror(errno)); - return EXIT_FAILURE; - } - (void)lseek(fd, 0, SEEK_SET); - - - /* - * Loop over hole&&data sections - * - * This loop must handle: - * - normal files with no holes - * - sparse files which start with data - * - sparse files which start with a hole - * - sparse files with multiple holes - * - sparse files which end with data - * - sparse files which end with a hole - * - * Note we start with index |1| for compatibility - * with SUN's original sparse file debuggung tools - * and Win32's - * $ /cygdrive/c/Windows/system32/fsutil sparse queryrange ... # - * output - */ -#define LSSPARSE_START_INDEX 1 - for (i=LSSPARSE_START_INDEX ;;) { - data_start = lseek(fd, offset, SEEK_DATA); - if (data_start == -1) - break; - hole_start = lseek(fd, data_start, SEEK_HOLE); - - if (hole_start == -1) { - if (errno == ENXIO) { - /* No more holes ? Use file site as pos of next hole */ - hole_start = file_size; - } else { - (void)fprintf(stderr, - "%s: lseek(..., SEEK_HOLE, ...) failed with [%s]\n", - progname, - strerror(errno)); - retval = EXIT_FAILURE; - goto done; - } - } - - if (print_holes && - (i == LSSPARSE_START_INDEX) && (data_start > 0)) { - (void)printf((pb == pb_hex)? - "Hole range[%ld]: offset=0x%llx,\tlength=0x%llx\n": - "Hole range[%ld]: offset=%lld,\tlength=%lld\n", - (long)i, - (long long)0, - (long long)data_start); - i++; - } - - data_len = hole_start - data_start; - - (void)printf((pb == pb_hex)? - "Data range[%ld]: offset=0x%llx,\tlength=0x%llx\n": - "Data range[%ld]: offset=%lld,\tlength=%lld\n", - (long)i, - (long long)data_start, - (long long)data_len); - i++; - - hole_end = lseek(fd, hole_start, SEEK_DATA); - - if (hole_end == -1) { - if (errno == ENXIO) { - /* No more holes ? */ - hole_end = file_size; - } else { - (void)fprintf(stderr, - "%s: lseek(..., SEEK_DATA, ...) failed with [%s]\n", - progname, - strerror(errno)); - retval = EXIT_FAILURE; - goto done; - } - } - - hole_len = hole_end - hole_start; - - if (print_holes && (hole_len > 0LL)) { - (void)printf((pb == pb_hex)? - "Hole range[%ld]: offset=0x%llx,\tlength=0x%llx\n": - "Hole range[%ld]: offset=%lld,\tlength=%lld\n", - (long)i, - (long long)hole_start, - (long long)hole_len); - i++; - } - - offset = hole_end; - } - - if ((data_start == -1) && (errno == ENXIO) && (offset == 0)) { - if (print_holes) { - (void)printf((pb == pb_hex)? - "Hole range[%ld]: offset=0x%llx,\tlength=0x%llx\n": - "Hole range[%ld]: offset=%lld,\tlength=%lld\n", - (long)LSSPARSE_START_INDEX, - (long long)0LL, - (long long)file_size); - } - - retval = EXIT_SUCCESS; - } else if ((data_start == -1) && (errno != ENXIO)) { - (void)fprintf(stderr, - "%s: lseek(..., SEEK_DATA, ...) failed with [%s]\n", - progname, - strerror(errno)); - retval = EXIT_FAILURE; - } - else { - retval = EXIT_SUCCESS; - } + /* Arguments */ + const char *progname = argv[0]; + printbase pb = pb_hex; + bool print_holes = false; + const char *filename; + + int retval; + int opt; + int fd; + + size_t i; + off_t offset = 0; + off_t data_start; + off_t hole_start; + off_t hole_end; + off_t file_size; + off_t data_len; + off_t hole_len; + + while ((opt = getopt(argc, argv, "hxdH")) != -1) { + switch (opt) { + case '?': + case 'h': + usage(progname); + return (EXIT_USAGE); + case 'x': + pb = pb_hex; + break; + case 'd': + pb = pb_dec; + break; + case 'H': + print_holes = true; + break; + default: + break; + } + } + + if (optind >= argc) { + usage(progname); + return (EXIT_USAGE); + } + + filename = argv[optind]; + + fd = open(filename, O_RDONLY); + if (fd == -1) { + (void) fprintf(stderr, "%s: Open file failed with [%s]\n", + progname, + strerror(errno)); + return (EXIT_FAILURE); + } + + /* Get file size */ + file_size = lseek(fd, 0, SEEK_END); + if (file_size == -1) { + (void) fprintf(stderr, "%s: Cannot seek [%s]\n", + progname, + strerror(errno)); + return (EXIT_FAILURE); + } + (void) lseek(fd, 0, SEEK_SET); + + + /* + * Loop over hole&&data sections + * + * This loop must handle: + * - normal files with no holes + * - sparse files which start with data + * - sparse files which start with a hole + * - sparse files with multiple holes + * - sparse files which end with data + * - sparse files which end with a hole + * + * Note we start with index |1| for compatibility + * with SUN's original sparse file debuggung tools + * and Win32's + * $ /cygdrive/c/Windows/system32/fsutil sparse queryrange ... # + * output + */ +#define LSSPARSE_START_INDEX 1 + for (i = LSSPARSE_START_INDEX ; ; ) { + data_start = lseek(fd, offset, SEEK_DATA); + if (data_start == -1) + break; + hole_start = lseek(fd, data_start, SEEK_HOLE); + + if (hole_start == -1) { + if (errno == ENXIO) { + /* + * No more holes ? Use file site as pos of + * next hole + */ + hole_start = file_size; + } else { + (void) fprintf(stderr, + "%s: " + "lseek(..., SEEK_HOLE, ...) failed with [%s]\n", + progname, + strerror(errno)); + retval = EXIT_FAILURE; + goto done; + } + } + + if (print_holes && + (i == LSSPARSE_START_INDEX) && (data_start > 0)) { + (void) printf((pb == pb_hex)? + "Hole range[%ld]: " + "offset=0x%llx,\tlength=0x%llx\n": + "Hole range[%ld]: " + "offset=%lld,\tlength=%lld\n", + (long)i, + (long long)0, + (long long)data_start); + i++; + } + + data_len = hole_start - data_start; + + (void) printf((pb == pb_hex)? + "Data range[%ld]: offset=0x%llx,\tlength=0x%llx\n": + "Data range[%ld]: offset=%lld,\tlength=%lld\n", + (long)i, + (long long)data_start, + (long long)data_len); + i++; + + hole_end = lseek(fd, hole_start, SEEK_DATA); + + if (hole_end == -1) { + if (errno == ENXIO) { + /* No more holes ? */ + hole_end = file_size; + } else { + (void) fprintf(stderr, + "%s: " + "lseek(..., SEEK_DATA, ...) failed with [%s]\n", + progname, + strerror(errno)); + retval = EXIT_FAILURE; + goto done; + } + } + + hole_len = hole_end - hole_start; + + if (print_holes && (hole_len > 0LL)) { + (void) printf((pb == pb_hex)? + "Hole range[%ld]: offset=0x%llx,\tlength=0x%llx\n": + "Hole range[%ld]: offset=%lld,\tlength=%lld\n", + (long)i, + (long long)hole_start, + (long long)hole_len); + i++; + } + + offset = hole_end; + } + + if ((data_start == -1) && (errno == ENXIO) && (offset == 0)) { + if (print_holes) { + (void) printf((pb == pb_hex)? + "Hole range[%ld]: " + "offset=0x%llx,\tlength=0x%llx\n": + "Hole range[%ld]: " + "offset=%lld,\tlength=%lld\n", + (long)LSSPARSE_START_INDEX, + (long long)0LL, + (long long)file_size); + } + + retval = EXIT_SUCCESS; + } else if ((data_start == -1) && (errno != ENXIO)) { + (void) fprintf(stderr, + "%s: lseek(..., SEEK_DATA, ...) failed with [%s]\n", + progname, + strerror(errno)); + retval = EXIT_FAILURE; + } else { + retval = EXIT_SUCCESS; + } done: - (void)close(fd); - return retval; + (void) close(fd); + return (retval); } -- 2.45.1 From cf2549ca6d0b9c201de9daec6c3baf9fbcb97ec5 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Wed, 23 Apr 2025 18:57:36 +0200 Subject: [PATCH 6/6] cygwin,tests: Add winclonefile utility for block cloning tests Add winclonefile utility for block cloning tests. Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- cygwin/Makefile | 4 + cygwin/Makefile.install | 7 + tests/winclonefile/Makefile | 23 +++ tests/winclonefile/winclonefile.c | 251 ++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+) create mode 100644 tests/winclonefile/Makefile create mode 100644 tests/winclonefile/winclonefile.c diff --git a/cygwin/Makefile b/cygwin/Makefile index 9ae5b84..f2aadb4 100644 --- a/cygwin/Makefile +++ b/cygwin/Makefile @@ -22,6 +22,7 @@ VS_BUILD_DIR64:=$(PROJECT_BASEDIR_DIR)/build.vc19/x64/Debug/ $(PROJECT_BASEDIR_DIR)/tests/ea/nfs_ea.exe \ $(PROJECT_BASEDIR_DIR)/tests/lssparse/lssparse.exe \ $(PROJECT_BASEDIR_DIR)/tests/winfsinfo1/winfsinfo.exe \ + $(PROJECT_BASEDIR_DIR)/tests/winclonefile/winclonefile.exe \ $(PROJECT_BASEDIR_DIR)/tests/winsg/winsg.exe: build_testutils # @@ -66,6 +67,7 @@ build_testutils: (cd "$(PROJECT_BASEDIR_DIR)/tests/ea" && make all) (cd "$(PROJECT_BASEDIR_DIR)/tests/lssparse" && make all) (cd "$(PROJECT_BASEDIR_DIR)/tests/winfsinfo1" && make all) + (cd "$(PROJECT_BASEDIR_DIR)/tests/winclonefile" && make all) (cd "$(PROJECT_BASEDIR_DIR)/tests/winsg" && make all) build: build_32bit_release build_32bit_debug build_64bit_release build_64bit_debug build_arm_64bit_debug build_testutils @@ -100,6 +102,7 @@ clean: (cd "$(PROJECT_BASEDIR_DIR)/tests/ea" && make clean) (cd "$(PROJECT_BASEDIR_DIR)/tests/lssparse" && make clean) (cd "$(PROJECT_BASEDIR_DIR)/tests/winfsinfo1" && make clean) + (cd "$(PROJECT_BASEDIR_DIR)/tests/winclonefile" && make clean) (cd "$(PROJECT_BASEDIR_DIR)/tests/winsg" && make clean) installdest_util: \ @@ -109,6 +112,7 @@ installdest_util: \ $(PROJECT_BASEDIR_DIR)/tests/ea/nfs_ea.exe \ $(PROJECT_BASEDIR_DIR)/tests/lssparse/lssparse.exe \ $(PROJECT_BASEDIR_DIR)/tests/winfsinfo1/winfsinfo.exe \ + $(PROJECT_BASEDIR_DIR)/tests/winclonefile/winclonefile.exe \ $(PROJECT_BASEDIR_DIR)/tests/winsg/winsg.exe \ $(CYGWIN_MAKEFILE_DIR)/devel/msnfs41client.bash diff --git a/cygwin/Makefile.install b/cygwin/Makefile.install index e274aa5..dcd6f13 100644 --- a/cygwin/Makefile.install +++ b/cygwin/Makefile.install @@ -103,6 +103,13 @@ installdest: else \ (cd $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/ && ln -sf winfsinfo.i686.exe winfsinfo.exe) \ fi + cp "$(PROJECT_BASEDIR_DIR)/tests/winclonefile/winclonefile.x86_64.exe" $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/winclonefile.x86_64.exe + cp "$(PROJECT_BASEDIR_DIR)/tests/winclonefile/winclonefile.i686.exe" $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/winclonefile.i686.exe + if [[ "$(CYGWIN_BASEPATH)" == *64* ]] ; then \ + (cd $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/ && ln -sf winclonefile.x86_64.exe winclonefile.exe) \ + else \ + (cd $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/ && ln -sf winclonefile.i686.exe winclonefile.exe) \ + fi cp "$(PROJECT_BASEDIR_DIR)/tests/lssparse/lssparse.x86_64.exe" $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/lssparse.x86_64.exe if [[ "$(CYGWIN_BASEPATH)" == *64* ]] ; then \ (cd $(DESTDIR)/$(CYGWIN_BASEPATH)/bin/ && ln -sf lssparse.x86_64.exe lssparse.exe) \ diff --git a/tests/winclonefile/Makefile b/tests/winclonefile/Makefile new file mode 100644 index 0000000..bf548b7 --- /dev/null +++ b/tests/winclonefile/Makefile @@ -0,0 +1,23 @@ +# +# Makefile for winclonefile +# + +# POSIX Makefile + +all: winclonefile.i686.exe winclonefile.x86_64.exe winclonefile.exe + +winclonefile.i686.exe: winclonefile.c + clang -target i686-pc-windows-gnu -Wall -Wextra -DUNICODE=1 -D_UNICODE=1 -I../../include -g winclonefile.c -lntdll -o winclonefile.i686.exe + +winclonefile.x86_64.exe: winclonefile.c + clang -target x86_64-pc-windows-gnu -Wall -Wextra -DUNICODE=1 -D_UNICODE=1 -I../../include -g winclonefile.c -lntdll -o winclonefile.x86_64.exe + +winclonefile.exe: winclonefile.x86_64.exe + ln -s winclonefile.x86_64.exe winclonefile.exe + +clean: + rm -fv \ + winclonefile.i686.exe \ + winclonefile.x86_64.exe \ + winclonefile.exe \ +# EOF. diff --git a/tests/winclonefile/winclonefile.c b/tests/winclonefile/winclonefile.c new file mode 100644 index 0000000..7d2d490 --- /dev/null +++ b/tests/winclonefile/winclonefile.c @@ -0,0 +1,251 @@ + +/* + * MIT License + * + * Copyright (c) 2024-2025 Roland Mainz <roland.mainz@nrubsig.org> + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +/* + * winclonefile.c - clone a file + * + * Written by Roland Mainz <roland.mainz@nrubsig.org> + */ + +#define WIN32_LEAN_AND_MEAN 1 + +#include <windows.h> +#include <winioctl.h> +#include <stdio.h> + +#define EXIT_USAGE (2) + +#if 1 +/* + * MinGW include do not define |DUPLICATE_EXTENTS_DATA| yet, + * see https://github.com/mingw-w64/mingw-w64/issues/90 + * ("[mingw-w64/mingw-w64] No |DUPLICATE_EXTENTS_DATA|/ + * |DUPLICATE_EXTENTS_DATA_EX| in MinGW includes") + */ +typedef struct _DUPLICATE_EXTENTS_DATA { + HANDLE FileHandle; + LARGE_INTEGER SourceFileOffset; + LARGE_INTEGER TargetFileOffset; + LARGE_INTEGER ByteCount; +} DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; +#endif + + +void +PrintWin32Error(const char *functionName, DWORD lasterrCode) +{ + char *lpMsgBuf; + + (void)FormatMessageA( + FORMAT_MESSAGE_ALLOCATE_BUFFER | + FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, + lasterrCode, + MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), + (LPSTR)&lpMsgBuf, + 0, NULL); + + (void)fprintf(stderr, + "%s failed with error %lu: %s\n", + functionName, + (unsigned long)lasterrCode, + lpMsgBuf); + + LocalFree(lpMsgBuf); +} + +int +main(int ac, char *av[]) +{ + const char* srcFileName; + const char* dstFileName; + HANDLE hSrc = INVALID_HANDLE_VALUE; + HANDLE hDst = INVALID_HANDLE_VALUE; + BOOL bResult = FALSE; + LARGE_INTEGER fileSize = { .QuadPart = 0LL }; + DUPLICATE_EXTENTS_DATA ded = { + .FileHandle = INVALID_HANDLE_VALUE, + .SourceFileOffset.QuadPart = 0LL, + .TargetFileOffset.QuadPart = 0LL, + .ByteCount.QuadPart = 0LL + }; + DWORD bytesReturnedDummy = 0; /* dummy var */ + + if (ac != 3) { + (void)fprintf(stderr, "Usage: %s <infile> <outfile>\n", av[0]); + return (EXIT_USAGE); + } + + srcFileName = av[1]; + dstFileName = av[2]; + + (void)printf("# Attempting to clone existing file '%s' to '%s' " + "using FSCTL_DUPLICATE_EXTENTS_TO_FILE...\n", + srcFileName, + dstFileName); + + hSrc = CreateFileA( + srcFileName, + GENERIC_READ, + FILE_SHARE_READ, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL, + NULL); + + if (hSrc == INVALID_HANDLE_VALUE) { + PrintWin32Error("CreateFileA (source read)", + GetLastError()); + goto cleanup; + } + (void)printf("# Successfully opened existing source file '%s'.\n", + srcFileName); + + bResult = GetFileSizeEx(hSrc, &fileSize); + if (!bResult) { + PrintWin32Error("GetFileSizeEx", + GetLastError()); + goto cleanup; + } + + if (fileSize.QuadPart == 0LL) { + (void)fprintf(stderr, + "# [NOTE] Source file '%s' is empty, " + "cloning will result in an empty file.\n", + srcFileName); + } + + (void)printf("Source file size: %lld bytes.\n", + fileSize.QuadPart); + + /* Get cluster size */ + FILE_STORAGE_INFO fsi = { 0 }; + bResult = GetFileInformationByHandleEx(hSrc, + FileStorageInfo, &fsi, sizeof(fsi)); + if (!bResult) { + PrintWin32Error("FileStorageInfo", + GetLastError()); + goto cleanup; + } + + unsigned long long srcClusterSize = + fsi.PhysicalBytesPerSectorForAtomicity; + (void)printf("src file cluster size=%llu\n", srcClusterSize); + + hDst = CreateFileA( + dstFileName, + GENERIC_ALL, + FILE_SHARE_DELETE|FILE_SHARE_WRITE, + NULL, + CREATE_ALWAYS, + FILE_ATTRIBUTE_NORMAL, + NULL); + + if (hDst == INVALID_HANDLE_VALUE) { + PrintWin32Error("CreateFileA (destination)", + GetLastError()); + goto cleanup; + } + + if (fileSize.QuadPart > 0LL) { + if (!SetFilePointerEx(hDst, fileSize, NULL, FILE_BEGIN)) { + PrintWin32Error("SetFilePointerEx (pre-allocate)", + GetLastError()); + goto cleanup; + } + + /* Sets the file size to the current position of the file pointer */ + bResult = SetEndOfFile(hDst); + if (!bResult) { + PrintWin32Error("SetEndOfFile (pre-allocate)", + GetLastError()); + goto cleanup; + } + + /* Reset file pointer to pos 0 */ + LARGE_INTEGER currentPos = { .QuadPart = 0LL }; + SetFilePointerEx(hDst, currentPos, NULL, FILE_BEGIN); + } + + ded.FileHandle = hSrc; + ded.SourceFileOffset.QuadPart = 0LL; + ded.TargetFileOffset.QuadPart = 0LL; + ded.ByteCount = fileSize; + + /* + * |FSCTL_DUPLICATE_EXTENTS_TO_FILE| spec requires that the src size + * is rounded to the filesytem's cluster size + * + * FIXME: What about the size of the destination file ? + */ + ded.ByteCount.QuadPart = + (ded.ByteCount.QuadPart+srcClusterSize) & ~(srcClusterSize-1); + + (void)printf("# DeviceIoControl(FSCTL_DUPLICATE_EXTENTS_TO_FILE)\n"); + + bResult = DeviceIoControl( + hDst, + FSCTL_DUPLICATE_EXTENTS_TO_FILE, + &ded, + sizeof(ded), + NULL, + 0, + &bytesReturnedDummy, + NULL); + + if (!bResult) { + PrintWin32Error("DeviceIoControl(FSCTL_DUPLICATE_EXTENTS_TO_FILE)", + GetLastError()); + goto cleanup; + } + + (void)printf("# Successfully cloned '%s' to '%s'!\n", + srcFileName, dstFileName); + + +cleanup: + if ((!bResult) && (hDst != INVALID_HANDLE_VALUE)) { + (void)printf("# Failure, deleting destination file...\n"); + + FILE_DISPOSITION_INFO di = { .DeleteFile = TRUE }; + bResult = SetFileInformationByHandle(hDst, + FileDispositionInfo, &di, sizeof(di)); + if (!bResult) { + PrintWin32Error("Delete destination file", + GetLastError()); + } + } + + (void)printf("# Cleaning up...\n"); + if (hSrc != INVALID_HANDLE_VALUE) { + (void)CloseHandle(hSrc); + } + if (hDst != INVALID_HANDLE_VALUE) { + (void)CloseHandle(hDst); + } + + return bResult ? EXIT_SUCCESS : EXIT_FAILURE; +} -- 2.45.1

Title:

Privacy: Public Private

How long should your post be retained?

15 mins an hour a day a week a month forever

All content is user-submitted.
The administrators of this site (kpaste.net) are not responsible for their content.
Abuse reports should be emailed to us at