Home
My Posts

Archive
Login
Signup

Recent Posts

Free subdomains

Want your own xyz.kpaste.net sub-domain for your community? Just type the address into your browser address bar (foo.xyz.kpaste.net is also supported).

Suggestions/Bugs?

Email us at . Try to include as much info as possible.

About

Pastebin is a tool for collaborative debugging or editing.

 

msnfs41client: Patches for |FSCTL_QUERY_ALLOCATED_RANGES| buffer size calculation, UNC path vs mkinstalldirs+misc, 2025-03-03
Posted by Anonymous on Mon 3rd Mar 2025 18:55
raw | new post

Submit a correction or amendment below (click here to make a fresh posting)
After submitting an amendment, you'll be able to view the differences between the old and new posts easily.

Syntax highlighting: None Bash C C++ CSS Diff HTML mIRC Script Perl PHP Python Smarty XML ---------------------------- ABAP Actionscript ADA Apache Log AppleScript APT sources.list ASM ASP AutoIT Bash C C# C++ C++ (with QT) CSS Diff HTML INI (Config Files) Java Javascript Lua Make mIRC Script MySQL NSIS Perl PHP Python Q(uick)BASIC robots Ruby Ruby on Rails SDLBasic Smarty SQL TCL Text VB.NET Visual BASIC Visual Fox Pro Visual Prolog Windows Registry Files XML Xorg.conf

To highlight particular lines, prefix each line with {%HIGHLIGHT}
From d7139291dd000f326aa5b40813f435ee7988a51d Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Mon, 3 Mar 2025 18:12:23 +0100 Subject: [PATCH 1/2] tests: Need workaround for GNU mkinstalldirs not able to handle UNC paths Need workaround for GNU mkinstalldirs not able to handle UNC paths. Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- tests/nfsbuildtest/nfsbuildtest.ksh93 | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/nfsbuildtest/nfsbuildtest.ksh93 b/tests/nfsbuildtest/nfsbuildtest.ksh93 index 92d0cf7..17eaef3 100644 --- a/tests/nfsbuildtest/nfsbuildtest.ksh93 +++ b/tests/nfsbuildtest/nfsbuildtest.ksh93 @@ -117,6 +117,10 @@ function gcc_build # patch sources and configure build # + # original mkinstalldirs cannot handle UNC paths + printf '#!/bin/sh\n# original mkinstalldirs cannot handle Cygwin UNC paths\nmkdir -p "$@"\nexit $?' >'mkinstalldirs' + chmod a+x 'mkinstalldirs' + # Cygwin/MSYS2: workaround for configure using cp -p where ln -s should be used # (this is an automake/autoconf issue, they should trust Cygwin and not use # ancient workarounds for issues which no longer exists) @@ -282,6 +286,10 @@ function bash_build # patch sources and configure build # + # original mkinstalldirs cannot handle UNC paths + printf '#!/bin/sh\n# original mkinstalldirs cannot handle Cygwin UNC paths\nmkdir -p "$@"\nexit $?' >'support/mkinstalldirs' + chmod a+x 'support/mkinstalldirs' + # disable loadable plugins sed -i -E 's/-\( cd \$\(LOADABLES_DIR\) && \$\(MAKE\) \$\(MFLAGS\) DESTDIR=\$\(DESTDIR\) \$@ \)//' Makefile.in -- 2.45.1 From 0fd8ca32556fb2fe23a43cc831d31f21d48b5ab0 Mon Sep 17 00:00:00 2001 From: Roland Mainz <roland.mainz@nrubsig.org> Date: Mon, 3 Mar 2025 18:16:11 +0100 Subject: [PATCH 2/2] daemon,sys,tests: RFE: |DeviceIoControl(...,FSCTL_QUERY_ALLOCATED_RANGES,...)| should support |ERROR_MORE_DATA| RFE: |DeviceIoControl(..., FSCTL_QUERY_ALLOCATED_RANGES, ...)| should support |ERROR_MORE_DATA| to query the buffer size required to list all data ranges in |FILE_ALLOCATED_RANGE_BUFFER|s in a given offset+length. This is partially ruined by a bug in Windows 10's $ fsutil sparse queryrange ... # which causes random data to be returned if |ERROR_MORE_DATA| returns a buffer size different than the buffer size passed in. fsutil passes 64 |FILE_ALLOCATED_RANGE_BUFFER| in, and expects that the size for 64 |FILE_ALLOCATED_RANGE_BUFFER| will be returned if the error code is |ERROR_MORE_DATA|. For now we define the cpp symbol |WINDOWS_FSUTILSPARSEQUERYRANGE_MOREDATA_BUG| to enable our workaround, until we get a fix for the issue from Microsoft. Reported-by: Lionel Cons <lionelcons1972@gmail.com> Signed-off-by: Cedric Blancher <cedric.blancher@gmail.com> --- daemon/fsctl.c | 69 +++++++++++++++++++++++---- daemon/upcall.h | 1 + sys/nfs41sys_driver.h | 3 +- sys/nfs41sys_fileinfo.c | 2 + sys/nfs41sys_fsctl.c | 29 +++++++++-- tests/sparsefiles/testsparsefile1.ksh | 4 +- tests/winfsinfo1/winfsinfo.c | 7 ++- 7 files changed, 96 insertions(+), 19 deletions(-) diff --git a/daemon/fsctl.c b/daemon/fsctl.c index 29092b2..bc74363 100644 --- a/daemon/fsctl.c +++ b/daemon/fsctl.c @@ -73,6 +73,7 @@ int query_sparsefile_datasections(nfs41_open_state *state, bool_t hole_seek_sr_eof; uint64_t hole_seek_sr_offset; size_t i; + bool error_more_data = false; stateid_arg stateid; @@ -91,7 +92,7 @@ int query_sparsefile_datasections(nfs41_open_state *state, next_offset = start_offset; *res_num_records = 0; - for (i=0 ; i < out_maxrecords ; i++) { + for (i=0 ; ; i++) { data_seek_status = nfs42_seek(state->session, &state->file, &stateid, @@ -172,16 +173,44 @@ int query_sparsefile_datasections(nfs41_open_state *state, (int)data_seek_sr_eof, (int)hole_seek_sr_eof)); - outbuffer[i].FileOffset.QuadPart = data_seek_sr_offset; - outbuffer[i].Length.QuadPart = data_size; + if (i < out_maxrecords) { + outbuffer[i].FileOffset.QuadPart = data_seek_sr_offset; + outbuffer[i].Length.QuadPart = data_size; + } + else { + error_more_data = true; + + /* Windows bug: + * Technically we should continue until we reach + * |end_offset| to return the correct size of the + * buffer with |DeviceIoControl(..., + * FSCTL_QUERY_ALLOCATED_RANGES,...)| == + * |ERROR_MORE_DATA|. + * + * Unfortunaely Win10 fsutil will then return + * random values after the first 64 entries, so the + * best we can do for bug-by-bug compatibility is + * to do the same: Stop counting here, and + * therefore return + * |out_maxrecords*sizeof(FILE_ALLOCATED_RANGE_BUFFER)|, + * which is exactly the same number of bytes of the + * original buffer passed as argument + */ +#define WINDOWS_FSUTILSPARSEQUERYRANGE_MOREDATA_BUG 1 + +#ifdef WINDOWS_FSUTILSPARSEQUERYRANGE_MOREDATA_BUG + break; +#endif /* WINDOWS_FSUTILSPARSEQUERYRANGE_MOREDATA_BUG */ + } + (*res_num_records)++; - if ((uint64_t)outbuffer[i].FileOffset.QuadPart > end_offset) { + if (data_seek_sr_offset > end_offset) { DPRINTF(QARLVL, ("end offset reached, " - "outbuffer[%d].FileOffset.QuadPart(=%lld) > end_offset(=%lld)\n", + "i=%d, data_seek_sr_offset(=%lld) > end_offset(=%lld)\n", (int)i, - (long long)outbuffer[i].FileOffset.QuadPart, + (long long)data_seek_sr_offset, (long long)end_offset)); break; } @@ -196,6 +225,12 @@ int query_sparsefile_datasections(nfs41_open_state *state, } out: + if (error_more_data) { + DPRINTF(QARLVL, ("returning ERROR_MORE_DATA, *res_num_records=%ld\n", + (long)*res_num_records)); + status = ERROR_MORE_DATA; + } + DPRINTF(QARLVL, ("<-- query_sparsefile_datasections(), status=0x%x\n", status)); return status; @@ -232,6 +267,7 @@ int handle_queryallocatedranges(void *daemon_context, "got space for %ld records\n", (int)num_records)); + args->buffer_overflow = FALSE; args->returned_size = 0; size_t res_num_records = 0; @@ -243,13 +279,23 @@ int handle_queryallocatedranges(void *daemon_context, num_records, &res_num_records); - if (!status) { - args->returned_size = - (ULONG)res_num_records*sizeof(FILE_ALLOCATED_RANGE_BUFFER); + /* + * Return buffer size, either on success, or to return the size + * of the buffer which would be needed. + */ + args->returned_size = + (ULONG)res_num_records*sizeof(FILE_ALLOCATED_RANGE_BUFFER); + + if (status == ERROR_MORE_DATA) { + status = NO_ERROR; + args->buffer_overflow = TRUE; } DPRINTF(QARLVL, - ("<-- handle_queryallocatedranges(), status=0x%lx\n", + ("<-- handle_queryallocatedranges(args->buffer_overflow=%d, args->returned_size=%ld), " + "status=0x%lx\n", + (int)args->buffer_overflow, + (long)args->returned_size, status)); return status; } @@ -260,8 +306,11 @@ static int marshall_queryallocatedranges(unsigned char *buffer, int status; queryallocatedranges_upcall_args *args = &upcall->args.queryallocatedranges; + status = safe_write(&buffer, length, &args->buffer_overflow, sizeof(args->buffer_overflow)); + if (status) goto out; status = safe_write(&buffer, length, &args->returned_size, sizeof(args->returned_size)); +out: return status; } diff --git a/daemon/upcall.h b/daemon/upcall.h index 19e36f8..fbed276 100644 --- a/daemon/upcall.h +++ b/daemon/upcall.h @@ -195,6 +195,7 @@ typedef struct __queryallocatedranges_upcall_args { FILE_ALLOCATED_RANGE_BUFFER inrange; HANDLE outbuffer; ULONG outbuffersize; + BOOLEAN buffer_overflow; ULONG returned_size; } queryallocatedranges_upcall_args; diff --git a/sys/nfs41sys_driver.h b/sys/nfs41sys_driver.h index 65884ca..9b2c08a 100644 --- a/sys/nfs41sys_driver.h +++ b/sys/nfs41sys_driver.h @@ -275,7 +275,8 @@ typedef struct _updowncall_entry { PMDL BufferMdl; ULONG BufferSize; PVOID Buffer; - LONGLONG returned_size; + BOOLEAN buffer_overflow; + ULONG returned_size; } QueryAllocatedRanges; struct { FILE_ZERO_DATA_INFORMATION setzerodata; diff --git a/sys/nfs41sys_fileinfo.c b/sys/nfs41sys_fileinfo.c index 07bb770..6efa067 100644 --- a/sys/nfs41sys_fileinfo.c +++ b/sys/nfs41sys_fileinfo.c @@ -487,6 +487,8 @@ NTSTATUS map_setfile_error( case ERROR_DISK_FULL: return STATUS_DISK_FULL; case ERROR_DISK_QUOTA_EXCEEDED: return STATUS_DISK_QUOTA_EXCEEDED; case ERROR_FILE_TOO_LARGE: return STATUS_FILE_TOO_LARGE; + case ERROR_INSUFFICIENT_BUFFER: return STATUS_BUFFER_TOO_SMALL; + case ERROR_MORE_DATA: return STATUS_BUFFER_OVERFLOW; case ERROR_INTERNAL_ERROR: return STATUS_INTERNAL_ERROR; default: print_error("map_setfile_error: " diff --git a/sys/nfs41sys_fsctl.c b/sys/nfs41sys_fsctl.c index 3157b54..274bf29 100644 --- a/sys/nfs41sys_fsctl.c +++ b/sys/nfs41sys_fsctl.c @@ -181,16 +181,31 @@ NTSTATUS nfs41_QueryAllocatedRanges( } entry->psec_ctx = NULL; - if (!entry->status) { + if (entry->status == NO_ERROR) { DbgP("nfs41_QueryAllocatedRanges: SUCCESS\n"); - RxContext->CurrentIrp->IoStatus.Status = STATUS_SUCCESS; - RxContext->IoStatusBlock.Information = - (ULONG_PTR)entry->u.QueryAllocatedRanges.returned_size; + + if (entry->u.QueryAllocatedRanges.buffer_overflow) { + DbgP("nfs41_QueryAllocatedRanges: buffer_overflow: " + "need at least a buffer with %ld bytes\n", + (long)entry->u.QueryAllocatedRanges.returned_size); + status = RxContext->CurrentIrp->IoStatus.Status = + STATUS_BUFFER_OVERFLOW; + RxContext->IoStatusBlock.Information = + (ULONG_PTR)entry->u.QueryAllocatedRanges.returned_size; + } + else { + status = RxContext->CurrentIrp->IoStatus.Status = + STATUS_SUCCESS; + RxContext->IoStatusBlock.Information = + (ULONG_PTR)entry->u.QueryAllocatedRanges.returned_size; + } } else { DbgP("nfs41_QueryAllocatedRanges: " "FAILURE, entry->status=0x%lx\n", entry->status); + status = map_setfile_error(entry->status); + RxContext->CurrentIrp->IoStatus.Status = status; RxContext->IoStatusBlock.Information = 0; } @@ -294,12 +309,16 @@ NTSTATUS unmarshal_nfs41_queryallocatedranges( goto out; } + RtlCopyMemory(&cur->u.QueryAllocatedRanges.buffer_overflow, + *buf, sizeof(BOOLEAN)); + *buf += sizeof(BOOLEAN); RtlCopyMemory(&cur->u.QueryAllocatedRanges.returned_size, *buf, sizeof(ULONG)); *buf += sizeof(ULONG); DbgP("unmarshal_nfs41_queryallocatedranges: " - "queryallocatedranges.returned_size=%llu\n", + "buffer_overflow=%d returned_size=%llu\n", + (int)cur->u.QueryAllocatedRanges.buffer_overflow, cur->u.QueryAllocatedRanges.returned_size); out: diff --git a/tests/sparsefiles/testsparsefile1.ksh b/tests/sparsefiles/testsparsefile1.ksh index b541849..2427af2 100644 --- a/tests/sparsefiles/testsparsefile1.ksh +++ b/tests/sparsefiles/testsparsefile1.ksh @@ -288,8 +288,8 @@ test_multihole_sparsefile1 1024 2 32 false test_multihole_sparsefile1 1024 0 4 true test_multihole_sparsefile1 1024 1 4 true -# 512 does not work, as Win10 fsutil can only handle 64 data sections -# test_multihole_sparsefile1 1024 2 512 false +# fsutil uses 64 entries per queryrange, so we test this here +test_multihole_sparsefile1 1024 2 256 false test_sparse_punchhole1 diff --git a/tests/winfsinfo1/winfsinfo.c b/tests/winfsinfo1/winfsinfo.c index 77cc036..de1d95e 100644 --- a/tests/winfsinfo1/winfsinfo.c +++ b/tests/winfsinfo1/winfsinfo.c @@ -1127,7 +1127,7 @@ retry_fsctl: * bytes we passed in, not the number of bytes which * we should allocate */ - max_ranges_per_query+=2; + max_ranges_per_query += 16; ranges = realloc(ranges, (sizeof(FILE_ALLOCATED_RANGE_BUFFER) * max_ranges_per_query)); if (ranges == NULL) { @@ -1142,6 +1142,11 @@ retry_fsctl: goto retry_fsctl; } + (void)fprintf(stderr, + "%s: DeviceIoControl() failed, lasterr=%d\n", + progname, + (int)lasterr); + retval = 1; goto out; } -- 2.45.1

Title:

Privacy: Public Private

How long should your post be retained?

15 mins an hour a day a week a month forever

All content is user-submitted.
The administrators of this site (kpaste.net) are not responsible for their content.
Abuse reports should be emailed to us at