<div dir="ltr"><div dir="ltr">On Tue, Feb 15, 2022 at 5:54 PM Richard W.M. Jones <<a href="mailto:rjones@redhat.com">rjones@redhat.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Pick the nbdcopy --requests parameter to target an implicit buffer<br>
size of 64M inside nbdcopy. However don't set nbdcopy --request < 64.<br>
<br>
If request_size == 256K (the default) => requests = 256<br>
If request_size == 8M => requests = 64 (buffer size 512M)<br></blockquote><div><br></div><div>Considering the total bytes buffered makes sense. I did the same in another</div><div>application that only reads from NBD using libnbd async API. I'm using:</div><div><br></div><div>max_requests = 16</div><div>max_bytes = 2m</div><div><br></div><div>So if you have small requests (e.g. 4k), you get 16 inflight requests per connection</div><div>and with 4 connections 64 inflight requests on the storage side.</div><div><br></div><div>But if you have large requests (256k), you get only 8 requests per connection and</div><div>32 requests on the storage side.</div><div><br></div><div>This was tested in a read-only case both on my laptop with fast NVMe</div><div>(Samsung 970 EVO Plus 1T) and with super fast NVMe on Dell server,</div><div>and with shared storage (NetApp iSCSI).</div><div><br></div><div>With fast NVMe, limiting the maximum buffered bytes to 1M is actually</div><div>~10% faster, but with shared storage using more requests is faster.</div><div><br></div><div>What you suggest here will result in:</div><div>small requests: 256 requests per connection, 1024 requests on storage side</div><div>large requests: 64 requests per connection, 156 requests on storage side.</div><div><br></div><div>I don't think any storage can handle such a large amount of connections better.</div><div><br></div><div>I think we should test --requests 8 first, it may show nice speedup comapred</div><div>to what we see in </div><div><a href="https://bugzilla.redhat.com/show_bug.cgi?id=2039255#c33">https://bugzilla.redhat.com/show_bug.cgi?id=2039255#c33</a><br></div><div><br></div><div>Looks like in </div><div><a href="https://bugzilla.redhat.com/show_bug.cgi?id=2039255#c32">https://bugzilla.redhat.com/show_bug.cgi?id=2039255#c32</a><br></div><div><br></div><div>We introduced 2 changes at the same time, which makes it impossible to tell</div><div>the effect of any single change.</div><div><br></div><div>Nir</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
---<br>
v2v/<a href="http://v2v.ml" rel="noreferrer" target="_blank">v2v.ml</a> | 13 +++++++++++++<br>
1 file changed, 13 insertions(+)<br>
<br>
diff --git a/v2v/<a href="http://v2v.ml" rel="noreferrer" target="_blank">v2v.ml</a> b/v2v/<a href="http://v2v.ml" rel="noreferrer" target="_blank">v2v.ml</a><br>
index cadf864d5c..7bd47c1e7e 100644<br>
--- a/v2v/<a href="http://v2v.ml" rel="noreferrer" target="_blank">v2v.ml</a><br>
+++ b/v2v/<a href="http://v2v.ml" rel="noreferrer" target="_blank">v2v.ml</a><br>
@@ -641,14 +641,27 @@ and nbdcopy ?request_size output_alloc input_uri output_uri =<br>
*)<br>
let cmd = ref [] in<br>
List.push_back_list cmd [ "nbdcopy"; input_uri; output_uri ];<br>
+<br>
(match request_size with<br>
| None -> ()<br>
| Some size -> List.push_back cmd (sprintf "--request-size=%d" size)<br>
);<br>
+ (* Choose max requests to target an implicit buffer size of 64M. *)<br>
+ let requests =<br>
+ let target_buffer_size = 64 * 1024 * 1024 in<br>
+ let request_size =<br>
+ match request_size with<br>
+ | None -> 256 * 1024 (* default in nbdcopy 1.10+ *)<br>
+ | Some size -> size in<br>
+ min 64 (target_buffer_size / request_size) in<br>
+ List.push_back cmd (sprintf "--requests=%d" requests);<br>
+<br>
List.push_back cmd "--flush";<br>
(*List.push_back cmd "--verbose";*)<br>
+<br>
if not (quiet ()) then List.push_back cmd "--progress";<br>
if output_alloc = Types.Preallocated then List.push_back cmd "--allocated";<br>
+<br>
let cmd = !cmd in<br>
<br>
if run_command cmd <> 0 then<br>
-- <br>
2.35.1<br>
<br>
</blockquote></div></div>