[Libguestfs] ANNOUNCE: libnbd 1.6 and nbdkit 1.24 released
Richard W.M. Jones
rjones at redhat.com
Thu Jan 7 17:47:43 UTC 2021
I'm pleased to announce the release of nbdkit 1.24, a high performance
plugin-based Network Block Device (NBD) server, and libnbd 1.6, a high
performance client library.
https://en.wikipedia.org/wiki/Network_block_device
Key features of nbdkit:
* Multithreaded NBD server written in C with good performance.
* Minimal dependencies for the basic server.
* Liberal license (BSD) allows nbdkit to be linked to proprietary
libraries or included in proprietary code.
* Well-documented, simple plugin API with a stable ABI guarantee.
Lets you export “unconventional” block devices easily.
* You can write plugins in C, Go, Lua, Perl, Python, OCaml, Ruby,
Rust, shell script or Tcl.
* Filters can be stacked in front of plugins to transform the output.
Key features of libnbd:
* Synchronous API for ease of use.
* Asynchronous API for writing non-blocking, multithreaded clients.
You can mix both APIs freely.
* High performance.
* Minimal dependencies for the basic library.
* Well-documented, stable API.
* Bindings in several programming languages.
* Shell (nbdsh) for command line and scripting.
* nbdfuse, nbdinfo and nbdcopy tools
Release notes for both are attached below.
Rich.
-------
These are the release notes for nbdkit stable release 1.24. This
describes the major changes since 1.22.
nbdkit 1.24.0 was released on 7 January 2021.
Security
There were no security issues found. All past security issues and
information about how to report new ones can be found in
nbdkit-security(1).
Server
The server and a subset of the plugins and filters have been ported to
Windows using the Win32 API. Only the mingw-w64 toolchain is supported
for building nbdkit (not MSVC). You can also cross-compile from Linux
and test under Wine. Some features are missing although it is very
usable.
New -D nbdkit.tls.log and -D nbdkit.tls.session debug flags which
enable additional debugging for TLS sessions.
The server now responds to "NBD_INFO_NAME" (Eric Blake).
Options --run and --vsock may now be used together.
In --run, $nbd is now a synonym for $uri instead of having confusing
magical properties.
Plugins
New nbdkit-S3-plugin(1) to access disks stored on AWS S3 and Ceph.
New nbdkit-sparse-random-plugin(1) which generates large disks sparsely
filled with random blocks of data. These look like virtual disk images
and can be used to benchmark copying tools.
nbdkit-data-plugin(1) has been greatly enhanced. New features include:
"(...)[N:M]" (slice syntax), assignments, comments, variables,
"<(SCRIPT)" (inlined shell script). In addition a parse tree is built,
allowing optimizations and other enhancements. The plugin is now 64
bit clean allowing patterns larger than 2^32 bytes to work correctly.
nbdkit-random-plugin(1) can now be used to test copying tools.
nbdkit-vddk-plugin(1) supports enabling compression between VDDK and
the VMware server which in some situations can improve performance.
Filters
New nbdkit-exportname-filter(1) can be used to extend, mask and modify
the exports exposed by a plugin (Eric Blake).
New nbdkit-exitwhen-filter(1) allowing the server to gracefully exit on
various events including external scripts.
New nbdkit-checkwrite-filter(1) can be used to benchmark copying tools.
nbdkit-ip-filter(1) now supports filtering clients by PID, user ID or
group ID, for clients connected over a Unix domain socket. Also you
can filter clients using "AF_VSOCK" and vsock port number.
nbdkit-log-filter(1) now logs "get_ready", "after_fork" and
"preconnect" methods, and there is a new "logscript" feature allowing
scripts to be run on logging events.
nbdkit-ext2-filter(1) new option "ext2file=exportname" can be used to
allow the client to select which file to see from the filesystem using
the NBD export name (Eric Blake).
Language bindings
Python style cleanups (Eric Blake).
The OCaml plugin now correctly acquires and releases the GC lock on all
paths in and out.
There is a new OCaml "NBDKit.version" function returning the version of
nbdkit that the plugin was compiled against.
Multiple enhancements to the Rust plugin (Alan Somers).
API
New ".list_exports", ".default_export" and ".export_description" plugin
methods allowing plugins to be queried about what export names they
support, and give additional descriptions of exports. New
"nbdkit_use_default_export" helper function (Eric Blake).
New "nbdkit_strdup_intern", "nbdkit_strndup_intern"
"nbdkit_printf_intern" and "nbdkit_vprintf_intern" functions which
allow you to allocate a string in a plugin without worrying about
freeing it (Eric Blake).
New "nbdkit_peer_pid", "nbdkit_peer_uid" and "nbdkit_peer_gid" allows
plugins and filters to read the client’s credentials, for clients
connected over a Unix domain socket.
Tests
Many tests now use libnbd(3), nbdsh(1), nbdinfo(1) and nbdcopy(1) as
lighter weight replacements for qemu tools.
Fix compatibility with qemu > 5.2.0.
Build
Downstream packagers can now use this to add extra downstream
information which is printed in version strings etc:
./configure --with-extra=...
If ocamldoc(1) is installed at build time, then it will be used to
build the NBDKit(3) manual page for OCaml documentation.
SEE ALSO
nbdkit(1).
AUTHORS
Authors of nbdkit 1.24:
• Alan Somers
• Eric Blake
• Richard W.M. Jones
COPYRIGHT
Copyright (C) 2020-2021 Red Hat Inc.
LICENSE
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
• Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the
distribution.
• Neither the name of Red Hat nor the names of its contributors may
be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
nbdkit-1.24.0 2021-01-07 nbdkit-release-notes-1.24(1)
-------
These are the release notes for libnbd stable release 1.6. This
describes the major changes since 1.4.
libnbd 1.6.0 was released on 7 January 2021.
Security
There were no security issues found in this release.
If you find a security issue, please read SECURITY in the source
(online here:
https://github.com/libguestfs/libnbd/blob/master/SECURITY). To find
out about previous security issues in libnbd, see libnbd-security(3).
New APIs
nbd_set_strict_mode(3)
nbd_get_strict_mode(3)
This API allows you to control how libnbd handles requests from the
program that it thinks would trigger undefined behaviour in the
server (such as sending unknown commands, zero-sized or out-of-
bounds requests, etc). Normally libnbd is very strict and will
prevent you from sending such requests, but you can use this API to
relax these restrictions, leaving it up to the server to handle or
reject them. Some servers may crash if you do this.
nbd_get_nr_meta_contexts(3)
nbd_get_meta_context(3)
nbd_clear_meta_contexts(3)
These APIs allow you to review or clear the requested metadata
contexts.
nbd_opt_list_meta_context(3)
nbd_aio_opt_list_meta_context(3)
Request that the server lists available metadata contexts. You can
only use this in opt mode.
Thanks: Eric Blake for all of the new APIs.
Enhancements to existing APIs
nbd_shutdown(3) new flag "LIBNBD_SHUTDOWN_ABANDON_PENDING" drops any
commands in flight which have not been sent to the server yet (Eric
Blake).
All APIs which have flag (bitmask) parameters now define a
"LIBNBD_*_MASK" constant allowing you to tell which bits represent
valid flags at compilation time (Eric Blake).
New features
The performance of the nbdcopy(1) tool has been greatly improved. Also
it supports numerous new features.
New nbdinfo(1) --map option showing the allocated and sparse extents of
an NBD server. This can also show other information such as dirty
bitmaps.
Tools
nbdfuse(1) can now mount the NBD export on top of any regular file, it
does not have to mount over a directory.
nbdsh(1) now produces a nice error message if the URI in the -u option
or the command in the -c option is wrong (Eric Blake).
New nbdsh(1) --opt-mode option which requests opt mode on the handle
before connecting.
nbdinfo(1) --list option now shows the supported metadata contexts of
the server (Eric Blake).
The output of nbdinfo(1) is now usually printed atomically, ie. it will
either print the complete output or an error message, but not partial
output followed by an error.
nbdinfo(1) can now decode "qemu:allocation-depth" metacontext (Eric
Blake).
Other improvements and bug fixes
Multiple improvements to handling of NBD protocol (Eric Blake).
Fix handling of booleans in Python bindings (Eric Blake).
Fix handling of flags parameters in OCaml bindings (Eric Blake).
Python style fixes (Eric Blake).
Fix multiple memory leaks (Eric Blake).
OCaml bindings now include ocamldoc-generated documentation, if
ocamldoc is installed at build time.
The "vector" library from nbdkit is now used by libnbd, simplifying
places where we used to use realloc(3).
SEE ALSO
libnbd(3).
AUTHORS
Eric Blake
Richard W.M. Jones
COPYRIGHT
Copyright (C) 2020-2021 Red Hat Inc.
LICENSE
This library is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This library is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301 USA
libnbd-1.6.0 2021-01-07 libnbd-release-notes-1.6(1)
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
More information about the Libguestfs
mailing list