[dm-devel] [RFC PATCH 1/3] block: add copy offload support

[Knight, Frederick]

Just FYI about copy domains.

SCSI already has copy domains defined (it is optional in the standard).  Third-party Copy Descriptor - Copy Group Identifier. Here's the SCSI text:

A Copy Group is a set of logical units that have a high probability of using high performance methods (e.g.,
copy on write snapshot) for third-party copy operations involving logical units that are in the same Copy
Group. Each logical unit may be a member of zero or one Copy Group. A logical unit indicates membership in
a Copy Group using the Copy Group Identifier third-party copy descriptor (see 7.7.18.13).

If a third-party copy operation involves logical units that are in different Copy Groups, then that third-party copy
operation has a high probability of using low performance methods (e.g., copy manager read operations from
the source CSCD or copy manager write operations to the destination CSCD).

NVMe today can only copy between LBAs on the SAME Namespace.  There is a new project just getting started to allow copy across namespaces in the same NVM subsystem (included as part of that project is to define how the copy domains will work).  So for NVMe, copy domains are still a work in progress.

Just FYI.

	Fred

-----Original Message-----
From: Keith Busch <kbusch at kernel.org> 
Sent: Wednesday, February 2, 2022 11:22 AM
To: Mikulas Patocka <mpatocka at redhat.com>
Cc: Javier González <javier at javigon.com>; Chaitanya Kulkarni <chaitanyak at nvidia.com>; linux-block at vger.kernel.org; linux-scsi at vger.kernel.org; dm-devel at redhat.com; linux-nvme at lists.infradead.org; linux-fsdevel <linux-fsdevel at vger.kernel.org>; Jens Axboe <axboe at kernel.dk>; msnitzer at redhat.com >> msnitzer at redhat.com <msnitzer at redhat.com>; Bart Van Assche <bvanassche at acm.org>; martin.petersen at oracle.com >> Martin K. Petersen <martin.petersen at oracle.com>; roland at purestorage.com; Hannes Reinecke <hare at suse.de>; Christoph Hellwig <hch at lst.de>; Knight, Frederick <Frederick.Knight at netapp.com>; zach.brown at ni.com; osandov at fb.com; lsf-pc at lists.linux-foundation.org; djwong at kernel.org; josef at toxicpanda.com; clm at fb.com; dsterba at suse.com; tytso at mit.edu; jack at suse.com; Kanchan Joshi <joshi.k at samsung.com>
Subject: Re: [RFC PATCH 1/3] block: add copy offload support

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




On Tue, Feb 01, 2022 at 01:32:29PM -0500, Mikulas Patocka wrote:
> +int blkdev_issue_copy(struct block_device *bdev1, sector_t sector1,
> +                   struct block_device *bdev2, sector_t sector2,
> +                   sector_t nr_sects, sector_t *copied, gfp_t 
> +gfp_mask) {
> +     struct page *token;
> +     sector_t m;
> +     int r = 0;
> +     struct completion comp;
> +
> +     *copied = 0;
> +
> +     m = min(bdev_max_copy_sectors(bdev1), bdev_max_copy_sectors(bdev2));
> +     if (!m)
> +             return -EOPNOTSUPP;
> +     m = min(m, (sector_t)round_down(UINT_MAX, PAGE_SIZE) >> 9);
> +
> +     if (unlikely(bdev_read_only(bdev2)))
> +             return -EPERM;
> +
> +     token = alloc_page(gfp_mask);
> +     if (unlikely(!token))
> +             return -ENOMEM;
> +
> +     while (nr_sects) {
> +             struct bio *read_bio, *write_bio;
> +             sector_t this_step = min(nr_sects, m);
> +
> +             read_bio = bio_alloc(gfp_mask, 1);
> +             if (unlikely(!read_bio)) {
> +                     r = -ENOMEM;
> +                     break;
> +             }
> +             bio_set_op_attrs(read_bio, REQ_OP_COPY_READ_TOKEN, REQ_NOMERGE);
> +             bio_set_dev(read_bio, bdev1);
> +             __bio_add_page(read_bio, token, PAGE_SIZE, 0);

You have this "token" payload as driver specific data, but there's no check that bdev1 and bdev2 subscribe to the same driver specific format.

I thought we discussed defining something like a "copy domain" that establishes which block devices can offload copy operations to/from each other, and that should be checked before proceeding with the copy operation.