[Cluster-devel] [DLM PATCH] DLM: Don't wait for resource library lookups if NOLOOKUP is specified

[Steven Whitehouse]

Hi,

On 01/10/14 18:21, Bob Peterson wrote:
> Hi,
>
> This patch adds a new lock flag, DLM_LKF_NOLOOKUP, which instructs DLM
> to refrain from sending lookup requests in cases where the lock library
> node is not the current node. This is similar to the DLM_LKF_NOQUEUE
> flag, except it fails locks that would require a lookup, with -EAGAIN.
>
> This is not just about saving a network operation. It allows callers
> like GFS2 to master locks for which they are the directory node. Each
> node can then "prefer" local locks, especially in the case of GFS2
> selecting resource groups for block allocations (implemented with a
> separate patch). This mastering of local locks distributes the locks
> between the nodes (at least until nodes enter or leave the cluster),
> which tends to make each node "keep to itself" when doing allocations.
> Thus, dlm communications are kept to a minimum, which results in
> significantly faster block allocations.
I think we need to do some more investigation here... how long do the 
lookups take? If the issue is just to create a list of perferred rgrps 
for each node, then there are various ways in which we might do that. 
That is not to say that this isn't a good way to do it, but I think we 
should try to understand the timings here first and make sure that we 
are solving the right problem,

Steve.

> Regards,
>
> Bob Peterson
> Red Hat File Systems
>
> Signed-off-by: Bob Peterson <rpeterso at redhat.com>
> ---
>   fs/dlm/lock.c                     | 16 ++++++++++++++--
>   include/uapi/linux/dlmconstants.h |  7 +++++++
>   2 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/fs/dlm/lock.c b/fs/dlm/lock.c
> index 83f3d55..f1e5b04 100644
> --- a/fs/dlm/lock.c
> +++ b/fs/dlm/lock.c
> @@ -222,6 +222,11 @@ static inline int can_be_queued(struct dlm_lkb *lkb)
>   	return !(lkb->lkb_exflags & DLM_LKF_NOQUEUE);
>   }
>   
> +static inline int can_be_looked_up(struct dlm_lkb *lkb)
> +{
> +	return !(lkb->lkb_exflags & DLM_LKF_NOLOOKUP);
> +}
> +
>   static inline int force_blocking_asts(struct dlm_lkb *lkb)
>   {
>   	return (lkb->lkb_exflags & DLM_LKF_NOQUEUEBAST);
> @@ -2745,6 +2750,11 @@ static int set_master(struct dlm_rsb *r, struct dlm_lkb *lkb)
>   		return 0;
>   	}
>   
> +	if (!can_be_looked_up(lkb)) {
> +		queue_cast(r, lkb, -EAGAIN);
> +		return -EAGAIN;
> +	}
> +
>   	wait_pending_remove(r);
>   
>   	r->res_first_lkid = lkb->lkb_id;
> @@ -2828,7 +2838,8 @@ static int set_lock_args(int mode, struct dlm_lksb *lksb, uint32_t flags,
>   	if (flags & DLM_LKF_CONVDEADLK && !(flags & DLM_LKF_CONVERT))
>   		goto out;
>   
> -	if (flags & DLM_LKF_CONVDEADLK && flags & DLM_LKF_NOQUEUE)
> +	if (flags & DLM_LKF_CONVDEADLK && (flags & (DLM_LKF_NOQUEUE |
> +						    DLM_LKF_NOLOOKUP)))
>   		goto out;
>   
>   	if (flags & DLM_LKF_EXPEDITE && flags & DLM_LKF_CONVERT)
> @@ -2837,7 +2848,8 @@ static int set_lock_args(int mode, struct dlm_lksb *lksb, uint32_t flags,
>   	if (flags & DLM_LKF_EXPEDITE && flags & DLM_LKF_QUECVT)
>   		goto out;
>   
> -	if (flags & DLM_LKF_EXPEDITE && flags & DLM_LKF_NOQUEUE)
> +	if (flags & DLM_LKF_EXPEDITE && (flags & (DLM_LKF_NOQUEUE |
> +						  DLM_LKF_NOLOOKUP)))
>   		goto out;
>   
>   	if (flags & DLM_LKF_EXPEDITE && mode != DLM_LOCK_NL)
> diff --git a/include/uapi/linux/dlmconstants.h b/include/uapi/linux/dlmconstants.h
> index 47bf08d..4b9ba15 100644
> --- a/include/uapi/linux/dlmconstants.h
> +++ b/include/uapi/linux/dlmconstants.h
> @@ -131,6 +131,12 @@
>    * Unlock the lock even if it is converting or waiting or has sublocks.
>    * Only really for use by the userland device.c code.
>    *
> + * DLM_LKF_NOLOOKUP
> + *
> + * Don't take any network time/bandwidth to do directory owner lookups.
> + * This is a lock for which we only care whether it's completely under
> + * local jurisdiction.
> + *
>    */
>   
>   #define DLM_LKF_NOQUEUE		0x00000001
> @@ -152,6 +158,7 @@
>   #define DLM_LKF_ALTCW		0x00010000
>   #define DLM_LKF_FORCEUNLOCK	0x00020000
>   #define DLM_LKF_TIMEOUT		0x00040000
> +#define DLM_LKF_NOLOOKUP	0x00080000
>   
>   /*
>    * Some return codes that are not in errno.h
>