[dm-devel] reinstate path not working
Benjamin Marzinski
bmarzins at redhat.com
Mon Jul 6 17:45:15 UTC 2015
On Tue, Jun 30, 2015 at 11:22:37AM +0530, Tejaswini Poluri wrote:
> Thanks Ben. So should I push the code to the upstream and get an approval?
> Regards,
Yes. Thanks!
-Ben
> Tejaswini
> On Tue, Jun 30, 2015 at 12:50 AM, Benjamin Marzinski
> <[1]bmarzins at redhat.com> wrote:
>
> On Fri, Jun 26, 2015 at 12:21:48PM +0530, Tejaswini Poluri wrote:
> > Yes I agree that having the same code in both cli_add_map() and
> ev_add_map
> > is not necessary. Hence I would suggest removing get_refwwid() code
> from
> > ev_add_map as it is not being used by anyone.
> >
> > ev_add_map(param, NULL, vecs) would create the multipath device by
> using
> > the get_refwwid() code and but all the functions above it like
> > (dm_get_minor, dm_get_major and dm_mapname) would fail and it
> wouldn't
> > enter any of the other code in ev_add_map like
> > 1.dm_map_present,
> > 2.add_map_without_path
> > 3. sync_map_state
> > which are responsible for registering the map and displaying it.
> >
> > So, I think moving the below code from ev_add_map to cli_add_map
> should be
> > a good idea right.
> >
> > r = get_refwwid(dev, DEV_DEVMAP, vecs->pathvec,&refwwid);
> > >
> > > if (refwwid) {
> > > r = coalesce_paths(vecs, NULL, refwwid,0);
> > > dm_lib_release();
> > > }
> > What do u think?
>
> I agree. We aren't using the code in ev_add_map, so it's presence there
> is simply confusing.
>
> -Ben
> >
> > Regards,
> > Tejaswini
> > On Fri, Jun 26, 2015 at 4:32 AM, Benjamin Marzinski
> > <[1][2]bmarzins at redhat.com> wrote:
> >
> > On Tue, Jun 23, 2015 at 03:48:26PM +0530, Tejaswini Poluri wrote:
> > > Hi Ben,
> > >
> > > This is regarding the add map issue I have been discussing.
> Posting
> > the
> > > issue again to remind.
> > >
> > > Case 1 : remove and add map.
> > > root at x86-generic-64:~# multipathd -k'show maps'
> > > name sysfs uuid
> > > dmpath0 dm-0 1IET_00010001
> > > root at x86-generic-64:~# multipathd -k'remove map dmpath0'
> > > ok
> > > root at x86-generic-64:~# multipathd -k'show maps'
> > > root at x86-generic-64:~# multipathd -k'add map dmpath0'
> > > ok
> > > root at x86-generic-64:~# multipathd -k'show maps'
> > > root at x86-generic-64:~#
> > > Once a map is removed, we are able to add it only using
> #multipath
> > > command and not using multipathd tools.
> > >
> > > I have fixed the problem with two approaches. I would like
> you to
> > review
> > > the same.
> > > Patch1 : By making 'remove map dmpath0' to remove only the
> map and
> > not the
> > > device. I have added extra functions discard_map and
> dm_remove_map
> > so as
> > > to not interfere with the existing code.
> > >
> > > Patch 2: The approach you have suggested.By getting wwid
> from the
> > mapname
> > > and doing coalesce_paths. I have just moved the following
> code in
> > > ev_add_map to cli_add_map.
> >
> > This is the general idea we'd like to go with. However, looking
> at the
> > latest upstream code, I don't think you should pull code in from
> > ev_add_map() to cli_add_map() like your patch does. cli_add_map()
> > already calls ev_add_map(), and ev_add_map() is certainly able to
> add
> > the map if it doesn't already exist.
> >
> > You would just need to call it with
> >
> > ev_add_map(param, NULL, vecs);
> >
> > and ev_add_map() will happily create you a new multipath device.
> All
> > you need to do is make sure that all the functions that
> ev_add_map()
> > calls with alias can accept a NULL value there.
> >
> > This might not be the best way to go about this, however. It
> turns out
> > that right now, even though ev_add_map() technically has the
> ability to
> > create new maps, nothing currently uses it, and it really doesn't
> make
> > sense for it to be there. Instead of just copying that code, you
> could
> > pull the map creation code out of ev_add_map() and add it to
> > cli_add_map(), for those situations where the requested device
> doesn't
> > already exist.
> >
> > But having the code in both cli_add_map() and ev_add_map() when
> one
> > already calls the other doesn't seem necessary.
> >
> > -Ben
> >
> > >
> > > r = get_refwwid(dev, DEV_DEVMAP, vecs->pathvec,&refwwid);
> > >
> > > if (refwwid) {
> > > r = coalesce_paths(vecs, NULL, refwwid,0);
> > > dm_lib_release();
> > > }
> > >
> > > changed dev to param.
> > >
> > > I have tested the same in all 3 versions -0.4.8, 0.4.9 and
> 0.5.0.
> > It would
> > > be great if you can review the same so that it doesn't cause
> any
> > extra
> > > side effects.
> > > I guess Patch2 is the way u have suggested me in the
> previous mail.
> > Please
> > > review it and share your views.
> > > Regards,
> > > Tejaswini
> > > On Fri, Jun 12, 2015 at 2:21 AM, Benjamin Marzinski
> > > <[1][2][3]bmarzins at redhat.com> wrote:
> > >
> > > On Wed, Jun 10, 2015 at 11:46:51AM +0530, Tejaswini Poluri
> wrote:
> > > >
> > > > > We are testing multipathd tools with all the
> possible
> > options
> > > and the
> > > > > following fails.
> > > > >
> > > > > Case 1 : remove and add map.
> > > > > root at x86-generic-64:~# multipathd -k'show maps'
> > > > > name sysfs uuid
> > > > > dmpath0 dm-0 1IET_00010001
> > > > > root at x86-generic-64:~# multipathd -k'remove map
> > dmpath0'
> > > > > ok
> > > > > root at x86-generic-64:~# multipathd -k'show maps'
> > > > > root at x86-generic-64:~# multipathd -k'add map
> dmpath0'
> > > > > ok
> > > > > root at x86-generic-64:~# multipathd -k'show maps'
> > > > > root at x86-generic-64:~#
> > > > > Once a map is removed, we are able to add it
> only using
> > > #multipath
> > > > > command and not using multipathd tools.
> > > >
> > > > It is working the way it was designed, but possibly
> it would
> > make
> > > sense
> > > > to change the design.
> > > >
> > > > You have mentioned that it would make sense to change
> the
> > design to
> > > add
> > > > map. Are there plans to change the design ?
> > > > I am trying to understand the code flow to change the
> > design. Can
> > > you
> > > > guide me if we should stop removing the device from
> in the
> > remove
> > > map code
> > > > flow or start adding the device and the map in the
> add map
> > code
> > > flow.
> > > >
> > > > have tried to understand the remove map code flow of
> > multipathd in
> > > 0.4.8
> > > > code.
> > >
> > > I think that we want multipath to actually remove the map
> > (instead of
> > > just not monitoring it) when you call "remove map <map>".
> We just
> > want
> > > "add map <map>" to try to create the map if it doesn't
> exist. To
> > do
> > > that, you would need to first firgure out what WWID is
> associated
> > with
> > > <map>. Presumably, <map> could either be an alias, wwid,
> or even
> > the
> > > name of a path in the map. Once you found the map, you
> would have
> > to
> > > call the code to create the map.
> > >
> > > Also, to answer your IRC question, no the 0.4.8 code is
> not still
> > being
> > > developed upstream. All upstream patches only go against
> the
> > current
> > > head. There are no other upstream branches.
> > >
> > > -Ben
> > > >
> > > > ev_remove_map (char * devname, struct vectors * vecs)
> > > >
> > > > flush_map(mpp, vecs);
> > > >
> > > > dm_flush_map(mpp->alias,
> > DEFAULT_TARGET);
> > > >
> > > > if
> > (!dm_map_present(mapname))
> > > >
> > > > return 0;
> > > >
> > > > if (dm_type(mapname, type) <= 0)
> > > >
> > > > return 1;
> > > >
> > > > if (dm_remove_partmaps(mapname))
> > > >
> > > > return 1;
> > > >
> > > > if (dm_get_opencount(mapname)) {
> > > >
> > > > condlog(2, "%s: map in use", mapname);
> > > >
> > > > return 1;
> > > >
> > > > }
> > > >
> > > > r = dm_simplecmd(DM_DEVICE_REMOVE, mapname);
> > > >
> > > > if (r) {
> > > >
> > > > condlog(4, "multipath map %s removed",
> > mapname);
> > > >
> > > > return
> 0;
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > > orphan_paths(vecs->pathvec,
> mpp);
> > > >
> > > > remove_map(mpp, vecs,
> > stop_waiter_thread,
> > > 1);
> > > >
> > > > Is removing this below line, the right step to stop
> removing
> > the
> > > device ?
> > > > r = dm_simplecmd(DM_DEVICE_REMOVE, mapname);
> > > >
> > > > Regards,
> > > >
> > > > Tejaswini
> > > >
> > > > On Mon, Jun 8, 2015 at 11:15 AM, Tejaswini Poluri
> > > > <[1][2][3][4]tejaswinipoluri3 at gmail.com> wrote:
> > > >
> > > > Thanks a lot Ben for the quick and detailed reply.
> I have
> > been
> > > > struggling to understand and conclude the issues
> with
> > multipath
> > > as I am
> > > > the only one working from my team. Your inputs help
> me a
> > lot.
> > > Thanks
> > > > again.
> > > > Regards,
> > > > Tejaswini
> > > > On Sat, Jun 6, 2015 at 3:36 AM, Benjamin Marzinski
> > > > <[2][3][4][5]bmarzins at redhat.com> wrote:
> > > >
> > > > On Fri, Jun 05, 2015 at 02:31:20PM +0530,
> Tejaswini
> > Poluri
> > > wrote:
> > > > > Hii Ben,
> > > > >
> > > > > We are testing multipathd tools with all the
> > possible
> > > options and
> > > > the
> > > > > following fails.
> > > > >
> > > > > Case 1 : remove and add map.
> > > > > root at x86-generic-64:~# multipathd -k'show
> maps'
> > > > > name sysfs uuid
> > > > > dmpath0 dm-0 1IET_00010001
> > > > > root at x86-generic-64:~# multipathd -k'remove
> map
> > dmpath0'
> > > > > ok
> > > > > root at x86-generic-64:~# multipathd -k'show
> maps'
> > > > > root at x86-generic-64:~# multipathd -k'add map
> > dmpath0'
> > > > > ok
> > > > > root at x86-generic-64:~# multipathd -k'show
> maps'
> > > > > root at x86-generic-64:~#
> > > > > Once a map is removed, we are able to add it
> only
> > using
> > > > #multipath
> > > > > command and not using multipathd tools.
> > > >
> > > > It is working the way it was designed, but
> possibly it
> > would
> > > make
> > > > sense
> > > > to change the design. The "remove map" command,
> not
> > only stops
> > > > multipathd from monitoring the multipath device,
> but it
> > removes
> > > it
> > > > from
> > > > the system as well. The "add map" command makes
> > multipath
> > > monitor an
> > > > already existing multipath device that in wasn't
> > previously
> > > > monitoring.
> > > > These commands do this for historical reasons.
> > multipathd
> > > wasn't
> > > > originally in charge of creating multipath
> devices,
> > multipath
> > > was.
> > > > Once
> > > > it had created the device, it ran
> > > >
> > > > multipathd -k"add map <MAP>"
> > > >
> > > > to make multipathd start monitoring it. However
> things
> > haven't
> > > worked
> > > > this way since RHEL4, so possibly "add map"
> should
> > actually
> > > create the
> > > > device if it doesn't currently exist.
> > > > > Case 2 : Active paths test case
> > > > > # while true ; do sleep 3 ; multipathd
> -k'remove
> > path sdb'
> > > ;
> > > > multipathd
> > > > > -k'add path sdb' ; multipathd -k'show maps
> status'
> > ; done
> > > > > ok
> > > > > ok
> > > > > name failback queueing paths dm-st
> > > > > dmpath0 - - 1 active // It should be 2.
> > > >
> > > > This is simply a timing issue. What you are
> seeing it
> > the
> > > number of
> > > > active paths. These are paths that the kernel
> can use.
> > The
> > > "add path"
> > > > command doesn't update the kernel state. This
> happens
> > later in
> > > > response
> > > > to the kernel reloading the device table. So, in
> a
> > second or
> > > two, this
> > > > will say 2, as expected.
> > > >
> > > > > We would like to know if the test cases are
> valid
> > and if
> > > these
> > > > are bugs or
> > > > > any design issues.
> > > > >
> > > > > Case 3 : Fail path and reinstate path
> > > > > root at x86-generic-64:~# multipathd -k"fail
> path
> > sdc";
> > > multipathd
> > > > > -k'reinstate path sdc'; multipathd -k"show
> paths";
> > > > > > [ 3962.708523] device-mapper:
> multipath:
> > Failing path
> > > 8:32.
> > > > > > ok
> > > > > > ok
> > > > > > hcil dev dev_t pri dm_st chk_st
> > next_check
> > > > > > 4:0:0:1 sdc 8:32 1 [active][faulty]
> > ..........
> > > 1/20
> > > > <==CHECK
> > > > > > 5:0:0:1 sdd 8:48 1 [active][ready]
> > XX........
> > > 4/20
> > > > > sdc path becomes [active][ready] only after
> the
> > polling
> > > interval
> > > > but not
> > > > > immediately after the reinstate path
> command.
> > > > > You have answered that this is a design
> issue. But
> > we have
> > > heard
> > > > from our
> > > > > test team that the same test case works in
> RHEL6.
> > Did you
> > > observe
> > > > it?
> > > > > I am also finding that the test cases fail
> because
> > we are
> > > trying
> > > > to do
> > > > > multiple commands at one shot. Please share
> your
> > thoughts
> > > so
> > > > that it
> > > > > could help me in debugging the issues
> further.
> > > > >
> > > >
> > > > It's totally possible that the checker state is
> > immediately
> > > updated in
> > > > RHEL6. Like I said before, what it currently
> does,
> > although
> > > correct,
> > > > is confusing, and perhaps we need a different
> checker
> > state for
> > > paths
> > > > where the "fail path" command has been used.
> > > >
> > > > -Ben
> > > > > Regards,
> > > > > Tejaswini
> > > > > On Tue, May 19, 2015 at 5:37 PM, Tejaswini
> Poluri
> > > > > <[1][3][4][5][6]tejaswinipoluri3 at gmail.com>
> wrote:
> > > > >
> > > > > Thanks a lot Ben. I will look into it
> more.
> > > > > On Mon, May 18, 2015 at 9:57 PM, Benjamin
> > Marzinski
> > > > > <[2][4][5][6][7]bmarzins at redhat.com>
> wrote:
> > > > >
> > > > > On Mon, May 18, 2015 at 02:09:27PM
> +0530,
> > Tejaswini
> > > Poluri
> > > > wrote:
> > > > > > Hii,
> > > > > > We are trying to test multipath
> setup in
> > our
> > > target and
> > > > tried the
> > > > > various
> > > > > > commands of multipathd demaon and
> we find
> > the
> > > following
> > > > error:
> > > > > > root at x86-generic-64:~# multipathd
> -k"fail
> > path
> > > sdc";
> > > > multipathd
> > > > > > -k'reinstate path
> > > > > > sdc'; multipathd -k"show paths";
> > > > > > [ 3962.708523] device-mapper:
> multipath:
> > Failing
> > > > path 8:32.
> > > > > > ok
> > > > > > ok
> > > > > > hcil dev dev_t pri dm_st
> chk_st
> > next_check
> > > > > > 4:0:0:1 sdc 8:32 1
> [active][faulty]
> > ..........
> > > 1/20
> > > > <<<===
> > > > > CHECK
> > > > > > 5:0:0:1 sdd 8:48 1
> [active][ready]
> > XX........
> > > 4/20
> > > > > > sdc path becomes [active][ready]
> only
> > after the
> > > polling
> > > > interval
> > > > > but not
> > > > > > immediately after the reinstate
> path
> > command.
> > > > > > I am observing this in latest
> multipath
> > tools in
> > > ubuntu
> > > > machine
> > > > > as well.
> > > > > > Please let me know if its a known
> issue or
> > if I
> > > am doing
> > > > > something wrong.
> > > > > > Regards.
> > > > > > Tejaswini
> > > > >
> > > > > the reinstate command is supposed to
> reinstate
> > the
> > > device
> > > > with the
> > > > > kernel, and it does that. The checker
> state
> > doesn't
> > > change
> > > > until the
> > > > > next time that the path is checked. I
> agree
> > that it's
> > > odd
> > > > that the
> > > > > check state switches to faulty as soon
> as you
> > fail the
> > > path,
> > > > but it
> > > > > doesn't switch back until the next check
> after
> > you
> > > reinistate
> > > > it.
> > > > >
> > > > > The issue is that multipathd needs to
> override
> > the
> > > checker
> > > > output,
> > > > > so that a failed path won't be
> immeditately
> > > reinstated. Once
> > > > the
> > > > > path comes back, multipathd wants to
> record the
> > switch
> > > in the
> > > > checker
> > > > > thread, so that it can refresh path
> information
> > what
> > > wasn't
> > > > > automatically done when the path was
> > reinstated.
> > > However, it
> > > > may make
> > > > > more sense to have a different checker
> state
> > for when
> > > the
> > > > device is
> > > > > in the failed state, so that it's
> obvious that
> > the
> > > checker
> > > > state is
> > > > > being overruled.
> > > > >
> > > > > -Ben
> > > > >
> > > > > > --
> > > > > > dm-devel mailing list
> > > > > > [3][5][6][7][8]dm-devel at redhat.com
> > > > > >
> > >
> [4][6][7][8][9]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > >
> > > > > --
> > > > > dm-devel mailing list
> > > > > [5][7][8][9][10]dm-devel at redhat.com
> > > > >
> > >
> [6][8][9][10][11]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > >
> > > > > References
> > > > >
> > > > > Visible links
> > > > > 1.
> mailto:[9][10][11][12]tejaswinipoluri3 at gmail.com
> > > > > 2.
> mailto:[10][11][12][13]bmarzins at redhat.com
> > > > > 3.
> mailto:[11][12][13][14]dm-devel at redhat.com
> > > > > 4.
> > >
> [12][13][14][15]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > > 5.
> mailto:[13][14][15][16]dm-devel at redhat.com
> > > > > 6.
> > >
> [14][15][16][17]https://www.redhat.com/mailman/listinfo/dm-devel
> > > >
> > > > References
> > > >
> > > > Visible links
> > > > 1. mailto:[16][17][18]tejaswinipoluri3 at gmail.com
> > > > 2. mailto:[17][18][19]bmarzins at redhat.com
> > > > 3. mailto:[18][19][20]tejaswinipoluri3 at gmail.com
> > > > 4. mailto:[19][20][21]bmarzins at redhat.com
> > > > 5. mailto:[20][21][22]dm-devel at redhat.com
> > > > 6.
> [21][22][23]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > 7. mailto:[22][23][24]dm-devel at redhat.com
> > > > 8.
> [23][24][25]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > 9. mailto:[24][25][26]tejaswinipoluri3 at gmail.com
> > > > 10. mailto:[25][26][27]bmarzins at redhat.com
> > > > 11. mailto:[26][27][28]dm-devel at redhat.com
> > > > 12.
> [27][28][29]https://www.redhat.com/mailman/listinfo/dm-devel
> > > > 13. mailto:[28][29][30]dm-devel at redhat.com
> > > > 14.
> [29][30][31]https://www.redhat.com/mailman/listinfo/dm-devel
> > >
> > > References
> > >
> > > Visible links
> > > 1. mailto:[31][32]bmarzins at redhat.com
> > > 2. mailto:[32][33]tejaswinipoluri3 at gmail.com
> > > 3. mailto:[33][34]bmarzins at redhat.com
> > > 4. mailto:[34][35]tejaswinipoluri3 at gmail.com
> > > 5. mailto:[35][36]bmarzins at redhat.com
> > > 6. mailto:[36][37]dm-devel at redhat.com
> > > 7. [37][38]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 8. mailto:[38][39]dm-devel at redhat.com
> > > 9. [39][40]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 10. mailto:[40][41]tejaswinipoluri3 at gmail.com
> > > 11. mailto:[41][42]bmarzins at redhat.com
> > > 12. mailto:[42][43]dm-devel at redhat.com
> > > 13. [43][44]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 14. mailto:[44][45]dm-devel at redhat.com
> > > 15. [45][46]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 16. mailto:[46][47]tejaswinipoluri3 at gmail.com
> > > 17. mailto:[47][48]bmarzins at redhat.com
> > > 18. mailto:[48][49]tejaswinipoluri3 at gmail.com
> > > 19. mailto:[49][50]bmarzins at redhat.com
> > > 20. mailto:[50][51]dm-devel at redhat.com
> > > 21. [51][52]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 22. mailto:[52][53]dm-devel at redhat.com
> > > 23. [53][54]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 24. mailto:[54][55]tejaswinipoluri3 at gmail.com
> > > 25. mailto:[55][56]bmarzins at redhat.com
> > > 26. mailto:[56][57]dm-devel at redhat.com
> > > 27. [57][58]https://www.redhat.com/mailman/listinfo/dm-devel
> > > 28. mailto:[58][59]dm-devel at redhat.com
> > > 29. [59][60]https://www.redhat.com/mailman/listinfo/dm-devel
> >
> > References
> >
> > Visible links
> > 1. mailto:[61]bmarzins at redhat.com
> > 2. mailto:[62]bmarzins at redhat.com
> > 3. mailto:[63]tejaswinipoluri3 at gmail.com
> > 4. mailto:[64]bmarzins at redhat.com
> > 5. mailto:[65]tejaswinipoluri3 at gmail.com
> > 6. mailto:[66]bmarzins at redhat.com
> > 7. mailto:[67]dm-devel at redhat.com
> > 8. [68]https://www.redhat.com/mailman/listinfo/dm-devel
> > 9. mailto:[69]dm-devel at redhat.com
> > 10. [70]https://www.redhat.com/mailman/listinfo/dm-devel
> > 11. mailto:[71]tejaswinipoluri3 at gmail.com
> > 12. mailto:[72]bmarzins at redhat.com
> > 13. mailto:[73]dm-devel at redhat.com
> > 14. [74]https://www.redhat.com/mailman/listinfo/dm-devel
> > 15. mailto:[75]dm-devel at redhat.com
> > 16. [76]https://www.redhat.com/mailman/listinfo/dm-devel
> > 17. mailto:[77]tejaswinipoluri3 at gmail.com
> > 18. mailto:[78]bmarzins at redhat.com
> > 19. mailto:[79]tejaswinipoluri3 at gmail.com
> > 20. mailto:[80]bmarzins at redhat.com
> > 21. mailto:[81]dm-devel at redhat.com
> > 22. [82]https://www.redhat.com/mailman/listinfo/dm-devel
> > 23. mailto:[83]dm-devel at redhat.com
> > 24. [84]https://www.redhat.com/mailman/listinfo/dm-devel
> > 25. mailto:[85]tejaswinipoluri3 at gmail.com
> > 26. mailto:[86]bmarzins at redhat.com
> > 27. mailto:[87]dm-devel at redhat.com
> > 28. [88]https://www.redhat.com/mailman/listinfo/dm-devel
> > 29. mailto:[89]dm-devel at redhat.com
> > 30. [90]https://www.redhat.com/mailman/listinfo/dm-devel
> > 31. mailto:[91]bmarzins at redhat.com
> > 32. mailto:[92]tejaswinipoluri3 at gmail.com
> > 33. mailto:[93]bmarzins at redhat.com
> > 34. mailto:[94]tejaswinipoluri3 at gmail.com
> > 35. mailto:[95]bmarzins at redhat.com
> > 36. mailto:[96]dm-devel at redhat.com
> > 37. [97]https://www.redhat.com/mailman/listinfo/dm-devel
> > 38. mailto:[98]dm-devel at redhat.com
> > 39. [99]https://www.redhat.com/mailman/listinfo/dm-devel
> > 40. mailto:[100]tejaswinipoluri3 at gmail.com
> > 41. mailto:[101]bmarzins at redhat.com
> > 42. mailto:[102]dm-devel at redhat.com
> > 43. [103]https://www.redhat.com/mailman/listinfo/dm-devel
> > 44. mailto:[104]dm-devel at redhat.com
> > 45. [105]https://www.redhat.com/mailman/listinfo/dm-devel
> > 46. mailto:[106]tejaswinipoluri3 at gmail.com
> > 47. mailto:[107]bmarzins at redhat.com
> > 48. mailto:[108]tejaswinipoluri3 at gmail.com
> > 49. mailto:[109]bmarzins at redhat.com
> > 50. mailto:[110]dm-devel at redhat.com
> > 51. [111]https://www.redhat.com/mailman/listinfo/dm-devel
> > 52. mailto:[112]dm-devel at redhat.com
> > 53. [113]https://www.redhat.com/mailman/listinfo/dm-devel
> > 54. mailto:[114]tejaswinipoluri3 at gmail.com
> > 55. mailto:[115]bmarzins at redhat.com
> > 56. mailto:[116]dm-devel at redhat.com
> > 57. [117]https://www.redhat.com/mailman/listinfo/dm-devel
> > 58. mailto:[118]dm-devel at redhat.com
> > 59. [119]https://www.redhat.com/mailman/listinfo/dm-devel
>
> References
>
> Visible links
> 1. mailto:bmarzins at redhat.com
> 2. mailto:bmarzins at redhat.com
> 3. mailto:bmarzins at redhat.com
> 4. mailto:tejaswinipoluri3 at gmail.com
> 5. mailto:bmarzins at redhat.com
> 6. mailto:tejaswinipoluri3 at gmail.com
> 7. mailto:bmarzins at redhat.com
> 8. mailto:dm-devel at redhat.com
> 9. https://www.redhat.com/mailman/listinfo/dm-devel
> 10. mailto:dm-devel at redhat.com
> 11. https://www.redhat.com/mailman/listinfo/dm-devel
> 12. mailto:tejaswinipoluri3 at gmail.com
> 13. mailto:bmarzins at redhat.com
> 14. mailto:dm-devel at redhat.com
> 15. https://www.redhat.com/mailman/listinfo/dm-devel
> 16. mailto:dm-devel at redhat.com
> 17. https://www.redhat.com/mailman/listinfo/dm-devel
> 18. mailto:tejaswinipoluri3 at gmail.com
> 19. mailto:bmarzins at redhat.com
> 20. mailto:tejaswinipoluri3 at gmail.com
> 21. mailto:bmarzins at redhat.com
> 22. mailto:dm-devel at redhat.com
> 23. https://www.redhat.com/mailman/listinfo/dm-devel
> 24. mailto:dm-devel at redhat.com
> 25. https://www.redhat.com/mailman/listinfo/dm-devel
> 26. mailto:tejaswinipoluri3 at gmail.com
> 27. mailto:bmarzins at redhat.com
> 28. mailto:dm-devel at redhat.com
> 29. https://www.redhat.com/mailman/listinfo/dm-devel
> 30. mailto:dm-devel at redhat.com
> 31. https://www.redhat.com/mailman/listinfo/dm-devel
> 32. mailto:bmarzins at redhat.com
> 33. mailto:tejaswinipoluri3 at gmail.com
> 34. mailto:bmarzins at redhat.com
> 35. mailto:tejaswinipoluri3 at gmail.com
> 36. mailto:bmarzins at redhat.com
> 37. mailto:dm-devel at redhat.com
> 38. https://www.redhat.com/mailman/listinfo/dm-devel
> 39. mailto:dm-devel at redhat.com
> 40. https://www.redhat.com/mailman/listinfo/dm-devel
> 41. mailto:tejaswinipoluri3 at gmail.com
> 42. mailto:bmarzins at redhat.com
> 43. mailto:dm-devel at redhat.com
> 44. https://www.redhat.com/mailman/listinfo/dm-devel
> 45. mailto:dm-devel at redhat.com
> 46. https://www.redhat.com/mailman/listinfo/dm-devel
> 47. mailto:tejaswinipoluri3 at gmail.com
> 48. mailto:bmarzins at redhat.com
> 49. mailto:tejaswinipoluri3 at gmail.com
> 50. mailto:bmarzins at redhat.com
> 51. mailto:dm-devel at redhat.com
> 52. https://www.redhat.com/mailman/listinfo/dm-devel
> 53. mailto:dm-devel at redhat.com
> 54. https://www.redhat.com/mailman/listinfo/dm-devel
> 55. mailto:tejaswinipoluri3 at gmail.com
> 56. mailto:bmarzins at redhat.com
> 57. mailto:dm-devel at redhat.com
> 58. https://www.redhat.com/mailman/listinfo/dm-devel
> 59. mailto:dm-devel at redhat.com
> 60. https://www.redhat.com/mailman/listinfo/dm-devel
> 61. mailto:bmarzins at redhat.com
> 62. mailto:bmarzins at redhat.com
> 63. mailto:tejaswinipoluri3 at gmail.com
> 64. mailto:bmarzins at redhat.com
> 65. mailto:tejaswinipoluri3 at gmail.com
> 66. mailto:bmarzins at redhat.com
> 67. mailto:dm-devel at redhat.com
> 68. https://www.redhat.com/mailman/listinfo/dm-devel
> 69. mailto:dm-devel at redhat.com
> 70. https://www.redhat.com/mailman/listinfo/dm-devel
> 71. mailto:tejaswinipoluri3 at gmail.com
> 72. mailto:bmarzins at redhat.com
> 73. mailto:dm-devel at redhat.com
> 74. https://www.redhat.com/mailman/listinfo/dm-devel
> 75. mailto:dm-devel at redhat.com
> 76. https://www.redhat.com/mailman/listinfo/dm-devel
> 77. mailto:tejaswinipoluri3 at gmail.com
> 78. mailto:bmarzins at redhat.com
> 79. mailto:tejaswinipoluri3 at gmail.com
> 80. mailto:bmarzins at redhat.com
> 81. mailto:dm-devel at redhat.com
> 82. https://www.redhat.com/mailman/listinfo/dm-devel
> 83. mailto:dm-devel at redhat.com
> 84. https://www.redhat.com/mailman/listinfo/dm-devel
> 85. mailto:tejaswinipoluri3 at gmail.com
> 86. mailto:bmarzins at redhat.com
> 87. mailto:dm-devel at redhat.com
> 88. https://www.redhat.com/mailman/listinfo/dm-devel
> 89. mailto:dm-devel at redhat.com
> 90. https://www.redhat.com/mailman/listinfo/dm-devel
> 91. mailto:bmarzins at redhat.com
> 92. mailto:tejaswinipoluri3 at gmail.com
> 93. mailto:bmarzins at redhat.com
> 94. mailto:tejaswinipoluri3 at gmail.com
> 95. mailto:bmarzins at redhat.com
> 96. mailto:dm-devel at redhat.com
> 97. https://www.redhat.com/mailman/listinfo/dm-devel
> 98. mailto:dm-devel at redhat.com
> 99. https://www.redhat.com/mailman/listinfo/dm-devel
> 100. mailto:tejaswinipoluri3 at gmail.com
> 101. mailto:bmarzins at redhat.com
> 102. mailto:dm-devel at redhat.com
> 103. https://www.redhat.com/mailman/listinfo/dm-devel
> 104. mailto:dm-devel at redhat.com
> 105. https://www.redhat.com/mailman/listinfo/dm-devel
> 106. mailto:tejaswinipoluri3 at gmail.com
> 107. mailto:bmarzins at redhat.com
> 108. mailto:tejaswinipoluri3 at gmail.com
> 109. mailto:bmarzins at redhat.com
> 110. mailto:dm-devel at redhat.com
> 111. https://www.redhat.com/mailman/listinfo/dm-devel
> 112. mailto:dm-devel at redhat.com
> 113. https://www.redhat.com/mailman/listinfo/dm-devel
> 114. mailto:tejaswinipoluri3 at gmail.com
> 115. mailto:bmarzins at redhat.com
> 116. mailto:dm-devel at redhat.com
> 117. https://www.redhat.com/mailman/listinfo/dm-devel
> 118. mailto:dm-devel at redhat.com
> 119. https://www.redhat.com/mailman/listinfo/dm-devel
More information about the dm-devel
mailing list