<div dir="ltr">We are seeing a substantial difference in performance when we perform a read/write to /dev/mapper/... vs. the specific device (/dev/sdXX)<div>What can we do to further isolate the issue?</div><div><br></div>
<div>We are using CentOS 6.4, with all updates, 2 CPUs, 4 FC ports:</div><div>Here's a table comparing the results:</div><div><table class="" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse">
<tbody><tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black"># of LUNs</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black"># of Paths per device</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Native Multipath Device</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">IO Pattern</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">IOPS</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Latency Micro</span></p>
</td>
<td nowrap valign="bottom" style="border-style:solid solid solid none;border-top-color:windowtext;border-right-color:windowtext;border-bottom-color:windowtext;border-top-width:1pt;border-right-width:1pt;border-bottom-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">BW KBps</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">No</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Read</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">605,661.4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">3,381</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">2,420,736</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">No</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Write</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">477,515.1</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">4,288</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,908,736</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">8</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">No</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Read</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">663,339.4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">6,174</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">2,650,112</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">8</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">No</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Write</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">536,936.9</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">7,628</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">2,146,304</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Yes</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Read</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">456,108.9</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,122</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,824,256</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Yes</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Write</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">371,665.8</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,377</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,486,336</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">8</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Yes</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Read</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">519,450.2</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,971</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">2,077,696</span></p>
</td>
</tr>
<tr style="height:15pt">
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">8</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">16</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">Yes</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class=""><span style="color:black">100% Write</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">448,840.4</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">2,281</span></p>
</td>
<td nowrap valign="bottom" style="border-style:none solid solid none;border-bottom-color:windowtext;border-bottom-width:1pt;border-right-color:windowtext;border-right-width:1pt;padding:0cm 5.4pt;height:15pt">
<p class="" align="right" style="text-align:right"><span style="color:black">1,795,072</span></p>
</td>
</tr>
</tbody></table></div></div>