<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:굴림;
panose-1:2 11 6 0 0 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@굴림";
panose-1:2 11 6 0 0 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-autospace:none;
word-break:break-hangul;
font-size:10.0pt;
font-family:"맑은 고딕";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"글자만 Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.Char
{mso-style-name:"글자만 Char";
mso-style-priority:99;
mso-style-link:글자만;
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"맑은 고딕";}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:3.0cm 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="KO" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoPlainText"><a name="_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">Hi Dave,<o:p></o:p></span></a></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">When errors started to come out, the system got slower (perf degraded) and lots of error messages showed up repeatedly.
Specifically, when the large amount of slab memory was reclaimed such as 9GB to 6GB, the about 30 error messages came out.<o:p></o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">‘send_repeat_remove’ messages were printed about 5 times intermittently as well. But the system didn’t get stuck.<o:p></o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">We are running JMeter tool to simulate the CDN workloads and there are 2 million files(3MB size per file) in my storage
that are read by 4 host servers.<o:p></o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">160Gbps bandwidth were reached using 16 client servers with 10Gb and 4 host servers with 40Gb that runs GFS. Hope
this helps you understand my usage. <o:p></o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:12.0pt;font-family:"Calibri",sans-serif">eric</span></span><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></span></p>
<p class="MsoPlainText"><span style="mso-bookmark:_MailEndCompose"><span lang="EN-US" style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></span></p>
<span style="mso-bookmark:_MailEndCompose"></span>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">-----Original Message-----<br>
From: David Teigland [mailto:teigland@redhat.com] <br>
Sent: Thursday, November 23, 2017 12:04 AM<br>
To: </span><span style="font-family:굴림">장홍석</span><span lang="EN-US" style="font-family:"Calibri",sans-serif">/SW-Defined Storage Lab <echang@sk.com><br>
Cc: linux-cluster@redhat.com; swhiteho@redhat.com; mferrell@redhat.com; </span><span style="font-family:굴림">성백재</span><span lang="EN-US" style="font-family:"Calibri",sans-serif">/SW-Defined Storage Lab <bj.sung@sk.com>;
</span><span style="font-family:굴림">윤진혁</span><span lang="EN-US" style="font-family:"Calibri",sans-serif">/SW-Defined Storage Lab <jhyoon01@sk.com>;
</span><span style="font-family:굴림">민항준</span><span lang="EN-US" style="font-family:"Calibri",sans-serif">/SW-Defined Storage Lab <hangjun.min@sk.com><br>
Subject: Re: [Linux-cluster] GFS2 DLM problem on NVMes<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">On Wed, Nov 22, 2017 at 04:32:13AM +0000, Eric H. Chang wrote:<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">> We ve tested with different toss_secs as advised. When we
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">> configured it as 1000, we saw the send_repeat_remove log after
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">> 1000sec. We can test with other values on toss_secs , but we think
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">> it would have the same problem potentially when freeing up the slab
<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">> after the configured sec.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif">Do you see many of these messages? Do gfs operations become stuck after they appear?<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="EN-US" style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
</div>
</body>
</html>