<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:880358522;
mso-list-type:hybrid;
mso-list-template-ids:-1289338858 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1
{mso-list-id:1317412700;
mso-list-type:hybrid;
mso-list-template-ids:1712773602 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Where to begin… Basically, I’m hosed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I performed an update to the latest versions of the Openstack distro and rebooted my hosts. Then problems started…<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo1"><![if !supportLists]><span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>Can’t launch new instances—the in progress star comes up and stays there almost forever.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo1"><![if !supportLists]><span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>Several of my instances went to an error state. When I tried to reset their state to “active” they went into shutdown/shutoff state. These are the instances on my controller/compute node. I typed “virsh list” and the list was
empty.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo1"><![if !supportLists]><span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>I tried to bring up a VNC Console on one of the running instances and after several minutes received this error in the browser:<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">-------------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">CommunicationError at /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><u><span style="color:red">Error communicating with http://10.10.12.245:9292 timed out</span></u><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Request Method: GET<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Request URL: http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/?tab=instance_details__vnc<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Django Version: 1.4.2<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Exception Type: CommunicationError<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Exception Value: <o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Error communicating with http://10.10.12.245:9292 timed out<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Exception Location: /usr/lib/python2.6/site-packages/glanceclient/common/http.py in _http_request, line 145<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Python Executable: /usr/bin/python<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Python Version: 2.6.6<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Python Path: <o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python26.zip',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/plat-linux2',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/lib-tk',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/lib-old',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/lib-dynload',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/site-packages',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib64/python2.6/site-packages/gtk-2.0',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib/python2.6/site-packages',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info',<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">'/usr/share/openstack-dashboard/openstack_dashboard']<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Server time: Thu, 21 Mar 2013 15:26:38 +0000<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">------------------------------------------------------------------------------------------------------------------------------------<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>I tried deleting iptables and rebooting but did not solve anything.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>I’m not seeing anything out of the ordinary in the nova logs.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas on where I should start looking?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for any help.<o:p></o:p></p>
<p class="MsoNormal">Rick<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><u><span style="font-size:14.0pt;color:#990000">Richard Minton<o:p></o:p></span></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt">LMICC Systems Administrator<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">4000 Geerdes Blvd, 13D31<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">King of Prussia, PA 19406<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Phone: 610-354-5482<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>