<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<STYLE type=text/css> <!--@import url(scrollbar.css); --></STYLE>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<STYLE> body{FONT-SIZE:12pt; FONT-FAMILY:宋体,serif;} </STYLE>
<META name=GENERATOR content="MSHTML 9.00.8112.16450"><BASE
target=_blank></HEAD>
<BODY
style="LINE-HEIGHT: 1.3; BORDER-RIGHT-WIDTH: 0px; MARGIN: 12px; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; BORDER-LEFT-WIDTH: 0px"
marginheight="0" marginwidth="0">
<DIV><FONT color=#000000 size=3 face=宋体>i do not think so.i am
asking ceph-developers,but no answers</FONT></DIV>
<DIV>exact info of my storage.ceph'data dir is in /home</DIV>
<DIV>1.df -lh</DIV>
<DIV>/dev/mapper/vg_ceph1-lv_home<BR>
143G 23G 114G 17% /home</DIV>
<DIV>2.df -lh</DIV>
<DIV>/dev/mapper/vg_ceph2-lv_home<BR>
144G 23G 114G 17% /home</DIV>
<DIV> </DIV>
<DIV>23+23=64G, it is impossible 489G.</DIV>
<DIV> </DIV>
<DIV>'rados df -p cloud ' can give right stat.</DIV>
<DIV>referring its implement maybe be right. </DIV>
<DIV> </DIV>
<DIV align=left><FONT color=#c0c0c0 size=2 face=Verdana>2012-11-01</FONT></DIV>
<DIV align=left><FONT size=2 face=Verdana>
<HR style="WIDTH: 122px; HEIGHT: 2px" id=SignNameHR align=left SIZE=2>
</FONT></DIV>
<DIV align=left><FONT color=#c0c0c0 size=2 face=Verdana><SPAN
id=_FlashSignName>libvirt</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Verdana>
<HR>
</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>发件人：</STRONG>Wido den
Hollander</FONT></DIV>
<DIV><FONT size=2
face=Verdana><STRONG>发送时间：</STRONG>2012-10-29 22:40</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>主题：</STRONG>Re: [libvirt] libvirt can not
get right stats of a rbd pool</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>收件人：</STRONG>"Daniel P.
Berrange"<berrange@redhat.com></FONT></DIV>
<DIV><FONT size=2
face=Verdana><STRONG>抄送：</STRONG>"yue"<libvirt@163.com>,"libvirt"<libvir-list@redhat.com></FONT></DIV>
<DIV><FONT size=2 face=Verdana></FONT> </DIV>
<DIV><FONT size=2 face=Verdana>
<DIV> </DIV>
<DIV> </DIV>
<DIV>On 10/29/2012 03:33 PM, Daniel P. Berrange wrote: </DIV>
<DIV>> On Fri, Oct 26, 2012 at 11:04:05AM +0800, yue wrote: </DIV>
<DIV>>> Allocation exceed Capacity ,but Available is not 0. </DIV>
<DIV>>> </DIV>
<DIV>>> #virsh pool-info 2361a6d4-0edc-3534-87ae-e7ee09199921 </DIV>
<DIV>>> Name: 2361a6d4-0edc-3534-87ae-e7ee09199921 </DIV>
<DIV>>> UUID: 2361a6d4-0edc-3534-87ae-e7ee09199921 </DIV>
<DIV>>> State: running </DIV>
<DIV>>> Persistent: yes </DIV>
<DIV>>> Autostart: no </DIV>
<DIV>>> Capacity: 285.57 GiB </DIV>
<DIV>>> Allocation: 489.89 GiB </DIV>
<DIV>>> Available: 230.59 GiB </DIV>
<DIV>> </DIV>
<DIV>> Hmm, these values do look a little bit suspect, but I don't know </DIV>
<DIV>> enough about RBD to suggest what might be going wrong. I'm copying </DIV>
<DIV>> Wido who wrote this code originally & thus might have an idea. </DIV>
<DIV>> </DIV>
<DIV> </DIV>
<DIV>I think I know where this is coming from, a little background about RBD. </DIV>
<DIV> </DIV>
<DIV>RBD is a disk device striped over 4MB RADOS objects inside a Ceph </DIV>
<DIV>cluster. RBD devices are sparse, which means that (RADOS) objects get </DIV>
<DIV>created whenever a write comes. </DIV>
<DIV> </DIV>
<DIV>When a read comes for a non-existing object 4MB of nothing is returned. </DIV>
<DIV> </DIV>
<DIV>However, when you do: $ rbd info disk1 </DIV>
<DIV> </DIV>
<DIV>You will get that the object COULD be 100GB, but that doesn't mean it </DIV>
<DIV>actually occupies 100GB of disk space. </DIV>
<DIV> </DIV>
<DIV>The problem is that you can't (at this point) find out how much space a </DIV>
<DIV>RBD device actually occupies. Yes, it can be done, but that should not </DIV>
<DIV>be done in the libvirt driver and it is pretty heavy for the Ceph cluster. </DIV>
<DIV> </DIV>
<DIV>Also, a RBD device of 100GB could take up 300GB of space when your </DIV>
<DIV>replication is set to 3x. </DIV>
<DIV> </DIV>
<DIV>What you are seeing there is that you over provisioned your Ceph cluster </DIV>
<DIV>by creating images which exceed 285GB in total. </DIV>
<DIV> </DIV>
<DIV>Wido </DIV>
<DIV> </DIV>
<DIV>> Daniel </DIV>
<DIV>> </DIV></FONT></DIV></BODY></HTML>