Global Filesystem Limits

[Jason Gray]

Markus,

I'm not currently using GFS but I do have large storage requirements.  
I'm using LVM to work around the 2TB limitations by creating multiple 
mount points (Well..okay you don't have to use LVM but I find it very 
useful when I have to expand the volume).  My storage devices are on an 
iSCSI network and presented as SCSI devices to the server.  Using an 
iSCSI portal I've carved up the RAID enclosures into multiple LUNS 
(virtualized Volumes) and then each is exposed as /dev/sda, /dev/sdb, 
etc.  All devices are varying sizes depending on the space 
requirements.  LVM has the same 2TB limitation so I just look at the 
folder structure and give space to specific folders: (total device = 14TB)

Parent Dir----> (100GB) /dev/lvm/lvm01(/mnt/mountpoint01)
   ----> folder 1 (10GB) /dev/lvm2/lvm201 (/mnt/mountpoint02)
   ----> folder 2 (space requirement falls under parent)
   ----> folder 3 (500GB) /dev/lvm3/lvm301 (/mnt/mountpoint03)
   ----> folderN .........

I can use the resize command in conjunction with LVM to add space to 
deivces as needed (if needed).  The whole idea is to create multiple 
mount points of varying sizes.  I suppose some people would not want to 
manage 10 or 20 volumes on a single server but I find that if one volume 
becomes problematic it is better to unmount one volume rather than the 
whole array.
Since I'm using iSCSI (not the native iSCSI driver but the adaptec 
asa72xx) the volumes are loaded after the driver is loaded.  There are 
ways (I believe) to have the iSCSI driver start up before the init 
scripts run during boot up.  To get things up and running I simply added 
the vgchange and mount commands to the rc.local file.  It's not very 
elegant but it works.

If you are using attached storage then the iSCSI issues won't be an 
issue.  Hope this was useful.

Cheers,

Jason

Markus Miller wrote:

>Hi Ben,
>
>you right, Kernel 2.6 sopports much larger devices, but it will take more than a couple of weeks unitl oracle certifies its database on RH 4. I also do not know if GFS will be awailable right away for RH 4.
>
>Anyway, if there would be only a limitation of 2 TB per device and if I could use serveral GFS in our cluster it would be ok. But if I could use only 2 TB as a total for the GFS Server, this would be fatal.
>
>So, does anybody have any experience with GFS and large storage?
>
>Thanks and regards,
>Markus
>
>-----Mensaje original-----
>De: Benjamin J. Weiss [mailto:benjamin at birdvet.org]
>Enviado el: Wednesday, December 29, 2004 6:01 PM
>Para: General Red Hat Linux discussion list
>Asunto: Re: Global Filesystem Limits
>
>
>Markus Miller wrote:
>
>  
>
>>Hi,
>>
>>we are going to buy a new database cluster. The two servers will be connected to a Dell CX700 (EMC Clarion) with about 4 TB of available storage. We want to uses Red Hat Global File System to avoid using RAW device. 
>>
>>Reviewing the GFS Admin Guide on page 12 it says:
>>---snip---
>>2 TB maximum, for total of all storage connected to a GFS cluster. Linux 2.4 kernels do not support devices larger than 2 TB; therefore the total size of storage available to GFS cannot exceed 2 TB.
>>---snip---
>>
>>I did some research in Internet and definitely the Kernel 2.4 does not sopport devices larger than 2 TB. This makes me think that each LUN presented to the server must be smaller than 2 TB and I would suppose that I could create various GFS with sizes smaller than 2 TB. Is that correct, or does the capacity of the storage itself has to be smaller than 2 TB to be used by GFS?
>>
>> 
>>
>>    
>>
>Why not wait for a couple of weeks?  RHEL 4 is in beta, and it uses the 
>2.6 kernel.  I don't know what it's max device size is, but I know that 
>it's bigger than 2TB.
>
>Ben
>
>  
>

-- 
Jason Gray
Bardel Entertainment Inc.
604-669-5589
jgray at bardel.ca