[rhos-list] More Swift questions - IBM HPSS VFS as backend storage

[David Hernandez]

Hello again.

As I mentioned in my last email, our VFS developer discovered that Swift 
does try to store binary extended attributes and HPSS VFS doesn't support 
them.  Our developer made a modification to our VFS client that converts 
all writes to VFS extended attributes to BASE64  It converts the BASE64 
xattrs back to their original format on reads.  I've run a few tests using 
the 'swift' command and it appears to be working.

I have a few more questions for the group.

What is the largest amount of extended attribute data Swift will try to 
store for an account/container/object?  The HPSS VFS default is set to 700 
bytes but can be increased if necessary.

I am currently using the swift command utility locally to upload and list 
containers and objects in my Swift (Essex Preview) install.  What changes 
do I need to do to my configuration to allow secure remote client access 
to my Swift storage.  I know I will need to create some self-signed 
certificates but don't know to get it to work with my current 
configuration.  I would like still be able to access locally using the 
swift command which uses port 8080.

How would I access my Swift storage through a web-server?  Is there 
anything readily available that I can use? 

Are there any command line utilities available that can be used by a 
remote client to access my Swift storage?  Something similar to FTP or 
even a FUSE type of mountable filesystem that can interface with Swift?

Regards. 

David Hernandez

Contractor / HPSS
IBM Global Business Services - US Federal
12301 Kurland Dr Suite 300
Houston, TX 77034-4812 
Mobile 713-444-5755



From:   Pete Zaitcev <zaitcev at redhat.com>
To:     David Hernandez/Houston/Contr/IBM at IBMUS
Cc:     Derek Higgins <derekh at redhat.com>, rhos-list at redhat.com, 
zaitcev at redhat.com
Date:   11/05/2012 08:35 PM
Subject:        Re: Unable to access Swift using Keystone on reboot



On Mon, 5 Nov 2012 17:22:48 -0600
David Hernandez <dhern at us.ibm.com> wrote:

> Does Swift try to write binary extended attributes?

It may if user asks it to, but I think not for itself normally.

> Does Swift use setfattr to write extended atrributes?

Swift uses the syscall fsetxattr(2), while setfattr(1) is a shell command.

With the defails aside, I thought about the problem of HPSS a little bit
in general.

I continue to think that your attempts of running Swift on top of
HPSS VFS are ill-advised. If I may, you really need a different thing:
a module for so-called "Swift LFS" (Local File System) which supports
HPSS natively. Unfortunately, LFS does not yet exist. The Red Hat Storage
team is working on an implementation of it, called "Gluster UFO" (Unified
Storage Objects). If someone wrote a plugin that worked as "HPSS UFO",
the issues of extended attributes, or any other kind of impedance
matching, would be contained within such plugin. It could even bypass
HPSS VFS altogether. I think it would be ideal for your needs --
as much as I understand them, of course.

So, if you want to deploy Swift today, please use tried and tested
storage back-ends. But by all means, feel free to help OpenStack community
to develop LFS/UFO for tomorrow.

Yours,
-- Pete


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/rhos-list/attachments/20121115/9b59abe6/attachment.htm>