<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<tt><span style="color: rgb(51, 51, 51); font-size: 12.7273px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: 20px; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
word-spacing: 0px; background-color: rgb(251, 251, 251);
display: inline ! important; float: none;">This is great! Thank
you. The only issue is that we use github as a public repo and
we do not use github for our normal development workflow. Do you
mind resubmitting the change using the development workflow as
described here:</span></tt><tt><a
href="https://github.com/gluster/gluster-swift/blob/master/doc/markdown/dev_guide.md"
style="color: rgb(65, 131, 196); text-decoration: none;
font-size: 12.7273px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: 20px;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px; background-color:
rgb(251, 251, 251);">https://github.com/gluster/gluster-swift/blob/master/doc/markdown/dev_guide.md</a></tt><tt>.
More information can be found at <a class="moz-txt-link-freetext" href="https://launchpad.net/gluster-swift">https://launchpad.net/gluster-swift</a></tt><br>
<br>
- Luis<br>
<br>
<div class="moz-cite-prefix">On 09/19/2013 05:23 PM, Paul Robert
Marino wrote:<br>
</div>
<blockquote
cite="mid:CAPJdpdCTDOYgQ4UspjoKdEJyBQ9j8=+W+5eE1sMwjN=E0yi-+g@mail.gmail.com"
type="cite">
<pre wrap="">Thank you every one for your help especially Louis
I tested the RPM and it went well every thing is working now.
I did have to use the tenant ID's as the volume names. I may submit an
update to the documentation to clarify this for people
So in other words the volume names have to match the output of
"
keystone tenant-list|grep -v + | \
grep -v -P '^\|\s+id\s+\|\s+name\s+\|\s+enabled\s+\|$' | \
grep -v -P '^\w+:' | awk '{print $2}'
"
I've created an updated copy of gluster-swift-gen-builders that grabs
the value of mount_ip from /etc/swift/fs.conf and posted it on github.
you should see a pull request on the site for the submission. of the
change
Thank you every one for your help
On Tue, Sep 17, 2013 at 4:38 PM, Paul Robert Marino <a class="moz-txt-link-rfc2396E" href="mailto:prmarino1@gmail.com"><prmarino1@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Luis
Thanks for the timely response.
On Tue, Sep 17, 2013 at 1:52 PM, Luis Pabon <a class="moz-txt-link-rfc2396E" href="mailto:lpabon@redhat.com"><lpabon@redhat.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
On 09/17/2013 11:13 AM, Paul Robert Marino wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Luis
well thats intresting because it was my impression that Gluster UFO
3.4 was based on the Grizzly version of Swift.
</pre>
</blockquote>
<pre wrap="">
[LP] Sorry, the gluster-ufo RPM is Essex only.
</pre>
</blockquote>
<pre wrap="">
[PRM] The source of my confusion was here
<a class="moz-txt-link-freetext" href="http://www.gluster.org/community/documentation/index.php/Features34">http://www.gluster.org/community/documentation/index.php/Features34</a>
and here <a class="moz-txt-link-freetext" href="http://www.gluster.org/2013/06/glusterfs-3-4-and-swift-where-are-all-the-pieces/">http://www.gluster.org/2013/06/glusterfs-3-4-and-swift-where-are-all-the-pieces/</a>
These pages on the gluster site should probably be updated to reflect
the changes.
</pre>
<blockquote type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">Also I was previously unaware of this new rpm which doesnt seem to be
in a repo any where.
</pre>
</blockquote>
<pre wrap="">
[LP] gluster-swift project RPMs have been submitted to Fedora and are
currently being reviewed.
</pre>
</blockquote>
<pre wrap="">
[PRM] Cool if they are in the EPEL testing repo Ill look for them
there because I would rather pull the properly EPEL signed RPMs if
they exist just to make node deployments easier. If not Ill ask some
of my friends offline if they can help expedite it.
</pre>
<blockquote type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">also there is a line in this new howto that is extreamly unclear
"
/usr/bin/gluster-swift-gen-builders test
"
in place of "test" what should go there is it the tenant ID string,
the tenant name, or just a generic volume you can name whatever you
want?
in other words how should the Gluster volumes be named?
</pre>
</blockquote>
<pre wrap="">
[LP] We will clarify that in the quick start guide. Thank you for pointing
it out. While we update the community site, please refer to the
documentation available here <a class="moz-txt-link-freetext" href="http://goo.gl/bQFI8o">http://goo.gl/bQFI8o</a> for a usage guide.
As for the tool, the format is:
gluster-swift-gen-buildes [VOLUME] [VOLUME...]
Where VOLUME is the name of the GlusterFS volume to use for object storage.
For example
if the following two GlusterFS volumes, volume1 and volume2, need to be
accessed over Swift,
then you can type the following:
# gluster-swift-gen-builders volume1 volume2
</pre>
</blockquote>
<pre wrap="">
[PRM] That part I understood however it doesn't answer the question exactly.
Correct me if I'm wrong but looking over the code briefly it looks as
though the volume name needs to be the same as the tenant ID number
like it did with Gluster UFO 3.3.
so for example
if I do a " keystone tenant-list" and a see tenant1 with an id of
"f6da0a8151ff43b7be10d961a20c94d6" then I would need to create a
volume named f6da0a8151ff43b7be10d961a20c94d6
If I can name the volumes whatever I want or give them the same name
as the tenant that would be great because it makes it easier for other
SA's who are not directly working with OpenStack but may need to mount
the volumes to comprehend, but its not urgently needed.
One thing I was glad to see is that with Gluster UFO 3.3 I had to add
mount points to /etc/fstab for each volume and manually create the
directories for the mount points this looks to have been corrected in
Gluster-Swift.
</pre>
<blockquote type="cite">
<pre wrap="">
For more information please read: <a class="moz-txt-link-freetext" href="http://goo.gl/gd8LkW">http://goo.gl/gd8LkW</a>
Let us know if you have any more questions or comments.
</pre>
</blockquote>
<pre wrap="">
[PRM] I may fork the Github repo and add some changes that may be
beneficial so they can be reviewed and possibly merged.
for example it would be nice if the gluster-swift-gen-buildes script
used the value of the mount_ip field in /etc/swift/fs.conf instead of
127.0.0.1 if its defined.
also I might make a more robust version that allows create, add,
remove, and list options.
Ill do testing tomorrow and let everyone know how it goes.
</pre>
<blockquote type="cite">
<pre wrap="">
- Luis
</pre>
<blockquote type="cite">
<pre wrap="">
On Tue, Sep 17, 2013 at 10:10 AM, Luis Pabon <a class="moz-txt-link-rfc2396E" href="mailto:lpabon@redhat.com"><lpabon@redhat.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
First thing I can see is that you have Essex based gluster-ufo-* which
has
been replaced by the gluster-swift project. We are currently in progress
of
replacing the gluster-ufo-* with RPMs from the gluster-swift project in
Fedora.
Please checkout the following quickstart guide which show how to download
the Grizzly version of gluster-swift:
<a class="moz-txt-link-freetext" href="https://github.com/gluster/gluster-swift/blob/master/doc/markdown/quick_start_guide.md">https://github.com/gluster/gluster-swift/blob/master/doc/markdown/quick_start_guide.md</a>
.
For more information please visit: <a class="moz-txt-link-freetext" href="https://launchpad.net/gluster-swift">https://launchpad.net/gluster-swift</a>
- Luis
On 09/16/2013 05:02 PM, Paul Robert Marino wrote:
Sorry for the delay on reporting the details. I got temporarily pulled
off the project and dedicated to a different project which was
considered higher priority by my employer. I'm just getting back to
doing my normal work today.
first here are the rpms I have installed
"
rpm -qa |grep -P -i '(gluster|swift)'
glusterfs-libs-3.4.0-8.el6.x86_64
glusterfs-server-3.4.0-8.el6.x86_64
openstack-swift-plugin-swift3-1.0.0-0.20120711git.el6.noarch
openstack-swift-proxy-1.8.0-2.el6.noarch
glusterfs-3.4.0-8.el6.x86_64
glusterfs-cli-3.4.0-8.el6.x86_64
glusterfs-geo-replication-3.4.0-8.el6.x86_64
glusterfs-api-3.4.0-8.el6.x86_64
openstack-swift-1.8.0-2.el6.noarch
openstack-swift-container-1.8.0-2.el6.noarch
openstack-swift-object-1.8.0-2.el6.noarch
glusterfs-fuse-3.4.0-8.el6.x86_64
glusterfs-rdma-3.4.0-8.el6.x86_64
openstack-swift-account-1.8.0-2.el6.noarch
glusterfs-ufo-3.4.0-8.el6.noarch
glusterfs-vim-3.2.7-1.el6.x86_64
python-swiftclient-1.4.0-1.el6.noarch
here are some key config files note I've changed the passwords I'm
using and hostnames
"
cat /etc/swift/account-server.conf
[DEFAULT]
mount_check = true
bind_port = 6012
user = root
log_facility = LOG_LOCAL2
devices = /swift/tenants/
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:gluster_swift_ufo#account
log_name = account-server
log_level = DEBUG
log_requests = true
[account-replicator]
vm_test_mode = yes
[account-auditor]
[account-reaper]
"
"
cat /etc/swift/container-server.conf
[DEFAULT]
devices = /swift/tenants/
mount_check = true
bind_port = 6011
user = root
log_facility = LOG_LOCAL2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:gluster_swift_ufo#container
[container-replicator]
vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync]
"
"
cat /etc/swift/object-server.conf
[DEFAULT]
mount_check = true
bind_port = 6010
user = root
log_facility = LOG_LOCAL2
devices = /swift/tenants/
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:gluster_swift_ufo#object
[object-replicator]
vm_test_mode = yes
[object-updater]
[object-auditor]
"
"
cat /etc/swift/proxy-server.conf
[DEFAULT]
bind_port = 8080
user = root
log_facility = LOG_LOCAL1
log_name = swift
log_level = DEBUG
log_headers = True
[pipeline:main]
pipeline = healthcheck cache authtoken keystone proxy-server
[app:proxy-server]
use = egg:gluster_swift_ufo#proxy
allow_account_management = true
account_autocreate = true
[filter:tempauth]
use = egg:swift#tempauth
# Here you need to add users explicitly. See the OpenStack Swift
Deployment
# Guide for more information. The user and user64 directives take the
# following form:
# user_<account>_<username> = <key> [group] [group] [...]
[storage_url]
# user64_<account_b64>_<username_b64> = <key> [group] [group]
[...] [storage_url]
# Where you use user64 for accounts and/or usernames that include
underscores.
#
# NOTE (and WARNING): The account name must match the device name
specified
# when generating the account, container, and object build rings.
#
# E.g.
# user_ufo0_admin = abc123 .admin
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
[filter:keystone]
use = egg:swift#keystoneauth
#paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = Member,admin,swiftoperator
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
auth_host = keystone01.vip.my.net
auth_port = 35357
auth_protocol = http
admin_user = swift
admin_password = PASSWORD
admin_tenant_name = service
signing_dir = /var/cache/swift
service_port = 5000
service_host = keystone01.vip.my.net
[filter:swiftauth]
use = egg:keystone#swiftauth
auth_host = keystone01.vip.my.net
auth_port = 35357
auth_protocol = http
keystone_url = <a class="moz-txt-link-freetext" href="https://keystone01.vip.my.net:5000/v2.0">https://keystone01.vip.my.net:5000/v2.0</a>
admin_user = swift
admin_password = PASSWORD
admin_tenant_name = service
signing_dir = /var/cache/swift
keystone_swift_operator_roles = Member,admin,swiftoperator
keystone_tenant_user_admin = true
[filter:catch_errors]
use = egg:swift#catch_errors
"
"
cat /etc/swift/swift.conf
[DEFAULT]
[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = gluster
#3d60c9458bb77abe
# The swift-constraints section sets the basic constraints on data
# saved in the swift cluster.
[swift-constraints]
# max_file_size is the largest "normal" object that can be saved in
# the cluster. This is also the limit on the size of each segment of
# a "large" object when using the large object manifest support.
# This value is set in bytes. Setting it to lower than 1MiB will cause
# some tests to fail. It is STRONGLY recommended to leave this value at
# the default (5 * 2**30 + 2).
# FIXME: Really? Gluster can handle a 2^64 sized file? And can the
fronting
# web service handle such a size? I think with UFO, we need to keep with
the
# default size from Swift and encourage users to research what size their
web
# services infrastructure can handle.
max_file_size = 18446744073709551616
# max_meta_name_length is the max number of bytes in the utf8 encoding
# of the name portion of a metadata header.
#max_meta_name_length = 128
# max_meta_value_length is the max number of bytes in the utf8 encoding
# of a metadata value
#max_meta_value_length = 256
# max_meta_count is the max number of metadata keys that can be stored
# on a single account, container, or object
#max_meta_count = 90
# max_meta_overall_size is the max number of bytes in the utf8 encoding
# of the metadata (keys + values)
#max_meta_overall_size = 4096
# max_object_name_length is the max number of bytes in the utf8 encoding
of
an
# object name: Gluster FS can handle much longer file names, but the
length
# between the slashes of the URL is handled below. Remember that most web
# clients can't handle anything greater than 2048, and those that do are
# rather clumsy.
max_object_name_length = 2048
# max_object_name_component_length (GlusterFS) is the max number of bytes
in
# the utf8 encoding of an object name component (the part between the
# slashes); this is a limit imposed by the underlying file system (for
XFS
it
# is 255 bytes).
max_object_name_component_length = 255
# container_listing_limit is the default (and max) number of items
# returned for a container listing request
#container_listing_limit = 10000
# account_listing_limit is the default (and max) number of items returned
# for an account listing request
#account_listing_limit = 10000
# max_account_name_length is the max number of bytes in the utf8 encoding
of
# an account name: Gluster FS Filename limit (XFS limit?), must be the
same
# size as max_object_name_component_length above.
max_account_name_length = 255
# max_container_name_length is the max number of bytes in the utf8
encoding
# of a container name: Gluster FS Filename limit (XFS limit?), must be
the
same
# size as max_object_name_component_length above.
max_container_name_length = 255
"
The volumes
"
gluster volume list
cindervol
unified-storage-vol
a07d2f39117c4e5abdeba722cf245828
bd74a005f08541b9989e392a689be2fc
f6da0a8151ff43b7be10d961a20c94d6
"
if I run the command
"
gluster-swift-gen-builders unified-storage-vol
a07d2f39117c4e5abdeba722cf245828 bd74a005f08541b9989e392a689be2fc
f6da0a8151ff43b7be10d961a20c94d6
"
because of a change in the script in this version as compaired to the
version I got from
<a class="moz-txt-link-freetext" href="http://repos.fedorapeople.org/repos/kkeithle/glusterfs/">http://repos.fedorapeople.org/repos/kkeithle/glusterfs/</a> the
gluster-swift-gen-builders script only takes the first option and
ignores the rest.
other than the location of the config files none of the changes Ive
made are functionally different than the ones mentioned in
<a class="moz-txt-link-freetext" href="http://www.gluster.org/2012/09/howto-using-ufo-swift-a-quick-and-dirty-setup-guide/">http://www.gluster.org/2012/09/howto-using-ufo-swift-a-quick-and-dirty-setup-guide/</a>
The result is that the first volume named "unified-storage-vol" winds
up being used for every thing regardless of the tenant, and users and
see and manage each others objects regardless of what tenant they are
members of.
through the swift command or via horizon.
In a way this is a good thing for me it simplifies thing significantly
and would be fine if it just created a directory for each tenant and
only allow the user to access the individual directories, not the
whole gluster volume.
by the way seeing every thing includes the service tenants data so
unprivileged users can delete glance images without being a member of
the service group.
On Mon, Sep 2, 2013 at 9:58 PM, Paul Robert Marino <a class="moz-txt-link-rfc2396E" href="mailto:prmarino1@gmail.com"><prmarino1@gmail.com></a>
wrote:
Well I'll give you the full details in the morning but simply I used the
stock cluster ring builder script that came with the 3.4 rpms and the old
version from 3.3 took the list of volumes and would add all of them the
version with 3.4 only takes the first one.
Well I ran the script expecting the same behavior but instead they all
used
the first volume in the list.
Now I knew from the docs I read that the per tenant directories in a
single
volume were one possible plan for 3.4 to deal with the scalding issue
with a
large number of tenants, so when I saw the difference in the script and
that
it worked I just assumed that this was done and I missed something.
-- Sent from my HP Pre3
________________________________
On Sep 2, 2013 20:55, Ramana Raja <a class="moz-txt-link-rfc2396E" href="mailto:rraja@redhat.com"><rraja@redhat.com></a> wrote:
Hi Paul,
Currently, gluster-swift doesn't support the feature of multiple
accounts/tenants accessing the same volume. Each tenant still needs his
own
gluster volume. So I'm wondering how you were able to observe the
reported
behaviour.
How did you prepare the ringfiles for the different tenants, which use
the
same gluster volume? Did you change the configuration of the servers?
Also,
how did you access the files that you mention? It'd be helpful if you
could
share the commands you used to perform these actions.
Thanks,
Ram
----- Original Message -----
From: "Vijay Bellur" <a class="moz-txt-link-rfc2396E" href="mailto:vbellur@redhat.com"><vbellur@redhat.com></a>
To: "Paul Robert Marino" <a class="moz-txt-link-rfc2396E" href="mailto:prmarino1@gmail.com"><prmarino1@gmail.com></a>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:rhos-list@redhat.com">rhos-list@redhat.com</a>, "Luis Pabon" <a class="moz-txt-link-rfc2396E" href="mailto:lpabon@redhat.com"><lpabon@redhat.com></a>, "Ramana Raja"
<a class="moz-txt-link-rfc2396E" href="mailto:rraja@redhat.com"><rraja@redhat.com></a>, "Chetan Risbud" <a class="moz-txt-link-rfc2396E" href="mailto:crisbud@redhat.com"><crisbud@redhat.com></a>
Sent: Monday, September 2, 2013 4:17:51 PM
Subject: Re: [rhos-list] Gluster UFO 3.4 swift Multi tenant question
On 09/02/2013 01:39 AM, Paul Robert Marino wrote:
I have Gluster UFO installed as a back end for swift from here
<a class="moz-txt-link-freetext" href="http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.0/RHEL/epel-6/">http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.0/RHEL/epel-6/</a>
with RDO 3
Its working well except for one thing. All of the tenants are seeing
one Gluster volume which is some what nice, especially when compared
to the old 3.3 behavior of creating one volume per tenant named after
the tenant ID number.
The problem is I expected to see is sub directory created under the
volume root for each tenant but instead what in seeing is that all of
the tenants can see the root of the Gluster volume. The result is that
all of the tenants can access each others files and even delete them.
even scarier is that the tennants can see and delete each others
glance images and snapshots.
Can any one suggest options to look at or documents to read to try to
figure out how to modify the behavior?
Adding gluster swift developers who might be able to help.
-Vijay
</pre>
</blockquote>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
</blockquote>
</blockquote>
<br>
</body>
</html>