From mnagy at redhat.com Sat Nov 1 01:55:56 2008 From: mnagy at redhat.com (Martin Nagy) Date: Sat, 1 Nov 2008 02:55:56 +0100 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <490B752B.9010305@redhat.com> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> <490B752B.9010305@redhat.com> Message-ID: <20081101025556.5dd493ee@notas> Some notes from me, sorry for the excessive length. Dmitri Pal wrote: > Simo Sorce wrote: > > On Fri, 2008-10-31 at 15:56 -0400, Dmitri Pal wrote: > > > > > >> *Length of line: * > >> > >> Not more than 100-110 characters. Just keep it reasonable. Longer > >> lines are harder to deal with due to different sized of monitors > >> and different editors used. > >> > > > > I am for strict adherence to not more than 80 chars per line. > > While I myself used to use up to 120 chars I found out that 80 > > lines is more usable in many situations, including debugging python > > code/C code on a console, letting people use their tools of choice, > > and yes, fitting 3 terminals in a row :-) > > > > > I think 80 is to short with the modern monitors and screens. > We should not limit us but try no to be overwhelmingly long I agree with Simo on this. Besides the reasons he stated, also think about your eyes. Take your favorite book and count the number of letters per line. It will be cca 66. That's because if it was more, your eyes would be tired or hurt after long hours of reading (more eyeball movement). Also, this convention is held by almost all projects I can think of and also our preferred style for python code. I don't have problems fitting my code into 80 characters even with 8 spaces of indentation, with 4 it will be very easy. > >> *Module structure: * > >> > >> The module should be structured in the following order: > >> * Copyright boilerplate > >> * Includes Let's also say that includes should be grouped properly. Standard headers and local headers should definitely be separated by a blank line. Other logical grouping should be reasonably done if needed. Files inside the groups should be sorted alphabetically, unless a specific order is required - this however is very rare, and must not happen with our own headers. Also, one shouldn't depend on the fact that one header file includes other one, unless it is really obvious and/or desirable, like in cases when one header file practically "enhances" the other one, for example with more error codes, etc. > >> * Local defines ? used only inside module. Global/shared defines > >> should be in the header > >> * Local module typedefs - Global/shared typedefs should be in the > >> header > > > > I would like to discourage use of typdefs unless really necessary, > > like when a very basic type is defined. I find the code is much more > > comprehensible if a struct is immediately recognizable as such as > > well as pointers. > > > > > I think this is should be formulated as preference with suggestion > not to use typedefs. I like using typedefs if the convention is to always end them with _t. However, this way you sometime need both struct and typedef (self reference, as in case of linear lists for example). I also like structs alone and this approach is much more clearer, because there's no need to mix typedefs and structs. And it's always apparent at first glance that we're dealing with a struct as Simo noted, so I'm vouching for structs. Also, let's not forget that hiding a pointer behind a typedef is a mortal sin. > >> * Module variable declarations ? variables that are global for the > >> module. > > > > Global variables should be discouraged as well, they make for very > > poor code. Should be used only if no other way can be found. They > > tend to be not thread/async safe > > > > > Agree. But if there are some they should be covered. Yup, global variables should be used only if necessary and when used in a threaded code should be lockable. But let's not be religiously against their use as many are in case of goto.. Anyway, they should be mentioned and I think that everyone knows he should use them only when really needed. > >> * Declarations of external functions > >> > > > > Should not be needed, usually headers should contain foreign > > functions definitions. > > > > > Agree. The word usually is crucial here. Sometimes you have > collisions on the headers in this case you need to include things > explicitly. But this is more an indication of the bad header design. I agree with Dmitri, sometimes it just happens. I wanted to tease Simo by mentioning ipa_pwd_extop.c, line 125, but then I realized I was the one who added the declaration :-) Oh well, but still better than to use the function without the declaration at all.. > >> * Declarations of static functions > >> > > > > I find it much easier to declare functions only if they need to be > > referenced before the body of the function is defined. Otherwise > > just use the ordering in the file to avoid declarations. > > > > > I agree but this should be a preference not a rule. > > >> * Implementation of the exposed functions from the module > >> * Implementation of the static functions > >> > > > > Given the above comment, static functions usually come first. > > > > > I kind of disagree. This should be a preference. > > > >> It is not required but recommended to group and position functions > >> in the module so that it is easier to understand. > >> > > > > This is very controversial, for me easier to understand is when the > > innermost function is first in the file and the outermost is last, > > but for some that seem an unnatural bottom to top ordering. > > > > > No it is not. I think this allows you to follow your natural flaw of > logic and me mine. > I prefer implementing the functions that are main to the module first > and stub out the functions that I need. > So naturally I read the code the same way. I do not care about the > details and low level functions that act as helpers. > I want to see and concentrate on the core functionality and not dig > it into the module to find it scattered somewhere in the middle or in > the bottom. > > I guess this is a personal preference so I would suggest leaving it > as it was spelled. On this subject, I strongly agree with Dmitri. I always found it unnatural that main function is on the bottom, it just doesn't make sense. I suspect people do this only because they are lazy to use forward declarations. This is not a good way. Simo please reconsider, I'm sure you will agree with me that code is much more read than written. By this logic, I think that if we can get a more easily readable code by making just a little more effort, it is worth it. And I definitely think this is more readable as it follows logic. You basically only need to read the main function to get the idea what the program does. You can then just go the way down as you'd read a book to get into more details of the implementation. Maybe you don't like the prototypes being on the beginning of the file? You can quickly skip them and read what's after them. Additionally, they serve as a nice reference. I actually always used to write the prototypes on the top. Also, if by mistake your declaration is different than the definition, the compiler will tell you. > >> *Comments: * > >> > >> Sections (groups of functions) can be separated with a line of > >> comments like this: > >> > >> /************ Main Interface Functions ****************/ > >> > >> Or this > >> > >> /********* Static Internal Functions **********/ > >> > > > > Is it really necessary to state this? Static functions are usually > > pretty obviously static ... Also I usually prefer to keep static > > functions related to a specific public function close to said > > function so that there is no need to jump around the file when > > checking what a related function does. > > > > > Yes. I think it will. It will help you with your approach to > processing code to read the code I write and vice versa. > These things are very helpful when you face the code that is > organized in the way it is not intuitive for you to digest. I agree with Dmitri, it's easier to find stuff in a file that has sections. Also (sorry about the wrapping made by my email client): /* * VERY important single-line comments look like this. */ /* Most single-line comments look like this. */ /* * Multi-line comments look like this. Make them real sentences. Fill * them so they look like real paragraphs. */ So ideally, no /* Multiline comments that look like this */ I always disliked any ascii art in my files, I always found them distracting. Couldn't we just separate the sections with the "VERY important" single-line comments? If one wants to really distinguish the comment from the others, he might add some empty lines around. > >> Each function must have a comment describing what this function > >> does and returns. > >> There is no specific format for this but it is a good practice to > >> describe what parameters mean and what return codes function > >> returns under what conditions. > >> > >> Inside the function comments should help reader to digest the > >> logic of the function. They should answer the main question ?why > >> is this line of code here and what it tries to accomplish??. Not > >> everything should be commented but practice shows that comments > >> giving the hint about what the developer what trying to accomplish > >> with this or that construct usually turn out to be pretty helpful. > >> > > > > This is fine as long as we don't have things like: > > > > int i = x +1; /* add 1 to x and assign it to i */ > > > > Let's avoid useless comments :-) > > > > > > Agree. +1, the general logic should be: Comment on *what* your code does, not *how* it does it. If you really must write how it works, you should probably rewrite it. > >> Do not use Hungarian notation (prefixing variable with the type). > >> The only exception to the rule is pointers. The following example > >> illustrates the case: > >> > >> void function_that_does_something(data_set inventory) { > >> int number_items =0; > >> > >> err=get_number_of_items_in_set(inventory, number_items); > >> ... > >> } > >> > >> int get_number_of_items_in_set(data_set inventory, int* > >> p_number_items) { ... > >> } > >> > > > > This definition is redundant, int * already defines the variable as > > pointer. Please let's avoid the insane Hungarian Notation > > completely. > > > > > >> In this case the variable in the second function is related to the > >> variable in the calling function but it is a pointer. This is > >> useful since naming variables same will cause confusion with type > >> (especially if there will be cutting and pasting of code which I > >> use to do a lot), and naming them differently will break the > >> relation. Like having number_items in one function and item_count > >> in the other and one is actually a pointer to another is hard to > >> work with. Prefixing with "p_" solves the issue nicely. > >> > > > > No, the compiler solves any issue nicely, it will tell you if you > > are using the wrong type. > > > > > > There are cases when these things caused a lot of trouble. Not > explicitly indicating pointers where variables are both pointers and > just variables IMO makes the code much harder to read and maintain. > I just makes things harder for no value but if others insist I would > not resist. Any other opinions? I agree there shouldn't be any Hungarian notation (and considering that my name is "Nagy", that's saying a lot!). Only exceptions should be cases similar to these: size_t strlen(const char *str); void free(void *ptr); Also, no one can complain about: size_t size; Which reminds me of another thing: If appropriate, always use the const modifier for pointers passed to the function. This makes the intentions of the function more clearer, plus allows the compiler to catch more bugs and make some optimizations. > >> *Names of the functions: * > >> > >> Looking at the code (server.c for example) I have seen all sorts > >> of names for functions used in one module: > >> > >> setup_signals(void) > >> BlockSignals(true, SIGPIPE); > >> poptGetNextOpt(pc) > >> > >> We should try to stick to one style which will be low case, multi > >> word, _ separated like: > >> monitor_task_init(struct task_server *task); > >> > > > > The later is the preferred form, poptGetNextOpt and BlockSignals > > come from external libraries, so we have no control over them. > > > > > Yes. +1, consistency in general should always be preferred, unless the breaking of it can actually make the code more cleaner and easier to read. > >> * Declarations * > >> Major declarations such as typedef's, stuct tags, etc. deserve to > >> be identifiable on their own in some fashion. > >> Often this was done with an initial cap, but since we're using > >> lower_case_with_underscores then I think the convention of > >> appending "_t" is a good way to go and is consistent with the > >> Posix and kernel conventions. > >> > > > > Only if we are defining a new type, something very rare. Again I am > > for avoiding typedefs altogether except for rare cases. > > > > If I need a structure to hold something a bout a socket this is how > > it should be defined: > > > > struct socket_stuff { > > int foo; > > long bar; > > struct baz; > > }; > > > > no typdefs. > > > > > Well... I kind of disagree but I can live with that. Again, preferably structs as noted above. > >> Other name formats should not be used unless it is an external > >> function we do not have control over. > >> > >> The wiki page will be updated based on this discussion. > >> > > > > Thanks. > > > > > > I'd like to add some more style related conventions I tend to use. > > > > Function definitions: > > > > Always define a function on one line with the opening parenthesis > > on the next line. > > Example: > > struct foobar *foo_bar_fn(int foo, int bar) > > { I disagree with the position of the type. There are a lot of types with variable length and with both the type and function name on one line, it's hard to quickly find the name of the function. In contrast, it is much easier to find it when it's on the beginning of a line and the type and name are easily distinguishable. IMO it also makes it easier to spot the header of a function. One disadvantage I can think of is the case when you need to know the return type in the debugger. But this IMO doesn't happen often and so I think it is not that important. Are there any arguments against? > > Do not put the "{" on the same line or the return type on a previous > > line like. > > > > > > +1 +1 > > Example of NOT ok declaration: > > struct foobar * > > foo_bar_fn(int foo, int bar) { As I mentioned above.. > > if the function declaration does not fit 80 chars, put arguments > > after the first on following lines and indent them to start after > > the "(" > > > > Example: > > int very_long_function_name_that_makes_arguments_not_fit_80(int foo, > > int bar) > > { > +1 +1 > > Do not put spaces before or after parenthesis: > > > > OK: int foo(int bar, int baz); > > NOT OK: bad ( arg1 , arg2 ); > > > > > > > +1 +1, but we should use space after standard C keywords, so: OK: if (expr) while (expr) NOT OK: if(expr) while(expr) etc. > > Avoid complex variable initializations (like calling functions) when > > declaring variables like: > > > > int foobar = get_foobar(baz); > > > > but split it in: > > int foobar; > > > > foobar = get_foobar(baz); > > > > > +1 +1, and don't initialize static or global variables to 0 or NULL. > > This makes it simpler to follow code in a debugger, and also avoid > > forgetting to test the return value of the function for errors. > > > > > > Always declare all variables at the top of the function, normally > > try to avoid declaring local variables in internal loops. > > > > > +1 Bjarne Stroustrup would concur. It's often much better to have local variables then to have 10 of them at the top of the function, regular human can't keep the track of them. Then again, it might be the case when breaking the function into smaller pieces is desirable. I would say the truth is somewhere in the middle. I wouldn't define this as a strict rule, only recommendation. The decision should be made in the interest of clarity and readability, and may wary from case to case. > > Use only C style comments /* blah */ and not C++ style comment: // > > blah > > > > > > > +1 +1 > > Add to these the coding style predicaments already available on > > freeipa.org :-) > > > > > > > I am planning to add this back to freeIPA.org > > Any other opinions and suggestions? Please let's wait some time so that other people get the chance to state their opinions. Some additional rules I think would be useful: Macros that are unsafe should be in upper-case. This also applies to macros that span multiple lines: #define MY_MACRO(a, b) do { \ foo((a) + (b)); \ bar(a); \ } while (0) Notice that arguments should be in parentheses if there's a risk, but in case of bar(a) this isn't the case (caution however won't hurt). Also notice that a is referenced two times, and hence the macro is dangerous. Wrapping the body in do { } while (0) makes it safe to use it like this: if (expr) MY_MACRO(x, y); Notice the semicolon is used after the invocation, not in the macro definition. Also notice the position of \'s. Otherwise, if a macro is safe (for example a simple wrapping function), then the case can be lower-case. Reasoning is that too much upper-case looks ugly. When using #ifdefs, it's nice to add comments in the pairing #endif: #ifndef _HEADER_H_ #define _HEADER_H_ /* something here */ #endif /* !_HEADER_H_ */ or: #ifdef HAVE_PTHREADS /* some code here */ #else /* !HAVE_PTHREADS */ /* some other code here */ #endif /* HAVE_PTHREADS */ Try to align members of union and struct definitions, but not at all costs: struct some_data { int size; /* Allocated bytes. */ char *buffer; /* Buffer containing the names. */ struct some_data *next; /* Next member. */ struct some_name_that_is_very_long var; /* Won't fit. */ }; Creating lists and queues was already done a lot of times. When possible, use some common functions for manipulating these to avoid mistakes. Don't use binary not (!) in tests, unless you test for a boolean: good: if (*str = '\0') bad: if (!*str) Switches: switch (var) { case 0: break; case 1: printf("meh.\n"); /* FALLTHROUGH */ case 2: printf("2\n"); break; default: /* Always have default */ break; } This is all from me right now. Martin From jdennis at redhat.com Sat Nov 1 14:32:05 2008 From: jdennis at redhat.com (John Dennis) Date: Sat, 01 Nov 2008 10:32:05 -0400 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <1225485318.5137.104.camel@localhost.localdomain> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> Message-ID: <490C6865.6030901@redhat.com> I'm going to respond to each persons email seperately, but only where I have something to add. Simo Sorce wrote: > Global variables should be discouraged as well, they make for very poor > code. Should be used only if no other way can be found. They tend to be > not thread/async safe > +1 > >> * Declarations of external functions >> > > Should not be needed, usually headers should contain foreign functions > definitions. > > +1 >> * Declarations of static functions >> > > I find it much easier to declare functions only if they need to be > referenced before the body of the function is defined. Otherwise just > use the ordering in the file to avoid declarations. > > -1 All functions should have prototypes declared at the top of the file. Depending on ordering is a side effect, side effects are to be avoided because of unintended consequences. Examples of non-obvious problems caused by this is reordering functions in a file or needing to call a function closer to the top of the file than was the case previously. It's just safer to always have a declared prototype for everything. > >> It is not required but recommended to group and position functions in >> the module so that it is easier to understand. >> > > This is very controversial, for me easier to understand is when the > innermost function is first in the file and the outermost is last, but > for some that seem an unnatural bottom to top ordering. > > You can't make a rule about stuff like this, rather just use your experience so the result is something readable to yourself and others. > >> Do not use Hungarian notation (prefixing variable with the type). The >> only exception to the rule is pointers. The following example >> illustrates the case: >> >> void function_that_does_something(data_set inventory) { >> int number_items =0; >> >> err=get_number_of_items_in_set(inventory, number_items); >> ... >> } >> >> int get_number_of_items_in_set(data_set inventory, int* p_number_items) { >> ... >> } >> > > This definition is redundant, int * already defines the variable as pointer. > Please let's avoid the insane Hungarian Notation completely. > > +1 Hungarian notation is a scourge. I have some sympathy for using a 'p' prefix for pointers in some limited circumstances where confusion might arise, use your judgement. > Function definitions: > > Always define a function on one line with the opening parenthesis on the > next line. > Example: > struct foobar *foo_bar_fn(int foo, int bar) > { > > Do not put the "{" on the same line or the return type on a previous > line like. > > Example of NOT ok declaration: > struct foobar * > foo_bar_fn(int foo, int bar) { > > if the function declaration does not fit 80 chars, put arguments after > the first on following lines and indent them to start after the "(" > > Example: > int very_long_function_name_that_makes_arguments_not_fit_80(int foo, > int bar) > { > > -1 I agree with Martin, the qualifiers and return type go on one line so that the function name can begin on the next line. It's way easier to visually spot function names when they begin on a line and some source code tools work better with this convention. > Do not put spaces before or after parenthesis: > > OK: int foo(int bar, int baz); > NOT OK: bad ( arg1 , arg2 ); > > +1 > Avoid complex variable initializations (like calling functions) when > declaring variables like: > > int foobar = get_foobar(baz); > > but split it in: > int foobar; > > foobar = get_foobar(baz); > > +1 -- John Dennis -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdennis at redhat.com Sat Nov 1 15:05:54 2008 From: jdennis at redhat.com (John Dennis) Date: Sat, 01 Nov 2008 11:05:54 -0400 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <20081101025556.5dd493ee@notas> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> <490B752B.9010305@redhat.com> <20081101025556.5dd493ee@notas> Message-ID: <490C7052.8000801@redhat.com> Martin Nagy wrote: > I like using typedefs if the convention is to always end them with _t. > However, this way you sometime need both struct and typedef (self > reference, as in case of linear lists for example). I also like structs > alone and this approach is much more clearer, because there's no need > to mix typedefs and structs. And it's always apparent at first glance > that we're dealing with a struct as Simo noted, so I'm vouching for > structs. > > Also, let's not forget that hiding a pointer behind a typedef is a > mortal sin. > > +1 >> No it is not. I think this allows you to follow your natural flaw of >> logic and me mine. >> I prefer implementing the functions that are main to the module first >> and stub out the functions that I need. >> So naturally I read the code the same way. I do not care about the >> details and low level functions that act as helpers. >> I want to see and concentrate on the core functionality and not dig >> it into the module to find it scattered somewhere in the middle or in >> the bottom. >> >> This ordering makes sense, but I've learned to read code in both styles so I don't really care. There really isn't a right or wrong way, both have valid reasons behind them so just let it be a personal preference of the author with the only caveat being the author is ultimately responsible for making the entire file readable and self consistent (the later being most important). > Which reminds me of another thing: If appropriate, always use the > const modifier for pointers passed to the function. This makes the > intentions of the function more clearer, plus allows the compiler to > catch more bugs and make some optimizations. > > +1 Not really a coding style but rather proper software engineering, but yes this should always be done. > I disagree with the position of the type. There are a lot of types with > variable length and with both the type and function name on one line, > it's hard to quickly find the name of the function. In contrast, it is > much easier to find it when it's on the beginning of a line and the > type and name are easily distinguishable. +1 see previous email > +1, but we should use space after standard C keywords, so: > OK: > if (expr) > while (expr) > > NOT OK: > if(expr) > while(expr) > +1 > Macros that are unsafe should be in upper-case. This also applies to > macros that span multiple lines: > > #define MY_MACRO(a, b) do { \ > foo((a) + (b)); \ > bar(a); \ > } while (0) > > Notice that arguments should be in parentheses if there's a risk, but > in case of bar(a) this isn't the case (caution however won't hurt). > Also notice that a is referenced two times, and hence the macro is > dangerous. Wrapping the body in do { } while (0) makes it safe to use > it like this: > > if (expr) > MY_MACRO(x, y); > > Notice the semicolon is used after the invocation, not in the macro > definition. Also notice the position of \'s. > > Otherwise, if a macro is safe (for example a simple wrapping function), > then the case can be lower-case. Reasoning is that too much upper-case > looks ugly. > > +1 > When using #ifdefs, it's nice to add comments in the pairing #endif: > > #ifndef _HEADER_H_ > #define _HEADER_H_ > > /* something here */ > > #endif /* !_HEADER_H_ */ > > or: > > #ifdef HAVE_PTHREADS > > /* some code here */ > > #else /* !HAVE_PTHREADS */ > > /* some other code here */ > > #endif /* HAVE_PTHREADS */ > > > +1 > > Creating lists and queues was already done a lot of times. When > possible, use some common functions for manipulating these to avoid > mistakes. > > +1 Actually it would be really nice we could agree on a basic helper library for implementing common data structures and operations. FWIW I've found glib to be pretty useful and widely available. But I'm sure others will have strong feelings about the merits of any such given library so let the flames begin :-) > Don't use binary not (!) in tests, unless you test for a boolean: > good: if (*str = '\0') > bad: if (!*str) > > I somewhat agree, I'm O.K. with "if (!*str)", it's compact (especially in loop constructs) and easy to read. But there are other cases where the ! operator is misleading. I find this coding style counter-intuitive to read: if (!strcmp(a,b)) I prefer if (strcmp(a,b) == 0) -- John Dennis From ssorce at redhat.com Sat Nov 1 16:51:39 2008 From: ssorce at redhat.com (Simo Sorce) Date: Sat, 01 Nov 2008 12:51:39 -0400 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <20081101025556.5dd493ee@notas> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> <490B752B.9010305@redhat.com> <20081101025556.5dd493ee@notas> Message-ID: <1225558299.5137.124.camel@localhost.localdomain> On Sat, 2008-11-01 at 02:55 +0100, Martin Nagy wrote: > Dmitri Pal wrote: > > Simo Sorce wrote: > > >> It is not required but recommended to group and position functions > > >> in the module so that it is easier to understand. > > >> > > > > > > This is very controversial, for me easier to understand is when the > > > innermost function is first in the file and the outermost is last, > > > but for some that seem an unnatural bottom to top ordering. > > > > > > > > No it is not. I think this allows you to follow your natural flaw of > > logic and me mine. > > I prefer implementing the functions that are main to the module first > > and stub out the functions that I need. > > So naturally I read the code the same way. I do not care about the > > details and low level functions that act as helpers. > > I want to see and concentrate on the core functionality and not dig > > it into the module to find it scattered somewhere in the middle or in > > the bottom. > > > > I guess this is a personal preference so I would suggest leaving it > > as it was spelled. > > On this subject, I strongly agree with Dmitri. I always found it > unnatural that main function is on the bottom, it just doesn't make > sense. I suspect people do this only because they are lazy to use > forward declarations. No, I find it easier to understand that dependencies come first, and forward declarations break this rule for me. It really is just 2 different ways to look at the code. > This is not a good way. Simo please reconsider, > I'm sure you will agree with me that code is much more read than > written. By this logic, I think that if we can get a more easily > readable code by making just a little more effort, it is worth it. And > I definitely think this is more readable as it follows logic. It really depends on what logic path you follow :-) > You > basically only need to read the main function to get the idea what the > program does. You can then just go the way down as you'd read a book to > get into more details of the implementation. s/down/up/ there fixed for ya :-P > Maybe you don't like the prototypes being on the beginning of the file? I find annoying to have to fill up prototypes, but I won't die if everybody prefer prototypes on top :) > You can quickly skip them and read what's after them. Additionally, > they serve as a nice reference. I actually always used to write the > prototypes on the top. Also, if by mistake your declaration is > different than the definition, the compiler will tell you. Compiler rules. > > >> *Comments: * > > >> > > >> Sections (groups of functions) can be separated with a line of > > >> comments like this: > > >> > > >> /************ Main Interface Functions ****************/ > > >> > > >> Or this > > >> > > >> /********* Static Internal Functions **********/ > > >> > > > > > > Is it really necessary to state this? Static functions are usually > > > pretty obviously static ... Also I usually prefer to keep static > > > functions related to a specific public function close to said > > > function so that there is no need to jump around the file when > > > checking what a related function does. > > > > > > > > Yes. I think it will. It will help you with your approach to > > processing code to read the code I write and vice versa. > > These things are very helpful when you face the code that is > > organized in the way it is not intuitive for you to digest. > > I agree with Dmitri, it's easier to find stuff in a file that has > sections. > > Also (sorry about the wrapping made by my email client): > > /* > * VERY important single-line comments look like this. > */ > > /* Most single-line comments look like this. */ > > /* > * Multi-line comments look like this. Make them real > sentences. Fill > * them so they look like real paragraphs. > */ > > So ideally, no > /* Multiline comments > that look like this */ +1 > I always disliked any ascii art in my files, I always found them > distracting. Couldn't we just separate the sections with the "VERY > important" single-line comments? If one wants to really distinguish the > comment from the others, he might add some empty lines around. +1 > > >> Each function must have a comment describing what this function > > >> does and returns. > > >> There is no specific format for this but it is a good practice to > > >> describe what parameters mean and what return codes function > > >> returns under what conditions. > > >> > > >> Inside the function comments should help reader to digest the > > >> logic of the function. They should answer the main question ?why > > >> is this line of code here and what it tries to accomplish??. Not > > >> everything should be commented but practice shows that comments > > >> giving the hint about what the developer what trying to accomplish > > >> with this or that construct usually turn out to be pretty helpful. > > >> > > > > > > This is fine as long as we don't have things like: > > > > > > int i = x +1; /* add 1 to x and assign it to i */ > > > > > > Let's avoid useless comments :-) > > > > > > > > > > Agree. > > +1, the general logic should be: > Comment on *what* your code does, not *how* it does it. If you really > must write how it works, you should probably rewrite it. It really depends on the code, sometimes something may seem wrong but it is right because it is clever, in that case you might also comment on why you did something that way. > Which reminds me of another thing: If appropriate, always use the > const modifier for pointers passed to the function. This makes the > intentions of the function more clearer, plus allows the compiler to > catch more bugs and make some optimizations. +1 > > > Function definitions: > > > > > > Always define a function on one line with the opening parenthesis > > > on the next line. > > > Example: > > > struct foobar *foo_bar_fn(int foo, int bar) > > > { > > I disagree with the position of the type. There are a lot of types with > variable length and with both the type and function name on one line, > it's hard to quickly find the name of the function. Syntax highlighting makes it easy. > In contrast, it is > much easier to find it when it's on the beginning of a line and the > type and name are easily distinguishable. IMO it also makes it easier > to spot the header of a function. One disadvantage I can think of is > the case when you need to know the return type in the debugger. But > this IMO doesn't happen often and so I think it is not that important. > > Are there any arguments against? I get sick when I see the type and the function on different lines :-) > +1, but we should use space after standard C keywords, so: > OK: > if (expr) > while (expr) > > NOT OK: > if(expr) > while(expr) +1 > etc. > > > > Avoid complex variable initializations (like calling functions) when > > > declaring variables like: > > > > > > int foobar = get_foobar(baz); > > > > > > but split it in: > > > int foobar; > > > > > > foobar = get_foobar(baz); > > > > > > > > +1 > > +1, and don't initialize static or global variables to 0 or NULL. right > Macros that are unsafe should be in upper-case. This also applies to > macros that span multiple lines: > > #define MY_MACRO(a, b) do { \ > foo((a) + (b)); \ > bar(a); \ > } while (0) > > Notice that arguments should be in parentheses if there's a risk, but > in case of bar(a) this isn't the case (caution however won't hurt). > Also notice that a is referenced two times, and hence the macro is > dangerous. Wrapping the body in do { } while (0) makes it safe to use > it like this: > > if (expr) > MY_MACRO(x, y); > > Notice the semicolon is used after the invocation, not in the macro > definition. Also notice the position of \'s. > > Otherwise, if a macro is safe (for example a simple wrapping function), > then the case can be lower-case. Reasoning is that too much upper-case > looks ugly. +1 on all except '\' position :) > When using #ifdefs, it's nice to add comments in the pairing #endif: > > #ifndef _HEADER_H_ > #define _HEADER_H_ > > /* something here */ > > #endif /* !_HEADER_H_ */ > > or: > > #ifdef HAVE_PTHREADS > > /* some code here */ > > #else /* !HAVE_PTHREADS */ > > /* some other code here */ > > #endif /* HAVE_PTHREADS */ +1 > Try to align members of union and struct definitions, but not at all > costs: > > struct some_data { > int size; /* Allocated bytes. */ > char *buffer; /* Buffer containing the names. */ > struct some_data *next; /* Next member. */ > struct some_name_that_is_very_long var; /* Won't fit. */ > }; I find alignment useless and generally harder too read. When you have more than a handful of variables and the names or values are far apart I generally find it more difficult to immediately grasp which refers to what. It makes my eyes keep going left-right many times to make sure I got the alignment right, waste of time. It also make you unnecessarily reformat code when you later on add a variable that moves the alignment further on the right of a few characters. I am sort of ok for the comment block though, mostly because it keeps it out of my way when reading a structure I know. > Creating lists and queues was already done a lot of times. When > possible, use some common functions for manipulating these to avoid > mistakes. +1 > Don't use binary not (!) in tests, unless you test for a boolean: > good: if (*str = '\0') > bad: if (!*str) I am bad, I tend to use both forms, but I use if (!foo), generally only to check memory allocations. I am flexible and we can certainly change to use if (NULL == foo) > Switches: > > switch (var) { > case 0: > break; > case 1: > printf("meh.\n"); > /* FALLTHROUGH */ > case 2: > printf("2\n"); > break; > default: > /* Always have default */ > break; > } +1 on switch statement alignment Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Sat Nov 1 16:58:24 2008 From: ssorce at redhat.com (Simo Sorce) Date: Sat, 01 Nov 2008 12:58:24 -0400 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <490C7052.8000801@redhat.com> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> <490B752B.9010305@redhat.com> <20081101025556.5dd493ee@notas> <490C7052.8000801@redhat.com> Message-ID: <1225558704.5137.127.camel@localhost.localdomain> On Sat, 2008-11-01 at 11:05 -0400, John Dennis wrote: > I somewhat agree, I'm O.K. with "if (!*str)", it's compact > (especially > in loop constructs) and easy to read. But there are other cases where > the ! operator is misleading. > > I find this coding style counter-intuitive to read: > > if (!strcmp(a,b)) > > I prefer > > if (strcmp(a,b) == 0) +1 -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Sat Nov 1 16:59:35 2008 From: ssorce at redhat.com (Simo Sorce) Date: Sat, 01 Nov 2008 12:59:35 -0400 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <490C6865.6030901@redhat.com> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> <490C6865.6030901@redhat.com> Message-ID: <1225558775.5137.129.camel@localhost.localdomain> On Sat, 2008-11-01 at 10:32 -0400, John Dennis wrote: > -1 > > I agree with Martin, the qualifiers and return type go on one line so > that the function name can begin on the next line. It's way easier to > visually spot function names when they begin on a line and some source > code tools work better with this convention. If everybody finds it better, I will adapt (sigh! :-) -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Sun Nov 2 17:27:08 2008 From: ssorce at redhat.com (Simo Sorce) Date: Sun, 02 Nov 2008 12:27:08 -0500 Subject: [Freeipa-devel] Re: [Freeipa-interest] [Roadmap] Evolution In-Reply-To: <81FA719A76839A4990A096D71D99A2E40308DC51@nlaexb12.neptune.dkcorp.net> References: <81FA719A76839A4990A096D71D99A2E40308DC51@nlaexb12.neptune.dkcorp.net> Message-ID: <1225646828.4565.1.camel@localhost.localdomain> On Tue, 2008-10-28 at 11:48 +0100, NOLLET Remy (CAMPUS) wrote: > > > Hi, > > I am very interested about this project to implement an Infra Server > on full Linux. > I know that the wishlist is growing but why do the roadmap going later > instead of having another release IPA3 ? > > What we have done separetly: > .DNS with LDAP Backend > .SUDERS with LDAP Backend > .a Mix POSIX/NIS schema on LDAP to allow group of users and > group of servers. Remy, would you mind moving this discussion on freeipa-devel ? (I took the liberty to reply there). We are planning on all 3 features for v2, although probably not implemented the way you did. But it would be nice to discuss merits and cons of your solution and see if we can learn in the process. Simo. -- Simo Sorce * Red Hat, Inc * New York From abartlet at samba.org Mon Nov 3 02:13:47 2008 From: abartlet at samba.org (Andrew Bartlett) Date: Mon, 03 Nov 2008 13:13:47 +1100 Subject: [Freeipa-devel] using with samba In-Reply-To: <490B1891.6060001@redhat.com> References: <48EB7B60.2010001@priefert.com> <1224559475.3494.49.camel@ruth> <49061961.2040306@priefert.com> <49063784.8010903@redhat.com> <490B1891.6060001@redhat.com> Message-ID: <1225678427.5890.11.camel@naomi.s4.naomi.abartlet.net> On Fri, 2008-10-31 at 10:39 -0400, Dmitri Pal wrote: > Hi William, > > I need to correct myself a bit. Samba 4 can use different back ends. The > primary back end it uses is the LDB - internal LDAP style storage. It is > fast and efficient. > The alternative back end Samba 4 can use is OpenLDAP. This work more > experimental but OpenLDAP currently seems to be more feature rich from > the prospective of Samba4 then Fedora DS. We also have (even more experimental) support for Fedora DS, handling some of easy stuff (schema format, automatic initialisation, and mapping of nsuiqueID to objectGUID). But this, like the OpenLDAP support replaces the schema. > The FreeIPA uses Fedora DS as a back end. For the task of bringing Samba > 4 and IPA together the options would be to try to use one and the same > back end or synch data if there will be two. > In both cases the mapping of entries, attributes and DITs would be the > first biggest problem to solve. Thanks Dmitiri, This captures the situation and requirements much more clearly. Figure out the mapping and the rest will be easy. (That does not mean the schema mapping will be easy however...) Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From abartlet at samba.org Mon Nov 3 02:16:21 2008 From: abartlet at samba.org (Andrew Bartlett) Date: Mon, 03 Nov 2008 13:16:21 +1100 Subject: [Freeipa-devel] Please review: more patches for winsync support In-Reply-To: <48E67FC0.1070908@redhat.com> References: <48E67FC0.1070908@redhat.com> Message-ID: <1225678581.5890.14.camel@naomi.s4.naomi.abartlet.net> On Fri, 2008-10-03 at 14:25 -0600, Rich Megginson wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=459729 > Resolves: bug 459729 > Bug Description: Windows sync support in IPA - account disable and force > sync > > During testing, I found a few issues: > > add winsync options to man page for ipa-replica-manage - > https://bugzilla.redhat.com/attachment.cgi?id=319412&action=diff > > ipa-replica-manage commands such as init, synch, and list did not work > with windows sync agreements - > https://bugzilla.redhat.com/attachment.cgi?id=319413&action=diff > > before installing the Windows CA cert, I stop the server - this just > makes the stop unconditional, since I don't care if the server is > running or not - > https://bugzilla.redhat.com/attachment.cgi?id=319414&action=diff > > the ntUniqueID and ntUserDomainID indexes exist by default, so we just > have to modify them to add eq,pres - > https://bugzilla.redhat.com/attachment.cgi?id=319415&action=diff Is it too late to hope that WinSync in FreeIPA might use the Samba3 schema for windows things (like the SID, username etc) rather than this netscape schema? It would make life easier for a possible future Samba4 backend (as it would consolidate work with the existing partial Samba3 -> Samba4 mapping). Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From remy.nollet at oxylane-group.com Mon Nov 3 09:30:52 2008 From: remy.nollet at oxylane-group.com (NOLLET Remy (CAMPUS)) Date: Mon, 3 Nov 2008 10:30:52 +0100 Subject: [Freeipa-devel] RE: [Freeipa-interest] [Roadmap] Evolution In-Reply-To: <1225646828.4565.1.camel@localhost.localdomain> Message-ID: <81FA719A76839A4990A096D71D99A2E4030DB93C@nlaexb12.neptune.dkcorp.net> Yes surely, We did this in the way to get a centralized infrastructure server with a uniq database as Active directory. We want to challenge AD on many Full Linux Infra domain. We need NTP relay, DNS server, Auth server, ACL server (for auth, right... ) etc... So first thing we did was to implement NTP relay and LDAP, SSH auth with a LDAP backend, and do a bind server with th same LDAP backend. For Sudoers, we test to implement it in LDAP Too. Fine. After the way was to use group of server, users and group of users from this LDAP to simplify management. Now we: - declare servers on DNS/LDAP, - declare posix users on LDAP, - put them on different groups NIS/POSIX - put ACLs in sudoers on LDAP - put ACLs for SSH Auth on LDAP too ( it's the beginning of major GPO ) We are looking on FreeIPA to get a friendly interface to manage all... We want to manage full/partial replication between LDAP. And manage user profile, printers etc... Maybe use this to integrate/authenticate Windows platform ;-)) Doesn't be anyway to split your roadmap on more fine phasis ? Kind Regards Remy -----Original Message----- From: Simo Sorce [mailto:ssorce at redhat.com] Sent: Sunday, November 02, 2008 6:27 PM To: NOLLET Remy (CAMPUS) Cc: freeipa-devel Subject: Re: [Freeipa-interest] [Roadmap] Evolution On Tue, 2008-10-28 at 11:48 +0100, NOLLET Remy (CAMPUS) wrote: > > > Hi, > > I am very interested about this project to implement an Infra Server > on full Linux. > I know that the wishlist is growing but why do the roadmap going later > instead of having another release IPA3 ? > > What we have done separetly: > .DNS with LDAP Backend > .SUDERS with LDAP Backend > .a Mix POSIX/NIS schema on LDAP to allow group of users and > group of servers. Remy, would you mind moving this discussion on freeipa-devel ? (I took the liberty to reply there). We are planning on all 3 features for v2, although probably not implemented the way you did. But it would be nice to discuss merits and cons of your solution and see if we can learn in the process. Simo. -- Simo Sorce * Red Hat, Inc * New York From sgallagh at redhat.com Mon Nov 3 13:31:34 2008 From: sgallagh at redhat.com (Stephen Gallagher) Date: Mon, 03 Nov 2008 08:31:34 -0500 Subject: [Freeipa-devel] Suggested additions to codding standard page on freeIPA wiki. In-Reply-To: <1225485318.5137.104.camel@localhost.localdomain> References: <490B62E3.2070705@redhat.com> <1225485318.5137.104.camel@localhost.localdomain> Message-ID: <490EFD36.4090202@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > if the function declaration does not fit 80 chars, put arguments after > the first on following lines and indent them to start after the "(" > > Example: > int very_long_function_name_that_makes_arguments_not_fit_80(int foo, > int bar) > { > > - -1 I'm a bigger fan of using the two-indent approach to this problem int very_long_function_name_that_makes_arguments_not_fit_80( int foo, int bar, void (*some_other_long_function_name)(int baz)) { Other general points: I tend to agree with the "always include a forward declaration" approach to coding. I disagree with the "mortal sin" of typedefing pointers in one case: I believe it's perfectly sane to typedef int (*long_function_name)(int foo, int bar, char *baz) foobar_fn Yes you lose the obvious pointer qualification, but foobar_fn is a LOT easier to add to subsequent argument lists than repeating the function pointer declaration. Hungarian notation should be grounds for revocation of commit privilege on FreeIPA. - From John: > Actually it would be really nice we could agree on a basic helper > library for implementing common data structures and operations. FWIW > I've found glib to be pretty useful and widely available. But I'm sure > others will have strong feelings about the merits of any such given > library so let the flames begin :-) The problem with using a common helper library is that our goal on the client-side is to be able to support as many client OSes as possible. Glib is not available on all platforms, so we're avoiding it. If you happen to know of a helper library that is available on all flavors of Linux, Solaris, HP-UX, AIX, OSX and BSD, then we'll talk :) - From Martin: > Don't use binary not (!) in tests, unless you test for a boolean: > good: if (*str = '\0') > bad: if (!*str) I don't really see a problem with the approach struct some_struct foo; foo = malloc(sizeof(struct some_struct)); if (!foo) { /* Handle OOM condition */ } !foo is at least as readable as (NULL == foo), if not more so. One might be able to make a stronger argument about this use when testing function arguments. - -- - -------------------- Stephen Gallagher RHCE 804006346421761 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkkO/TMACgkQc7MaxVic+2oVigCfd8j+HMMNeVOBUrO3I6TfxQ+O XacAn1lAcGFknnFuIbtNUvdZEozzkUjD =4mrr -----END PGP SIGNATURE----- From rmeggins at redhat.com Mon Nov 3 16:33:15 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 03 Nov 2008 09:33:15 -0700 Subject: [Freeipa-devel] Please review: more patches for winsync support In-Reply-To: <1225678581.5890.14.camel@naomi.s4.naomi.abartlet.net> References: <48E67FC0.1070908@redhat.com> <1225678581.5890.14.camel@naomi.s4.naomi.abartlet.net> Message-ID: <490F27CB.9070902@redhat.com> Andrew Bartlett wrote: > On Fri, 2008-10-03 at 14:25 -0600, Rich Megginson wrote: > >> https://bugzilla.redhat.com/show_bug.cgi?id=459729 >> Resolves: bug 459729 >> Bug Description: Windows sync support in IPA - account disable and force >> sync >> >> During testing, I found a few issues: >> >> add winsync options to man page for ipa-replica-manage - >> https://bugzilla.redhat.com/attachment.cgi?id=319412&action=diff >> >> ipa-replica-manage commands such as init, synch, and list did not work >> with windows sync agreements - >> https://bugzilla.redhat.com/attachment.cgi?id=319413&action=diff >> >> before installing the Windows CA cert, I stop the server - this just >> makes the stop unconditional, since I don't care if the server is >> running or not - >> https://bugzilla.redhat.com/attachment.cgi?id=319414&action=diff >> >> the ntUniqueID and ntUserDomainID indexes exist by default, so we just >> have to modify them to add eq,pres - >> https://bugzilla.redhat.com/attachment.cgi?id=319415&action=diff >> > > Is it too late to hope that WinSync in FreeIPA might use the Samba3 > schema for windows things (like the SID, username etc) rather than this > netscape schema? > Yes, it's too late, at least for this feature. The purpose of this is just to be able to support some sort of windows sync with IPA for user entries. This isn't really any sort of enhancement or new feature (except for the bit about adding some additional schema for IPA user entries). It's really just making the existing Fedora DS winsync work with IPA. > It would make life easier for a possible future Samba4 backend (as it > would consolidate work with the existing partial Samba3 -> Samba4 > mapping). > > Andrew Bartlett > > From dpal at redhat.com Mon Nov 3 20:58:06 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 03 Nov 2008 15:58:06 -0500 Subject: [Freeipa-devel] RE: [Freeipa-interest] [Roadmap] Evolution In-Reply-To: <81FA719A76839A4990A096D71D99A2E4030DB93C@nlaexb12.neptune.dkcorp.net> References: <81FA719A76839A4990A096D71D99A2E4030DB93C@nlaexb12.neptune.dkcorp.net> Message-ID: <490F65DE.7090708@redhat.com> NOLLET Remy (CAMPUS) wrote: > Yes surely, > > We did this in the way to get a centralized infrastructure server with a > uniq database as Active directory. > We want to challenge AD on many Full Linux Infra domain. > > We need NTP relay, DNS server, Auth server, ACL server (for auth, > right... ) etc... > So first thing we did was to implement NTP relay and LDAP, SSH auth with > a LDAP backend, and do a bind server with th same LDAP backend. > For Sudoers, we test to implement it in LDAP Too. Fine. > After the way was to use group of server, users and group of users from > this LDAP to simplify management. > Now we: > - declare servers on DNS/LDAP, > - declare posix users on LDAP, > - put them on different groups NIS/POSIX > - put ACLs in sudoers on LDAP > - put ACLs for SSH Auth on LDAP too > ( it's the beginning of major GPO ) > > We are looking on FreeIPA to get a friendly interface to manage all... > We want to manage full/partial replication between LDAP. > And manage user profile, printers etc... > > Maybe use this to integrate/authenticate Windows platform ;-)) > > Doesn't be anyway to split your roadmap on more fine phasis ? > > Kind Regards > > Remy > Hm, we are doing pretty much the same thing. We are embedding DNS, we are manging users, user groups, hosts, host groups... We plan to manage policies for host based access, sudoers, SELinux etc. And we also looking at Samba integration down the road. In v2 we really need to make a leap from LDAP+Kerberos to: LDAP, Kerberos, CA, DNS, Policy engine, basic audit log collection , host management, client etc. At least this is what we got from different sources to be a minimal subset of functionality we need to provide. Without it it won't be attractive. Are you saying that a subset would be interesting in your case? Ho would you slice this in different releases to meet your needs? Thanks Dmitri From rmeggins at redhat.com Tue Nov 4 23:36:26 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Tue, 04 Nov 2008 16:36:26 -0700 Subject: [Freeipa-devel] Please review: should use ipautil.CalledProcessError instead of just CalledProcessError Message-ID: <4910DC7A.6000008@redhat.com> Dumb python programmer error :P -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-use-ipautil.CalledProcessError-instead-of-CalledProc.patch Type: text/x-patch Size: 927 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 5 02:58:51 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 04 Nov 2008 21:58:51 -0500 Subject: [Freeipa-devel] Please review: should use ipautil.CalledProcessError instead of just CalledProcessError In-Reply-To: <4910DC7A.6000008@redhat.com> References: <4910DC7A.6000008@redhat.com> Message-ID: <49110BEB.6010106@redhat.com> Rich Megginson wrote: > Dumb python programmer error :P > ack I also fixed a typo and removed a trailing space (elsewhere in the file) and pushed to master and ipa-1-2. rob From rcritten at redhat.com Wed Nov 5 04:22:48 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 04 Nov 2008 23:22:48 -0500 Subject: [Freeipa-devel] automount in LDAP Message-ID: <49111F98.4020806@redhat.com> I'm trying to wrap my head around automount over LDAP and how we would want an API over XML-RPC to support it. At its core there are 2 types of objects we'll deal with: maps and keys That part is easy enough. I can implement add-map and add-key methods. The problem is producing something actually usable to a client. The thing is, the admin will need a certain amount of automount knowledge to create a usable setup. Do we want to mirror the LDAP objects or provide an interface to be useful, or both? So an example is in order. Here is an example of a simple non-home shared directory. It creates the maps auto.master and auto.share and exports the NFS share /share/builds. dn: automountmapname=auto.master,cn=accounts,dc=example,dc=com objectClass: automountMap objectClass: top automountMapName: auto.master dn: automountkey=/share,automountmapname=auto.master,cn=accounts,dc=example,dc =com objectClass: automount objectClass: top automountKey: /share automountInformation: ldap:automountmapname=auto.share, cn=accounts,dc=example ,dc=com dn: automountmapname=auto.share,cn=accounts,dc=example,dc=com objectClass: automountMap objectClass: top automountMapName: auto.share dn: automountkey=builds,automountmapname=auto.share,cn=accounts,dc=example,dc= com automountInformation: somehost:/share/builds objectClass: automount objectClass: top automountKey: builds So assuming I'm reading this right, and that is a bit of a leap, this shows there are two kinds of keys. One defines a search point (/share) and one defines a mount point (builds). And it requires knowledge of how automount wants to format shares. And also, this assumes we use a schema with automountkey/automountmap instead of cn and ou. Adding shares can be left as an exercise to the user by just exporting a way to add maps and keys, or we can try to impose some sort of order on this. What that might look like I don't know. Feedback welcome. rob From rcritten at redhat.com Wed Nov 5 14:49:57 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 05 Nov 2008 09:49:57 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <57127b5f0811050456i6cedda85k2c3a9786f0511ba4@mail.gmail.com> References: <49111F98.4020806@redhat.com> <57127b5f0811050456i6cedda85k2c3a9786f0511ba4@mail.gmail.com> Message-ID: <4911B295.60804@redhat.com> Deependra Singh Shekhawat wrote: > Hi, > > This sounds different from what I see here > > http://directory.fedoraproject.org/wiki/Howto:Automount Is is similar but I'm using different attributes. The sample there is mounting home directories which is a bit of a harder problem (requires an entry for every user) and is using the cn and ou attributes. The general format is the same. > Doesn't client need to do configuration changes in /etc/sysconfig/autofs ? > > And if that's true it will be good to have some automation about it in > FreeIPA-client? Yes, some client-side changes are required as well to get automount working. /etc/sysconfig/autofs needs to say which LDAP server to use, what its basedn is and what attributes should be used. Additionally /etc/nsswitch.conf needs to be modified to use ldap for automount. rob > > Thanks > Deependra > > On Wed, Nov 5, 2008 at 9:52 AM, Rob Crittenden > wrote: > > I'm trying to wrap my head around automount over LDAP and how we > would want an API over XML-RPC to support it. > > At its core there are 2 types of objects we'll deal with: maps and keys > > That part is easy enough. I can implement add-map and add-key > methods. The problem is producing something actually usable to a client. > > The thing is, the admin will need a certain amount of automount > knowledge to create a usable setup. Do we want to mirror the LDAP > objects or provide an interface to be useful, or both? > > So an example is in order. Here is an example of a simple non-home > shared directory. It creates the maps auto.master and auto.share and > exports the NFS share /share/builds. > > dn: automountmapname=auto.master,cn=accounts,dc=example,dc=com > objectClass: automountMap > objectClass: top > automountMapName: auto.master > > dn: > automountkey=/share,automountmapname=auto.master,cn=accounts,dc=example,dc > =com > objectClass: automount > objectClass: top > automountKey: /share > automountInformation: ldap:automountmapname=auto.share, > cn=accounts,dc=example > ,dc=com > > dn: automountmapname=auto.share,cn=accounts,dc=example,dc=com > objectClass: automountMap > objectClass: top > automountMapName: auto.share > > dn: > automountkey=builds,automountmapname=auto.share,cn=accounts,dc=example,dc= > com > automountInformation: somehost:/share/builds > objectClass: automount > objectClass: top > automountKey: builds > > So assuming I'm reading this right, and that is a bit of a leap, > this shows there are two kinds of keys. One defines a search point > (/share) and one defines a mount point (builds). And it requires > knowledge of how automount wants to format shares. > > And also, this assumes we use a schema with > automountkey/automountmap instead of cn and ou. > > Adding shares can be left as an exercise to the user by just > exporting a way to add maps and keys, or we can try to impose some > sort of order on this. What that might look like I don't know. > > Feedback welcome. > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > > -- > Type bits /keyID Date User ID > pub 1024D/483B234C 2007/06/29 Deependra Singh Shekhawat (Fedora > Project) > > Key fingerprint = ED45 62EA A4D7 53FB 44C7 774A D55B F3F0 483B 234C > From rcritten at redhat.com Wed Nov 5 16:09:04 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 05 Nov 2008 11:09:04 -0500 Subject: [Freeipa-devel] [PATCH] fix winsync replica ID Message-ID: <4911C520.7080809@redhat.com> A change I made recently to getting unique replica IDs broke winsync. It was trying to connect to the AD server to get the next nsDS5replicaId which obviously won't work. So this change will make it connect to itself instead. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-100-winsync.patch Type: text/x-patch Size: 1258 bytes Desc: not available URL: From rmeggins at redhat.com Wed Nov 5 16:12:56 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 05 Nov 2008 09:12:56 -0700 Subject: [Freeipa-devel] [PATCH] fix winsync replica ID In-Reply-To: <4911C520.7080809@redhat.com> References: <4911C520.7080809@redhat.com> Message-ID: <4911C608.3050502@redhat.com> Rob Crittenden wrote: > A change I made recently to getting unique replica IDs broke winsync. > > It was trying to connect to the AD server to get the next > nsDS5replicaId which obviously won't work. So this change will make it > connect to itself instead. ack > > rob > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From nalin at redhat.com Wed Nov 5 16:54:54 2008 From: nalin at redhat.com (Nalin Dahyabhai) Date: Wed, 5 Nov 2008 11:54:54 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <4911B295.60804@redhat.com> References: <49111F98.4020806@redhat.com> <57127b5f0811050456i6cedda85k2c3a9786f0511ba4@mail.gmail.com> <4911B295.60804@redhat.com> Message-ID: <20081105165454.GA10119@redhat.com> On Wed, Nov 05, 2008 at 09:49:57AM -0500, Rob Crittenden wrote: > Is is similar but I'm using different attributes. The sample there is > mounting home directories which is a bit of a harder problem (requires > an entry for every user) and is using the cn and ou attributes. The > general format is the same. If the per-user entries follow a pattern, you can use a wildcard entry and the substitution character (&) to stand-in for the key in the automountInformation attribute, which greatly reduces the size of the map. The autofs(5) man page covers its use in a bit more depth. HTH, Nalin From rcritten at redhat.com Wed Nov 5 17:04:20 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 05 Nov 2008 12:04:20 -0500 Subject: [Freeipa-devel] [PATCH] fix winsync replica ID In-Reply-To: <4911C608.3050502@redhat.com> References: <4911C520.7080809@redhat.com> <4911C608.3050502@redhat.com> Message-ID: <4911D214.8030801@redhat.com> Rich Megginson wrote: > Rob Crittenden wrote: >> A change I made recently to getting unique replica IDs broke winsync. >> >> It was trying to connect to the AD server to get the next >> nsDS5replicaId which obviously won't work. So this change will make it >> connect to itself instead. > ack >> Pushed to ipa-1-2 and master rob From nalin at redhat.com Wed Nov 5 17:20:13 2008 From: nalin at redhat.com (Nalin Dahyabhai) Date: Wed, 5 Nov 2008 12:20:13 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49111F98.4020806@redhat.com> References: <49111F98.4020806@redhat.com> Message-ID: <20081105172013.GB10119@redhat.com> On Tue, Nov 04, 2008 at 11:22:48PM -0500, Rob Crittenden wrote: > I'm trying to wrap my head around automount over LDAP and how we would > want an API over XML-RPC to support it. > > At its core there are 2 types of objects we'll deal with: maps and keys > > That part is easy enough. I can implement add-map and add-key methods. > The problem is producing something actually usable to a client. > > The thing is, the admin will need a certain amount of automount > knowledge to create a usable setup. Do we want to mirror the LDAP > objects or provide an interface to be useful, or both? I know it's going to be more work, but you may end up having to do both. > So an example is in order. Here is an example of a simple non-home > shared directory. It creates the maps auto.master and auto.share and > exports the NFS share /share/builds. > > dn: automountmapname=auto.master,cn=accounts,dc=example,dc=com > objectClass: automountMap > objectClass: top > automountMapName: auto.master > > dn: > automountkey=/share,automountmapname=auto.master,cn=accounts,dc=example,dc > =com > objectClass: automount > objectClass: top > automountKey: /share > automountInformation: ldap:automountmapname=auto.share, > cn=accounts,dc=example > ,dc=com > > dn: automountmapname=auto.share,cn=accounts,dc=example,dc=com > objectClass: automountMap > objectClass: top > automountMapName: auto.share > > dn: > automountkey=builds,automountmapname=auto.share,cn=accounts,dc=example,dc= > com > automountInformation: somehost:/share/builds > objectClass: automount > objectClass: top > automountKey: builds > > So assuming I'm reading this right, and that is a bit of a leap, this > shows there are two kinds of keys. One defines a search point (/share) > and one defines a mount point (builds). And it requires knowledge of how > automount wants to format shares. Looks right to me. You'll see fewer maps storing directory paths (only auto.master and auto.direct come to mind) than subdirectory names. The first group are typically known as "direct" maps, and the latter group as "indirect" maps. > And also, this assumes we use a schema with automountkey/automountmap > instead of cn and ou. Yes, and it's actually what I prefer. Using these particular attributes means that keys can be case-sensitive (automountKey is defined to be so in the schema, while cn and ou are not). Automount map information stored in files and NIS maps treats the keys as case-sensitive, too, and any deviation from that behavior when you're moving to LDAP is surprising (and, it turns out, often upsetting). > Adding shares can be left as an exercise to the user by just exporting a > way to add maps and keys, or we can try to impose some sort of order on > this. What that might look like I don't know. > > Feedback welcome. The only opinion I have to offer is this: people do exceptionally weird things with autofs maps, so the less constraints you force, the happier people will be. If you can manage to offer default settings that don't push people toward weirdness, though, the situation might look better in the future. Nalin From dpal at redhat.com Wed Nov 5 20:13:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 05 Nov 2008 15:13:07 -0500 Subject: [Freeipa-devel] DNS and replication Message-ID: <4911FE53.6080608@redhat.com> Hi, According to design the DNS would be able to write to the DS storage. What is the impact of the DNS updates to the replication? How frequent are these updates? If we in future move to the read only replicas how would DNS updates be affected? Should we think about this situation now and at least assess how we are going to configure DNS in case of the RO replica? Will we have clients to update only DNS on masters or we will use forwarding but them if the volume is high would there be a performance issue. Martin, please track this discussion and update wiki page accordingly. Thanks Dmitri From mnagy at redhat.com Wed Nov 5 20:24:39 2008 From: mnagy at redhat.com (Martin Nagy) Date: Wed, 5 Nov 2008 21:24:39 +0100 Subject: [Freeipa-devel] DNS and replication In-Reply-To: <4911FE53.6080608@redhat.com> References: <4911FE53.6080608@redhat.com> Message-ID: <20081105212439.453032e7@notas> Dmitri Pal wrote: > Hi, > > According to design the DNS would be able to write to the DS storage. > What is the impact of the DNS updates to the replication? How > frequent are these updates? That depends on where IPA is deployed, but generally, there should be a lot less writing than reading. > If we in future move to the read only replicas how would DNS updates > be affected? > Should we think about this situation now and at least assess how we > are going to configure DNS in case of the RO replica? AFAIK, this should not be a concern for us. If you try to write to a RO replica, it should redirect you to a master. This should be made transparently. Someone please correct me if I'm wrong about this :) > Will we have clients to update only DNS on masters or we will use > forwarding but them if the volume is high would there be a > performance issue. Clients themselves will use GSS-TSIG and talk to the DNS server. The DNS server itself will talk to LDAP. Martin From dpal at redhat.com Wed Nov 5 20:27:27 2008 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 05 Nov 2008 15:27:27 -0500 Subject: [Freeipa-devel] DNS and replication In-Reply-To: <20081105212439.453032e7@notas> References: <4911FE53.6080608@redhat.com> <20081105212439.453032e7@notas> Message-ID: <491201AF.2070708@redhat.com> >> If we in future move to the read only replicas how would DNS updates >> be affected? >> Should we think about this situation now and at least assess how we >> are going to configure DNS in case of the RO replica? >> > > AFAIK, this should not be a concern for us. If you try to write to a RO > replica, it should redirect you to a master. This should be made > transparently. Someone please correct me if I'm wrong about this :) > > You are correct. I am just concerned that if DNS produces a big volume of the updates such redirection might become a real bottleneck. The question is really "should we worry about this becoming a problem"? Is there any concern about this? If not - the discussion is closed. Thanks Dmitri From dpal at redhat.com Wed Nov 5 20:49:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 05 Nov 2008 15:49:07 -0500 Subject: [Freeipa-devel] Need help with Solaris Host Based access control Message-ID: <491206C3.9070204@redhat.com> Hello, As a part of the IPA client configuration in IPA v1.x we allow implementing host based access control. We provide the instructions on how to configure client (actually PAM and NSS) to allow or deny user access to a host based on the information in the IPA back end. The example of such instructions for Linux is: You can configure Linux to allow or deny access to IPA resources and services based on the configuration of the host from which access is attempted. This requires modification to the |/etc/security/access.conf| and |/etc/pam.d/system-auth| files, as described below. 1. Modify the |/etc/security/access.conf| file to include the following lines: + : root : ALL + : ipausers : ALL - : ALL : ALL 2. Modify the |/etc/pam.d/system-auth| file to include the following line: account required pam_access.so This configuration specifies that: * The |root| user can log in. * All members of the |ipausers| group can log in. * IPA administrators can not log in (because the |admin| account is not a member of the |ipausers| group). ========= The instructions are based on the ability of the pam_access PAM module to check the access control rules specified in the access.conf. The group information can be retrieved from the IPA server via nss_ldap. We tried to find similar functionality on other OS's. We spotted PAM modules on HP-UX and AIX that are responsible for the similar authorization checks. But we are stuck with Solaris. All our investigations about similar functionality in Solaris bear no fruits. We saw pam_roles and pam_unix_account on Solaris but they do not seem to accomplish what we are trying to do. We are looking for some help and advice from Solaris experts on this functionality. Thank you, Dmitri From rcritten at redhat.com Wed Nov 5 21:49:57 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 05 Nov 2008 16:49:57 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <20081105172013.GB10119@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> Message-ID: <49121505.8040501@redhat.com> Nalin Dahyabhai wrote: > On Tue, Nov 04, 2008 at 11:22:48PM -0500, Rob Crittenden wrote: >> I'm trying to wrap my head around automount over LDAP and how we would >> want an API over XML-RPC to support it. >> >> At its core there are 2 types of objects we'll deal with: maps and keys >> >> That part is easy enough. I can implement add-map and add-key methods. >> The problem is producing something actually usable to a client. >> >> The thing is, the admin will need a certain amount of automount >> knowledge to create a usable setup. Do we want to mirror the LDAP >> objects or provide an interface to be useful, or both? > > I know it's going to be more work, but you may end up having to do both. > >> So an example is in order. Here is an example of a simple non-home >> shared directory. It creates the maps auto.master and auto.share and >> exports the NFS share /share/builds. >> >> dn: automountmapname=auto.master,cn=accounts,dc=example,dc=com >> objectClass: automountMap >> objectClass: top >> automountMapName: auto.master >> >> dn: >> automountkey=/share,automountmapname=auto.master,cn=accounts,dc=example,dc >> =com >> objectClass: automount >> objectClass: top >> automountKey: /share >> automountInformation: ldap:automountmapname=auto.share, >> cn=accounts,dc=example >> ,dc=com >> >> dn: automountmapname=auto.share,cn=accounts,dc=example,dc=com >> objectClass: automountMap >> objectClass: top >> automountMapName: auto.share >> >> dn: >> automountkey=builds,automountmapname=auto.share,cn=accounts,dc=example,dc= >> com >> automountInformation: somehost:/share/builds >> objectClass: automount >> objectClass: top >> automountKey: builds >> >> So assuming I'm reading this right, and that is a bit of a leap, this >> shows there are two kinds of keys. One defines a search point (/share) >> and one defines a mount point (builds). And it requires knowledge of how >> automount wants to format shares. > > Looks right to me. You'll see fewer maps storing directory paths (only > auto.master and auto.direct come to mind) than subdirectory names. The > first group are typically known as "direct" maps, and the latter group > as "indirect" maps. > >> And also, this assumes we use a schema with automountkey/automountmap >> instead of cn and ou. > > Yes, and it's actually what I prefer. Using these particular attributes > means that keys can be case-sensitive (automountKey is defined to be so > in the schema, while cn and ou are not). Automount map information > stored in files and NIS maps treats the keys as case-sensitive, too, and > any deviation from that behavior when you're moving to LDAP is > surprising (and, it turns out, often upsetting). > >> Adding shares can be left as an exercise to the user by just exporting a >> way to add maps and keys, or we can try to impose some sort of order on >> this. What that might look like I don't know. >> >> Feedback welcome. > > The only opinion I have to offer is this: people do exceptionally weird > things with autofs maps, so the less constraints you force, the happier > people will be. If you can manage to offer default settings that don't > push people toward weirdness, though, the situation might look better in > the future. > Ok, here is some LDIF that will define some defaults. I haven't tackled auto.home yet. dn: cn=automount,$SUFFIX objectClass: nsContainer cn: automount dn: automountmapname=auto.master,cn=automount,$SUFFIX objectClass: automountMap automountMapName: auto.master dn: automountkey=/-,automountmapname=auto.master,cn=automount,$SUFFIX objectClass: automount automountKey: /- automountInformation: ldap:automountmapname=auto.direct,cn=automount,$SUFFIX dn: automountmapname=auto.direct,cn=automount,$SUFFIX objectClass: automountMap automountMapName: auto.direct So adding a new direct mount is as simple as: ipa automount-addkey --key=share --info=server:/share auto.direct Nalin, if I create an auto.home map that handles /home is that going to confuse local accounts who exist on /home? That happens on Solaris and has always driven me nuts. rob From nalin at redhat.com Wed Nov 5 22:34:53 2008 From: nalin at redhat.com (Nalin Dahyabhai) Date: Wed, 5 Nov 2008 17:34:53 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49121505.8040501@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> Message-ID: <20081105223453.GD10119@redhat.com> On Wed, Nov 05, 2008 at 04:49:57PM -0500, Rob Crittenden wrote: > Ok, here is some LDIF that will define some defaults. I haven't tackled > auto.home yet. [snip] > So adding a new direct mount is as simple as: > > ipa automount-addkey --key=share --info=server:/share auto.direct Slick! > Nalin, if I create an auto.home map that handles /home is that going to > confuse local accounts who exist on /home? That happens on Solaris and > has always driven me nuts. Yes. On Linux, at least, /home becomes a mount point for an autofs filesystem, and behaves like other mount points (meaning whatever was there before the filesystem was mounted is hidden until it's unmounted again). You could get around that using the direct map, but that requires more resources, maybe too many. I don't have hard numbers to back it up, but I expect that heavy users of the automounter make sure to either store home directories for local users elsewhere on the filesystem (/export?), or to use some other location for the mountpoint (in school, we used /ncsu). HTH, Nalin From dpal at redhat.com Thu Nov 6 00:02:48 2008 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 05 Nov 2008 19:02:48 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <20081105223453.GD10119@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> Message-ID: <49123428.2050302@redhat.com> Nalin Dahyabhai wrote: > On Wed, Nov 05, 2008 at 04:49:57PM -0500, Rob Crittenden wrote: > >> Ok, here is some LDIF that will define some defaults. I haven't tackled >> auto.home yet. >> > [snip] > >> So adding a new direct mount is as simple as: >> >> ipa automount-addkey --key=share --info=server:/share auto.direct >> > > Slick! > > >> Nalin, if I create an auto.home map that handles /home is that going to >> confuse local accounts who exist on /home? That happens on Solaris and >> has always driven me nuts. >> > > Yes. On Linux, at least, /home becomes a mount point for an autofs > filesystem, and behaves like other mount points (meaning whatever was > there before the filesystem was mounted is hidden until it's unmounted > again). > > You could get around that using the direct map, but that requires more > resources, maybe too many. > > I don't have hard numbers to back it up, but I expect that heavy users > of the automounter make sure to either store home directories for local > users elsewhere on the filesystem (/export?), or to use some other > location for the mountpoint (in school, we used /ncsu). > > In my previous place of work we used to have home directories be mounted to /tmp_mnt/home//.... The /home//... was for local place for local homes on the machine itself. Thanks Dmitri > HTH, > > Nalin > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > From dpal at redhat.com Thu Nov 6 00:14:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 05 Nov 2008 19:14:07 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA Message-ID: <491236CF.6090508@redhat.com> Hello, One of the ideas that we are exploring for IPA v2 is to have sort of ?commit comment? feature. The examples of the use cases are: * Administrator changed and pushed out the policy. As he saves it or applies it (pushes it) he will be prompted to make a comment mainly describing why the operation was performed. * Administrator changed mapping of the users and hosts to a role. He is prompted to make a comment about the operation he performed. * Administrator changed host based access control rule. Same thing as above. These are just several examples of where the commenting on the operation performed might be very important for audit purposes. The main differentiator between the logging of different activity and such feature is that it allows administrator to leave some tips, reasons and answers to questions ?why this or that operation was performed and why it was done this way?. We believe that having such functionality would be very valuable for IPA customers. Bugzilla bug has a very similar functionality. It records not only comments themselves, but who left the comment and when. It also allows modifying comments later on. I have seen the implementations where the comments can't be modified but just added. In our case we would have to implement some similar functionality but in DS. That pauses a challenge. Here are some ideas that I wanted to run by you: 1. I do not think that it would be a good idea to keep the comments separately from the entries being modified. It can be done but I think it would be a database approach to solving the problem, not a DS one. So I suggest that attributes that would be involved in storing this information would be stored in the same entry that gets modified. 2. If it is the same entry but there might be different kinds of entries the attributes that would store comments would be combined into an auxiliary object class. This way we would be able to do comments on any kind of entry we want. It can be role mapping, access control rule, policy or something else we do not have in the system yet. 3. One of the important aspects of such commit log is that it should be ordered can be easily orderable. It is known that multi value attributes can't be ordered. So we have several options: 1. Use one structured attribute and stick everything into it. Though it solves the problem of sorting it makes replication less efficient. One and the same data will be replicated over and over again. 2. Use dynamic number of attributes ? one attribute for a comment. I do not know if it is possible but even sounds like a bad practice even if possible. 3. Use a multi value attribute. If we have a header at the beginning of the mv-attribute that would have a time stamp and sequence number or both, then we would be able to easily sort the attribute values and present them in the right order. We would also minimize replication burden and have a pretty easy reporting mechanism 4. So based on analyses above the MV approach seems like a good starting point. Ok now it makes sense to enforce same behavior related to this attribute(s) regardless whether one uses management interface (UI/CLI) or direct LDAP calls. This means that all the logic related to our commit log object class should be handled via DS plugin. 5. Let us list the operations that the DS should support against the attribute(s) that would store the commit log data: Add: Add a new comment to the end of the list (low level privilege operation) Modify: Change comment in the list (high level privilege operation) Delete: Comment from the list (high level privilege operation) List: Return all entries sorted the order they have been added. (low level privilege operation) Get: Get specific item from the list by some sort of identifier ? needed to be able to modify/replace the right comment. (low level privilege operation) If we do not allow modification of the list then we do not need ?modify?, ?delete? and ?get? operations. Is this acceptable? Then it would be much simpler to create a DS plugin that will just respond to add, and search requests and would reject attempts to delete or modify value. One would be able to delete the whole entry though. 6. The value would be structured like this: 1. sequence number (integer as string left zero padded to 10 characters) 2. length of user CN as left zero padded 4 digit string 3. user CN of the user who created a comment 4. time stamp of creation in YYYYMMDDHHMMSS in UTC 5. time stamp of last modification in YYYYMMDDHHMMSS in UTC 6. comment itself Since the size of the items is fixed or can be easily determined we do not need any separators thus there is no need to escape anything in data itself. 7. To be able to efficiently maintain the list we would need another attribute. This attribute will contain a current max sequence number. That attribute would be updated each time a new comment is added to the list. The plugin will control access to it and prevent any direct modification of this attribute. 8. In future is we decide to change the format we would be able to add other attributes that would contain a version or a format specifier. The logic of the plugin would then be changed to check those attributes and if they are not present it will fall back to original (this format). So I do not see a need to add any format or version attribute in the first implementation. 9. The ldap search will return all the values of the mv-attribute already sorted. It would be a responsibility of the caller to parse parts of it out and present the information to the user in a usable format. Any ideas or comments or suggestions are appreciated. Thank you Dmitri From ssorce at redhat.com Thu Nov 6 06:39:10 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 01:39:10 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491236CF.6090508@redhat.com> References: <491236CF.6090508@redhat.com> Message-ID: <1225953550.10160.99.camel@localhost.localdomain> On Wed, 2008-11-05 at 19:14 -0500, Dmitri Pal wrote: > Hello, > > One of the ideas that we are exploring for IPA v2 is to have sort of > ?commit comment? feature. The examples of the use cases are: > * Administrator changed and pushed out the policy. As he saves it or > applies it (pushes it) he will be prompted to make a comment mainly > describing why the operation was performed. > * Administrator changed mapping of the users and hosts to a role. He is > prompted to make a comment about the operation he performed. > * Administrator changed host based access control rule. Same thing as > above. > > These are just several examples of where the commenting on the operation > performed might be very important for audit purposes. > The main differentiator between the logging of different activity and > such feature is that it allows administrator to leave some tips, reasons > and answers to questions ?why this or that operation was performed and > why it was done this way?. We believe that having such functionality > would be very valuable for IPA customers. > > Bugzilla bug has a very similar functionality. It records not only > comments themselves, but who left the comment and when. It also allows > modifying comments later on. I have seen the implementations where the > comments can't be modified but just added. Up to here I follow you, and I think some users would find somewhat useful to have some sort of process management around high level UI functions. I am not sure a mandatory comment on most operations would be welcome, I think most Admins would just hate such a feature, as it would get in the way of their job, this unless, perhaps, it pops up only once per "macro" operation. If it is not mandatory admins will simply skip it. Suppose Admin A has to change some 20 policies to fix a minor policy bug. If you ask him once to leave a comment, he may grumble, but he would probably do it. If you ask him to leave a comment for each of the 20 policies, he would probably A) throw the keyboard at the monitor after the 3rd one, B) try to find and kill us :), C) post the following comment for all changes: "asdfg". > In our case we would have to implement some similar functionality but in > DS. That pauses a challenge. Ok, here I don't follow you anymore. It's understandable to have audit trails on some operations, but why would we want to store them in DS ?? Can you provide what are the reasons that would make a good idea to store this kind of data in DS instead of a log file or a log database ? I honestly see no reason, nothing but a human operator would make any use of such attributes, they are not useful to any machine which is the real consumer of DS data. As you pointed on it would add an unnecessary amount of data to replicate around. Also because DS is not a logging server but a directory server there are many other problems in trying to use it to store logs. So, while I may agree that some kind of commenting feature could be useful, I really do not see at all DS as the right storage for this stuff. Far from. If comments are required for audit trails I think they should just go in the auditing system and marked as special "comment" logs. Otherwise they should probably simply go in a normal log file or a relational log database (the latter in case online searches are required). Now to some of the reasons why I don't see DS as a viable option: - Multi-value attributes are not ordered, so you need to invent some scheme to store this data structured so that ordering can be preserved. Sure probably using the "posting" date before the content is all is needed, but that makes attributes not searcheable. - You would have to create a clean up process that removes old stuff, I don't think that keeping around a hundred entries log for years would make sense. - We would need to index yet another attribute if you want to make searches on it, note also that just consulting the log would require searches on the identity store increasing its load, something a log file/database would avoid completely. - If you invent a complex format you loose the capability to do decent filtering on searches, meaning you will often need to do wide scope searches and implement filtering in the UI (slooow, and loads DS) - You have no relation of events (ldap not beeing a relational database makes it particularly difficult indeed). - If a single UI command changes many different objects, where do you store the comment? In one? All of them? If in one how do you relate changes to others ? If you replicate it an all objects how do you deal with access to all entries? (see below) See also scenario above about angry admins if you require a comment for each object being changed. - Comments may contain sensitive information that should not be leaked, so comments should not me generally available for search on ldap. This would require to add (on the fly?) ACIs on objects that get comments. - Some ACI may allow a lower level admin to perform an operation on some attributes, but not add objectclasses or new attributes. We loose the comments in this case ? - Anyone with write access to the attribute will be able to change the contents, making them generally completely useless as audit trails. Delegation of any minor task would require write access to comments all over the place. Simo. P.S: please remove the reply-to if you want replies to go to the mailing list by default :-) -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Thu Nov 6 06:45:49 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 06:45:49 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1225953550.10160.99.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> Message-ID: <1225953949.10160.105.camel@localhost.localdomain> On Thu, 2008-11-06 at 01:39 -0500, Simo Sorce wrote: > On Wed, 2008-11-05 at 19:14 -0500, Dmitri Pal wrote: > > Hello, > > > > One of the ideas that we are exploring for IPA v2 is to have sort of > > ?commit comment? feature. The examples of the use cases are: > > * Administrator changed and pushed out the policy. As he saves it or > > applies it (pushes it) he will be prompted to make a comment mainly > > describing why the operation was performed. > > * Administrator changed mapping of the users and hosts to a role. He is > > prompted to make a comment about the operation he performed. > > * Administrator changed host based access control rule. Same thing as > > above. > > > > These are just several examples of where the commenting on the operation > > performed might be very important for audit purposes. > > The main differentiator between the logging of different activity and > > such feature is that it allows administrator to leave some tips, reasons > > and answers to questions ?why this or that operation was performed and > > why it was done this way?. We believe that having such functionality > > would be very valuable for IPA customers. > > > > Bugzilla bug has a very similar functionality. It records not only > > comments themselves, but who left the comment and when. It also allows > > modifying comments later on. I have seen the implementations where the > > comments can't be modified but just added. > > Up to here I follow you, and I think some users would find somewhat > useful to have some sort of process management around high level UI > functions. > I am not sure a mandatory comment on most operations would be welcome, I > think most Admins would just hate such a feature, as it would get in the > way of their job, this unless, perhaps, it pops up only once per "macro" > operation. > If it is not mandatory admins will simply skip it. > > Suppose Admin A has to change some 20 policies to fix a minor policy > bug. If you ask him once to leave a comment, he may grumble, but he > would probably do it. If you ask him to leave a comment for each of the > 20 policies, he would probably A) throw the keyboard at the monitor > after the 3rd one, B) try to find and kill us :), C) post the following > comment for all changes: "asdfg". > > > In our case we would have to implement some similar functionality but in > > DS. That pauses a challenge. > > Ok, here I don't follow you anymore. It's understandable to have audit > trails on some operations, but why would we want to store them in DS ?? > > Can you provide what are the reasons that would make a good idea to > store this kind of data in DS instead of a log file or a log database ? > > I honestly see no reason, nothing but a human operator would make any > use of such attributes, they are not useful to any machine which is the > real consumer of DS data. > As you pointed on it would add an unnecessary amount of data to > replicate around. > > Also because DS is not a logging server but a directory server there are > many other problems in trying to use it to store logs. > > > So, while I may agree that some kind of commenting feature could be > useful, I really do not see at all DS as the right storage for this > stuff. Far from. > > If comments are required for audit trails I think they should just go in > the auditing system and marked as special "comment" logs. Otherwise they > should probably simply go in a normal log file or a relational log > database (the latter in case online searches are required). > > > Now to some of the reasons why I don't see DS as a viable option: > > - Multi-value attributes are not ordered, so you need to invent some > scheme to store this data structured so that ordering can be preserved. > Sure probably using the "posting" date before the content is all is > needed, but that makes attributes not searcheable. > > - You would have to create a clean up process that removes old stuff, I > don't think that keeping around a hundred entries log for years would > make sense. > > - We would need to index yet another attribute if you want to make > searches on it, note also that just consulting the log would require > searches on the identity store increasing its load, something a log > file/database would avoid completely. > > - If you invent a complex format you loose the capability to do decent > filtering on searches, meaning you will often need to do wide scope > searches and implement filtering in the UI (slooow, and loads DS) > > - You have no relation of events (ldap not beeing a relational database > makes it particularly difficult indeed). > > - If a single UI command changes many different objects, where do you > store the comment? In one? All of them? > If in one how do you relate changes to others ? > If you replicate it an all objects how do you deal with access to all > entries? (see below) > See also scenario above about angry admins if you require a comment for > each object being changed. > > - Comments may contain sensitive information that should not be leaked, > so comments should not me generally available for search on ldap. > This would require to add (on the fly?) ACIs on objects that get > comments. > > - Some ACI may allow a lower level admin to perform an operation on some > attributes, but not add objectclasses or new attributes. > We loose the comments in this case ? > > - Anyone with write access to the attribute will be able to change the > contents, making them generally completely useless as audit trails. > Delegation of any minor task would require write access to comments all > over the place. Forgot another important few: - It would make extremely difficult for people to extract this information. Instead of connecting to a well known relational database with well known tools used for reporting, they would have to build a custom parser that speaks LDAP. This thing alone would be a deadly one imo. - The time it will take us to build all the necessary machinery around managing such attributes (I see you even envision a plugin :-O ) would be considerable, and would probably be much better spent on more critical features at this stage imo. (Piping this data in a db from python will take no more than a day or two, building schema, plugin, and all the testing will require weeks). Simo. -- Simo Sorce * Red Hat, Inc * New York From sgallagh at redhat.com Thu Nov 6 12:47:59 2008 From: sgallagh at redhat.com (Stephen Gallagher) Date: Thu, 06 Nov 2008 07:47:59 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49123428.2050302@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <49123428.2050302@redhat.com> Message-ID: <4912E77F.8070705@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> I don't have hard numbers to back it up, but I expect that heavy users >> of the automounter make sure to either store home directories for local >> users elsewhere on the filesystem (/export?), or to use some other >> location for the mountpoint (in school, we used /ncsu). >> >> > In my previous place of work we used to have home directories be mounted > to /tmp_mnt/home//.... > The /home//... was for local place for local homes on the machine > itself. > In most systems I've seen, the home directory server will maintain directories in /export/home/ and they will mount to client machines as /home/ or as /home//. Local home directories tend to either be avoided completely or stored in /opt/home. Now, what we REALLY need (though not for V2) is support for offline home directories. I had an idea about this. If we require that home directories have their own partition, we can configure it such that when online, we mount it to /opt/home and mount the automount directory in /home. We could then run an rsync behind the scenes to keep the two directories in sync. Once the SSSD detects that we have gone offline, it would immediately remount /opt/home as /home and the user could continue. - -- - -------------------- Stephen Gallagher RHCE 804006346421761 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkkS53wACgkQc7MaxVic+2rj2gCeL1vntUQaBPPrU7AASaLcFLvA zSYAnjZU/KeGkpXJaM8NWYwg/e/paJK9 =5H9l -----END PGP SIGNATURE----- From rcritten at redhat.com Thu Nov 6 13:52:20 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 06 Nov 2008 08:52:20 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <20081105223453.GD10119@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> Message-ID: <4912F694.8090309@redhat.com> Nalin Dahyabhai wrote: > On Wed, Nov 05, 2008 at 04:49:57PM -0500, Rob Crittenden wrote: >> Ok, here is some LDIF that will define some defaults. I haven't tackled >> auto.home yet. > [snip] >> So adding a new direct mount is as simple as: >> >> ipa automount-addkey --key=share --info=server:/share auto.direct > > Slick! > >> Nalin, if I create an auto.home map that handles /home is that going to >> confuse local accounts who exist on /home? That happens on Solaris and >> has always driven me nuts. > > Yes. On Linux, at least, /home becomes a mount point for an autofs > filesystem, and behaves like other mount points (meaning whatever was > there before the filesystem was mounted is hidden until it's unmounted > again). > > You could get around that using the direct map, but that requires more > resources, maybe too many. > > I don't have hard numbers to back it up, but I expect that heavy users > of the automounter make sure to either store home directories for local > users elsewhere on the filesystem (/export?), or to use some other > location for the mountpoint (in school, we used /ncsu). > Ok, so how about an API to create indirect maps. Then all one would need to do is add the keys (mounts) needed. As I understand it a new indirect map requires 2 entries, one that defines the map name itself and one that associates the mount point with that map. So it might look something like: % ipa automount-addindirect --root=/export auto.export This will create these two entries: dn: automountmapname=auto.export, cn=automount, $SUFFIX objectclass: automountmap dn: automountkey=/export, automountmapname=auto.master, cn=automount, $SUFFIX objectclass: automount automountinformation: ldap:automountmapname=auth.export, cn=automount, $SUFFIX automountkey=/export Then you could add a new key with: % ipa automount-addkey --key=src --info=server:/exports/stuff/src auto.export And then have /export/src available. If I'm reading this right this should be equivalent to: auto.master: /export auto.share auto.export: src server:/exports/stuff/src What this buys one is that the admin adding the indirect map doesn't need to worry about the details of the automountinformation attribute. Does this make sense? And one more question. If I want to provide some mount options, say -ro,soft, where are those applied? I assume it goes into the attribute automountInformation but is it a prefix or a suffix: So: automountInformation: server:/exports/stuff/src -ro,soft OR automountInformation: -ro,soft server:/exports/stuff/src thanks rob From ssorce at redhat.com Thu Nov 6 14:39:34 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 09:39:34 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <4912F694.8090309@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> Message-ID: <1225982374.10160.117.camel@localhost.localdomain> On Thu, 2008-11-06 at 08:52 -0500, Rob Crittenden wrote: > Nalin Dahyabhai wrote: > > On Wed, Nov 05, 2008 at 04:49:57PM -0500, Rob Crittenden wrote: > >> Ok, here is some LDIF that will define some defaults. I haven't tackled > >> auto.home yet. > > [snip] > >> So adding a new direct mount is as simple as: > >> > >> ipa automount-addkey --key=share --info=server:/share auto.direct > > > > Slick! > > > >> Nalin, if I create an auto.home map that handles /home is that going to > >> confuse local accounts who exist on /home? That happens on Solaris and > >> has always driven me nuts. > > > > Yes. On Linux, at least, /home becomes a mount point for an autofs > > filesystem, and behaves like other mount points (meaning whatever was > > there before the filesystem was mounted is hidden until it's unmounted > > again). > > > > You could get around that using the direct map, but that requires more > > resources, maybe too many. > > > > I don't have hard numbers to back it up, but I expect that heavy users > > of the automounter make sure to either store home directories for local > > users elsewhere on the filesystem (/export?), or to use some other > > location for the mountpoint (in school, we used /ncsu). > > > > Ok, so how about an API to create indirect maps. Then all one would need > to do is add the keys (mounts) needed. > > As I understand it a new indirect map requires 2 entries, one that > defines the map name itself and one that associates the mount point with > that map. > > So it might look something like: > > % ipa automount-addindirect --root=/export auto.export > > This will create these two entries: > > dn: automountmapname=auto.export, cn=automount, $SUFFIX > objectclass: automountmap > > dn: automountkey=/export, automountmapname=auto.master, cn=automount, > $SUFFIX > objectclass: automount > automountinformation: ldap:automountmapname=auth.export, cn=automount, > $SUFFIX > automountkey=/export Where is the server being specified ? > Then you could add a new key with: > > % ipa automount-addkey --key=src --info=server:/exports/stuff/src > auto.export > > And then have /export/src available. > > If I'm reading this right this should be equivalent to: > > auto.master: > /export auto.share what is auto.share? > auto.export: > src server:/exports/stuff/src > > What this buys one is that the admin adding the indirect map doesn't > need to worry about the details of the automountinformation attribute. > > Does this make sense? So to recap, this basically "remaps" where exports appears ? > And one more question. If I want to provide some mount options, say > -ro,soft, where are those applied? I assume it goes into the attribute > automountInformation but is it a prefix or a suffix: > > So: > > automountInformation: server:/exports/stuff/src -ro,soft > > OR > > automountInformation: -ro,soft server:/exports/stuff/src Looking at this discussion, it is unclear to me how geographical situations are handled. If I have IPA server in 3 continents, are we making all clients still try to connect to servers in other continents ? How do clients choose which server to connect to ? Is there any concept like that in automount ? Should we care ? Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Thu Nov 6 15:02:24 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 06 Nov 2008 10:02:24 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1225982374.10160.117.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> Message-ID: <49130700.7040909@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 08:52 -0500, Rob Crittenden wrote: >> Nalin Dahyabhai wrote: >>> On Wed, Nov 05, 2008 at 04:49:57PM -0500, Rob Crittenden wrote: >>>> Ok, here is some LDIF that will define some defaults. I haven't tackled >>>> auto.home yet. >>> [snip] >>>> So adding a new direct mount is as simple as: >>>> >>>> ipa automount-addkey --key=share --info=server:/share auto.direct >>> Slick! >>> >>>> Nalin, if I create an auto.home map that handles /home is that going to >>>> confuse local accounts who exist on /home? That happens on Solaris and >>>> has always driven me nuts. >>> Yes. On Linux, at least, /home becomes a mount point for an autofs >>> filesystem, and behaves like other mount points (meaning whatever was >>> there before the filesystem was mounted is hidden until it's unmounted >>> again). >>> >>> You could get around that using the direct map, but that requires more >>> resources, maybe too many. >>> >>> I don't have hard numbers to back it up, but I expect that heavy users >>> of the automounter make sure to either store home directories for local >>> users elsewhere on the filesystem (/export?), or to use some other >>> location for the mountpoint (in school, we used /ncsu). >>> >> Ok, so how about an API to create indirect maps. Then all one would need >> to do is add the keys (mounts) needed. >> >> As I understand it a new indirect map requires 2 entries, one that >> defines the map name itself and one that associates the mount point with >> that map. >> >> So it might look something like: >> >> % ipa automount-addindirect --root=/export auto.export >> >> This will create these two entries: >> >> dn: automountmapname=auto.export, cn=automount, $SUFFIX >> objectclass: automountmap >> >> dn: automountkey=/export, automountmapname=auto.master, cn=automount, >> $SUFFIX >> objectclass: automount >> automountinformation: ldap:automountmapname=auth.export, cn=automount, >> $SUFFIX >> automountkey=/export > > Where is the server being specified ? On the client. This LDAP notation is defined by autofs. If I understand this correctly (and that is not necessarily a safe assumption), this says "the map is stored in LDAP, so use the LDAP information from your autofs configuration to retrieve it." The format is ldap:dn_of_map >> Then you could add a new key with: >> >> % ipa automount-addkey --key=src --info=server:/exports/stuff/src >> auto.export >> >> And then have /export/src available. >> >> If I'm reading this right this should be equivalent to: >> >> auto.master: >> /export auto.share > > what is auto.share? An indirect automount map. See autofs(5) >> auto.export: >> src server:/exports/stuff/src >> >> What this buys one is that the admin adding the indirect map doesn't >> need to worry about the details of the automountinformation attribute. >> >> Does this make sense? > > So to recap, this basically "remaps" where exports appears ? Yes, this is fairly basic automount configuration. >> And one more question. If I want to provide some mount options, say >> -ro,soft, where are those applied? I assume it goes into the attribute >> automountInformation but is it a prefix or a suffix: >> >> So: >> >> automountInformation: server:/exports/stuff/src -ro,soft >> >> OR >> >> automountInformation: -ro,soft server:/exports/stuff/src > > > Looking at this discussion, it is unclear to me how geographical > situations are handled. If I have IPA server in 3 continents, are we > making all clients still try to connect to servers in other continents ? > > How do clients choose which server to connect to ? Hardcoded in /etc/sysconfig/autofs on Fedora and RHEL. > Is there any concept like that in automount ? Should we care ? AFAIK there is no accomodation for this. We'd either have to provide separate areas (in the dn) to store the maps or the end-user would need to carefully configure things. LDAP for autofs is configured on Fedora (the only thing I've experimented with so far) in /etc/sysconfig/autofs. You can specify the server and the search base. To accomodate geographic areas we could use a separate basedn for each one, something like: cn=australia,cn=autoumount,$SUFFIX cn=baltimore,cn=automount,$SUFFIX autofs issues a query like this when starting up: SRCH base="$SEARCH_BASE_FROM_AUTOFS_CONF" scope=2 filter="(&(objectClass=automountMap)(automountMapName=auto.master))" attrs="automountMapName" So this still wouldn't work if you travel from Europe to the U.S. You wouldn't automatically mount the local servers. rob From jdennis at redhat.com Thu Nov 6 15:49:11 2008 From: jdennis at redhat.com (John Dennis) Date: Thu, 06 Nov 2008 10:49:11 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491236CF.6090508@redhat.com> References: <491236CF.6090508@redhat.com> Message-ID: <491311F7.1090808@redhat.com> I agree capturing this information is useful however I also agree with Simo's comments in his follow-up. We do have to make sure the comment capture is friendly for admins. DS is the wrong place to store this information for all the reasons Simo pointed out. A relational database might make more sense as a repository but that is extremely heavy weight just for this problem, we really don't want to be installing and maintaining a database just for this purpose. I think Simo's idea of logging the comments to a log file is the right idea. It's simple, easy to manage, could be served up by a simple CGI script via the Web UI and can be a candidate for auditing by the IPA Audit component which would allow for subsequent indexing and search. -- John Dennis From dpal at redhat.com Thu Nov 6 15:56:42 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 10:56:42 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1225953949.10160.105.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> Message-ID: <491313BA.1060303@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 01:39 -0500, Simo Sorce wrote: > >> On Wed, 2008-11-05 at 19:14 -0500, Dmitri Pal wrote: >> >>> Hello, >>> >>> One of the ideas that we are exploring for IPA v2 is to have sort of >>> ?commit comment? feature. The examples of the use cases are: >>> * Administrator changed and pushed out the policy. As he saves it or >>> applies it (pushes it) he will be prompted to make a comment mainly >>> describing why the operation was performed. >>> * Administrator changed mapping of the users and hosts to a role. He is >>> prompted to make a comment about the operation he performed. >>> * Administrator changed host based access control rule. Same thing as >>> above. >>> >>> These are just several examples of where the commenting on the operation >>> performed might be very important for audit purposes. >>> The main differentiator between the logging of different activity and >>> such feature is that it allows administrator to leave some tips, reasons >>> and answers to questions ?why this or that operation was performed and >>> why it was done this way?. We believe that having such functionality >>> would be very valuable for IPA customers. >>> >>> Bugzilla bug has a very similar functionality. It records not only >>> comments themselves, but who left the comment and when. It also allows >>> modifying comments later on. I have seen the implementations where the >>> comments can't be modified but just added. >>> >> Up to here I follow you, and I think some users would find somewhat >> useful to have some sort of process management around high level UI >> functions. >> Yes I am talking about the situations when a change requires a formal process. I was surprised when I faced the reality: big companies especially banks do not do any changes until it has been approved, scheduled, verified etc. It is a complex process. This is the situation when the "commit log" feature is mostly valuable. Put it differently it is for the environment where the administrator is not allowed to do the changes to the system when and how he thinks appropriate but rather has to follow a special procedure. There are a lot of such companies. >> I am not sure a mandatory comment on most operations would be welcome, I >> think most Admins would just hate such a feature, as it would get in the >> way of their job, this unless, perhaps, it pops up only once per "macro" >> operation. >> If it is not mandatory admins will simply skip it. >> >> Suppose Admin A has to change some 20 policies to fix a minor policy >> bug. If you ask him once to leave a comment, he may grumble, but he >> would probably do it. If you ask him to leave a comment for each of the >> 20 policies, he would probably A) throw the keyboard at the monitor >> after the 3rd one, B) try to find and kill us :), C) post the following >> comment for all changes: "asdfg". >> As I mentioned above if administrator has the right to make changes without comments this feature should be turned off. But I am more focusing on the cases when the feature is need. Then it can be potentially configured as mandatory comment at least for some critical operations. Another point is that as administrator if you went into the system and changed SUDO policy for the enterprise that created a securtiy hole you better leave a comment explaining why you did this change. Just a precaution to :-) Definitely it should be selective and configurable. I might not been clear about this. So: 1) The feature is optional 2) It can be configured for which DS objects it should be mandatory and for which not. I think that policy, HBAC rule or role mapping objects are good candidates to be "commentable" out of box. Probably not mandatory comments though. >> >>> In our case we would have to implement some similar functionality but in >>> DS. That pauses a challenge. >>> >> Ok, here I don't follow you anymore. It's understandable to have audit >> trails on some operations, but why would we want to store them in DS ?? >> >> Well, this is the main point. It would have been great if we had a product that would be able to act as DS and DB at the same time. I agree that this information would easier be managed in a DB. But we do not have one. We can create one but then we face all the same issues as we face with policies: a) Replication b) Referential integrity across different data stores c) Backup - restore With policies the decision was made to store them in DS not to face these problems. I think that creating a parallel infrastructure for policies or for commit logs is a huge overhead. >> Can you provide what are the reasons that would make a good idea to >> store this kind of data in DS instead of a log file or a log database ? >> >> Log file is not manageable. It is hard to pull and query. It is good as a stream of data but when you need to analyze it you use other tools. Log database is different. It is for different purpose. Log database is for analysis. It would be hard to relate the history or a DS entry to the entry itself and present them in proper way in UI. It would be much more complex than you think. >> I honestly see no reason, nothing but a human operator would make any >> use of such attributes, they are not useful to any machine which is the >> real consumer of DS data. >> I completely disagree with trhis statement. DS is a data store for all data that needs to be stored unless it does not belong there. The "commit log" data is on the edge but there is unfortunately no better place to put it. >> As you pointed on it would add an unnecessary amount of data to >> replicate around. >> >> If we use MV attributes the replication problem will be solved. >> Also because DS is not a logging server but a directory server there are >> many other problems in trying to use it to store logs. >> >> >> This is not a log. I think this is a fundamental difference. It is a set of comments. IMO they should be tightly related to the object they refer to and not be somewhere else in the log server. The log will contain who did what when. But it never answers the question "why". The "commit comment" feature is designed for that use case. May be the word selection confused you but it is not a "log". Think about comments in bugzilla bug. Bugzilla bug is not a log and never perceived as such. "commit comment" feature should not be too. >> So, while I may agree that some kind of commenting feature could be >> useful, I really do not see at all DS as the right storage for this >> stuff. Far from. >> I agree but I do not see a better alternative. Implementing an external DB synchronized with DS is much more complex than you would think. >> If comments are required for audit trails I think they should just go in >> the auditing system and marked as special "comment" logs. Otherwise they >> should probably simply go in a normal log file or a relational log >> database (the latter in case online searches are required). >> As I said they are not a log and this approach will make the feature useless. >> >> Now to some of the reasons why I don't see DS as a viable option: >> >> - Multi-value attributes are not ordered, so you need to invent some >> scheme to store this data structured so that ordering can be preserved. >> Sure probably using the "posting" date before the content is all is >> needed, but that makes attributes not searcheable. >> >> I think that creating a generic plugin that would allow storing ordered MV attributes would be a big benefit for everybody. We can use the "commit comment" use case to create one. I would think that DS folks would find such plugin pretty valuable. I am actually surprised to the fact that one does not exist yet. It would have solved a lot of different issues and paved a way for even broader adoption of the DS. I would start with implementing "commit comment" feature as a step towards a generic ordered MV value plugin solution. This would actually mean turning a MV attribute into sort of mini table with records associated with an entry in this table. I think it is a very cool feature. >> - You would have to create a clean up process that removes old stuff, I >> don't think that keeping around a hundred entries log for years would >> make sense. >> I am not sure it should be done day one. But I will think more about cleanup. I view this as a second tear functionality on top of the original feature. >> - We would need to index yet another attribute if you want to make >> searches on it, note also that just consulting the log would require >> searches on the identity store increasing its load, something a log >> file/database would avoid completely. >> >> The whole value of the feature is to have the whole list. There is no need to index it since the search by this attribute does not make sense. This is not a valid concern. >> - If you invent a complex format you loose the capability to do decent >> filtering on searches, meaning you will often need to do wide scope >> searches and implement filtering in the UI (slooow, and loads DS) >> >> Again as I said: there is no need to filter by this attribute. You will pull this MV attribute if you need it. Searching inside the values pulled out is up to the application not to DS. >> - You have no relation of events (ldap not beeing a relational database >> makes it particularly difficult indeed). >> >> There is no relation to the event. There is a relation the object itself since the attribute that will contain the commit comments will be a part of the same entry. There is no need to have any kind of cross references if it is done this way. >> - If a single UI command changes many different objects, where do you >> store the comment? In one? All of them? >> If in one how do you relate changes to others ? >> If you replicate it an all objects how do you deal with access to all >> entries? (see below) >> See also scenario above about angry admins if you require a comment for >> each object being changed. >> >> The "commit comment" feature by nature makes sense in the context of the top level object as i mentioned above. If change happens to several entries at once the designer of the schema and UI should decide what would be the best approach and what entry the "commit comment" should be applied to. If we do it as auxiliary class we have a flexibility to use it in multiple places. I really do not see a problem here. You just pick the main entry you are dealing with. If the UI touches multiple entries of different kind in one step it is a subject for deep though and reevaluation. Such operations especially with DS where transactions are not supported should be avoided. >> - Comments may contain sensitive information that should not be leaked, >> so comments should not me generally available for search on ldap. >> This would require to add (on the fly?) ACIs on objects that get >> comments. >> >> The ACI defines the access control rules for the MV attribute. I doubt that thew comments would contain sensitive information. This is a matter of administrative policies not software. The first step is at least to treat it as a MV attribute with ordering that can't be modified or deleted. Later as a second tear feature we can start thinking about more discretionary read access control. Again I do not see a big issue here for the first implementation. >> - Some ACI may allow a lower level admin to perform an operation on some >> attributes, but not add objectclasses or new attributes. >> We loose the comments in this case ? >> No. I will dig into that but the plan is to have consistent ACI rules. I hope that DS specialists will chime in and confirm that the ACI has enough flexibility to deal with the object I am suggesting. >> - Anyone with write access to the attribute will be able to change the >> contents, making them generally completely useless as audit trails. >> Delegation of any minor task would require write access to comments all >> over the place. >> > > No. The whole idea is to make it non-editable at all. Only add. Only later we might start diving into ACIs and deal with the complexity of editing the data by admins that have different levels of privileges. > Forgot another important few: > > - It would make extremely difficult for people to extract this > information. Instead of connecting to a well known relational database > with well known tools used for reporting, they would have to build a > custom parser that speaks LDAP. This thing alone would be a deadly one > imo. > > This is not a log. This is a comment on the entry. I think that if down the road we create a mean to store ordered lists in DS people would take advantage of that. I disagree that it is hard to extract. It is the same as any other MV attribute except that there is some header inside that prefixes the data that need to be skipped. There are so many LDAP attributes that have special internal formats that I read about in different RFC I really do not see my approach being against any main stream ways of doing things. There can be a helper library written for an easier adoption later. This is really not an issue at all imo. > - The time it will take us to build all the necessary machinery around > managing such attributes (I see you even envision a plugin :-O ) would > be considerable, and would probably be much better spent on more > critical features at this stage imo. (Piping this data in a db from > python will take no more than a day or two, building schema, plugin, and > all the testing will require weeks). > Based on the experience with Nalin and my reading of DS documentation and reading DS plugin code it seems a pretty straightforward task. It is powers of magnitude simpler than any external database. I think that it is worth a try. If I see that it takes much more time than I think I might defer this till later. But a decision needs to be made pretty soon. This is why I am bringing it up. Mike Langlie is building the UI screens prototype and we (he and I) need to understand whether this feature will be a part of the UI or not so that we can prepare properly for usability testing we are planning to conduct. Thanks Dmitri > Simo. > > From rmeggins at redhat.com Thu Nov 6 16:13:51 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 06 Nov 2008 09:13:51 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491236CF.6090508@redhat.com> References: <491236CF.6090508@redhat.com> Message-ID: <491317BF.6060108@redhat.com> Dmitri Pal wrote: > Hello, > > One of the ideas that we are exploring for IPA v2 is to have sort of > ?commit comment? feature. The examples of the use cases are: > * Administrator changed and pushed out the policy. As he saves it or > applies it (pushes it) he will be prompted to make a comment mainly > describing why the operation was performed. > * Administrator changed mapping of the users and hosts to a role. He > is prompted to make a comment about the operation he performed. > * Administrator changed host based access control rule. Same thing as > above. > > These are just several examples of where the commenting on the > operation performed might be very important for audit purposes. > The main differentiator between the logging of different activity and > such feature is that it allows administrator to leave some tips, > reasons and answers to questions ?why this or that operation was > performed and why it was done this way?. We believe that having such > functionality would be very valuable for IPA customers. > > Bugzilla bug has a very similar functionality. It records not only > comments themselves, but who left the comment and when. It also allows > modifying comments later on. I have seen the implementations where the > comments can't be modified but just added. > > In our case we would have to implement some similar functionality but > in DS. That pauses a challenge. > Here are some ideas that I wanted to run by you: > > 1. I do not think that it would be a good idea to keep the comments > separately from the entries being modified. It can be done but I > think it would be a database approach to solving the problem, not > a DS one. So I suggest that attributes that would be involved in > storing this information would be stored in the same entry that > gets modified. > 2. If it is the same entry but there might be different kinds of > entries the attributes that would store comments would be combined > into an auxiliary object class. This way we would be able to do > comments on any kind of entry we want. It can be role mapping, > access control rule, policy or something else we do not have in > the system yet. > 3. One of the important aspects of such commit log is that it should > be ordered can be easily orderable. It is known that multi value > attributes can't be ordered. So we have several options: > 1. Use one structured attribute and stick everything into it. > Though it solves the problem of sorting it makes replication > less efficient. One and the same data will be replicated > over and over again. > 2. Use dynamic number of attributes ? one attribute for a > comment. I do not know if it is possible but even sounds > like a bad practice even if possible. > 3. Use a multi value attribute. If we have a header at the > beginning of the mv-attribute that would have a time stamp > and sequence number or both, then we would be able to easily > sort the attribute values and present them in the right > order. We would also minimize replication burden and have a > pretty easy reporting mechanism 3.4. OpenLDAP supports an extension by which you can create an ordered list of values in attributes - each value is prefixed by {N} where N is a number. I think it's done in such a way that clients that understand ordering can request the values with the prefix, and dumb clients get the raw values. Fedora DS does not support this, but perhaps we should? There is an Internet Draft (or perhaps an informational RFC) about this feature. > > 4. So based on analyses above the MV approach seems like a good > starting point. Ok now it makes sense to enforce same behavior > related to this attribute(s) regardless whether one uses > management interface (UI/CLI) or direct LDAP calls. This means > that all the logic related to our commit log object class should > be handled via DS plugin. > 5. Let us list the operations that the DS should support against the > attribute(s) that would store the commit log data: > > Add: Add a new comment to the end of the list (low level privilege > operation) > Modify: Change comment in the list (high level privilege operation) > Delete: Comment from the list (high level privilege operation) > List: Return all entries sorted the order they have been added. (low > level privilege operation) > Get: Get specific item from the list by some sort of identifier ? > needed to be able to modify/replace the right comment. (low level > privilege operation) > If we do not allow modification of the list then we do not need > ?modify?, ?delete? and ?get? operations. Is this acceptable? Then it > would be much simpler to create a DS plugin that will just respond to > add, and search requests and would reject attempts to delete or modify > value. One would be able to delete the whole entry though. > > 6. The value would be structured like this: > 1. sequence number (integer as string left zero padded to 10 > characters) > 2. length of user CN as left zero padded 4 digit string > 3. user CN of the user who created a comment > 4. time stamp of creation in YYYYMMDDHHMMSS in UTC > 5. time stamp of last modification in YYYYMMDDHHMMSS in UTC > 6. comment itself > Since the size of the items is fixed or can be easily determined > we do not need any separators thus there is no need to escape > anything in data itself. > 7. To be able to efficiently maintain the list we would need another > attribute. This attribute will contain a current max sequence > number. That attribute would be updated each time a new comment is > added to the list. The plugin will control access to it and > prevent any direct modification of this attribute. > 8. In future is we decide to change the format we would be able to > add other attributes that would contain a version or a format > specifier. The logic of the plugin would then be changed to check > those attributes and if they are not present it will fall back to > original (this format). So I do not see a need to add any format > or version attribute in the first implementation. > 9. The ldap search will return all the values of the mv-attribute > already sorted. It would be a responsibility of the caller to > parse parts of it out and present the information to the user in a > usable format. > > Any ideas or comments or suggestions are appreciated. This sounds a lot like the information we keep in the replication changelog, and to a certain extent the replication state information we keep in the entry itself. Also in the retro changelog. If we were to do something like this, it would be worth revisiting the current internal change notification code that we use in various places (post ops, persistent search, replication changelog, retro changelog, audit log, etc.) so that we could have a general purpose internal change notification service to be used by all of these various "sinks" and more. > > Thank you > Dmitri > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From dpal at redhat.com Thu Nov 6 17:13:11 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 12:13:11 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491317BF.6060108@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> Message-ID: <491325A7.7030704@redhat.com> > 3.4. OpenLDAP supports an extension by which you can create an > ordered list of values in attributes - each value is prefixed by {N} > where N is a number. I think it's done in such a way that clients > that understand ordering can request the values with the prefix, and > dumb clients get the raw values. Fedora DS does not support this, but > perhaps we should? There is an Internet Draft (or perhaps an > informational RFC) about this feature. I will explore this more. I think the fact that OpenLDAP has it proves that there is a demand for such feature. It would be nice for DS to support it. But I think it would be harder to deal with if in the first implementation we push it to DS to take care of. The better approach IMO would be to create an IPA plugin in v2 and then use it as a base for DS plugin in follow up DS version. This way we will avoid creating dependencies on DS and forcing the plugin to be perfect from day one. It would be much harder to fix issues with it if it is a part of DS. >> > This sounds a lot like the information we keep in the replication > changelog, and to a certain extent the replication state information > we keep in the entry itself. Also in the retro changelog. > > If we were to do something like this, it would be worth revisiting the > current internal change notification code that we use in various > places (post ops, persistent search, replication changelog, retro > changelog, audit log, etc.) so that we could have a general purpose > internal change notification service to be used by all of these > various "sinks" and more. May be. But I am a bit scared going that far right away. This starts to sound like a major feature. Let us take incremental steps and start with small tasks. I do not want to boil an ocean. Eventually I might but this is to far fetched. I want to solve a specific problem and implement a specific feature. But I want to write it so that it can be reused for other purposes and become more generic. I view the "commit comment" feature as a send box for a more generic implementation down the road. Rich, would you agree with such approach? >> >> Thank you >> Dmitri >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > From ssorce at redhat.com Thu Nov 6 17:13:21 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 17:13:21 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491313BA.1060303@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> Message-ID: <1225991601.10160.212.camel@localhost.localdomain> On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: > Simo Sorce wrote: > Well, this is the main point. It would have been great if we had a > product that would be able to act as DS and DB at the same time. > I agree that this information would easier be managed in a DB. But we do > not have one. We can create one but then we face all the same issues as > we face with policies: > a) Replication Why do we need to replicate a log database? It is not vital for the normal functioning of the Identity solution so it can well be a database or a file kept on a single server > b) Referential integrity across different data stores Use object GUIDs, that's all is needed, seriously, it is very simple to keep referential integrity when GUIDs are used. > c) Backup - restore Not sure why this would make any difference. We do not offer any special backup/restore feature with IPA so it would be just business as usual. > With policies the decision was made to store them in DS not to face > these problems. Policies will be constantly queried by clients, that's why you need to replicate them on multiple servers to be able to scale up when the number of clients grow. A log does not need that. > I think that creating a parallel infrastructure for policies or for > commit logs is a huge overhead. I think logs and policies are completely different in nature, these logs are just audit trails in nature, and should be rather be developed as part of the audit system. > >> Can you provide what are the reasons that would make a good idea to > >> store this kind of data in DS instead of a log file or a log database ? > >> > >> > Log file is not manageable. It is hard to pull and query. About the same complexity of pulling an query a strctured attribute in LDAP. To be honesyt for most people a file is *much* more easier to pull and parse than an ldap attribute. > It is good as > a stream of data but when you need to analyze it you use other tools. > Log database is different. It is for different purpose. Log database is > for analysis. It would be hard to relate the history or a DS entry to > the entry itself and present them in proper way in UI. I don't think so, as long as you store the GUID of the entry it refers to as part of the metadata of the comment/log entry it is quite easy, just a single ldap search (or vice versa, just a single "grep") > It would be much more complex than you think. I honestly think you greatly over-estimate the complexity of parsing a log file in a modern scripting language like python, or even just old stuff like grep, cut or awk. > >> I honestly see no reason, nothing but a human operator would make any > >> use of such attributes, they are not useful to any machine which is the > >> real consumer of DS data. > >> > > I completely disagree with trhis statement. DS is a data store for all > data that needs to be stored unless it does not belong there. The point is that comments or log do not belong there, they just don't. > The "commit log" data is on the edge but there is unfortunately no > better place to put it. A log file or a log database, it's auditing related we will have auditing facilities, use them. > >> As you pointed on it would add an unnecessary amount of data to > >> replicate around. > >> > >> > If we use MV attributes the replication problem will be solved. NO, it will just be softened, you still replicate around data useless 99% of the time. > >> Also because DS is not a logging server but a directory server there are > >> many other problems in trying to use it to store logs. > >> > >> > >> > This is not a log. I think this is a fundamental difference. It is a set > of comments. Sorry but I see no difference between a set of "comments" or a log, a log is just a set of "comments" about what is going on in some generic "process". > IMO they should be tightly related to the object they refer > to and not be somewhere else in the log server. Why? > The log will contain who did what when. But it never answers the > question "why". If you put a comment in a log it can also answer why, I really do not see your point here. > The "commit comment" feature is designed for that use case. > May be the word selection confused you but it is not a "log". No, the nature of a comment is the same as that of a log, it is just that the "comment" is user generated instead of automatically generated but it really bears no relevant semantic difference. > Think about comments in bugzilla bug. Bugzilla bug is not a log and > never perceived as such. "commit comment" feature should not be too. Bugzilla is used to have "conversations" about topics. The "commit comment" is instead just the same as a commit message in an SCM system. Not surprisingly to see the list of commit comments in cvs, svn,git, etc.. the command is "log" (cvs log, svn log, git log ....) They are just logs. > I agree but I do not see a better alternative. Implementing an external > DB synchronized with DS is much more complex than you would think. I beg to strongly disagree. All you need is to use GUIDs on objects, that is your "join" key. It makes thing extremely simple to manage. > >> If comments are required for audit trails I think they should just go in > >> the auditing system and marked as special "comment" logs. Otherwise they > >> should probably simply go in a normal log file or a relational log > >> database (the latter in case online searches are required). > >> > > As I said they are not a log and this approach will make the feature > useless. Evidently there is a problem in understanding what this feature is useful for. >From your premises about why customers need it, it is useful as an audit trail log to know why an object is in that state. Is there any other use to it ? > >> Now to some of the reasons why I don't see DS as a viable option: > >> > >> - Multi-value attributes are not ordered, so you need to invent some > >> scheme to store this data structured so that ordering can be preserved. > >> Sure probably using the "posting" date before the content is all is > >> needed, but that makes attributes not searcheable. > >> > >> > > I think that creating a generic plugin that would allow storing ordered > MV attributes would be a big benefit for everybody. The problem is that ordered MV are not defined in the LDAP protocol, you would need to come up with a standardizable way to manage how to add entries so that they fall in the precise order you need them to. This would mean modifying the add operation either with a control or by creating an extended operation. > We can use the "commit comment" use case to create one. I would think > that DS folks would find such plugin pretty valuable. > I am actually surprised to the fact that one does not exist yet. It > would have solved a lot of different issues and paved a way for even > broader adoption of the DS. This is *way* more work than you think it is. And the reason why nobody did it and no clear standard has still been proposed. > I would start with implementing "commit comment" feature as a step > towards a generic ordered MV value plugin solution. This alone would require considerable time. > This would actually mean turning a MV attribute into sort of mini table > with records associated with an entry in this table. > I think it is a very cool feature. Yes, but would mean modification of the LDAP protocol, you do not want to do that outside of a standardization body, or it will most probably be just a dead end. > >> - You would have to create a clean up process that removes old stuff, I > >> don't think that keeping around a hundred entries log for years would > >> make sense. > >> > I am not sure it should be done day one. But I will think more about > cleanup. > I view this as a second tear functionality on top of the original feature. > > >> - We would need to index yet another attribute if you want to make > >> searches on it, note also that just consulting the log would require > >> searches on the identity store increasing its load, something a log > >> file/database would avoid completely. > >> > >> > > The whole value of the feature is to have the whole list. There is no > need to index it since the search by this attribute does not make sense. > This is not a valid concern. Can you explain what this list is used for actually? And why someone should be interested in a comment written 5 years earlier and that is completely outdated as the data in the object has no bearing with that original comment anymore? > >> - If you invent a complex format you loose the capability to do decent > >> filtering on searches, meaning you will often need to do wide scope > >> searches and implement filtering in the UI (slooow, and loads DS) > >> > >> > Again as I said: there is no need to filter by this attribute. You will > pull this MV attribute if you need it. Searching inside the values > pulled out is up to the application not to DS. This just tells that storing it in DS is really not required. If you need it you can pull it from a DB or a file equally well. > >> - You have no relation of events (ldap not beeing a relational database > >> makes it particularly difficult indeed). > >> > >> > > There is no relation to the event. There is a relation the object itself > since the attribute that will contain the commit comments will be a part > of the same entry. A comment is related to a "change" in an object, therefore it is logically related to an event (the change), insomuch that you require ordering of the entries. Now assume I have a set of events and related comments, like the following: "add foo to sudoers file" "foo need access to x as admin" "add bar to sudoers file" "bar need access to x as admin" "remove foo from sudoers file" "foo need no more access as admin" "add baz to sudoers file" "baz need access as admin" "remove baz from sudoers file" "baz need no more access as admin" ... Now who needs the full list ? After 30-40 changes why do you care about something like lines 4 and 5 except for auditing purposes ? And if you do not have a date associated what is it useful for ? > There is no need to have any kind of cross references if it is done this > way. If there are no cross references like the date the event/change happened or who made the operation it would be quite useless (see above example). > >> - If a single UI command changes many different objects, where do you > >> store the comment? In one? All of them? > >> If in one how do you relate changes to others ? > >> If you replicate it an all objects how do you deal with access to all > >> entries? (see below) > >> See also scenario above about angry admins if you require a comment for > >> each object being changed. > >> > >> > The "commit comment" feature by nature makes sense in the context of the > top level object as i mentioned above. > If change happens to several entries at once the designer of the schema > and UI should decide what would be the best approach and what entry the > "commit comment" should be applied to. This is an operational policy, can't be decided at the schema level, it completely depends on what kind of event the specific security administrator wants in the specific deployment. > If we do it as auxiliary class we have a flexibility to use it in > multiple places. SCNR but I read this like: we can clutter the DS as much as we want :) > I really do not see a problem here. You just pick the main entry you are > dealing with. If the UI touches multiple entries of different kind in > one step it is a subject for deep though and reevaluation. It will, I am trying to evaluate this *before* we waste a lot of time in something that seem wrong to me on way too many levels to easily express them all. for some aspects this is just a feeling, but so far, the further we dig into it the more the feeling grows. > Such operations especially with DS where transactions are not supported > should be avoided. Reality is people will touch multiple objects when changing stuff around, and transactions really do not matter in this case (see the angry-admin example I posted in the previous email). > >> - Comments may contain sensitive information that should not be leaked, > >> so comments should not me generally available for search on ldap. > >> This would require to add (on the fly?) ACIs on objects that get > >> comments. > >> > >> > The ACI defines the access control rules for the MV attribute. I doubt > that thew comments would contain sensitive information. I don't think ACIs can express something like "append-only". > This is a matter of administrative policies not software. Sorry I do not get what this means. We are talking about access authorization to an attribute that seem to me to be framed in a way the software can't cope with, I think software matters. > The first step > is at least to treat it as a MV attribute with ordering that can't be > modified or deleted. If you can write it you can modify it. > Later as a second tear feature we can start thinking about more > discretionary read access control. Again I do not see a big issue here > for the first implementation. > > >> - Some ACI may allow a lower level admin to perform an operation on some > >> attributes, but not add objectclasses or new attributes. > >> We loose the comments in this case ? > >> > No. I will dig into that but the plan is to have consistent ACI rules. > I hope that DS specialists will chime in and confirm that the ACI has > enough flexibility to deal with the object I am suggesting. I don't think so, if I correctly understood how you want to manage it. And btw the way you seem to be willing to manage it once again resembles an audit trail log, and that's I think because it ultimately is just an audit trail log. > >> - Anyone with write access to the attribute will be able to change the > >> contents, making them generally completely useless as audit trails. > >> Delegation of any minor task would require write access to comments all > >> over the place. > >> > > > > > No. The whole idea is to make it non-editable at all. Only add. Exactly this is something that does not exist in the LDAP model, nor in the ACI model we have. > Only > later we might start diving into ACIs and deal with the complexity of > editing the data by admins that have different levels of privileges. No, architectural problems must be evaluated first in this case, because you are trying to construct something that is so off the way LDAP works for something that seem so irrelevant for the general functioning that I want to understand exactly why it is so important, Because from this reading I just re-evaluated the time need to "adjust" DS to handle this single attribute in term of several weeks, and that's a lot of effort, just to keep around some log that can instead very easily be piped in a file or in a database with an effort that will take a lot less time. > > Forgot another important few: > > > > - It would make extremely difficult for people to extract this > > information. Instead of connecting to a well known relational database > > with well known tools used for reporting, they would have to build a > > custom parser that speaks LDAP. This thing alone would be a deadly one > > imo. > > > > > > This is not a log. This is a comment on the entry. I am sorry, the description you give: ordered list, immutable list, add only, all scream this is a log. If it were just a comment about the object we would need nothing more that the "description" attribute already available in ldap, and there would be no ordering nor immutability problem. I think you might be trying to conflate together the concepts of "description of the object" and "log of changes", I think this is a very wrong approach. > I think that if down > the road we create a mean to store ordered lists in DS people would take > advantage of that. > I disagree that it is hard to extract. It is the same as any other MV > attribute except that there is some header inside that prefixes the data > that need to be skipped. > There are so many LDAP attributes that have special internal formats > that I read about in different RFC I really do not see my approach being > against any main stream ways of doing things. > There can be a helper library written for an easier adoption later. This > is really not an issue at all imo. The issue here is why we want to waste a lot of time and effort to implement just a per entry log that is useful only when some auditing need to be performed. Seriously I think the effort is not worth the value from what I have seen so far. > > - The time it will take us to build all the necessary machinery around > > managing such attributes (I see you even envision a plugin :-O ) would > > be considerable, and would probably be much better spent on more > > critical features at this stage imo. (Piping this data in a db from > > python will take no more than a day or two, building schema, plugin, and > > all the testing will require weeks). > > > Based on the experience with Nalin and my reading of DS documentation > and reading DS plugin code it seems a pretty straightforward task. > It is powers of magnitude simpler than any external database. I think you really are greatly underestimating the complexity of writing a DS plugin, the complexity of dealing with changing LDAP semantics. And the development cycle involved with all the bugs custom code would entail. A plugin also always adds security concerns as it runs as a privileged process wrt DS data. It add stability concerns as it runs inside a threaded application (one segfault and goodbye LDAP server). For a database all you need is a schema and a few SQL queries to insert/extract data. you don't have ordering problems there, you can split data and metadata and have own tables. Writing an small database schema with a couple of tables is honestly orders of magnitude simpler than writing a plugin in C that has to do what you would like it to do. > I think that it is worth a try. If I see that it takes much more time > than I think I might defer this till later. > But a decision needs to be made pretty soon. This is why I am bringing > it up. So far none of the arguments given convince me the huge effort required make sense. If I'd have to vote now, I'd say no. > Mike Langlie is building the UI screens prototype and we (he and I) need > to understand whether this feature will be a part of the UI or not so > that we can prepare properly for usability testing we are planning to > conduct. Maybe you can explain how the UI would use this information, that may shed some more light on what is the appropriate way to manage these user-generated-logs. Simo. -- Simo Sorce * Red Hat, Inc * New York From nkinder at redhat.com Thu Nov 6 17:18:31 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Thu, 06 Nov 2008 09:18:31 -0800 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491313BA.1060303@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> Message-ID: <491326E7.6050403@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Thu, 2008-11-06 at 01:39 -0500, Simo Sorce wrote: >> >>> On Wed, 2008-11-05 at 19:14 -0500, Dmitri Pal wrote: >>> >>> >>> Now to some of the reasons why I don't see DS as a viable option: >>> >>> - Multi-value attributes are not ordered, so you need to invent some >>> scheme to store this data structured so that ordering can be preserved. >>> Sure probably using the "posting" date before the content is all is >>> needed, but that makes attributes not searcheable. >>> >>> > > I think that creating a generic plugin that would allow storing > ordered MV attributes would be a big benefit for everybody. > We can use the "commit comment" use case to create one. I would think > that DS folks would find such plugin pretty valuable. > I am actually surprised to the fact that one does not exist yet. It > would have solved a lot of different issues and paved a way for even > broader adoption of the DS. There is a proposed standard for ordering of attributes: https://opends.dev.java.net/public/standards/draft-chu-ldap-xordered.txt > > I would start with implementing "commit comment" feature as a step > towards a generic ordered MV value plugin solution. > This would actually mean turning a MV attribute into sort of mini > table with records associated with an entry in this table. > I think it is a very cool feature. > > > > Thanks > Dmitri > >> Simo. >> >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ssorce at redhat.com Thu Nov 6 17:18:39 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 12:18:39 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49130700.7040909@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> Message-ID: <1225991919.10160.217.camel@localhost.localdomain> On Thu, 2008-11-06 at 10:02 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > How do clients choose which server to connect to ? > > Hardcoded in /etc/sysconfig/autofs on Fedora and RHEL. > > > Is there any concept like that in automount ? Should we care ? > > AFAIK there is no accomodation for this. We'd either have to provide > separate areas (in the dn) to store the maps or the end-user would need > to carefully configure things. I think providing separate areas then is paramount. Admins should be able to define "locations" and the maps would be created inside these location. This way admins can set different automount options for clients located in different places. I am sure clients in Australia are not going to use the same automount maps as for clients in Baltimore. > LDAP for autofs is configured on Fedora (the only thing I've > experimented with so far) in /etc/sysconfig/autofs. You can specify the > server and the search base. > > To accomodate geographic areas we could use a separate basedn for each > one, something like: > > cn=australia,cn=autoumount,$SUFFIX > cn=baltimore,cn=automount,$SUFFIX Yes I think this is needed. After if someone wants to use the same configuration for all client he can create a "cn=default" area and just configure all clients to use it. > autofs issues a query like this when starting up: > > SRCH base="$SEARCH_BASE_FROM_AUTOFS_CONF" scope=2 > filter="(&(objectClass=automountMap)(automountMapName=auto.master))" > attrs="automountMapName" > > So this still wouldn't work if you travel from Europe to the U.S. You > wouldn't automatically mount the local servers. Well maybe SSSD can be used to fetch the proper basedn at startup, modify /etc/sysconfig/autofs and restart autofs before the user logs in. I think that might work, and might even be a policy connected to the location the client is in. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Thu Nov 6 17:20:28 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 06 Nov 2008 10:20:28 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491325A7.7030704@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> <491325A7.7030704@redhat.com> Message-ID: <4913275C.40906@redhat.com> Dmitri Pal wrote: > >> 3.4. OpenLDAP supports an extension by which you can create an >> ordered list of values in attributes - each value is prefixed by {N} >> where N is a number. I think it's done in such a way that clients >> that understand ordering can request the values with the prefix, and >> dumb clients get the raw values. Fedora DS does not support this, >> but perhaps we should? There is an Internet Draft (or perhaps an >> informational RFC) about this feature. > > I will explore this more. I think the fact that OpenLDAP has it proves > that there is a demand for such feature. > It would be nice for DS to support it. But I think it would be harder > to deal with if in the first implementation we push it to DS to take > care of. > The better approach IMO would be to create an IPA plugin in v2 and > then use it as a base for DS plugin in follow up DS version. > This way we will avoid creating dependencies on DS and forcing the > plugin to be perfect from day one. It would be much harder to fix > issues with it if it is a part of DS. Ok, but I'm not sure if the plugin API exposes all of the hooks necessary to support such a feature. I think one big missing piece is the ability to edit search entries before they are returned to clients. But adding those hooks would be generally useful for other plugins as well. > >>> >> This sounds a lot like the information we keep in the replication >> changelog, and to a certain extent the replication state information >> we keep in the entry itself. Also in the retro changelog. >> >> If we were to do something like this, it would be worth revisiting >> the current internal change notification code that we use in various >> places (post ops, persistent search, replication changelog, retro >> changelog, audit log, etc.) so that we could have a general purpose >> internal change notification service to be used by all of these >> various "sinks" and more. > > May be. But I am a bit scared going that far right away. This starts > to sound like a major feature. Let us take incremental steps and start > with small tasks. Ok. I just don't like having all of this redundant code (there's actually not that much) - plus people keep asking for all sorts of general purpose notification mechanisms (e.g. fire off a shell script/email when an event happens, add/mod/delete a row in a database, etc.). > I do not want to boil an ocean. Eventually I might but this is to far > fetched. > I want to solve a specific problem and implement a specific feature. > But I want to write it so that it can be reused for other purposes and > become more generic. > I view the "commit comment" feature as a send box for a more generic > implementation down the road. > > Rich, would you agree with such approach? Yes. For starters, I would suggest the Retro Changelog code and the MMR backend postop code (the code that adds the state information to entries). > >>> >>> Thank you >>> Dmitri >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> > From jdennis at redhat.com Thu Nov 6 17:39:55 2008 From: jdennis at redhat.com (John Dennis) Date: Thu, 06 Nov 2008 12:39:55 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491325A7.7030704@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> <491325A7.7030704@redhat.com> Message-ID: <49132BEB.4080607@redhat.com> LDAP is not the right tool/technology for storing change log information. Directories are optimized for particular uses, this is not one of them. There is a reason why directories coexist with databases, they solve different problems. Changelog entries of the type you envision are not bound to a single object in the directory, rather they are a logical unit of work which may affect multiple directory entries. Which entries in the directory are you going to tag with the comment? This is really a problem which needs to be solved at a different level and a different place. It is closely related to an audit problem. The change needs to be given a transaction id which encapsulates the various component changes and binds it with a comment and other meta data (e.g. user id, timestamp, etc.). This is then logged somewhere (but not in the directory). Audit analysis should be able to correlate the changelog transaction with other auditable events (e.g. directory audit logs). For now I would suggest the log destination be a file and to accommodate structured information it should be written in XML. This might be a reasonable feature for v2, anything beyond that should be postponed. Just this limited functionality (write the changelog in xml) would meet a lot of needs, get current v2 users used to providing changelog information, provides a reasonable way to view the changelog, and we get all this for not a lot of work (a heck of a lot less work than the other ideas). -- John Dennis From rmeggins at redhat.com Thu Nov 6 17:55:38 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 06 Nov 2008 10:55:38 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49132BEB.4080607@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> <491325A7.7030704@redhat.com> <49132BEB.4080607@redhat.com> Message-ID: <49132F9A.90204@redhat.com> John Dennis wrote: > LDAP is not the right tool/technology for storing change log > information. Directories are optimized for particular uses, this is > not one of them. There is a reason why directories coexist with > databases, they solve different problems. The directory is already used for storing "change log" information. It can be configured to record every single operation, along with a globally unique identifier for that operation, the timestamp, and the identity who made that change. Replication in essence just replays this "change log". Maybe I am misunderstanding what you mean by "change log" - could you define what you mean by that term? > Changelog entries of the type you envision are not bound to a single > object in the directory, rather they are a logical unit of work which > may affect multiple directory entries. Which entries in the directory > are you going to tag with the comment? That is a big problem for DS. We have no way to create a transaction that spans several operations/entries. There is an internet draft for LDAP transactions, and the IBM Tivoli DS supports them. I'm not sure how the DS could know which operations to group - the user would necessarily have to provide a "transaction ID" for the change set either explicity (as a separate attribute) or implicitly (if we supported transactions). > > This is really a problem which needs to be solved at a different level > and a different place. It is closely related to an audit problem. The > change needs to be given a transaction id which encapsulates the > various component changes and binds it with a comment and other meta > data (e.g. user id, timestamp, etc.). This is then logged somewhere > (but not in the directory). Audit analysis should be able to correlate > the changelog transaction with other auditable events (e.g. directory > audit logs). The directory server already logs the raw data - we just need some way to collect it and group it. > > For now I would suggest the log destination be a file and to > accommodate structured information it should be written in XML. The changelog db is easily convertible to LDIF - the DS audit log is in LDIF - LDIF is convertible to XML (we already have ldif2dsml tools). > This might be a reasonable feature for v2, anything beyond that should > be postponed. Just this limited functionality (write the changelog in > xml) would meet a lot of needs, get current v2 users used to providing > changelog information, provides a reasonable way to view the > changelog, and we get all this for not a lot of work (a heck of a lot > less work than the other ideas). > From ssorce at redhat.com Thu Nov 6 18:15:46 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 18:15:46 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49132F9A.90204@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> <491325A7.7030704@redhat.com> <49132BEB.4080607@redhat.com> <49132F9A.90204@redhat.com> Message-ID: <1225995346.10160.227.camel@localhost.localdomain> Lets not make confusion, the change log we are talking about in this thread is not the low level change log in DS. It is a very high level log about why a (set of) operation(s) has been performed in IPA (and I say IPA and not DS, because I believe it could span more than just modifications in the Directory Server). On Thu, 2008-11-06 at 10:55 -0700, Rich Megginson wrote: > John Dennis wrote: > > LDAP is not the right tool/technology for storing change log > > information. Directories are optimized for particular uses, this is > > not one of them. There is a reason why directories coexist with > > databases, they solve different problems. > The directory is already used for storing "change log" information. It > can be configured to record every single operation, along with a > globally unique identifier for that operation, the timestamp, and the > identity who made that change. Replication in essence just replays this > "change log". Maybe I am misunderstanding what you mean by "change log" > - could you define what you mean by that term? See the title, it is a comment set interactively by a human operator on the reasons why he did some high level operation. > > Changelog entries of the type you envision are not bound to a single > > object in the directory, rather they are a logical unit of work which > > may affect multiple directory entries. Which entries in the directory > > are you going to tag with the comment? > That is a big problem for DS. We have no way to create a transaction > that spans several operations/entries. There is an internet draft for > LDAP transactions, and the IBM Tivoli DS supports them. This really goes beyond transactions. If I change 2 policies because they are somewhat related I really do not need an LDAP transaction to bind the 2 operations together. > I'm not sure how the DS could know which operations to group - the user > would necessarily have to provide a "transaction ID" for the change set > either explicity (as a separate attribute) or implicitly (if we > supported transactions). > > > > This is really a problem which needs to be solved at a different level > > and a different place. It is closely related to an audit problem. The > > change needs to be given a transaction id which encapsulates the > > various component changes and binds it with a comment and other meta > > data (e.g. user id, timestamp, etc.). This is then logged somewhere > > (but not in the directory). Audit analysis should be able to correlate > > the changelog transaction with other auditable events (e.g. directory > > audit logs). > The directory server already logs the raw data - we just need some way > to collect it and group it. Yes but that's at a completely different level. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Thu Nov 6 18:43:25 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 06 Nov 2008 13:43:25 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1225991919.10160.217.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> Message-ID: <49133ACD.3080306@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 10:02 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> How do clients choose which server to connect to ? >> Hardcoded in /etc/sysconfig/autofs on Fedora and RHEL. >> >>> Is there any concept like that in automount ? Should we care ? >> AFAIK there is no accomodation for this. We'd either have to provide >> separate areas (in the dn) to store the maps or the end-user would need >> to carefully configure things. > > I think providing separate areas then is paramount. Admins should be > able to define "locations" and the maps would be created inside these > location. This way admins can set different automount options for > clients located in different places. I am sure clients in Australia are > not going to use the same automount maps as for clients in Baltimore. > >> LDAP for autofs is configured on Fedora (the only thing I've >> experimented with so far) in /etc/sysconfig/autofs. You can specify the >> server and the search base. >> >> To accomodate geographic areas we could use a separate basedn for each >> one, something like: >> >> cn=australia,cn=autoumount,$SUFFIX >> cn=baltimore,cn=automount,$SUFFIX > > Yes I think this is needed. > After if someone wants to use the same configuration for all client he > can create a "cn=default" area and just configure all clients to use it. Ok, I think that should be simple enough to achieve. I think what I'll do is add an optional attribute 'location' and if that is not set, put the entry into 'default'. When someone does automount-newlocation I'll create the default auto_master and /- entries. I should be able to use the ldap-updater code to make this easy (thanks to Martin for suggesting making it a library). > Well maybe SSSD can be used to fetch the proper basedn at startup, > modify /etc/sysconfig/autofs and restart autofs before the user logs in. > > I think that might work, and might even be a policy connected to the > location the client is in. Yes, I think laptops put the only kink into this from a user-perspective. SSSD will need to recognize that they are in a different location and configure the machine accordingly. rob From dpal at redhat.com Thu Nov 6 18:47:39 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 13:47:39 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1225991601.10160.212.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> Message-ID: <49133BCB.7070308@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: > >> Simo Sorce wrote: >> > > >> Well, this is the main point. It would have been great if we had a >> product that would be able to act as DS and DB at the same time. >> I agree that this information would easier be managed in a DB. But we do >> not have one. We can create one but then we face all the same issues as >> we face with policies: >> a) Replication >> > > Why do we need to replicate a log database? It is not vital for the > normal functioning of the Identity solution so it can well be a database > or a file kept on a single server > > You are continuing to view it from the wrong angle. It is not a log database. It is not a log file. It is a property of the entry. It should be stored close to the object it is related to. It is not an audit data. You want this data to be viewable when you are viewing or modifying the entry. Making the UI go and pull this data from the external database would be an huge overhead. This data should be available regardless of which master you are connecting to. By all its characteristics it is not a log data so please stop thinking about it from that perspective. >> b) Referential integrity across different data stores >> > > Use object GUIDs, that's all is needed, seriously, it is very simple to > keep referential integrity when GUIDs are used. > Across different data stores? It does not matter what you use as a foreign key. The whole notion of having two data stores and creating referential integrity between the two make be cry. Been there, done that. It is extremely complex and nearly impossible to make it right regardless whether it is a DB+DB or DB+DS or DS+File or any other combination. It is always more complex than a synch task. I strongly believe that having a DS plugin is a small, simple and what is more important self-contained solution that does not create any dependencies . > >> c) Backup - restore >> > > Not sure why this would make any difference. We do not offer any special > backup/restore feature with IPA so it would be just business as usual. > > If you have more than one data stores you have to make sure that your backup and restore are also synched otherwise the referential integrity goes out of the window. We do not have a problem in IPA because we do not have any storage that should synched with other storage (except AD sycnh and you know how not simple it is :-) ). This is why I was strongly for storing policies in the DS itself. It solves a lot of problems you never even want to think about. Same is here. >> With policies the decision was made to store them in DS not to face >> these problems. >> > > Policies will be constantly queried by clients, that's why you need to > replicate them on multiple servers to be able to scale up when the > number of clients grow. A log does not need that. > It is not a log. It is a property of the object. It will be queried and displayed in the UI any time the object is viewed or modified. > >> I think that creating a parallel infrastructure for policies or for >> commit logs is a huge overhead. >> > > I think logs and policies are completely different in nature, these logs > are just audit trails in nature, and should be rather be developed as > part of the audit system. > They are not audit trails. They are comments that contain information. Like this. For example on the SUDO policy object: "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS policy according to the decision made by the security board on 10/28/08" "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS policy to fix a security problem reported in ticket #2131209" Is this audit? No. It looks like one but it is not. It has one important piece of information: why? who authorized the change to the the critical infrastructure. If we treat it as pure audit log record we will loose its relevance. It will go into the audit server and it would be possible to pull it from there on demand but not in the context of modification of the entry in DS. Making the UI do this search against the audit server would be an overhead and there is no guarantee that this data is not archived or cleaned from audit DB. When it is in DS and close to the object we can guarantee that it is complete, relevant and always accessible when the entry is viewed and administrator is about to make a critical change to the system. I think that since we a re a security product any changes to the policies that define access rules should have a comment facility like this. >>>> Can you provide what are the reasons that would make a good idea to >>>> store this kind of data in DS instead of a log file or a log database ? >>>> >>>> >>>> >> Log file is not manageable. It is hard to pull and query. >> > > About the same complexity of pulling an query a strctured attribute in > LDAP. To be honesyt for most people a file is *much* more easier to pull > and parse than an ldap attribute. > > I suggest not pursue this argument - it is talking about apples and oranges. >> It is good as >> a stream of data but when you need to analyze it you use other tools. >> Log database is different. It is for different purpose. Log database is >> for analysis. It would be hard to relate the history or a DS entry to >> the entry itself and present them in proper way in UI. >> > > I don't think so, as long as you store the GUID of the entry it refers > to as part of the metadata of the comment/log entry it is quite easy, > just a single ldap search (or vice versa, just a single "grep") > > See replication issues above. Log file or external DB just do not work. >> It would be much more complex than you think. >> > > I honestly think you greatly over-estimate the complexity of parsing a > log file in a modern scripting language like python, or even just old > stuff like grep, cut or awk. > > Parsing is not a problem. Manging is the problem. Access control, replication, backup etc. I do not want to duplicate all the arguments again. >>>> I honestly see no reason, nothing but a human operator would make any >>>> use of such attributes, they are not useful to any machine which is the >>>> real consumer of DS data. >>>> >>>> >> I completely disagree with trhis statement. DS is a data store for all >> data that needs to be stored unless it does not belong there. >> > > The point is that comments or log do not belong there, they just don't. > This is a property not a log. > >> The "commit log" data is on the edge but there is unfortunately no >> better place to put it. >> > > A log file or a log database, it's auditing related we will have > auditing facilities, use them. > Just disagree. See comments above. > >>>> As you pointed on it would add an unnecessary amount of data to >>>> replicate around. >>>> >>>> >>>> >> If we use MV attributes the replication problem will be solved. >> > > NO, it will just be softened, you still replicate around data useless > 99% of the time. > > No I checked. According to DS gurus the operation is replicated not the data so it is not an issue. Check with Nathan and Rich if you do not believe me. This was one of the first things I checked because i was concerned about this too. >>>> Also because DS is not a logging server but a directory server there are >>>> many other problems in trying to use it to store logs. >>>> >>>> >>>> >>>> >> This is not a log. I think this is a fundamental difference. It is a set >> of comments. >> > > Sorry but I see no difference between a set of "comments" or a log, a > log is just a set of "comments" about what is going on in some generic > "process". > The log is a set of events generated by softwere in repsonse to user actions. Comments are not a log IMO. They can be BLOG but this is in other context :-) > >> IMO they should be tightly related to the object they refer >> to and not be somewhere else in the log server. >> > > Why? > > Because when something goes wrong it should be there. The policies are too critical to the entire enterprise. It is much easier to find what was going on right there with that kind of comments . >> The log will contain who did what when. But it never answers the >> question "why". >> > > If you put a comment in a log it can also answer why, I really do not > see your point here. > > We view it from different angles. You are are stuck with the log approach. >> The "commit comment" feature is designed for that use case. >> May be the word selection confused you but it is not a "log". >> > > No, the nature of a comment is the same as that of a log, it is just > that the "comment" is user generated instead of automatically generated > but it really bears no relevant semantic difference. > Disagree. > >> Think about comments in bugzilla bug. Bugzilla bug is not a log and >> never perceived as such. "commit comment" feature should not be too. >> > > Bugzilla is used to have "conversations" about topics. The "commit > comment" is instead just the same as a commit message in an SCM system. > Not surprisingly to see the list of commit comments in cvs, svn,git, > etc.. the command is "log" (cvs log, svn log, git log ....) > > They are just logs. > > NO. >> I agree but I do not see a better alternative. Implementing an external >> DB synchronized with DS is much more complex than you would think. >> > > I beg to strongly disagree. All you need is to use GUIDs on objects, > that is your "join" key. It makes thing extremely simple to manage. > Have you ever done this? I did. It is not that simple. > >>>> If comments are required for audit trails I think they should just go in >>>> the auditing system and marked as special "comment" logs. Otherwise they >>>> should probably simply go in a normal log file or a relational log >>>> database (the latter in case online searches are required). >>>> >>>> >> As I said they are not a log and this approach will make the feature >> useless. >> > > Evidently there is a problem in understanding what this feature is > useful for. > > >From your premises about why customers need it, it is useful as an audit > trail log to know why an object is in that state. Is there any other use > to it ? > > The main reason to have it is to have an answer to "why somone did what he did". And it should be there at hand so that when next guy comes in and tries to clean the mess he knows why the previous change was made and who authorized it. Then re can turn to audit system and dig more but the recorded comment will give him a very good starting point. This is all about authority and responsibility. >>>> Now to some of the reasons why I don't see DS as a viable option: >>>> >>>> - Multi-value attributes are not ordered, so you need to invent some >>>> scheme to store this data structured so that ordering can be preserved. >>>> Sure probably using the "posting" date before the content is all is >>>> needed, but that makes attributes not searcheable. >>>> >>>> >>>> >> I think that creating a generic plugin that would allow storing ordered >> MV attributes would be a big benefit for everybody. >> > > The problem is that ordered MV are not defined in the LDAP protocol, you > would need to come up with a standardizable way to manage how to add > entries so that they fall in the precise order you need them to. This > would mean modifying the add operation either with a control or by > creating an extended operation. > > I will see what RFCs exist for this case. Rich pointed out that OpenLDAP has it. >> We can use the "commit comment" use case to create one. I would think >> that DS folks would find such plugin pretty valuable. >> I am actually surprised to the fact that one does not exist yet. It >> would have solved a lot of different issues and paved a way for even >> broader adoption of the DS. >> > > This is *way* more work than you think it is. And the reason why nobody > did it and no clear standard has still been proposed. > > That is why I do not want to do it in one step. I want to do commit comment plugin first focusing on specific functionality and then reuse it to solve a more generic problem. >> I would start with implementing "commit comment" feature as a step >> towards a generic ordered MV value plugin solution. >> > > This alone would require considerable time. > Probably a week for expert like Nalin and up to 3 weeks for novice like me. :-) > >> This would actually mean turning a MV attribute into sort of mini table >> with records associated with an entry in this table. >> I think it is a very cool feature. >> > > Yes, but would mean modification of the LDAP protocol, you do not want > to do that outside of a standardization body, or it will most probably > be just a dead end. > I do not see what you are talking about. There is no change to the protocol at all. There are tons of RFCs that store complex structures in the attribute. I do not see how my solution is different from those. > >>>> - You would have to create a clean up process that removes old stuff, I >>>> don't think that keeping around a hundred entries log for years would >>>> make sense. >>>> >>>> >> I am not sure it should be done day one. But I will think more about >> cleanup. >> I view this as a second tear functionality on top of the original feature. >> >> >>>> - We would need to index yet another attribute if you want to make >>>> searches on it, note also that just consulting the log would require >>>> searches on the identity store increasing its load, something a log >>>> file/database would avoid completely. >>>> >>>> >>>> >> The whole value of the feature is to have the whole list. There is no >> need to index it since the search by this attribute does not make sense. >> This is not a valid concern. >> > > Can you explain what this list is used for actually? And why someone > should be interested in a comment written 5 years earlier and that is > completely outdated as the data in the object has no bearing with that > original comment anymore? > > We can show last X comments in the UI. This is irrelevant. You want to know how this whole thing ended up in the state it is in and why. Then you can come up with the effective remediation. You can look at just last X comments or the whole stack is up to you. Whatever is needed for you as admin to make the right decision. >>>> - If you invent a complex format you loose the capability to do decent >>>> filtering on searches, meaning you will often need to do wide scope >>>> searches and implement filtering in the UI (slooow, and loads DS) >>>> >>>> >>>> >> Again as I said: there is no need to filter by this attribute. You will >> pull this MV attribute if you need it. Searching inside the values >> pulled out is up to the application not to DS. >> > > This just tells that storing it in DS is really not required. > If you need it you can pull it from a DB or a file equally well. > > External source is a problem. >>>> - You have no relation of events (ldap not beeing a relational database >>>> makes it particularly difficult indeed). >>>> >>>> >>>> >> There is no relation to the event. There is a relation the object itself >> since the attribute that will contain the commit comments will be a part >> of the same entry. >> > > A comment is related to a "change" in an object, therefore it is > logically related to an event (the change), insomuch that you require > ordering of the entries. Now assume I have a set of events and related > comments, like the following: > > "add foo to sudoers file" "foo need access to x as admin" > "add bar to sudoers file" "bar need access to x as admin" > "remove foo from sudoers file" "foo need no more access as admin" > "add baz to sudoers file" "baz need access as admin" > "remove baz from sudoers file" "baz need no more access as admin" > ... > > Now who needs the full list ? > After 30-40 changes why do you care about something like lines 4 and 5 > except for auditing purposes ? And if you do not have a date associated > what is it useful for ? > > These comments are useless. They do not answer "why". I did the change because my manager authorized me or I responded to ticket #XXXXXX or security board authorized me - this is what should be put into the comment. Not "foo needs access to x as admin". Such comment is the duplication of the even and bears no value. The comment should contain information that links the event (changing of the policy) to the formal process that authorized this change. Hope this clarifies the purpose and the difference. >> There is no need to have any kind of cross references if it is done this >> way. >> > > If there are no cross references like the date the event/change happened > or who made the operation it would be quite useless (see above example). > > >>>> - If a single UI command changes many different objects, where do you >>>> store the comment? In one? All of them? >>>> If in one how do you relate changes to others ? >>>> If you replicate it an all objects how do you deal with access to all >>>> entries? (see below) >>>> See also scenario above about angry admins if you require a comment for >>>> each object being changed. >>>> >>>> >>>> >> The "commit comment" feature by nature makes sense in the context of the >> top level object as i mentioned above. >> If change happens to several entries at once the designer of the schema >> and UI should decide what would be the best approach and what entry the >> "commit comment" should be applied to. >> > > This is an operational policy, can't be decided at the schema level, it > completely depends on what kind of event the specific security > administrator wants in the specific deployment. > > We can attach it to multiple different objects but the system administrator - the "root" of the whole IPA deployment - will have a way to say where this is mandatory, where optional, and where it should be hidden. The UI and CLI will respect these settings. >> If we do it as auxiliary class we have a flexibility to use it in >> multiple places. >> > > SCNR but I read this like: we can clutter the DS as much as we want :) > > We can store whatever we want in the DS. IMO the 2307 is a good example of cluttering LDAP especially netgroups schema :-) I think we are playing much more nicely than this and other RFCs. >> I really do not see a problem here. You just pick the main entry you are >> dealing with. If the UI touches multiple entries of different kind in >> one step it is a subject for deep though and reevaluation. >> > > It will, I am trying to evaluate this *before* we waste a lot of time in > something that seem wrong to me on way too many levels to easily express > them all. for some aspects this is just a feeling, but so far, the > further we dig into it the more the feeling grows. > > NO comments. >> Such operations especially with DS where transactions are not supported >> should be avoided. >> > > Reality is people will touch multiple objects when changing stuff > around, and transactions really do not matter in this case (see the > angry-admin example I posted in the previous email). > > This is why we have CLI tools and UI so that things can be done properly. Messing with raw data is always dangerous if you do not know what you are doing. >>>> - Comments may contain sensitive information that should not be leaked, >>>> so comments should not me generally available for search on ldap. >>>> This would require to add (on the fly?) ACIs on objects that get >>>> comments. >>>> >>>> >>>> >> The ACI defines the access control rules for the MV attribute. I doubt >> that thew comments would contain sensitive information. >> > > I don't think ACIs can express something like "append-only". > > Can Nathan or Rich comment on that? >> This is a matter of administrative policies not software. >> > > Sorry I do not get what this means. We are talking about access > authorization to an attribute that seem to me to be framed in a way the > software can't cope with, I think software matters. > Anyone can read, the one who can edit the entry can add. The one who can delete the entry can delete all values together. Noone can modify. I think plugin can easily enforce this logic. > >> The first step >> is at least to treat it as a MV attribute with ordering that can't be >> modified or deleted. >> > > If you can write it you can modify it. > This is where plugin will reject the attempt to write. > >> Later as a second tear feature we can start thinking about more >> discretionary read access control. Again I do not see a big issue here >> for the first implementation. >> >> >>>> - Some ACI may allow a lower level admin to perform an operation on some >>>> attributes, but not add objectclasses or new attributes. >>>> We loose the comments in this case ? >>>> >>>> >> No. I will dig into that but the plan is to have consistent ACI rules. >> I hope that DS specialists will chime in and confirm that the ACI has >> enough flexibility to deal with the object I am suggesting. >> > > I don't think so, if I correctly understood how you want to manage it. > And btw the way you seem to be willing to manage it once again resembles > an audit trail log, and that's I think because it ultimately is just an > audit trail log. > > See the logic in the previous comment. >>>> - Anyone with write access to the attribute will be able to change the >>>> contents, making them generally completely useless as audit trails. >>>> Delegation of any minor task would require write access to comments all >>>> over the place. >>>> >>>> >>> >>> >> No. The whole idea is to make it non-editable at all. Only add. >> > > Exactly this is something that does not exist in the LDAP model, nor in > the ACI model we have. > I am pretty sure the plugin can take care of that. If not I would agree that this is not a good idea. Nathan? Rich? > >> Only >> later we might start diving into ACIs and deal with the complexity of >> editing the data by admins that have different levels of privileges. >> > > No, architectural problems must be evaluated first in this case, because > you are trying to construct something that is so off the way LDAP works > for something that seem so irrelevant for the general functioning that I > want to understand exactly why it is so important, Because from this > reading I just re-evaluated the time need to "adjust" DS to handle this > single attribute in term of several weeks, and that's a lot of effort, > just to keep around some log that can instead very easily be piped in a > file or in a database with an effort that will take a lot less time. > > The solution if possible is self contained and does not rely on any external piece of functionality - audit server (that would not be up to the task for quite some time). Adjusting DS is much simpler (I think) than building a logic of logging this into external store (file or DB) and then pulling it out when we need to edit policy and see who was messing with it and why. I have shown all the arguments about this at the top of this thread. >>> Forgot another important few: >>> >>> - It would make extremely difficult for people to extract this >>> information. Instead of connecting to a well known relational database >>> with well known tools used for reporting, they would have to build a >>> custom parser that speaks LDAP. This thing alone would be a deadly one >>> imo. >>> >>> >>> >> This is not a log. This is a comment on the entry. >> > > I am sorry, the description you give: ordered list, immutable list, add > only, all scream this is a log. > If it were just a comment about the object we would need nothing more > that the "description" attribute already available in ldap, and there > would be no ordering nor immutability problem. > View this as a history of the description attribute. Does that help? :-) > I think you might be trying to conflate together the concepts of > "description of the object" and "log of changes", I think this is a very > wrong approach. > > I think that this is a value add and something that is neede for the project to be successful. Otherwise I would not have suggested that. You know I am against any unnecessary work myself. I see that without this feature the adoption of the IPA will be slower since the feature allows tying the data to the formal processes established in the company. >> I think that if down >> the road we create a mean to store ordered lists in DS people would take >> advantage of that. >> I disagree that it is hard to extract. It is the same as any other MV >> attribute except that there is some header inside that prefixes the data >> that need to be skipped. >> There are so many LDAP attributes that have special internal formats >> that I read about in different RFC I really do not see my approach being >> against any main stream ways of doing things. >> There can be a helper library written for an easier adoption later. This >> is really not an issue at all imo. >> > > The issue here is why we want to waste a lot of time and effort to > implement just a per entry log that is useful only when some auditing > need to be performed. Seriously I think the effort is not worth the > value from what I have seen so far. > I would agree that if we had a robust audit server capable of doing real time searches now I would have explored the log approach and would have considered trying to hook into it. But it is not there and it would take quite a while to be there so log approach is IMO a non starter. > >>> - The time it will take us to build all the necessary machinery around >>> managing such attributes (I see you even envision a plugin :-O ) would >>> be considerable, and would probably be much better spent on more >>> critical features at this stage imo. (Piping this data in a db from >>> python will take no more than a day or two, building schema, plugin, and >>> all the testing will require weeks). >>> >>> >> Based on the experience with Nalin and my reading of DS documentation >> and reading DS plugin code it seems a pretty straightforward task. >> It is powers of magnitude simpler than any external database. >> > > I think you really are greatly underestimating the complexity of writing > a DS plugin, I have seen Nalin do it. > the complexity of dealing with changing LDAP semantics. No changes to LDAP semantics. No changes to protocol > And > the development cycle involved with all the bugs custom code would > entail. All the code we write is a custom code. This is not an argument :-) > A plugin also always adds security concerns as it runs as a > privileged process wrt DS data. Yes let us remove all the plugins. They are potentially insecure! Let us not publish the APIs so that noone can create one... This argument sounds really funny from you :-) > It add stability concerns as it runs > inside a threaded application (one segfault and goodbye LDAP server). > > It is as stable as any other plugin. And it is much simpler than the NIS plugin Nalin put together. No caching or memory pools. I do not see any complexity to worry about. May be I am wrong but unless I dive deeper I do not see a problem you are talking about. Just follow the rules of plugin development and do the right thing. There is no need to to get or update multiple entries - just couple attributes in one and the same entry - what a big deal? > For a database all you need is a schema and a few SQL queries to > insert/extract data. you don't have ordering problems there, you can > split data and metadata and have own tables. Writing an small database > schema with a couple of tables is honestly orders of magnitude simpler > than writing a plugin in C that has to do what you would like it to do. > > Really? And then you need to write installation scripts, dump and load utilities, access control rights, backup and restore, replication and other utilities. Even if the DS provider has them you now suggenly have to deal with all this and manager and document and test... This is far more than you think. I know what that means to use an embedded DB. The whole server I worked with for 10 years had an embedded DB. Creating schema and getting data is small part of the puzzle other utilities are the main burden. >> I think that it is worth a try. If I see that it takes much more time >> than I think I might defer this till later. >> But a decision needs to be made pretty soon. This is why I am bringing >> it up. >> > > So far none of the arguments given convince me the huge effort required > make sense. If I'd have to vote now, I'd say no. > > I want to hear other opinions. >> Mike Langlie is building the UI screens prototype and we (he and I) need >> to understand whether this feature will be a part of the UI or not so >> that we can prepare properly for usability testing we are planning to >> conduct. >> > > Maybe you can explain how the UI would use this information, that may > shed some more light on what is the appropriate way to manage these > user-generated-logs. > > See example above. > Simo. > > From dpal at redhat.com Thu Nov 6 18:58:56 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 13:58:56 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49132BEB.4080607@redhat.com> References: <491236CF.6090508@redhat.com> <491317BF.6060108@redhat.com> <491325A7.7030704@redhat.com> <49132BEB.4080607@redhat.com> Message-ID: <49133E70.8030605@redhat.com> John Dennis wrote: > LDAP is not the right tool/technology for storing change log > information. Directories are optimized for particular uses, this is > not one of them. There is a reason why directories coexist with > databases, they solve different problems. > > Changelog entries of the type you envision are not bound to a single > object in the directory, rather they are a logical unit of work which > may affect multiple directory entries. Which entries in the directory > are you going to tag with the comment? > The ones we decide. > This is really a problem which needs to be solved at a different level > and a different place. It is closely related to an audit problem. The > change needs to be given a transaction id which encapsulates the > various component changes and binds it with a comment and other meta > data (e.g. user id, timestamp, etc.). This is then logged somewhere > (but not in the directory). Audit analysis should be able to correlate > the changelog transaction with other auditable events (e.g. directory > audit logs). > As I said in other response. I would have explored the audit path if the audit server would have been available at the moment. Unfortunately it is not and I doubt it will be capable to this kind of functionality in reasonable future (v2 may be even v3). > For now I would suggest the log destination be a file and to > accommodate structured information it should be written in XML. This > might be a reasonable feature for v2, anything beyond that should be > postponed. If you store it in file how you replicate it? If we reuse the DS to replicate is we still would have to use a plugin and we will be replicating the whole file. If we do not replicate it it won't be transparently available when it is needed by UI. > Just this limited functionality (write the changelog in xml) would > meet a lot of needs, I doubt it would in the for you suggest. > get current v2 users used to providing changelog information, provides > a reasonable way to view the changelog, and we get all this for not a > lot of work (a heck of a lot less work than the other ideas). > Thanks Dmitri From ssorce at redhat.com Thu Nov 6 19:16:06 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 19:16:06 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49133BCB.7070308@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> Message-ID: <1225998966.10160.240.camel@localhost.localdomain> Ok let me reply first on the "log" nature of this information, will answer to other stuff in a separate email. > On Thu, 2008-11-06 at 13:47 -0500, Dmitri Pal wrote: > > "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS > policy according to the decision made by the security board on 10/28/08" > "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS > policy to fix a security problem reported in ticket #2131209" This loloks like, a log, smells like a log, there fore it is a log. It is probbaly not an "application generated log", but it is at all effects what in english you would call a log [1]. So please let's call stuff with its name. It is a log, it maybe some special kind of log, but it is nothing more than a log. It maybe so special that it will require a special way to handle it, I perfectly agree not all logs are created equal, but let's not fight on a definition. [1] http://dictionary.reference.com/browse/log Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Thu Nov 6 19:17:09 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 06 Nov 2008 12:17:09 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49133BCB.7070308@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> Message-ID: <491342B5.8090207@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: >> >>> Simo Sorce wrote: >>> >> >> >>> Well, this is the main point. It would have been great if we had a >>> product that would be able to act as DS and DB at the same time. >>> I agree that this information would easier be managed in a DB. But >>> we do not have one. We can create one but then we face all the same >>> issues as we face with policies: >>> a) Replication >>> >> >> Why do we need to replicate a log database? It is not vital for the >> normal functioning of the Identity solution so it can well be a database >> or a file kept on a single server >> >> > You are continuing to view it from the wrong angle. > It is not a log database. It is not a log file. It is a property of > the entry. It should be stored close to the object it is related to. > It is not an audit data. You want this data to be viewable when you > are viewing or modifying the entry. > Making the UI go and pull this data from the external database would > be an huge overhead. > This data should be available regardless of which master you are > connecting to. > By all its characteristics it is not a log data so please stop > thinking about it from that perspective. This is similar to the passwordHistory we store in each entry, and similar to the CSN state information in each entry. > > > > >>> b) Referential integrity across different data stores >>> >> >> Use object GUIDs, that's all is needed, seriously, it is very simple to >> keep referential integrity when GUIDs are used. >> > > Across different data stores? It does not matter what you use as a > foreign key. The whole notion of having two data stores and creating > referential integrity between the two make be cry. > Been there, done that. It is extremely complex and nearly impossible > to make it right regardless whether it is a DB+DB or DB+DS or DS+File > or any other combination. It is always more complex than a synch task. > I strongly believe that having a DS plugin is a small, simple and what > is more important self-contained solution that does not create any > dependencies . > > >> >>> c) Backup - restore >>> >> >> Not sure why this would make any difference. We do not offer any special >> backup/restore feature with IPA so it would be just business as usual. >> >> > If you have more than one data stores you have to make sure that your > backup and restore are also synched otherwise the referential > integrity goes out of the window. > We do not have a problem in IPA because we do not have any storage > that should synched with other storage (except AD sycnh and you know > how not simple it is :-) ). > This is why I was strongly for storing policies in the DS itself. It > solves a lot of problems you never even want to think about. > Same is here. > > > >>> With policies the decision was made to store them in DS not to face >>> these problems. >>> >> >> Policies will be constantly queried by clients, that's why you need to >> replicate them on multiple servers to be able to scale up when the >> number of clients grow. A log does not need that. >> > > It is not a log. It is a property of the object. It will be queried > and displayed in the UI any time the object is viewed or modified. > > > >> >>> I think that creating a parallel infrastructure for policies or for >>> commit logs is a huge overhead. >>> >> >> I think logs and policies are completely different in nature, these logs >> are just audit trails in nature, and should be rather be developed as >> part of the audit system. >> > > They are not audit trails. They are comments that contain information. > Like this. > For example on the SUDO policy object: > > "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS > policy according to the decision made by the security board on 10/28/08" > "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS > policy to fix a security problem reported in ticket #2131209" > > Is this audit? No. It looks like one but it is not. It has one > important piece of information: why? who authorized the change to the > the critical infrastructure. > If we treat it as pure audit log record we will loose its relevance. > It will go into the audit server and it would be possible to pull it > from there on demand but not in the context of modification of the > entry in DS. Making the UI do this search against the audit server > would be an overhead and there is no guarantee that this data is not > archived or cleaned from audit DB. > When it is in DS and close to the object we can guarantee that it is > complete, relevant and always accessible when the entry is viewed and > administrator is about to make a critical change to the system. > > I think that since we a re a security product any changes to the > policies that define access rules should have a comment facility like > this. > >>>>> Can you provide what are the reasons that would make a good idea to >>>>> store this kind of data in DS instead of a log file or a log >>>>> database ? >>>>> >>>>> >>> Log file is not manageable. It is hard to pull and query. >>> >> >> About the same complexity of pulling an query a strctured attribute in >> LDAP. To be honesyt for most people a file is *much* more easier to pull >> and parse than an ldap attribute. >> >> > I suggest not pursue this argument - it is talking about apples and > oranges. If there are different datum (timestamp, user id, comment) stored in a structured way as an attribute value, it will be harder to perform certain types of queries. e.g. if you have comment: 20081106121212Z rmeggins at REALM added aci to allow managers to edit their subordinates' information as required by dpal at REALM How would I search for all comments by rmeggins at REALM? If the data is structured "enough", we could write custom matching rules to allow such searches to be indexed. > > > >>> It is good as a stream of data but when you need to analyze it you >>> use other tools. >>> Log database is different. It is for different purpose. Log database >>> is for analysis. It would be hard to relate the history or a DS >>> entry to the entry itself and present them in proper way in UI. >>> >> >> I don't think so, as long as you store the GUID of the entry it refers >> to as part of the metadata of the comment/log entry it is quite easy, >> just a single ldap search (or vice versa, just a single "grep") >> >> > > See replication issues above. Log file or external DB just do not work. > > >>> It would be much more complex than you think. >>> >> >> I honestly think you greatly over-estimate the complexity of parsing a >> log file in a modern scripting language like python, or even just old >> stuff like grep, cut or awk. >> >> > Parsing is not a problem. Manging is the problem. Access control, > replication, backup etc. > I do not want to duplicate all the arguments again. > > > >>>>> I honestly see no reason, nothing but a human operator would make any >>>>> use of such attributes, they are not useful to any machine which >>>>> is the >>>>> real consumer of DS data. >>>>> >>> I completely disagree with trhis statement. DS is a data store for >>> all data that needs to be stored unless it does not belong there. >>> >> >> The point is that comments or log do not belong there, they just don't. >> > > This is a property not a log. > >> >>> The "commit log" data is on the edge but there is unfortunately no >>> better place to put it. >>> >> >> A log file or a log database, it's auditing related we will have >> auditing facilities, use them. >> > > Just disagree. See comments above. > >> >>>>> As you pointed on it would add an unnecessary amount of data to >>>>> replicate around. >>>>> >>>>> >>> If we use MV attributes the replication problem will be solved. >>> >> >> NO, it will just be softened, you still replicate around data useless >> 99% of the time. >> >> > > No I checked. According to DS gurus the operation is replicated not > the data so it is not an issue. > Check with Nathan and Rich if you do not believe me. This was one of > the first things I checked because i was concerned about this too. Just the modify-add operation will be replicated. > >>>>> Also because DS is not a logging server but a directory server >>>>> there are >>>>> many other problems in trying to use it to store logs. >>>>> >>>>> >>>>> >>> This is not a log. I think this is a fundamental difference. It is a >>> set of comments. >>> >> >> Sorry but I see no difference between a set of "comments" or a log, a >> log is just a set of "comments" about what is going on in some generic >> "process". >> > > The log is a set of events generated by softwere in repsonse to user > actions. > Comments are not a log IMO. They can be BLOG but this is in other > context :-) >> >>> IMO they should be tightly related to the object they refer to and >>> not be somewhere else in the log server. >>> >> >> Why? >> >> > > Because when something goes wrong it should be there. The policies are > too critical to the entire enterprise. > It is much easier to find what was going on right there with that > kind of comments . > >>> The log will contain who did what when. But it never answers the >>> question "why". >>> >> >> If you put a comment in a log it can also answer why, I really do not >> see your point here. >> >> > > We view it from different angles. You are are stuck with the log > approach. > >>> The "commit comment" feature is designed for that use case. >>> May be the word selection confused you but it is not a "log". >>> >> >> No, the nature of a comment is the same as that of a log, it is just >> that the "comment" is user generated instead of automatically generated >> but it really bears no relevant semantic difference. >> > > Disagree. > >> >>> Think about comments in bugzilla bug. Bugzilla bug is not a log and >>> never perceived as such. "commit comment" feature should not be too. >>> >> >> Bugzilla is used to have "conversations" about topics. The "commit >> comment" is instead just the same as a commit message in an SCM system. >> Not surprisingly to see the list of commit comments in cvs, svn,git, >> etc.. the command is "log" (cvs log, svn log, git log ....) >> >> They are just logs. >> >> > > NO. > >>> I agree but I do not see a better alternative. Implementing an >>> external DB synchronized with DS is much more complex than you would >>> think. >>> >> >> I beg to strongly disagree. All you need is to use GUIDs on objects, >> that is your "join" key. It makes thing extremely simple to manage. >> > > Have you ever done this? I did. It is not that simple. > >> >>>>> If comments are required for audit trails I think they should just >>>>> go in >>>>> the auditing system and marked as special "comment" logs. >>>>> Otherwise they >>>>> should probably simply go in a normal log file or a relational log >>>>> database (the latter in case online searches are required). >>>>> >>> As I said they are not a log and this approach will make the feature >>> useless. >>> >> >> Evidently there is a problem in understanding what this feature is >> useful for. >> >> >From your premises about why customers need it, it is useful as an >> audit >> trail log to know why an object is in that state. Is there any other use >> to it ? >> >> > > The main reason to have it is to have an answer to "why somone did > what he did". And it should be there at hand so that when next guy > comes in and tries to clean the mess he knows why the previous change > was made and who authorized it. Then re can turn to audit system and > dig more but the recorded comment will give him a very good starting > point. > > This is all about authority and responsibility. > > > >>>>> Now to some of the reasons why I don't see DS as a viable option: >>>>> >>>>> - Multi-value attributes are not ordered, so you need to invent some >>>>> scheme to store this data structured so that ordering can be >>>>> preserved. >>>>> Sure probably using the "posting" date before the content is all is >>>>> needed, but that makes attributes not searcheable. >>>>> >>>>> >>> I think that creating a generic plugin that would allow storing >>> ordered MV attributes would be a big benefit for everybody. >>> >> >> The problem is that ordered MV are not defined in the LDAP protocol, you >> would need to come up with a standardizable way to manage how to add >> entries so that they fall in the precise order you need them to. This >> would mean modifying the add operation either with a control or by >> creating an extended operation. >> > I will see what RFCs exist for this case. Rich pointed out that > OpenLDAP has it. > > >>> We can use the "commit comment" use case to create one. I would >>> think that DS folks would find such plugin pretty valuable. >>> I am actually surprised to the fact that one does not exist yet. It >>> would have solved a lot of different issues and paved a way for even >>> broader adoption of the DS. >>> >> >> This is *way* more work than you think it is. And the reason why nobody >> did it and no clear standard has still been proposed. >> >> > > That is why I do not want to do it in one step. I want to do commit > comment plugin first focusing on specific functionality and then reuse > it to solve a more generic problem. > > > >>> I would start with implementing "commit comment" feature as a step >>> towards a generic ordered MV value plugin solution. >>> >> >> This alone would require considerable time. >> > > Probably a week for expert like Nalin and up to 3 weeks for novice > like me. :-) > > >> >>> This would actually mean turning a MV attribute into sort of mini >>> table with records associated with an entry in this table. >>> I think it is a very cool feature. >>> >> >> Yes, but would mean modification of the LDAP protocol, you do not want >> to do that outside of a standardization body, or it will most probably >> be just a dead end. >> > > I do not see what you are talking about. There is no change to the > protocol at all. > There are tons of RFCs that store complex structures in the attribute. > I do not see how my solution is different from those. There are matching rules that allow you to parse and index individual fields in an X.509 cert stored as a single userCertificate attribute value - openldap has or is implementing support for them. We could do the same. > >> >>>>> - You would have to create a clean up process that removes old >>>>> stuff, I >>>>> don't think that keeping around a hundred entries log for years would >>>>> make sense. >>>>> >>> I am not sure it should be done day one. But I will think more about >>> cleanup. >>> I view this as a second tear functionality on top of the original >>> feature. >>> >>> >>>>> - We would need to index yet another attribute if you want to make >>>>> searches on it, note also that just consulting the log would require >>>>> searches on the identity store increasing its load, something a log >>>>> file/database would avoid completely. >>>>> >>>>> >>> The whole value of the feature is to have the whole list. There is >>> no need to index it since the search by this attribute does not make >>> sense. >>> This is not a valid concern. >>> >> >> Can you explain what this list is used for actually? And why someone >> should be interested in a comment written 5 years earlier and that is >> completely outdated as the data in the object has no bearing with that >> original comment anymore? >> >> > > We can show last X comments in the UI. This is irrelevant. You want to > know how this whole thing ended up in the state it is in and why. > Then you can come up with the effective remediation. You can look at > just last X comments or the whole stack is up to you. > Whatever is needed for you as admin to make the right decision. > > > >>>>> - If you invent a complex format you loose the capability to do >>>>> decent >>>>> filtering on searches, meaning you will often need to do wide scope >>>>> searches and implement filtering in the UI (slooow, and loads DS) >>>>> >>>>> >>> Again as I said: there is no need to filter by this attribute. You >>> will pull this MV attribute if you need it. Searching inside the >>> values pulled out is up to the application not to DS. >>> >> >> This just tells that storing it in DS is really not required. >> If you need it you can pull it from a DB or a file equally well. >> >> > External source is a problem. > > >>>>> - You have no relation of events (ldap not beeing a relational >>>>> database >>>>> makes it particularly difficult indeed). >>>>> >>>>> >>> There is no relation to the event. There is a relation the object >>> itself since the attribute that will contain the commit comments >>> will be a part of the same entry. >>> >> >> A comment is related to a "change" in an object, therefore it is >> logically related to an event (the change), insomuch that you require >> ordering of the entries. Now assume I have a set of events and related >> comments, like the following: >> >> "add foo to sudoers file" "foo need access to x as admin" >> "add bar to sudoers file" "bar need access to x as admin" >> "remove foo from sudoers file" "foo need no more access as admin" >> "add baz to sudoers file" "baz need access as admin" >> "remove baz from sudoers file" "baz need no more access as admin" >> ... >> >> Now who needs the full list ? >> After 30-40 changes why do you care about something like lines 4 and 5 >> except for auditing purposes ? And if you do not have a date associated >> what is it useful for ? >> >> > These comments are useless. They do not answer "why". I did the change > because my manager authorized me or I responded to ticket #XXXXXX or > security board authorized me - this is what should be put into the > comment. Not "foo needs access to x as admin". Such comment is the > duplication of the even and bears no value. The comment should contain > information that links the event (changing of the policy) to the > formal process that authorized this change. Hope this clarifies the > purpose and the difference. > > > >>> There is no need to have any kind of cross references if it is done >>> this way. >>> >> >> If there are no cross references like the date the event/change happened >> or who made the operation it would be quite useless (see above example). >> >> >>>>> - If a single UI command changes many different objects, where do you >>>>> store the comment? In one? All of them? >>>>> If in one how do you relate changes to others ? >>>>> If you replicate it an all objects how do you deal with access to all >>>>> entries? (see below) >>>>> See also scenario above about angry admins if you require a >>>>> comment for >>>>> each object being changed. >>>>> >>>>> >>> The "commit comment" feature by nature makes sense in the context of >>> the top level object as i mentioned above. >>> If change happens to several entries at once the designer of the >>> schema and UI should decide what would be the best approach and what >>> entry the "commit comment" should be applied to. >>> >> >> This is an operational policy, can't be decided at the schema level, it >> completely depends on what kind of event the specific security >> administrator wants in the specific deployment. >> >> > > We can attach it to multiple different objects but the system > administrator - the "root" of the whole IPA deployment - will have a > way to say where this is mandatory, where optional, and where it > should be hidden. The UI and CLI will respect these settings. > > >>> If we do it as auxiliary class we have a flexibility to use it in >>> multiple places. >>> >> >> SCNR but I read this like: we can clutter the DS as much as we want :) >> >> > > We can store whatever we want in the DS. IMO the 2307 is a good > example of cluttering LDAP especially netgroups schema :-) > I think we are playing much more nicely than this and other RFCs. > > >>> I really do not see a problem here. You just pick the main entry you >>> are dealing with. If the UI touches multiple entries of different >>> kind in one step it is a subject for deep though and reevaluation. >>> >> >> It will, I am trying to evaluate this *before* we waste a lot of time in >> something that seem wrong to me on way too many levels to easily express >> them all. for some aspects this is just a feeling, but so far, the >> further we dig into it the more the feeling grows. >> >> > NO comments. > >>> Such operations especially with DS where transactions are not >>> supported should be avoided. >>> >> >> Reality is people will touch multiple objects when changing stuff >> around, and transactions really do not matter in this case (see the >> angry-admin example I posted in the previous email). >> >> > > This is why we have CLI tools and UI so that things can be done > properly. Messing with raw data is always dangerous if you do not know > what you are doing. > >>>>> - Comments may contain sensitive information that should not be >>>>> leaked, >>>>> so comments should not me generally available for search on ldap. >>>>> This would require to add (on the fly?) ACIs on objects that get >>>>> comments. >>>>> >>>>> >>> The ACI defines the access control rules for the MV attribute. I >>> doubt that thew comments would contain sensitive information. >>> >> >> I don't think ACIs can express something like "append-only". >> >> > > Can Nathan or Rich comment on that? You can use targattrfilters with the add keyword to do something similar - http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Managing_Access_Control-Creating_ACIs_Manually.html - not sure if it would handle every case - we might need to add another type of Right for append or modify/add or something like that. > >>> This is a matter of administrative policies not software. >>> >> >> Sorry I do not get what this means. We are talking about access >> authorization to an attribute that seem to me to be framed in a way the >> software can't cope with, I think software matters. >> > Anyone can read, the one who can edit the entry can add. > The one who can delete the entry can delete all values together. > Noone can modify. > > I think plugin can easily enforce this logic. > > > >> >>> The first step is at least to treat it as a MV attribute with >>> ordering that can't be modified or deleted. >>> >> >> If you can write it you can modify it. >> > > This is where plugin will reject the attempt to write. If ACIs cannot currently handle this case, then we should extend ACIs. Since targattrfilters already does something similar, the code is already there. > >> >>> Later as a second tear feature we can start thinking about more >>> discretionary read access control. Again I do not see a big issue >>> here for the first implementation. >>> >>> >>>>> - Some ACI may allow a lower level admin to perform an operation >>>>> on some >>>>> attributes, but not add objectclasses or new attributes. >>>>> We loose the comments in this case ? >>>>> >>> No. I will dig into that but the plan is to have consistent ACI rules. >>> I hope that DS specialists will chime in and confirm that the ACI >>> has enough flexibility to deal with the object I am suggesting. >>> >> >> I don't think so, if I correctly understood how you want to manage it. >> And btw the way you seem to be willing to manage it once again resembles >> an audit trail log, and that's I think because it ultimately is just an >> audit trail log. >> >> > See the logic in the previous comment. > > >>>>> - Anyone with write access to the attribute will be able to change >>>>> the >>>>> contents, making them generally completely useless as audit trails. >>>>> Delegation of any minor task would require write access to >>>>> comments all >>>>> over the place. >>>>> >>>> >>> No. The whole idea is to make it non-editable at all. Only add. >>> >> >> Exactly this is something that does not exist in the LDAP model, nor in >> the ACI model we have. >> > I am pretty sure the plugin can take care of that. If not I would > agree that this is not a good idea. > Nathan? Rich? I'm not sure what you mean by "does not exist in the LDAP model" here. ACIs are mentioned above. > >> >>> Only later we might start diving into ACIs and deal with the >>> complexity of editing the data by admins that have different levels >>> of privileges. >>> >> >> No, architectural problems must be evaluated first in this case, because >> you are trying to construct something that is so off the way LDAP works >> for something that seem so irrelevant for the general functioning that I >> want to understand exactly why it is so important, Because from this >> reading I just re-evaluated the time need to "adjust" DS to handle this >> single attribute in term of several weeks, and that's a lot of effort, >> just to keep around some log that can instead very easily be piped in a >> file or in a database with an effort that will take a lot less time. >> >> > The solution if possible is self contained and does not rely on any > external piece of functionality - audit server (that would not be up > to the task for quite some time). > Adjusting DS is much simpler (I think) than building a logic of > logging this into external store (file or DB) and then pulling it out > when we need to edit policy and see who was messing with it and why. > I have shown all the arguments about this at the top of this thread. > >>>> Forgot another important few: >>>> >>>> - It would make extremely difficult for people to extract this >>>> information. Instead of connecting to a well known relational database >>>> with well known tools used for reporting, they would have to build a >>>> custom parser that speaks LDAP. This thing alone would be a deadly one >>>> imo. >>>> >>>> >>> This is not a log. This is a comment on the entry. >>> >> >> I am sorry, the description you give: ordered list, immutable list, add >> only, all scream this is a log. >> If it were just a comment about the object we would need nothing more >> that the "description" attribute already available in ldap, and there >> would be no ordering nor immutability problem. >> > View this as a history of the description attribute. Does that help? :-) > > >> I think you might be trying to conflate together the concepts of >> "description of the object" and "log of changes", I think this is a very >> wrong approach. >> >> > I think that this is a value add and something that is neede for the > project to be successful. > Otherwise I would not have suggested that. You know I am against any > unnecessary work myself. > I see that without this feature the adoption of the IPA will be slower > since the feature allows tying > the data to the formal processes established in the company. > >>> I think that if down the road we create a mean to store ordered >>> lists in DS people would take advantage of that. >>> I disagree that it is hard to extract. It is the same as any other >>> MV attribute except that there is some header inside that prefixes >>> the data that need to be skipped. >>> There are so many LDAP attributes that have special internal formats >>> that I read about in different RFC I really do not see my approach >>> being against any main stream ways of doing things. >>> There can be a helper library written for an easier adoption later. >>> This is really not an issue at all imo. >>> >> >> The issue here is why we want to waste a lot of time and effort to >> implement just a per entry log that is useful only when some auditing >> need to be performed. Seriously I think the effort is not worth the >> value from what I have seen so far. >> > I would agree that if we had a robust audit server capable of doing > real time searches now I would have explored the log approach and > would have considered trying to hook into it. > But it is not there and it would take quite a while to be there so log > approach is IMO a non starter. > > >> >>>> - The time it will take us to build all the necessary machinery around >>>> managing such attributes (I see you even envision a plugin :-O ) would >>>> be considerable, and would probably be much better spent on more >>>> critical features at this stage imo. (Piping this data in a db from >>>> python will take no more than a day or two, building schema, >>>> plugin, and >>>> all the testing will require weeks). >>>> >>> Based on the experience with Nalin and my reading of DS >>> documentation and reading DS plugin code it seems a pretty >>> straightforward task. >>> It is powers of magnitude simpler than any external database. >>> >> >> I think you really are greatly underestimating the complexity of writing >> a DS plugin, > I have seen Nalin do it. It's still not that easy :P > >> the complexity of dealing with changing LDAP semantics. > No changes to LDAP semantics. No changes to protocol I'm not sure what you mean by "changing LDAP semantics" here. > >> And >> the development cycle involved with all the bugs custom code would >> entail. > All the code we write is a custom code. This is not an argument :-) > > >> A plugin also always adds security concerns as it runs as a >> privileged process wrt DS data. > > Yes let us remove all the plugins. They are potentially insecure! Let > us not publish the APIs so that noone can create one... > This argument sounds really funny from you :-) > >> It add stability concerns as it runs >> inside a threaded application (one segfault and goodbye LDAP server). >> >> > It is as stable as any other plugin. And it is much simpler than the > NIS plugin Nalin put together. No caching or memory pools. > I do not see any complexity to worry about. May be I am wrong but > unless I dive deeper I do not see a problem you are talking about. > Just follow the rules of plugin development and do the right thing. > There is no need to to get or update multiple entries - just couple > attributes in one and the same entry - what a big deal? > >> For a database all you need is a schema and a few SQL queries to >> insert/extract data. you don't have ordering problems there, you can >> split data and metadata and have own tables. Writing an small database >> schema with a couple of tables is honestly orders of magnitude simpler >> than writing a plugin in C that has to do what you would like it to do. >> >> > Really? And then you need to write installation scripts, dump and load > utilities, access control rights, backup and restore, replication and > other utilities. Even if the DS provider has them you now suggenly > have to deal with all this and manager and document and test... This > is far more than you think. I know what that means to use an embedded > DB. The whole server I worked with for 10 years had an embedded DB. > Creating schema and getting data is small part of the puzzle other > utilities are the main burden. > > >>> I think that it is worth a try. If I see that it takes much more >>> time than I think I might defer this till later. >>> But a decision needs to be made pretty soon. This is why I am >>> bringing it up. >>> >> >> So far none of the arguments given convince me the huge effort required >> make sense. If I'd have to vote now, I'd say no. >> >> > I want to hear other opinions. > >>> Mike Langlie is building the UI screens prototype and we (he and I) >>> need to understand whether this feature will be a part of the UI or >>> not so that we can prepare properly for usability testing we are >>> planning to conduct. >>> >> >> Maybe you can explain how the UI would use this information, that may >> shed some more light on what is the appropriate way to manage these >> user-generated-logs. >> >> > > See example above. > >> Simo. >> >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From nkinder at redhat.com Thu Nov 6 19:21:04 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Thu, 06 Nov 2008 11:21:04 -0800 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49133BCB.7070308@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> Message-ID: <491343A0.9090402@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: >> >>> Simo Sorce wrote: >>> >> >>>>> - Anyone with write access to the attribute will be able to change >>>>> the >>>>> contents, making them generally completely useless as audit trails. >>>>> Delegation of any minor task would require write access to >>>>> comments all >>>>> over the place. >>>>> >>>> >>> No. The whole idea is to make it non-editable at all. Only add. >>> >> >> Exactly this is something that does not exist in the LDAP model, nor in >> the ACI model we have. >> > I am pretty sure the plugin can take care of that. If not I would > agree that this is not a good idea. > Nathan? Rich? The plug-in could easily take care of this by intercepting operations on the comment attribute at the pre-op stage. That said, I belive that this can be accomplished by using the "targattrfilters" keyword in an ACI. You can allow one to add values for a targetted attribute, but not delete and values. Doing a replace operation requires both "add" and "del" privileges, so simply giving someone "add" privileges only for the comment attribute would acheive the desired result. > >> >>> Only later we might start diving into ACIs and deal with the >>> complexity of editing the data by admins that have different levels >>> of privileges. >>> >> Simo. >> >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From chorn at fluxcoil.net Thu Nov 6 20:14:39 2008 From: chorn at fluxcoil.net (Christian Horn) Date: Thu, 6 Nov 2008 21:14:39 +0100 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1225953550.10160.99.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> Message-ID: <20081106201439.GA11765@fluxcoil.net> Hi, On Thu, Nov 06, 2008 at 01:39:10AM -0500, Simo Sorce wrote: > On Wed, 2008-11-05 at 19:14 -0500, Dmitri Pal wrote: > > > > One of the ideas that we are exploring for IPA v2 is to have sort of > > ?commit comment? feature. The examples of the use cases are: Really good idea from ipa-user perspective, mainly for - making relations to troubleticket-system entries - making other admins aware on why something happened > I am not sure a mandatory comment on most operations would be welcome, I > think most Admins would just hate such a feature, as it would get in the > way of their job, this unless, perhaps, it pops up only once per "macro" > operation. > If it is not mandatory admins will simply skip it. One could use policys that say what can/has to be commented. Apart from knowledge that ldap-db are optimized for 95% reading oper- ations etc. i vote for storage directly in the DS. Cant help but thinking of subversion/cvs where that feature is of undisputed importance. I dont see a need for transactions yet that include several operations on the whole DS yet. One comment on was on commenting several operations grouped in one logical operation, one could comment this with the first operation, then get a link to that stored comment back from DS (i.e. a DN of the comment) and use that as comment for the other operations. A bit offtopic: really nice to see something like this getting created, good work :) Christian From dpal at redhat.com Thu Nov 6 21:00:45 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 16:00:45 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1225998966.10160.240.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1225998966.10160.240.camel@localhost.localdomain> Message-ID: <49135AFD.8080103@redhat.com> Simo Sorce wrote: > Ok let me reply first on the "log" nature of this information, will > answer to other stuff in a separate email. > > >> On Thu, 2008-11-06 at 13:47 -0500, Dmitri Pal wrote: >> >> "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS >> policy according to the decision made by the security board on 10/28/08" >> "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS >> policy to fix a security problem reported in ticket #2131209" >> > > This loloks like, a log, smells like a log, there fore it is a log. > > It is probbaly not an "application generated log", but it is at all > effects what in english you would call a log [1]. > > So please let's call stuff with its name. It is a log, it maybe some > special kind of log, but it is nothing more than a log. > > It maybe so special that it will require a special way to handle it, I > perfectly agree not all logs are created equal, but let's not fight on a > definition. > > [1] http://dictionary.reference.com/browse/log > > Simo. > > Ok. You are right but this is semantics. The main point is the handling. DS already has similar kind of data. Just one more - not a big deal. Let us not waste more time on this now. I guess I will explore it more and come up with prototype some time later. It I see that it is much harder than I anticipate I will step back and admit. If it deems to be simple we will continue this discussion. Thank you everybody for the input. Dmitri From mbooth at redhat.com Thu Nov 6 21:01:42 2008 From: mbooth at redhat.com (Matthew Booth) Date: Thu, 06 Nov 2008 21:01:42 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491313BA.1060303@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> Message-ID: <49135B36.8020704@redhat.com> Dmitri Pal wrote: > Yes I am talking about the situations when a change requires a formal > process. I was surprised when I faced the reality: big companies > especially banks do not do any changes until it has been approved, > scheduled, verified etc. It is a complex process. This is the situation > when the "commit log" feature is mostly valuable. > Put it differently it is for the environment where the administrator is > not allowed to do the changes to the system when and how he thinks > appropriate but rather has to follow a special procedure. > There are a lot of such companies. This is an excellent suggestion, Dmitri. This information can be crucial to auditors. In general, Windows does this much better than we do, and the auditors appreciate it. You make a critical point in the above, which is that the primary purpose of this feature is to integrate with an existing corporate change management solution. Given this, it would be quite exceptionally useful to be able to include site-specific, custom, structured data in the log, which can be correlated against an existing corporate system. For example, if a customer uses HP's ServiceDesk, the ServiceDesk reference number would be invaluable. This will contain the complete log of change request, to discussion to authorisation. There are a great many such systems in wide use, so the ability to integrate effectively with any of them would be a powerful feature. Matt -- Matthew Booth, RHCA, RHCSS Red Hat, Global Professional Services M: +44 (0)7977 267231 GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490 From dpal at redhat.com Thu Nov 6 21:11:50 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 16:11:50 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49135B36.8020704@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <49135B36.8020704@redhat.com> Message-ID: <49135D96.4010509@redhat.com> Matthew Booth wrote: > Dmitri Pal wrote: > >> Yes I am talking about the situations when a change requires a formal >> process. I was surprised when I faced the reality: big companies >> especially banks do not do any changes until it has been approved, >> scheduled, verified etc. It is a complex process. This is the situation >> when the "commit log" feature is mostly valuable. >> Put it differently it is for the environment where the administrator is >> not allowed to do the changes to the system when and how he thinks >> appropriate but rather has to follow a special procedure. >> There are a lot of such companies. >> > > This is an excellent suggestion, Dmitri. This information can be crucial > to auditors. In general, Windows does this much better than we do, and > the auditors appreciate it. > > You make a critical point in the above, which is that the primary > purpose of this feature is to integrate with an existing corporate > change management solution. Given this, it would be quite exceptionally > useful to be able to include site-specific, custom, structured data in > the log, which can be correlated against an existing corporate system. > For example, if a customer uses HP's ServiceDesk, the ServiceDesk > reference number would be invaluable. This will contain the complete log > of change request, to discussion to authorisation. There are a great > many such systems in wide use, so the ability to integrate effectively > with any of them would be a powerful feature. > > Matt > Thank for support. I think the main argument was not about the need of such information but rather where it belongs. Ultimately it belongs in good robust audit subsystem fully integrated into IPA. Here I agree with John. But we are not there yet. The only other reasonable place to put was to leave it in DS and that is where the whole argument started. I think that this is a really simple, fast, interim solution with a lot of benefits. I would be interesting to see if this argument is important from the perspective of the user of the IPA or not? Thanks Dmitri From ssorce at redhat.com Thu Nov 6 21:16:21 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 06 Nov 2008 21:16:21 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49133BCB.7070308@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> Message-ID: <1226006181.10160.338.camel@localhost.localdomain> On Thu, 2008-11-06 at 13:47 -0500, Dmitri Pal wrote: > Simo Sorce wrote: > > On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: > > > >> Simo Sorce wrote: > >> > > > > > >> Well, this is the main point. It would have been great if we had a > >> product that would be able to act as DS and DB at the same time. > >> I agree that this information would easier be managed in a DB. But we do > >> not have one. We can create one but then we face all the same issues as > >> we face with policies: > >> a) Replication > >> > > > > Why do we need to replicate a log database? It is not vital for the > > normal functioning of the Identity solution so it can well be a database > > or a file kept on a single server > > > > > You are continuing to view it from the wrong angle. Err sorry lets assume each one has his own Point of view, ok? > It is not a log database. It is not a log file. It is a property of the > entry. It should be stored close to the object it is related to. It is a log related to an entry (or potentially multiple entries). > It is not an audit data. You want this data to be viewable when you are > viewing or modifying the entry. Really? I don't think that normally I would care much about the change log. I do not look at the SCM log each time I modify something, I look at it only when something does not add up and I want to know what happened. > Making the UI go and pull this data from the external database would be > an huge overhead. No, because any sane admin will just pull this data when needed, it makes no sense to me to show the log of changes each time I edit an entry, it's not interesting when all goes well. This kind of data is interesting to retrieve only when there is reason to suspect something is wrong and you want to check what has been done. In this case having to wait a second or 2 for the interface to retrieve this information is really completely acceptable. > This data should be available regardless of which master you are > connecting to. Yes, but because it is not vital for the normal functionality of the server, but only to "check", generally after the fact or before an important modification, what is going on, this kind of information does not have the same kind of requirements the rest of the data has. Assuming you put this data in a database, even if this data is not available for some hours, it is really not important. > Across different data stores? It does not matter what you use as a > foreign key. The whole notion of having two data stores and creating > referential integrity between the two make be cry. Yes, but I think it is a better solution and would help shaping how the audit database could work too. > Been there, done that. It is extremely complex and nearly impossible to > make it right regardless whether it is a DB+DB or DB+DS or DS+File or > any other combination. It's not like this is something new in our field, I've seen this done in various products. > It is always more complex than a synch task. IT is not a synchronization task. > >> c) Backup - restore > >> > > > > Not sure why this would make any difference. We do not offer any special > > backup/restore feature with IPA so it would be just business as usual. > > > > > If you have more than one data stores you have to make sure that your > backup and restore are also synched otherwise the referential integrity > goes out of the window. No, at most you may loose some of the logs, that would be regrettable, but given the fact this information is not necessary for the server to work it is not that important. Besides, a "restore" in the IPA world is a "end of the world" situation, it should really be a "disaster recovery" situation, therefore if the log database is not perfectly inline (some more entries since the last backup or lost some) it is really a very minor concern. In any case given the nature of the log (entry-bound) at most you have a problem of missing the very latest logs, or having logs for operations that have been rolled back by the restore. In the first case you can just live with the fact, you can also probably reconstruct most of it, in the second it will A) actually help you understand what is left to do to get back on par with the previous situation, then it can be wipe out based on the date the DS backup was made. > We do not have a problem in IPA because we do not have any storage that > should synched with other storage (except AD sycnh and you know how not > simple it is :-) ). AD synch is a completely different kind of problem (quoting you, let's not dive into apple to oranges comparisons). > This is why I was strongly for storing policies in the DS itself. It > solves a lot of problems you never even want to think about. > Same is here. Policies are different, that's why, even if I do not like it too much, I am ok with storing them in DS, they are a vital part of the information we need to distribute, and we need to scale in the way we provide them. These logs are neither vital, nor will break anything even if they suddenly vanish. > >> I think that creating a parallel infrastructure for policies or for > >> commit logs is a huge overhead. > >> > > > > I think logs and policies are completely different in nature, these logs > > are just audit trails in nature, and should be rather be developed as > > part of the audit system. > > > > They are not audit trails. They are comments that contain information. > Like this. > For example on the SUDO policy object: > > "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS > policy according to the decision made by the security board on 10/28/08" > "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS > policy to fix a security problem reported in ticket #2131209" > > Is this audit? No. It looks like one but it is not. It has one important > piece of information: why? who authorized the change to the the critical > infrastructure. It looks like a log, smells like a log ... > If we treat it as pure audit log record we will loose its relevance. Why ? What matters is what information we store, not how we store it. > It > will go into the audit server and it would be possible to pull it from > there on demand but not in the context of modification of the entry in > DS. I really fail to see why this is relevant. The data itself is not necessary for the modification you are going to make. > Making the UI do this search against the audit server would be an > overhead and there is no guarantee that this data is not archived or > cleaned from audit DB. It would be an overhead only if you fetch it every time you search the object, I really do no think that is necessary, I even think it is really not what admins want. But even if that is what some admin want, we can use caches to solve performances problems, let's not try to optimize first, let's first use the right mechanisms and optimize later. > When it is in DS and close to the object we can guarantee that it is > complete, relevant and always accessible when the entry is viewed and > administrator is about to make a critical change to the system. The change maybe critical, but the log is certainly not. Some place may "require" the admin to fill a change log, but yet in itself it is not at all critical for the functioning of the server. > I think that since we a re a security product any changes to the > policies that define access rules should have a comment facility like this. It is a "nice to have", but not a requirement, other popular solutions in this area do not have it and they still thrive. > Parsing is not a problem. Manging is the problem. Access control, > replication, backup etc. > I do not want to duplicate all the arguments again. Sorry but these are some of the core arguments, you give some of them for granted while I do not so maybe we should come to an agreement about them first, or we will just keep disagreeing on the consequences because we have different premises. Access control: as you said there is no special access control needed, it should be just read and append only. This is something quite simple to achieve, it is what every single log facility I know implements. Replication: again replication is important when data availability is important for the normal operation of a server. This log data is important for someone that needs to check the history of some changes, but not for the availability of the identity or policy service, so even a single point of failure server is probably good enough in most cases. > > NO, it will just be softened, you still replicate around data useless > > 99% of the time. > > > > > > No I checked. According to DS gurus the operation is replicated not the > data so it is not an issue. What I meant to say is that you are still sending this operation to all masters and replicas, and you are sending the full set of data when you create a new replica. > Check with Nathan and Rich if you do not believe me. This was one of the > first things I checked because i was concerned about this too. I've read the replication code in DS and studied ldap replication for some time myself, I know how it works, you misunderstood what I wrote. > >> IMO they should be tightly related to the object they refer > >> to and not be somewhere else in the log server. > > > > Why? > > Because when something goes wrong it should be there. The policies are > too critical to the entire enterprise. > It is much easier to find what was going on right there with that kind > of comments . It depends on what is going wrong, but unless you put your database on the most unreliable server you could find this data will be there. This data is not critical for the identity or policy distribution services, so I really fail to see how the log database availability would really impact any *important* operation. > Have you ever done this? I did. It is not that simple. It's not that complex either *for this specific tasks in the way you have described it*. > The main reason to have it is to have an answer to "why somone did what > he did". And it should be there at hand so that when next guy comes in > and tries to clean the mess he knows why the previous change was made > and who authorized it. Then re can turn to audit system and dig more but > the recorded comment will give him a very good starting point. Yes, but this information is *not* critical for the service. Some people may see it as critical on an information management level, but it is not at the data layer level, and from the point of view of the importance of the information can equally be stored somewhere else. > This is all about authority and responsibility. It's a nice to have yes, but not critical, it can be safely stored in another data storage. Another storage also allows you to separate privileges, so that if one storage is compromised the other is not necessarily compromised as well, and this is usually more important for critical logs. > >>>> Now to some of the reasons why I don't see DS as a viable option: > >>>> > >>>> - Multi-value attributes are not ordered, so you need to invent some > >>>> scheme to store this data structured so that ordering can be preserved. > >>>> Sure probably using the "posting" date before the content is all is > >>>> needed, but that makes attributes not searcheable. > >>>> > >>>> > >>>> > >> I think that creating a generic plugin that would allow storing ordered > >> MV attributes would be a big benefit for everybody. > >> > > > > The problem is that ordered MV are not defined in the LDAP protocol, you > > would need to come up with a standardizable way to manage how to add > > entries so that they fall in the precise order you need them to. This > > would mean modifying the add operation either with a control or by > > creating an extended operation. > > > > > I will see what RFCs exist for this case. Rich pointed out that OpenLDAP > has it. It's a proposal, it is not an official RFC yet AFAIK. Anyway Rich also pointed out that it may not be a trivial task to implement it because the slapi API may not exposes enough hooks deep into DS core for it. I totally agree that it would be a nice feature to have in general, would be even better if formalized in an official approved RFC though. > I do not see what you are talking about. There is no change to the > protocol at all. I am sorry I used "protocol" in a broad way in this sentence, meaning that currently there is no official way to store ordered multi-value attributes. At the very least it requires creating a convention like it has been done in the OpenLDAP extension, but then you need to make sure the client code you use is able to understand it and provide you functions to use that convention. Current client libraries are built with the knowledge that multivalue attributes order is not important so some bindings may decide to reorder elements at will before returning them to the caller. > There are tons of RFCs that store complex structures in the attribute. > I do not see how my solution is different from those. To be honest I think that implementing ordering is not even necessary to fulfill your goal, all you need is to store a date in the single attribute and use it to reorder entries in the UI for display. The only reason to build a plugin, I think, would be to assure "append-only" behavior, if really required. But given the fact you said this is not an audit trail I am not even sure we should enforce that at the DS level, it could just be an XML-RPC interface standard practice to just append and never delete old comments. > We can show last X comments in the UI. This is irrelevant. You want to > know how this whole thing ended up in the state it is in and why. > Then you can come up with the effective remediation. You can look at > just last X comments or the whole stack is up to you. > Whatever is needed for you as admin to make the right decision. Except to cast blame I do not think that going back years in the comments would be all that useful, but I don't have data to back this impression so that is not important, any storage can be made to keep comments indefinitely. > These comments are useless. They do not answer "why". I did the change > because my manager authorized me or I responded to ticket #XXXXXX or > security board authorized me - this is what should be put into the > comment. Are you going to enforce a format that grants information is stored this way? If not just face the fact that lot of people will not be so disciplinate and will put in garbage like that. > Not "foo needs access to x as admin". Such comment is the > duplication of the even and bears no value. The comment should contain > information that links the event (changing of the policy) to the formal > process that authorized this change. Hope this clarifies the purpose and > the difference. Sure, but this is kind of stuff is a policy that will certainly differ per environment. You have no control on what people will put in it, and if you try to force a too strict control on the format you will probably alienate all the ones that need something slightly different (or they will just hack it in). As far as I can see all you can record is: - who did the change - when the change was made - which objects were affected and finally - a user comment that *hopefully* describes why Did I say this looks like a log, smells.. :-) > We can attach it to multiple different objects but the system > administrator - the "root" of the whole IPA deployment - will have a way > to say where this is mandatory, where optional, and where it should be > hidden. The UI and CLI will respect these settings. Are you thinking of rejecting changes to an ldap object if this attribute is not added, ie enforcing comments at the ldap operation level ? This proposal would require a host of new considerations to make, and it would seem to me completely disproportionate and perhaps even dangerous. > We can store whatever we want in the DS. IMO the 2307 is a good > example > of cluttering LDAP especially netgroups schema :-) Yeah I have a special personal grief against some of the schemas we inherited ... my latest one is the automount one I guess :-) > I think we are playing much more nicely than this and other RFCs. I think it depends on the point of view. I think it is not appropriate to store this kind of data in LDAP itself. It should be implemented at a higher level imo, we are already putting a lot of complexity in DS lately, we need to be more moderate, too much change will certainly bite back. Even if we get to the conclusion that this is something we want to do, I think it is way too much for v2, I would strongly suggest to postpone it to later. > > Reality is people will touch multiple objects when changing stuff > > around, and transactions really do not matter in this case (see the > > angry-admin example I posted in the previous email). > > > > > > This is why we have CLI tools and UI so that things can be done > properly. Messing with raw data is always dangerous if you do not know > what you are doing. Uhm sorry but I do not understand the context of this comment. > Anyone can read, the one who can edit the entry can add. > The one who can delete the entry can delete all values together. > Noone can modify. > > I think plugin can easily enforce this logic. > This is where plugin will reject the attempt to write. Why would you need to enforce it at the DS level if this is not an audit trail ? > > I don't think so, if I correctly understood how you want to manage it. > > And btw the way you seem to be willing to manage it once again resembles > > an audit trail log, and that's I think because it ultimately is just an > > audit trail log. > > > > > See the logic in the previous comment. I am really sorry, but I think I still fail to fully see it. > I am pretty sure the plugin can take care of that. If not I would > agree that this is not a good idea. > Nathan? Rich? I can tell you there is no concept of append-only attribute in LDAP. But see above, why should we enforce this at the LDAP level ? > The solution if possible is self contained and does not rely on any > external piece of functionality - audit server (that would not be up to > the task for quite some time). Then lets delay it. > Adjusting DS is much simpler (I think) than building a logic of logging > this into external store (file or DB) and then pulling it out when we > need to edit policy and see who was messing with it and why. It might be simpler for a limited task, but it is the "wrong way" to do it long term IMO, in fact no other solution does something like this, they all rely on external storage for auditing purposes or external applications if a company wants to implement a formal protocol. > I think that this is a value add and something that is neede for the > project to be successful. I seriously see it just as a marginal nice to have at this stage. No one of the people I have been talking to ever raised the necessity to have something like this implemented at a so low level. And we have a lot more to do to have the basic foundations up running then caring about something like this imo. > Otherwise I would not have suggested that. You know I am against any > unnecessary work myself. I know, but I still think this is not something necessary at this stage and implemented this way. > I see that without this feature the adoption of the IPA will be slower > since the feature allows tying > the data to the formal processes established in the company. No other Identity management tool at our stage of development has it, and none have it at the LDAP level, I think, with good reason. > I would agree that if we had a robust audit server capable of doing real > time searches now I would have explored the log approach and would have > considered trying to hook into it. > But it is not there and it would take quite a while to be there so log > approach is IMO a non starter. Then we will wait until it becomes available and concentrate our efforts on it. Short term quick hacks should not drive core changes in DS, really I am quite opposed to this line of reasoning. > > I think you really are greatly underestimating the complexity of writing > > a DS plugin, > I have seen Nalin do it. I have done it myself! More than once. And also seen some unfortunate cases also quite recently where we had to go in and almost rewrite the entire plugin because it came out fundamentally flawed on the first implementation (dna, memberof). > > the complexity of dealing with changing LDAP semantics. > No changes to LDAP semantics. No changes to protocol Changing a multivalue attribute to be ordered is a change in semantics, so far LDAP multivalued attributes are not ordered, and applications rely/account/exploit/endure that. > > > And > > the development cycle involved with all the bugs custom code would > > entail. > All the code we write is a custom code. This is not an argument :-) Unfortunate choice of wording on my side, but adding code to DS shouldn't be taken lightly. It is a critical piece of infrastructure, every line of code we add is much more critical then any 100 lines of python we add in the UI. > > A plugin also always adds security concerns as it runs as a > > privileged process wrt DS data. > > Yes let us remove all the plugins. They are potentially insecure! Let us > not publish the APIs so that noone can create one... > This argument sounds really funny from you :-) I don't find funny to be cautious and avoid unnecessary risk in a critical piece of code. Code that run in DS and code that runs in the interface have completely different ciriticality*. Therefore only "necessary" plugins see my approval, "handful" ones don't. * (is this an english word? :-). > > It add stability concerns as it runs > > inside a threaded application (one segfault and goodbye LDAP server). > > > > > It is as stable as any other plugin. Until it fails it is stable as anything else, but right now it is nothing it does not exists :) > And it is much simpler than the NIS > plugin Nalin put together. No caching or memory pools. The NIS plugin could not be easily be built as an external process. > I do not see any complexity to worry about. May be I am wrong but unless > I dive deeper I do not see a problem you are talking about. It's a matter of risk and long term support, the more the code the more the risk and the support burden, amplified by the fact that this code runs in a critical service. > Just follow the rules of plugin development and do the right thing. That does not mitigate risk, we always try to make perfect code, but it is clear that perfect code does not exist or we would have no bugs reported ever. > There is no need to to get or update multiple entries - just couple > attributes in one and the same entry - what a big deal? A matter of perspective. > > For a database all you need is a schema and a few SQL queries to > > insert/extract data. you don't have ordering problems there, you can > > split data and metadata and have own tables. Writing an small database > > schema with a couple of tables is honestly orders of magnitude simpler > > than writing a plugin in C that has to do what you would like it to do. > > > > > Really? And then you need to write installation scripts, dump and load > utilities, access control rights, backup and restore, replication and > other utilities. Every decent database provide most of this already. > Even if the DS provider has them you now suggenly have > to deal with all this and manager and document and test... This is far > more than you think. I know what that means to use an embedded DB. The > whole server I worked with for 10 years had an embedded DB. Creating > schema and getting data is small part of the puzzle other utilities are > the main burden. I am not proposing an embedded DB, it would make no sense. I would agree with you if I were to propose something like that, but I was thinking of just a basic mysql, postgresql or something like that, nothing scary or complex. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Thu Nov 6 21:38:10 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 06 Nov 2008 14:38:10 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1226006181.10160.338.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> Message-ID: <491363C2.9010306@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 13:47 -0500, Dmitri Pal wrote: > >> Simo Sorce wrote: >> >>> On Thu, 2008-11-06 at 10:56 -0500, Dmitri Pal wrote: >>> >>> >>>> Simo Sorce wrote: >>>> >>>> >>> >>> >>>> Well, this is the main point. It would have been great if we had a >>>> product that would be able to act as DS and DB at the same time. >>>> I agree that this information would easier be managed in a DB. But we do >>>> not have one. We can create one but then we face all the same issues as >>>> we face with policies: >>>> a) Replication >>>> >>>> >>> Why do we need to replicate a log database? It is not vital for the >>> normal functioning of the Identity solution so it can well be a database >>> or a file kept on a single server >>> >>> >>> >> You are continuing to view it from the wrong angle. >> > > Err sorry lets assume each one has his own Point of view, ok? > > >> It is not a log database. It is not a log file. It is a property of the >> entry. It should be stored close to the object it is related to. >> > > It is a log related to an entry (or potentially multiple entries). > > >> It is not an audit data. You want this data to be viewable when you are >> viewing or modifying the entry. >> > > Really? I don't think that normally I would care much about the change > log. I do not look at the SCM log each time I modify something, I look > at it only when something does not add up and I want to know what > happened. > There are several ways to hide this data from "normal" searches 1) make them operational attributes - this is how passwordHistory works 2) use ACIs to hide them 3) have a custom search plugin intercept and hide them unless a control or something like that is present > >> Making the UI go and pull this data from the external database would be >> an huge overhead. >> > > No, because any sane admin will just pull this data when needed, it > makes no sense to me to show the log of changes each time I edit an > entry, it's not interesting when all goes well. > See above > This kind of data is interesting to retrieve only when there is reason > to suspect something is wrong and you want to check what has been done. > In this case having to wait a second or 2 for the interface to retrieve > this information is really completely acceptable. > > >> This data should be available regardless of which master you are >> connecting to. >> > > Yes, but because it is not vital for the normal functionality of the > server, but only to "check", generally after the fact or before an > important modification, what is going on, this kind of information does > not have the same kind of requirements the rest of the data has. > > Assuming you put this data in a database, even if this data is not > available for some hours, it is really not important. > > >> Across different data stores? It does not matter what you use as a >> foreign key. The whole notion of having two data stores and creating >> referential integrity between the two make be cry. >> > > Yes, but I think it is a better solution and would help shaping how the > audit database could work too. > > >> Been there, done that. It is extremely complex and nearly impossible to >> make it right regardless whether it is a DB+DB or DB+DS or DS+File or >> any other combination. >> > > It's not like this is something new in our field, I've seen this done in > various products. > > >> It is always more complex than a synch task. >> > > IT is not a synchronization task. > > >>>> c) Backup - restore >>>> >>>> >>> Not sure why this would make any difference. We do not offer any special >>> backup/restore feature with IPA so it would be just business as usual. >>> >>> >>> >> If you have more than one data stores you have to make sure that your >> backup and restore are also synched otherwise the referential integrity >> goes out of the window. >> > > No, at most you may loose some of the logs, that would be regrettable, > but given the fact this information is not necessary for the server to > work it is not that important. > Besides, a "restore" in the IPA world is a "end of the world" situation, > it should really be a "disaster recovery" situation, therefore if the > log database is not perfectly inline (some more entries since the last > backup or lost some) it is really a very minor concern. > In any case given the nature of the log (entry-bound) at most you have a > problem of missing the very latest logs, or having logs for operations > that have been rolled back by the restore. In the first case you can > just live with the fact, you can also probably reconstruct most of it, > in the second it will A) actually help you understand what is left to do > to get back on par with the previous situation, then it can be wipe out > based on the date the DS backup was made. > > >> We do not have a problem in IPA because we do not have any storage that >> should synched with other storage (except AD sycnh and you know how not >> simple it is :-) ). >> > > AD synch is a completely different kind of problem (quoting you, let's > not dive into apple to oranges comparisons). > > >> This is why I was strongly for storing policies in the DS itself. It >> solves a lot of problems you never even want to think about. >> Same is here. >> > > Policies are different, that's why, even if I do not like it too much, I > am ok with storing them in DS, they are a vital part of the information > we need to distribute, and we need to scale in the way we provide them. > > These logs are neither vital, nor will break anything even if they > suddenly vanish. > > >>>> I think that creating a parallel infrastructure for policies or for >>>> commit logs is a huge overhead. >>>> >>>> >>> I think logs and policies are completely different in nature, these logs >>> are just audit trails in nature, and should be rather be developed as >>> part of the audit system. >>> >>> >> They are not audit trails. They are comments that contain information. >> Like this. >> For example on the SUDO policy object: >> >> "DPal 11/05/08 12:15PM: Added new group "contractors" to the SUDOERS >> policy according to the decision made by the security board on 10/28/08" >> "SSorce 11/17/08 2:47PM: Removed group "contractors" from the SUDOERS >> policy to fix a security problem reported in ticket #2131209" >> >> Is this audit? No. It looks like one but it is not. It has one important >> piece of information: why? who authorized the change to the the critical >> infrastructure. >> > > It looks like a log, smells like a log ... > > >> If we treat it as pure audit log record we will loose its relevance. >> > > Why ? What matters is what information we store, not how we store it. > > >> It >> will go into the audit server and it would be possible to pull it from >> there on demand but not in the context of modification of the entry in >> DS. >> > > I really fail to see why this is relevant. The data itself is not > necessary for the modification you are going to make. > > >> Making the UI do this search against the audit server would be an >> overhead and there is no guarantee that this data is not archived or >> cleaned from audit DB. >> > > It would be an overhead only if you fetch it every time you search the > object, I really do no think that is necessary, I even think it is > really not what admins want. But even if that is what some admin want, > we can use caches to solve performances problems, let's not try to > optimize first, let's first use the right mechanisms and optimize later. > There are ways to do this that do not involve every search operation retrieving all of this data. > >> When it is in DS and close to the object we can guarantee that it is >> complete, relevant and always accessible when the entry is viewed and >> administrator is about to make a critical change to the system. >> > > The change maybe critical, but the log is certainly not. Some place may > "require" the admin to fill a change log, but yet in itself it is not at > all critical for the functioning of the server. > Sure - critical to directory server operation != critical to enterprise operation > >> I think that since we a re a security product any changes to the >> policies that define access rules should have a comment facility like this. >> > > It is a "nice to have", but not a requirement, other popular solutions > in this area do not have it and they still thrive. > > >> Parsing is not a problem. Manging is the problem. Access control, >> replication, backup etc. >> I do not want to duplicate all the arguments again. >> > > Sorry but these are some of the core arguments, you give some of them > for granted while I do not so maybe we should come to an agreement about > them first, or we will just keep disagreeing on the consequences because > we have different premises. > > Access control: as you said there is no special access control needed, > it should be just read and append only. This is something quite simple > to achieve, it is what every single log facility I know implements. > > Replication: again replication is important when data availability is > important for the normal operation of a server. This log data is > important for someone that needs to check the history of some changes, > but not for the availability of the identity or policy service, so even > a single point of failure server is probably good enough in most cases. > > >>> NO, it will just be softened, you still replicate around data useless >>> 99% of the time. >>> >>> >>> >> No I checked. According to DS gurus the operation is replicated not the >> data so it is not an issue. >> > > What I meant to say is that you are still sending this operation to all > masters and replicas, and you are sending the full set of data when you > create a new replica. > Right. > >> Check with Nathan and Rich if you do not believe me. This was one of the >> first things I checked because i was concerned about this too. >> > > I've read the replication code in DS and studied ldap replication for > some time myself, I know how it works, you misunderstood what I wrote. > > >>>> IMO they should be tightly related to the object they refer >>>> to and not be somewhere else in the log server. >>>> >>> Why? >>> >> Because when something goes wrong it should be there. The policies are >> too critical to the entire enterprise. >> It is much easier to find what was going on right there with that kind >> of comments . >> > > It depends on what is going wrong, but unless you put your database on > the most unreliable server you could find this data will be there. > This data is not critical for the identity or policy distribution > services, so I really fail to see how the log database availability > would really impact any *important* operation. > > >> Have you ever done this? I did. It is not that simple. >> > > It's not that complex either *for this specific tasks in the way you > have described it*. > > >> The main reason to have it is to have an answer to "why somone did what >> he did". And it should be there at hand so that when next guy comes in >> and tries to clean the mess he knows why the previous change was made >> and who authorized it. Then re can turn to audit system and dig more but >> the recorded comment will give him a very good starting point. >> > > Yes, but this information is *not* critical for the service. > Some people may see it as critical on an information management level, > but it is not at the data layer level, and from the point of view of the > importance of the information can equally be stored somewhere else. > > >> This is all about authority and responsibility. >> > > It's a nice to have yes, but not critical, it can be safely stored in > another data storage. > > Another storage also allows you to separate privileges, so that if one > storage is compromised the other is not necessarily compromised as well, > and this is usually more important for critical logs. > > >>>>>> Now to some of the reasons why I don't see DS as a viable option: >>>>>> >>>>>> - Multi-value attributes are not ordered, so you need to invent some >>>>>> scheme to store this data structured so that ordering can be preserved. >>>>>> Sure probably using the "posting" date before the content is all is >>>>>> needed, but that makes attributes not searcheable. >>>>>> >>>>>> >>>>>> >>>>>> >>>> I think that creating a generic plugin that would allow storing ordered >>>> MV attributes would be a big benefit for everybody. >>>> >>>> >>> The problem is that ordered MV are not defined in the LDAP protocol, you >>> would need to come up with a standardizable way to manage how to add >>> entries so that they fall in the precise order you need them to. This >>> would mean modifying the add operation either with a control or by >>> creating an extended operation. >>> >>> >>> >> I will see what RFCs exist for this case. Rich pointed out that OpenLDAP >> has it. >> > > It's a proposal, it is not an official RFC yet AFAIK. > Anyway Rich also pointed out that it may not be a trivial task to > implement it because the slapi API may not exposes enough hooks deep > into DS core for it. > I totally agree that it would be a nice feature to have in general, > would be even better if formalized in an official approved RFC though. > > >> I do not see what you are talking about. There is no change to the >> protocol at all. >> > > I am sorry I used "protocol" in a broad way in this sentence, meaning > that currently there is no official way to store ordered multi-value > attributes. At the very least it requires creating a convention like it > has been done in the OpenLDAP extension, but then you need to make sure > the client code you use is able to understand it and provide you > functions to use that convention. Current client libraries are built > with the knowledge that multivalue attributes order is not important so > some bindings may decide to reorder elements at will before returning > them to the caller. > It could be done solely on the server side without using the openldap proposed extension e.g. just have the server store them in timestamp order and /or replay them in that order in search operations. That doesn't break LDAP afaik, but would run afoul of clients who might order the values in some other way (which would be none of our clients). > >> There are tons of RFCs that store complex structures in the attribute. >> I do not see how my solution is different from those. >> > > To be honest I think that implementing ordering is not even necessary to > fulfill your goal, all you need is to store a date in the single > attribute and use it to reorder entries in the UI for display. > > The only reason to build a plugin, I think, would be to assure > "append-only" behavior, if really required. > But given the fact you said this is not an audit trail I am not even > sure we should enforce that at the DS level, it could just be an XML-RPC > interface standard practice to just append and never delete old > comments. > > >> We can show last X comments in the UI. This is irrelevant. You want to >> know how this whole thing ended up in the state it is in and why. >> Then you can come up with the effective remediation. You can look at >> just last X comments or the whole stack is up to you. >> Whatever is needed for you as admin to make the right decision. >> > > Except to cast blame I do not think that going back years in the > comments would be all that useful, but I don't have data to back this > impression so that is not important, any storage can be made to keep > comments indefinitely. > > >> These comments are useless. They do not answer "why". I did the change >> because my manager authorized me or I responded to ticket #XXXXXX or >> security board authorized me - this is what should be put into the >> comment. >> > > Are you going to enforce a format that grants information is stored this > way? > If not just face the fact that lot of people will not be so disciplinate > and will put in garbage like that. > Not if it is organizational policy that says all changes must have a reason. Then it's not really up to us to enforce it. > >> Not "foo needs access to x as admin". Such comment is the >> duplication of the even and bears no value. The comment should contain >> information that links the event (changing of the policy) to the formal >> process that authorized this change. Hope this clarifies the purpose and >> the difference. >> > > Sure, but this is kind of stuff is a policy that will certainly differ > per environment. You have no control on what people will put in it, and > if you try to force a too strict control on the format you will probably > alienate all the ones that need something slightly different (or they > will just hack it in). > > As far as I can see all you can record is: > - who did the change > - when the change was made > - which objects were affected > and finally > - a user comment that *hopefully* describes why > Again, the user comment content is probably unenforceable from the server standpoint (I suppose we could add simple rule checking as is done for cvs commit log messages, for example e.g. must contain a bz number). But I don't see why that invalidates the concept altogether. > Did I say this looks like a log, smells.. :-) > > >> We can attach it to multiple different objects but the system >> administrator - the "root" of the whole IPA deployment - will have a way >> to say where this is mandatory, where optional, and where it should be >> hidden. The UI and CLI will respect these settings. >> > > Are you thinking of rejecting changes to an ldap object if this attribute is not added, ie enforcing comments at the ldap operation level ? > This proposal would require a host of new considerations to make, and it would seem to me completely disproportionate and perhaps even dangerous. > > >> We can store whatever we want in the DS. IMO the 2307 is a good >> example >> of cluttering LDAP especially netgroups schema :-) >> > > Yeah I have a special personal grief against some of the schemas we > inherited ... my latest one is the automount one I guess :-) > > >> I think we are playing much more nicely than this and other RFCs. >> > > I think it depends on the point of view. I think it is not appropriate > to store this kind of data in LDAP itself. It should be implemented at a > higher level imo, we are already putting a lot of complexity in DS > lately, we need to be more moderate, too much change will certainly bite > back. > > Even if we get to the conclusion that this is something we want to do, I > think it is way too much for v2, I would strongly suggest to postpone it > to later. > > >>> Reality is people will touch multiple objects when changing stuff >>> around, and transactions really do not matter in this case (see the >>> angry-admin example I posted in the previous email). >>> >>> >>> >> This is why we have CLI tools and UI so that things can be done >> properly. Messing with raw data is always dangerous if you do not know >> what you are doing. >> > > Uhm sorry but I do not understand the context of this comment. > > >> Anyone can read, the one who can edit the entry can add. >> The one who can delete the entry can delete all values together. >> Noone can modify. >> >> I think plugin can easily enforce this logic. >> > > >> This is where plugin will reject the attempt to write. >> > > Why would you need to enforce it at the DS level if this is not an audit > trail ? > > >>> I don't think so, if I correctly understood how you want to manage it. >>> And btw the way you seem to be willing to manage it once again resembles >>> an audit trail log, and that's I think because it ultimately is just an >>> audit trail log. >>> >>> >>> >> See the logic in the previous comment. >> > > I am really sorry, but I think I still fail to fully see it. > > >> I am pretty sure the plugin can take care of that. If not I would >> agree that this is not a good idea. >> Nathan? Rich? >> > > I can tell you there is no concept of append-only attribute in LDAP. > There is no standard access control in LDAP either. > But see above, why should we enforce this at the LDAP level ? > > >> The solution if possible is self contained and does not rely on any >> external piece of functionality - audit server (that would not be up to >> the task for quite some time). >> > > Then lets delay it. > > >> Adjusting DS is much simpler (I think) than building a logic of logging >> this into external store (file or DB) and then pulling it out when we >> need to edit policy and see who was messing with it and why. >> > > It might be simpler for a limited task, but it is the "wrong way" to do > it long term IMO, in fact no other solution does something like this, > they all rely on external storage for auditing purposes or external > applications if a company wants to implement a formal protocol. > > >> I think that this is a value add and something that is neede for the >> project to be successful. >> > > I seriously see it just as a marginal nice to have at this stage. No one > of the people I have been talking to ever raised the necessity to have > something like this implemented at a so low level. And we have a lot > more to do to have the basic foundations up running then caring about > something like this imo. > > >> Otherwise I would not have suggested that. You know I am against any >> unnecessary work myself. >> > > I know, but I still think this is not something necessary at this stage > and implemented this way. > > >> I see that without this feature the adoption of the IPA will be slower >> since the feature allows tying >> the data to the formal processes established in the company. >> > > No other Identity management tool at our stage of development has it, > and none have it at the LDAP level, I think, with good reason. > > >> I would agree that if we had a robust audit server capable of doing real >> time searches now I would have explored the log approach and would have >> considered trying to hook into it. >> But it is not there and it would take quite a while to be there so log >> approach is IMO a non starter. >> > > Then we will wait until it becomes available and concentrate our efforts > on it. Short term quick hacks should not drive core changes in DS, > really I am quite opposed to this line of reasoning. > > >>> I think you really are greatly underestimating the complexity of writing >>> a DS plugin, >>> >> I have seen Nalin do it. >> > > I have done it myself! More than once. And also seen some unfortunate > cases also quite recently where we had to go in and almost rewrite the > entire plugin because it came out fundamentally flawed on the first > implementation (dna, memberof). > > >>> the complexity of dealing with changing LDAP semantics. >>> >> No changes to LDAP semantics. No changes to protocol >> > > Changing a multivalue attribute to be ordered is a change in semantics, > so far LDAP multivalued attributes are not ordered, and applications > rely/account/exploit/endure that. > But having the server return ordered attributes does not break LDAP in any way, it will only confuse apps that expect them to be returned in random order. > >>> And >>> the development cycle involved with all the bugs custom code would >>> entail. >>> >> All the code we write is a custom code. This is not an argument :-) >> > > Unfortunate choice of wording on my side, but adding code to DS > shouldn't be taken lightly. It is a critical piece of infrastructure, > every line of code we add is much more critical then any 100 lines of > python we add in the UI. > > >>> A plugin also always adds security concerns as it runs as a >>> privileged process wrt DS data. >>> >> Yes let us remove all the plugins. They are potentially insecure! Let us >> not publish the APIs so that noone can create one... >> This argument sounds really funny from you :-) >> > > I don't find funny to be cautious and avoid unnecessary risk in a > critical piece of code. Code that run in DS and code that runs in the > interface have completely different ciriticality*. Therefore only > "necessary" plugins see my approval, "handful" ones don't. > > * (is this an english word? :-). > criticality > >>> It add stability concerns as it runs >>> inside a threaded application (one segfault and goodbye LDAP server). >>> >>> >>> >> It is as stable as any other plugin. >> > Until it fails it is stable as anything else, but right now it is > nothing it does not exists :) > > >> And it is much simpler than the NIS >> plugin Nalin put together. No caching or memory pools. >> > > The NIS plugin could not be easily be built as an external process. > > >> I do not see any complexity to worry about. May be I am wrong but unless >> I dive deeper I do not see a problem you are talking about. >> > > It's a matter of risk and long term support, the more the code the more > the risk and the support burden, amplified by the fact that this code > runs in a critical service. > > >> Just follow the rules of plugin development and do the right thing. >> > > That does not mitigate risk, we always try to make perfect code, but it > is clear that perfect code does not exist or we would have no bugs > reported ever. > > >> There is no need to to get or update multiple entries - just couple >> attributes in one and the same entry - what a big deal? >> > > A matter of perspective. > > >>> For a database all you need is a schema and a few SQL queries to >>> insert/extract data. you don't have ordering problems there, you can >>> split data and metadata and have own tables. Writing an small database >>> schema with a couple of tables is honestly orders of magnitude simpler >>> than writing a plugin in C that has to do what you would like it to do. >>> >>> >>> >> Really? And then you need to write installation scripts, dump and load >> utilities, access control rights, backup and restore, replication and >> other utilities. >> > > Every decent database provide most of this already. > > >> Even if the DS provider has them you now suggenly have >> to deal with all this and manager and document and test... This is far >> more than you think. I know what that means to use an embedded DB. The >> whole server I worked with for 10 years had an embedded DB. Creating >> schema and getting data is small part of the puzzle other utilities are >> the main burden. >> > > I am not proposing an embedded DB, it would make no sense. I would agree > with you if I were to propose something like that, but I was thinking of > just a basic mysql, postgresql or something like that, nothing scary or > complex. > > Simo. > > From dpal at redhat.com Thu Nov 6 22:53:20 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 17:53:20 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1226006181.10160.338.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> Message-ID: <49137560.7030705@redhat.com> Summary: 1) We all agree that: a) Providing commit comments is valuable feature for users of the IPA b) We can't force (and should not) the user to put some meaningful data in such comments. This is the responsibility of the corporate policy - not software. c) It should be flexible so that only in the cases when the corporate policy requires that kind of comment it would be enforced. Otherwise it should be optional or even hidden to avoid annoying administrator of the system. 2) We disagree mainly on the mean where this data should be stored. Main point is does it belong to DS or not. a) The argument to not put it in DS is that this data does not belong there. It is perceived as a log and thus should be stored in the audit system. b) Other arguments include the fact that we should avoid developing unnecessary DS plugins until there is absolute need because the bugs in plugins can bring the whole server down. c) This data is not critical for functioning of the server so should not be in DS etc. The argument for storing in DS is: a) There is no other place to store it. Audit system will not be robust enough soon enough to fit the bill (especially real time lookups) b) Many other features require plugins so what a big deal about one more c) The amount of work (may be erroneously is perceived as smaller than using other alternatives). I will not explore that more. One can read the thread. d) There are already similar things in the DS that do things in pretty much the same way e) The DS experts do not see a big issue with the approach and see a value down the road f) The company policies might require that the changes to the critical object be commented . Without this feature and DS plugin this can't be enforced. If it is done in UI or CLI the admin might circumvent it by using ldap calls directly. So DS is the only common denominator. I strongly believe that based on the last reason it should be done in DS plugin and only there. It can be done in different ways though. For example one could suggest that the DS plugin can just require the comment to be inserted on each add/modify of an object and save it to a log file that then can be processed by the audit system. We can do this but if we agree that DS plugin is anyway inevitable then I would rather do a plugin that I originally proposed since it would have more value for DS in future. If it deems to be more complex than expected we can always fall back to the logging to file from the plugin. Seems like a compromise to me :-) Thanks Dmitri From jdennis at redhat.com Fri Nov 7 00:08:18 2008 From: jdennis at redhat.com (John Dennis) Date: Thu, 06 Nov 2008 19:08:18 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49137560.7030705@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> Message-ID: <491386F2.4010800@redhat.com> Dmitri Pal wrote: > The argument for storing in DS is: > a) There is no other place to store it. Nonsense, there are many other good places, but see my comment below. > f) The company policies might require that the changes to the critical > object be commented . Without this feature and DS plugin this can't be > enforced. If it is done in UI or CLI the admin might circumvent it by > using ldap calls directly. So DS is the only common denominator. > I strongly believe that based on the last reason it should be done in > DS plugin and only there. It can be done in different ways though. I wrote an email earlier about how we have conceptual impedance illustrated with Aseoph's Fable about the 5 Blind Men and the Elephant. I elected not to send it. But I want to point out something out which has been worrying me, we might be locking ourselves into a series of bad design decisions. The problem goes like this: We've put DS at the center of the IPA universe. We've built an entire management system on top of it which we then say is completely optional to use, it's just sugar, feel free reach inside and use ldapmodify and do whatever you want behind our backs. Because we've deliberately picked one data store and provided a wide open back door to it we constantly find ourselves in the situation of having to enforce our business logic inside that data store. Perhaps I'm the only one, but that strikes me as crazy and it won't scale. IPA is going to keep growing features, for goodness sake not everything can nor should be implemented in the directory server. I think we would be better off to say our XMLRPC server is what implements IPA functionality, we provide multiple ways to interact with it, it implements all our business logic, it is a heavy user of LDAP but NOT AN EXCLUSIVE user of LDAP. You're free to use the ldapmodify backdoor if you want but that comes with all the caveats about backdoor out-of-band updates. To be a good IPA citizen use the IPA interfaces. > We can do this but if we agree that DS plugin is anyway inevitable ... No! Endless DS plugins are not inevitable. They are only inevitable if you buy into a flawed model which says everything in IPA needs to be implemented in DS. LDAP makes a lot of sense for what IPA does with the "I" of IPA, but let's have the freedom to use other tools and technologies for the remainder. If you do buy into the fact we need more flexibility then there has to be a "central agent" which interacts with all the components and enforces business logic (I am not convinced that central agent can or should be a LDAP server). With this in mind capturing comments becomes trivial. Use the IPA interface to make the change, it optionally captures the comment. IPA then updates DS and every other relevant component AND enforces the integrity of our business logic. IPA generates an ID for the entire set of operations it performed (e.g. a transaction), IPA then writes the comment tagged with the ID wherever it wants to. When we need to present the comment to a user the GUI, command line tool, or whatever calls the defined IPA interface (e.g. XMLRPC). Which retrieves the comment from wherever the heck it wants to (in v2 it might be a file, in v3 it might be a database). You only need to store everything in DS and enforce all business logic in DS if you believe ldapmodify is the one true IPA interface (which IMHO is driving us to make bad design decisions). BTW, if someone uses ldapmodify IPA can still identify when and how the change occurred because it will be in an audit log of DS modifications. -- John Dennis From dpal at redhat.com Fri Nov 7 00:38:29 2008 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 06 Nov 2008 19:38:29 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491386F2.4010800@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <491386F2.4010800@redhat.com> Message-ID: <49138E05.9060607@redhat.com> John Dennis wrote: > Dmitri Pal wrote: >> The argument for storing in DS is: >> a) There is no other place to store it. > > Nonsense, there are many other good places, but see my comment below. >> f) The company policies might require that the changes to the >> critical object be commented . Without this feature and DS plugin >> this can't be enforced. If it is done in UI or CLI the admin might >> circumvent it by using ldap calls directly. So DS is the only common >> denominator. >> I strongly believe that based on the last reason it should be done in >> DS plugin and only there. It can be done in different ways though. > I wrote an email earlier about how we have conceptual impedance > illustrated with Aseoph's Fable about the 5 Blind Men and the > Elephant. I elected not to send it. But I want to point out something > out which has been worrying me, we might be locking ourselves into a > series of bad design decisions. > > The problem goes like this: We've put DS at the center of the IPA > universe. We've built an entire management system on top of it which > we then say is completely optional to use, it's just sugar, feel free > reach inside and use ldapmodify and do whatever you want behind our > backs. Because we've deliberately picked one data store and provided a > wide open back door to it we constantly find ourselves in the > situation of having to enforce our business logic inside that data > store. Perhaps I'm the only one, but that strikes me as crazy and it > won't scale. IPA is going to keep growing features, for goodness sake > not everything can nor should be implemented in the directory server. > I think we would be better off to say our XMLRPC server is what > implements IPA functionality, we provide multiple ways to interact > with it, it implements all our business logic, it is a heavy user of > LDAP but NOT AN EXCLUSIVE user of LDAP. You're free to use the > ldapmodify backdoor if you want but that comes with all the caveats > about backdoor out-of-band updates. To be a good IPA citizen use the > IPA interfaces. >> We can do this but if we agree that DS plugin is anyway inevitable ... > No! Endless DS plugins are not inevitable. They are only inevitable if > you buy into a flawed model which says everything in IPA needs to be > implemented in DS. > > LDAP makes a lot of sense for what IPA does with the "I" of IPA, but > let's have the freedom to use other tools and technologies for the > remainder. If you do buy into the fact we need more flexibility then > there has to be a "central agent" which interacts with all the > components and enforces business logic (I am not convinced that > central agent can or should be a LDAP server). > > With this in mind capturing comments becomes trivial. Use the IPA > interface to make the change, it optionally captures the comment. IPA > then updates DS and every other relevant component AND enforces the > integrity of our business logic. IPA generates an ID for the entire > set of operations it performed (e.g. a transaction), IPA then writes > the comment tagged with the ID wherever it wants to. When we need to > present the comment to a user the GUI, command line tool, or whatever > calls the defined IPA interface (e.g. XMLRPC). Which retrieves the > comment from wherever the heck it wants to (in v2 it might be a file, > in v3 it might be a database). > > You only need to store everything in DS and enforce all business logic > in DS if you believe ldapmodify is the one true IPA interface (which > IMHO is driving us to make bad design decisions). > > BTW, if someone uses ldapmodify IPA can still identify when and how > the change occurred because it will be in an audit log of DS > modifications. > John, This is a very good point. I think we should clarify this and come to terms about it. If we are not on the same page with fundamental approach to data store than we should come to agreement (or at least state the rule that everybody would have to comply - I wonder who has such authority :-)). I am operating under the following assumptions: a) DS is our data store and whatever we need to store there we will store there unless there is a better place for it b) There is a better place for audit data so we won't store it there but for most other things it the store. c) We allow direct access by LDAP . Even if we state that we does not allow it the LDAP channel will be there. No security operation check or enforcement should be bypassed on this route. Otherwise there is no sense of building such checks. DS has to be the common denominator unless we create some other abstraction layer as you suggest and stop exposing LDAP interface at all. The abstraction layer (agent as you called it) reminds me of the effort that was undertaken by my previous company. They spent 2 years and 20 people building such layer. Did it wrong, threw it away and started over using different technologies. Spent another 3 years, doubled team and finally got the product out. We just can't afford this. :-) I think it is too late to change the course. DS is our data store and we need to be prepared for the long list of data going into it together with a long list of the integrity, consistency and security plugins. XML-RPC is just a convenience layer. We have to admit that. From this perspective all attempt to keep DS "pure" seems at least strange. Based on the assumptions above my original proposal seems perfectly reasonable. If we can't agree to DS being the core and being the common low level denominator we would have to start from day 1. Any other opinions are welcome. Thanks Dmitri From chorn at fluxcoil.net Fri Nov 7 08:13:30 2008 From: chorn at fluxcoil.net (Christian Horn) Date: Fri, 7 Nov 2008 09:13:30 +0100 Subject: [Freeipa-devel] Re: [Freeipa-users] Need help with Solaris Host Based access control In-Reply-To: <491206C3.9070204@redhat.com> References: <491206C3.9070204@redhat.com> Message-ID: <20081107081330.GA13820@fluxcoil.net> Mornings, On Wed, Nov 05, 2008 at 03:49:07PM -0500, Dmitri Pal wrote: > > The instructions are based on the ability of the pam_access PAM module > to check the access control rules specified in the access.conf. > The group information can be retrieved from the IPA server via nss_ldap. > > We tried to find similar functionality on other OS's. We spotted PAM > modules on HP-UX and AIX that are responsible for the similar > authorization checks. > > But we are stuck with Solaris. All our investigations about similar > functionality in Solaris bear no fruits. We saw pam_roles and > pam_unix_account on Solaris but they do not seem to accomplish what we > are trying to do. > > We are looking for some help and advice from Solaris experts on this > functionality. Checked with solaris-guys, this is in use for pure ldap-authentication/ authorization. Apparently just after hooking up a solaris-box to an ldap no user is allowed to login. The permissions to login are handled by this: a) entries in /etc/passwd, containing names of NIS-netgroups whose members are allowed to log in, i.e. + at netgroup1:::::: b) entries in /etc/shadow, containing names of NIS-netgroups whose members are allowed to log in, i.e. + at netgroup1:::::::: (thats 8 colons vs. 6 on the /etcx/passwd-entries) c) entries in /etc/nsswitch.conf for this to work: passwd: compat passwd_compat: ldap [NOTFOUND=return] I dont use this myself on Solaris-boxen but should be enough to see the Solaris-way to handle those login-authorizations. Christian From sbose at redhat.com Fri Nov 7 09:35:36 2008 From: sbose at redhat.com (Sumit Bose) Date: Fri, 07 Nov 2008 10:35:36 +0100 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49137560.7030705@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> Message-ID: <49140BE8.9060100@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, here are my 0.02$ - - for the policies it would be possible to have the commit comments inside the XML file, like ....... - - although I have not made up my mind if I like the idea of storing the commit comments in the DS or not, I would suggest to think about storing the data not in a multi value attribute of the object, but in child objects with an own objectclass. Then you can store who and when and what into specific attributes where it would be easy to search or select specific data. bye, Sumit Dmitri Pal schrieb: > Summary: > 1) We all agree that: > a) Providing commit comments is valuable feature for users of the IPA > b) We can't force (and should not) the user to put some meaningful > data in such comments. This is the responsibility of the corporate > policy - not software. > c) It should be flexible so that only in the cases when the corporate > policy requires that kind of comment it would be enforced. Otherwise it > should be optional or even hidden to avoid annoying administrator of the > system. > > 2) We disagree mainly on the mean where this data should be stored. Main > point is does it belong to DS or not. > > a) The argument to not put it in DS is that this data does not belong > there. It is perceived as a log and thus should be stored in the audit > system. > b) Other arguments include the fact that we should avoid developing > unnecessary DS plugins until there is absolute need because the bugs in > plugins can bring the whole server down. > c) This data is not critical for functioning of the server so should not > be in DS etc. > > The argument for storing in DS is: > a) There is no other place to store it. Audit system will not be robust > enough soon enough to fit the bill (especially real time lookups) > b) Many other features require plugins so what a big deal about one more > c) The amount of work (may be erroneously is perceived as smaller than > using other alternatives). I will not explore that more. One can read > the thread. d) There are already similar things in the DS that do things > in pretty much the same way > e) The DS experts do not see a big issue with the approach and see a > value down the road > f) The company policies might require that the changes to the critical > object be commented . Without this feature and DS plugin this can't be > enforced. If it is done in UI or CLI the admin might circumvent it by > using ldap calls directly. So DS is the only common denominator. > I strongly believe that based on the last reason it should be done in DS > plugin and only there. It can be done in different ways though. > For example one could suggest that the DS plugin can just require the > comment to be inserted on each add/modify of an object and save it to a > log file that then can be processed by the audit system. > We can do this but if we agree that DS plugin is anyway inevitable then > I would rather do a plugin that I originally proposed since it would > have more value for DS in future. If it deems to be more complex than > expected we can always fall back to the logging to file from the plugin. > Seems like a compromise to me :-) > > Thanks > Dmitri > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkkUC+QACgkQUDGHpI6P4rr7ZgCeM1G6WSLN6wygcb8HjB42xHrW w3QAnA7ZGGfh4xZgxrVvipO/wsly8SOM =M5hq -----END PGP SIGNATURE----- From dpal at redhat.com Fri Nov 7 14:23:38 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 09:23:38 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49140BE8.9060100@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> Message-ID: <49144F6A.8030905@redhat.com> Sumit Bose wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > here are my 0.02$ > > - - for the policies it would be possible to have the commit comments > inside the XML file, like > ....... > > Why do we need the comments inside the policy itself? Comments inside the policy do not male much sense to me. The policies are sent to the clients. So comments would be extraneous information that will be sent to clients for no value. It will also cause more replication since policies a compressed XMLs. Change to a comment will trigger the update of the whole attribute. I was thinking to have the comments stored in the object that is a link to the policy. I think they belong there as a part of an entry or a child entry (as you suggested), if put into DS at all. We also have description attribute of the link so some comments can be put there. > - - although I have not made up my mind if I like the idea of storing the > commit comments in the DS or not, I would suggest to think about storing > the data not in a multi value attribute of the object, but in child > objects with an own objectclass. Then you can store who and when and > what into specific attributes where it would be easy to search or select > specific data. > Nathan, Rich. Is it a better approach from a pure technical angle? > bye, > Sumit > > From rcritten at redhat.com Fri Nov 7 14:25:47 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 07 Nov 2008 09:25:47 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1225991919.10160.217.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> Message-ID: <49144FEB.8000406@redhat.com> Simo Sorce wrote: > On Thu, 2008-11-06 at 10:02 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> How do clients choose which server to connect to ? >> Hardcoded in /etc/sysconfig/autofs on Fedora and RHEL. >> >>> Is there any concept like that in automount ? Should we care ? >> AFAIK there is no accomodation for this. We'd either have to provide >> separate areas (in the dn) to store the maps or the end-user would need >> to carefully configure things. > > I think providing separate areas then is paramount. Admins should be > able to define "locations" and the maps would be created inside these > location. This way admins can set different automount options for > clients located in different places. I am sure clients in Australia are > not going to use the same automount maps as for clients in Baltimore. If we add some sort of location identifier for automount what implications does this have for other features? Do we want to have per-location settings for anything else? My plan is to embed the location into the DN of the automount map and key names using cn. For the UI we'll need some method of selecting/managing this list of locations (drop-down box comes to mind). I'm not sure if storing this separately is a good idea or not. I just want to avoid any short-term choices I make don't have long-term consequences. rob From dpal at redhat.com Fri Nov 7 14:27:00 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 09:27:00 -0500 Subject: [Freeipa-devel] Re: [Freeipa-users] Need help with Solaris Host Based access control In-Reply-To: <20081107081330.GA13820@fluxcoil.net> References: <491206C3.9070204@redhat.com> <20081107081330.GA13820@fluxcoil.net> Message-ID: <49145034.8030409@redhat.com> Thank you Christian! I will dig more into it. Dmitri Christian Horn wrote: > Mornings, > > On Wed, Nov 05, 2008 at 03:49:07PM -0500, Dmitri Pal wrote: > >> The instructions are based on the ability of the pam_access PAM module >> to check the access control rules specified in the access.conf. >> The group information can be retrieved from the IPA server via nss_ldap. >> >> We tried to find similar functionality on other OS's. We spotted PAM >> modules on HP-UX and AIX that are responsible for the similar >> authorization checks. >> >> But we are stuck with Solaris. All our investigations about similar >> functionality in Solaris bear no fruits. We saw pam_roles and >> pam_unix_account on Solaris but they do not seem to accomplish what we >> are trying to do. >> >> We are looking for some help and advice from Solaris experts on this >> functionality. >> > > Checked with solaris-guys, this is in use for pure ldap-authentication/ > authorization. > Apparently just after hooking up a solaris-box to an ldap no user > is allowed to login. > > The permissions to login are handled by this: > > a) entries in /etc/passwd, containing names of NIS-netgroups > whose members are allowed to log in, i.e. > > + at netgroup1:::::: > > b) entries in /etc/shadow, containing names of NIS-netgroups > whose members are allowed to log in, i.e. > > + at netgroup1:::::::: > (thats 8 colons vs. 6 on the /etcx/passwd-entries) > > c) entries in /etc/nsswitch.conf for this to work: > > passwd: compat > passwd_compat: ldap [NOTFOUND=return] > > > I dont use this myself on Solaris-boxen but should be enough to see > the Solaris-way to handle those login-authorizations. > > > Christian > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > From ssorce at redhat.com Fri Nov 7 14:46:02 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 07 Nov 2008 14:46:02 +0000 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49144F6A.8030905@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> Message-ID: <1226069162.10160.369.camel@localhost.localdomain> On Fri, 2008-11-07 at 09:23 -0500, Dmitri Pal wrote: > Sumit Bose wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Hi, > > > > here are my 0.02$ > > > > - - for the policies it would be possible to have the commit comments > > inside the XML file, like > > ....... > > > > > > Why do we need the comments inside the policy itself? Comments inside > the policy do not male much sense to me. > The policies are sent to the clients. So comments would be extraneous > information that will be sent to clients for no value. I think comments may contain information that should not be widespread, so I do not think we should put them in the policy file (at least not in the clear, maybe encrypted). If disclosing comments to clients is not a problem, Sumit's suggestion seem to me *much* more appropriate. It will preserve all comments when you do changes to a policy on a staging test installation and then later on you transport them to the production environment (export policy, import policy). This would be an extremely valuable feature imo. > It will also cause more replication since policies a compressed XMLs. No, you add a comment when you change a policy, so you are already rewriting the XML file. > Change to a comment will trigger the update of the whole attribute. Your description of the feature sounded like comments cannot be changed once the operation is committed so this would not be a problem. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Fri Nov 7 15:23:25 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Fri, 07 Nov 2008 08:23:25 -0700 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49144F6A.8030905@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> Message-ID: <49145D6D.4020509@redhat.com> Dmitri Pal wrote: > Sumit Bose wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Hi, >> >> here are my 0.02$ >> >> - - for the policies it would be possible to have the commit comments >> inside the XML file, like >> ....... >> >> > > Why do we need the comments inside the policy itself? Comments inside > the policy do not male much sense to me. > The policies are sent to the clients. So comments would be extraneous > information that will be sent to clients for no value. > It will also cause more replication since policies a compressed XMLs. > Change to a comment will trigger the update of the whole attribute. > I was thinking to have the comments stored in the object that is a > link to the policy. > I think they belong there as a part of an entry or a child entry (as > you suggested), if put into DS at all. > We also have description attribute of the link so some comments can be > put there. > >> - - although I have not made up my mind if I like the idea of storing >> the >> commit comments in the DS or not, I would suggest to think about storing >> the data not in a multi value attribute of the object, but in child >> objects with an own objectclass. Then you can store who and when and >> what into specific attributes where it would be easy to search or select >> specific data. >> > Nathan, Rich. Is it a better approach from a pure technical angle? That would give you a lot more flexibility in terms of schema, indexing, sorting, searching, etc. And the comments would still be associated with the data, although not as tightly coupled. Minuses: * comments not included directly with the data - not as tightly coupled as having the comments in the same entry as the data - operations to the same entry in LDAP are atomic, but operations to multiple entries are not atomic (we have no LDAP transactions) * the sub-entries might turn up in subtree searches, which may confuse some applications - we might be able to "hide" these entries by making them a subclass of ldapSubEntry objectclass - then you would only see them if you ask for them explicitly * more data to manage - larger databases, indexes, more entries, etc. > > >> bye, >> Sumit >> >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From dpal at redhat.com Fri Nov 7 16:33:16 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 11:33:16 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <1226069162.10160.369.camel@localhost.localdomain> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> <1226069162.10160.369.camel@localhost.localdomain> Message-ID: <49146DCC.20109@redhat.com> Simo Sorce wrote: > > I think comments may contain information that should not be widespread, > so I do not think we should put them in the policy file (at least not in > the clear, maybe encrypted). > Yes they might contain information that should not be sent to clients. > If disclosing comments to clients is not a problem, Sumit's IMO it is problem. > suggestion > seem to me *much* more appropriate. It will preserve all comments when > you do changes to a policy on a staging test installation and then later > on you transport them to the production environment (export policy, > import policy). This would be an extremely valuable feature imo. > > In the test environment you are playing with the policy. You are polishing it so that it does what you need. I would say that in this case the mandatory commenting should be disabled. It might take multiple iterations and since it is not a production environment to come to a good state. You do not want to be annoyed with these comments. But once you import it into the production system it all becomes formal. When you apply policy to the production system that is when you are asked to provide a comment. So really based on this comments inside the policy do not have much value IMO. >> It will also cause more replication since policies a compressed XMLs. >> > > No, you add a comment when you change a policy, so you are already > rewriting the XML file. > No - you add a comment when you apply the policy not when you change it. You can be editing and saving policy several times but until you click the "apply" button it is not offical. Clicking apply button will just change the link object not the policy itself. More details about these things are coming in the DS schema spec I am writing so stay tuned. > >> Change to a comment will trigger the update of the whole attribute. >> > > Your description of the feature sounded like comments cannot be changed > once the operation is committed so this would not be a problem. > > I was planning to use the comments not only with policies but with role mapping and HBAC rules. When you edit configuration policy (as I said above) you are not asked to provide a comment - only when you apply it. HBAC rules and role mapping are different they become "official" when you save them so comment might be required when you save them. > Simo. > > Any design solution I suggest is based on several principles: a) User of the system goes first. Requirements take precedence over our convenience or desire to create a nice architecture. b) Creating nice and logical architecture. c) Use what you already have and do not invent the wheel (we have DS it is our data store - let us deal with it) d) Keep things simple and self contained so that you do not create more dependencies out of box e) Create a vision where you want to be from a big picture and go to it in incremental steps keeping in mind the ultimate goal. Do not try to boil an ocean. But do not grind everything to very small pieces this is unreasonable too. f) Do not re-argue the decisions that were already made regardless whether you like them or not - they are a base for the new designs. You really need to have a solid foundation to steadily move forward. g) While designing something I consider that: 1) One data store is always better than more than one. It is always powers of magnitude more work to have data spread around. 2) Do not send around extra data if you do not need. Keep protocols crisp to contain only data needed. 3) Follow standards where it makes sense. Cut the corners where it does not. 4) Performance and scalability is important 5) Early optimization is bad but design that does not require any further optimization is a good design. This is not for discussion (or at least we can discuss it in a separate thread and share our contradicting positions). This is just a statement to clarify my point of view and explain where I am coming from and why I am suggesting what I am suggesting. Thanks Dmitri From dpal at redhat.com Fri Nov 7 16:45:34 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 11:45:34 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <49145D6D.4020509@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> <49145D6D.4020509@redhat.com> Message-ID: <491470AE.5050509@redhat.com> >>> >> Nathan, Rich. Is it a better approach from a pure technical angle? > That would give you a lot more flexibility in terms of schema, > indexing, sorting, searching, etc. And the comments would still be > associated with the data, although not as tightly coupled. > > Minuses: > * comments not included directly with the data - not as tightly > coupled as having the comments in the same entry as the data - > operations to the same entry in LDAP are atomic, but operations to > multiple entries are not atomic (we have no LDAP transactions) > * the sub-entries might turn up in subtree searches, which may confuse > some applications - we might be able to "hide" these entries by making > them a subclass of ldapSubEntry objectclass - then you would only see > them if you ask for them explicitly > * more data to manage - larger databases, indexes, more entries, etc. >> Thanks Rich. I am still struggling with understanding why we would need search and indexing capabilities for these comments. With generic ordered attribute we probably need. But with comments we do not. As I see the use case one would be interested in commit comments for an entry in general and would not want to search an entry in the DS that contained some special string (for example ticket number) by that string. The case is the reverse. I have the entry and what to know how its evolution was authorized. I guess I am struggling with comparing the benefits of this approach and minuses to the mine original proposal. One of the reasons I can't assess the amount of work. I will think more about it. It seems a bit too heavy but one would argue that plugins are too heavy. May be. That is why I need to give it a try and play with it more. Thanks Dmitri From sbose at redhat.com Fri Nov 7 16:55:43 2008 From: sbose at redhat.com (Sumit Bose) Date: Fri, 07 Nov 2008 17:55:43 +0100 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <491470AE.5050509@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> <49145D6D.4020509@redhat.com> <491470AE.5050509@redhat.com> Message-ID: <4914730F.9030706@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dmitri Pal schrieb: > >>>> >>> Nathan, Rich. Is it a better approach from a pure technical angle? >> That would give you a lot more flexibility in terms of schema, >> indexing, sorting, searching, etc. And the comments would still be >> associated with the data, although not as tightly coupled. >> >> Minuses: >> * comments not included directly with the data - not as tightly >> coupled as having the comments in the same entry as the data - >> operations to the same entry in LDAP are atomic, but operations to >> multiple entries are not atomic (we have no LDAP transactions) >> * the sub-entries might turn up in subtree searches, which may confuse >> some applications - we might be able to "hide" these entries by making >> them a subclass of ldapSubEntry objectclass - then you would only see >> them if you ask for them explicitly >> * more data to manage - larger databases, indexes, more entries, etc. >>> > Thanks Rich. I am still struggling with understanding why we would need > search and indexing capabilities for these comments. With generic To be able to search for dates or user could have benefits with respect to: - - Configuration management: if something broke and I knew it works a week ago, I can search for changes happen during last week and with the comments I can hopefully see which change broke my system. - - Security/Audit: if the account of an administration was compromised I can check what changes were made under this account > ordered attribute we probably need. But with comments we do not. As I > see the use case one would be interested in commit comments for an entry > in general and would not want to search an entry in the DS that > contained some special string (for example ticket number) by that > string. The case is the reverse. I have the entry and what to know how > its evolution was authorized. > > I guess I am struggling with comparing the benefits of this approach and > minuses to the mine original proposal. One of the reasons I can't assess > the amount of work. > I will think more about it. It seems a bit too heavy but one would argue > that plugins are too heavy. May be. That is why I need to give it a try > and play with it more. > > > Thanks > Dmitri -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkkUcwsACgkQUDGHpI6P4rr/EQCgkP0hncDRHugCRrJ6xSslMSBL +YoAoPDqr2+1rp1SZhbNdEIT46gnPTrJ =LcP4 -----END PGP SIGNATURE----- From ssorce at redhat.com Fri Nov 7 17:00:36 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 07 Nov 2008 12:00:36 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49144FEB.8000406@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> Message-ID: <1226077236.10160.385.camel@localhost.localdomain> On Fri, 2008-11-07 at 09:25 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Thu, 2008-11-06 at 10:02 -0500, Rob Crittenden wrote: > >> Simo Sorce wrote: > >>> How do clients choose which server to connect to ? > >> Hardcoded in /etc/sysconfig/autofs on Fedora and RHEL. > >> > >>> Is there any concept like that in automount ? Should we care ? > >> AFAIK there is no accomodation for this. We'd either have to provide > >> separate areas (in the dn) to store the maps or the end-user would need > >> to carefully configure things. > > > > I think providing separate areas then is paramount. Admins should be > > able to define "locations" and the maps would be created inside these > > location. This way admins can set different automount options for > > clients located in different places. I am sure clients in Australia are > > not going to use the same automount maps as for clients in Baltimore. > > If we add some sort of location identifier for automount what > implications does this have for other features? Do we want to have > per-location settings for anything else? We might going on I think. > My plan is to embed the location into the DN of the automount map and > key names using cn. Seem sensible, another, perhaps better, way could be to have the location as an attribute in the object so that the same maps could be used in multiple locations by simply adding a new value. But for this to work automount's ldap code would need to be able to use additional custom filters, is that possible? > For the UI we'll need some method of selecting/managing this list of > locations (drop-down box comes to mind). I'm not sure if storing this > separately is a good idea or not. I am not sure what is the best method from a UI perspective, how would you like to manage this property ? I think you may want to see all data and then just have a list of locations it applies to ("list" in case assuming we use a multivalue attribute to store the location and not embed it in the DN). > I just want to avoid any short-term choices I make don't have long-term > consequences. Yes, thanks for doing that, it is important to always keep in mind future developments, as data migration is always a pain if you later have to change something, even more so in a distributed environment where there are multiple servers involved. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Fri Nov 7 17:07:00 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 12:07:00 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49144FEB.8000406@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> Message-ID: <491475B4.3050604@redhat.com> > > If we add some sort of location identifier for automount what > implications does this have for other features? Do we want to have > per-location settings for anything else? > > My plan is to embed the location into the DN of the automount map and > key names using cn. > > For the UI we'll need some method of selecting/managing this list of > locations (drop-down box comes to mind). I'm not sure if storing this > separately is a good idea or not. > > I just want to avoid any short-term choices I make don't have > long-term consequences. > Yes I agree . We need to have a more generic approach to that. I am struggling with understanding how client would determine which location he is in. I do not think that it should be based on the server the client is connected to. This approach would require servers to serve out data different from data being served by other replica. But what if the client fails over? Does not seem like a good approach imo. So may be using IP address range would be a better indication? May be we have a part of client policy that would say : Location A: range1, ragne2, range 3 Location B: range4, ragne5, range 6 This approach works if you can determine the right IP to check. From my past experience answering the question "which of my IPs is the IP that server sees" is not a trivial answer. But let us say we dealt with that. Then as soon as client gets online it would know its IP so it will know its location so it will be able to construct a search filter and add it to the search string. I really like the idea of having location as an attribute. Then for any object that client needs to fetch and it can be location specific it would just add to the search filter: ( | (location=) (!(location))) This will pull down all entries that are specific to the location or entries that are not location specific. This logic can be then reused across different entities not only mounts. Thanks Dmitri > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From dpal at redhat.com Fri Nov 7 17:30:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 12:30:07 -0500 Subject: [Freeipa-devel] "Commit comments log" functionality in IPA In-Reply-To: <4914730F.9030706@redhat.com> References: <491236CF.6090508@redhat.com> <1225953550.10160.99.camel@localhost.localdomain> <1225953949.10160.105.camel@localhost.localdomain> <491313BA.1060303@redhat.com> <1225991601.10160.212.camel@localhost.localdomain> <49133BCB.7070308@redhat.com> <1226006181.10160.338.camel@localhost.localdomain> <49137560.7030705@redhat.com> <49140BE8.9060100@redhat.com> <49144F6A.8030905@redhat.com> <49145D6D.4020509@redhat.com> <491470AE.5050509@redhat.com> <4914730F.9030706@redhat.com> Message-ID: <49147B1F.8020001@redhat.com> > To be able to search for dates or user could have benefits with respect to: > - - Configuration management: if something broke and I knew it works a > week ago, I can search for changes happen during last week and with the > comments I can hopefully see which change broke my system. > If something broke you rollback. That is why we will have a quick rollback mechanism. And then you pull the commit comment and see the history. You will not see the changes. > - - Security/Audit: if the account of an administration was compromised I > can check what changes were made under this account > > In this case audit is your tool. Commit comments would not help much. May be the use case is still not clear. The comments are need only in the big companies that have a "paper" process of approving any changes. In such companies the administrators do not have flexibility of just changing something. Any change goes through the lengthy review and approval process. Trust me there are tons of those companies. The commit comment is the link between the event of applying or rolling back the change and this process. The contents of the comment can't be enforced by software but usually mandated by the company policies. It becomes extremely important in case of fire when something went seriously wrong and CIO is on the quest for heads. It will provide fast and convenient way to find the guilty party that authorized the change. This is why this data becomes really critical. If it is glued to the object it is related too it is easier to extract (in UI) and it is more trustable. One can still argue that audit would be a better place for it. I see the point but I do not believe that audit will have enough search capabilities to deal with this data. May it will. I am thinking about it. We will see. Thanks Dmitri From ssorce at redhat.com Fri Nov 7 19:17:33 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 07 Nov 2008 14:17:33 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <491475B4.3050604@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> Message-ID: <1226085453.10160.400.camel@localhost.localdomain> On Fri, 2008-11-07 at 12:07 -0500, Dmitri Pal wrote: > > > > If we add some sort of location identifier for automount what > > implications does this have for other features? Do we want to have > > per-location settings for anything else? > > > > My plan is to embed the location into the DN of the automount map and > > key names using cn. > > > > For the UI we'll need some method of selecting/managing this list of > > locations (drop-down box comes to mind). I'm not sure if storing this > > separately is a good idea or not. > > > > I just want to avoid any short-term choices I make don't have > > long-term consequences. > > > > Yes I agree . We need to have a more generic approach to that. > > I am struggling with understanding how client would determine which > location he is in. > I do not think that it should be based on the server the client is > connected to. > This approach would require servers to serve out data different from > data being served by other replica. But what if the client fails over? > Does not seem like a good approach imo. > So may be using IP address range would be a better indication? > May be we have a part of client policy that would say : > > Location A: range1, ragne2, range 3 > Location B: range4, ragne5, range 6 > > This approach works if you can determine the right IP to check. From my > past experience answering the question "which of my IPs is the IP that > server sees" is not a trivial answer. > But let us say we dealt with that. Then as soon as client gets online it > would know its IP so it will know its location so it will be able to > construct a search filter and add it to the search string. It is not a simple task, that said I have been working on a document on DNS based location discovery for some time, I will try to publish it as soon as I have some time. > I really like the idea of having location as an attribute. Then for any > object that client needs to fetch and it can be location specific it > would just add to the search filter: > ( | (location=) (!(location))) This will pull down > all entries that are specific to the location or entries that are not > location specific. > > This logic can be then reused across different entities not only mounts. It is fundamental for SSSD to be able to discover the closest IPA service and fallback to the next closest in case of problems and so on. It's a fundamental feature we have been discussing for some time months ago, I will publish some docs as soon as possible. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Fri Nov 7 20:18:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 15:18:07 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226085453.10160.400.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> Message-ID: <4914A27F.7090109@redhat.com> Simo Sorce wrote: > On Fri, 2008-11-07 at 12:07 -0500, Dmitri Pal wrote: > >>> If we add some sort of location identifier for automount what >>> implications does this have for other features? Do we want to have >>> per-location settings for anything else? >>> >>> My plan is to embed the location into the DN of the automount map and >>> key names using cn. >>> >>> For the UI we'll need some method of selecting/managing this list of >>> locations (drop-down box comes to mind). I'm not sure if storing this >>> separately is a good idea or not. >>> >>> I just want to avoid any short-term choices I make don't have >>> long-term consequences. >>> >>> >> Yes I agree . We need to have a more generic approach to that. >> >> I am struggling with understanding how client would determine which >> location he is in. >> I do not think that it should be based on the server the client is >> connected to. >> This approach would require servers to serve out data different from >> data being served by other replica. But what if the client fails over? >> Does not seem like a good approach imo. >> So may be using IP address range would be a better indication? >> May be we have a part of client policy that would say : >> >> Location A: range1, ragne2, range 3 >> Location B: range4, ragne5, range 6 >> >> This approach works if you can determine the right IP to check. From my >> past experience answering the question "which of my IPs is the IP that >> server sees" is not a trivial answer. >> But let us say we dealt with that. Then as soon as client gets online it >> would know its IP so it will know its location so it will be able to >> construct a search filter and add it to the search string. >> > > It is not a simple task, that said I have been working on a document on > DNS based location discovery for some time, I will try to publish it as > soon as I have some time. > > >> I really like the idea of having location as an attribute. Then for any >> object that client needs to fetch and it can be location specific it >> would just add to the search filter: >> ( | (location=) (!(location))) This will pull down >> all entries that are specific to the location or entries that are not >> location specific. >> >> This logic can be then reused across different entities not only mounts. >> > > It is fundamental for SSSD to be able to discover the closest IPA > service and fallback to the next closest in case of problems and so on. > It's a fundamental feature we have been discussing for some time months > ago, I will publish some docs as soon as possible. > > Simo. > > Simo I remember and do not discard that. But I was talking about the fact that affinity and closest server do not mean that the location of the client is the determined correctly. What if the APAC replica is down and all clients now to use one in Europe. But they are still in APAC so they need to use APAC mount shares. For the visiting salesman with laptop from Europe his mounts should be APAC is he visits APAC right though he is affiliated naturally with European server? These are the scenarios I was talking about . I think determining location of the client is different from the affinity and failover logic. Thanks Dmitri From rmeggins at redhat.com Fri Nov 7 20:19:42 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Fri, 07 Nov 2008 13:19:42 -0700 Subject: [Freeipa-devel] Please review: Bug 470379 - winsync requires updated dna plugin to handle internal operations Message-ID: <4914A2DE.5030604@redhat.com> https://bugzilla.redhat.com/show_bug.cgi?id=470379 Resolves: bug 470379 Description: winsync requires updated dna plugin to handle internal operations Fix Description: Nathan already did most of the work, I just made some minor tweaks. This allows the DNA plugin in IPA to handle internal operations such as those from WinSync. Tested on: Fedora 9 with IPA WinSync https://bugzilla.redhat.com/attachment.cgi?id=322896&action=diff From nkinder at redhat.com Fri Nov 7 21:33:43 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Fri, 07 Nov 2008 13:33:43 -0800 Subject: [Freeipa-devel] Please review: Bug 470379 - winsync requires updated dna plugin to handle internal operations In-Reply-To: <4914A2DE.5030604@redhat.com> References: <4914A2DE.5030604@redhat.com> Message-ID: <4914B437.4090007@redhat.com> Rich Megginson wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=470379 > Resolves: bug 470379 > Description: winsync requires updated dna plugin to handle internal > operations > Fix Description: Nathan already did most of the work, I just made some > minor tweaks. This allows the DNA plugin in IPA to handle internal > operations such as those from WinSync. > Tested on: Fedora 9 with IPA WinSync > https://bugzilla.redhat.com/attachment.cgi?id=322896&action=diff ack > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From dpal at redhat.com Fri Nov 7 22:13:45 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 17:13:45 -0500 Subject: [Freeipa-devel] Minor change to host based access control time stamp related logic Message-ID: <4914BD99.9000905@redhat.com> Hi, After some evaluation and research the original idea of including timezone into time stamps stored in HBAC rules does not fly well since different platforms have different level of support for modern time zone definitions. See details on the following page: http://www.freeipa.org/page/Representing_Objects_in_DS_%28Part_1_-_Base_objects%29#Expressing_Time_in_a_Rule For changes to the design see the history of the page. Thanks Dmitri From ssorce at redhat.com Fri Nov 7 22:36:39 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 07 Nov 2008 17:36:39 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <4914A27F.7090109@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> Message-ID: <1226097399.10160.405.camel@localhost.localdomain> On Fri, 2008-11-07 at 15:18 -0500, Dmitri Pal wrote: > Simo Sorce wrote: > > On Fri, 2008-11-07 at 12:07 -0500, Dmitri Pal wrote: > > > >>> If we add some sort of location identifier for automount what > >>> implications does this have for other features? Do we want to have > >>> per-location settings for anything else? > >>> > >>> My plan is to embed the location into the DN of the automount map and > >>> key names using cn. > >>> > >>> For the UI we'll need some method of selecting/managing this list of > >>> locations (drop-down box comes to mind). I'm not sure if storing this > >>> separately is a good idea or not. > >>> > >>> I just want to avoid any short-term choices I make don't have > >>> long-term consequences. > >>> > >>> > >> Yes I agree . We need to have a more generic approach to that. > >> > >> I am struggling with understanding how client would determine which > >> location he is in. > >> I do not think that it should be based on the server the client is > >> connected to. > >> This approach would require servers to serve out data different from > >> data being served by other replica. But what if the client fails over? > >> Does not seem like a good approach imo. > >> So may be using IP address range would be a better indication? > >> May be we have a part of client policy that would say : > >> > >> Location A: range1, ragne2, range 3 > >> Location B: range4, ragne5, range 6 > >> > >> This approach works if you can determine the right IP to check. From my > >> past experience answering the question "which of my IPs is the IP that > >> server sees" is not a trivial answer. > >> But let us say we dealt with that. Then as soon as client gets online it > >> would know its IP so it will know its location so it will be able to > >> construct a search filter and add it to the search string. > >> > > > > It is not a simple task, that said I have been working on a document on > > DNS based location discovery for some time, I will try to publish it as > > soon as I have some time. > > > > > >> I really like the idea of having location as an attribute. Then for any > >> object that client needs to fetch and it can be location specific it > >> would just add to the search filter: > >> ( | (location=) (!(location))) This will pull down > >> all entries that are specific to the location or entries that are not > >> location specific. > >> > >> This logic can be then reused across different entities not only mounts. > >> > > > > It is fundamental for SSSD to be able to discover the closest IPA > > service and fallback to the next closest in case of problems and so on. > > It's a fundamental feature we have been discussing for some time months > > ago, I will publish some docs as soon as possible. > > > > Simo. > > > > > Simo I remember and do not discard that. > But I was talking about the fact that affinity and closest server do not > mean that the location of the client is the determined correctly. > > What if the APAC replica is down and all clients now to use one in > Europe. But they are still in APAC so they need to use APAC mount shares. > For the visiting salesman with laptop from Europe his mounts should be > APAC is he visits APAC right though he is affiliated naturally with > European server? > > These are the scenarios I was talking about . I think determining > location of the client is different from the affinity and failover logic. Determining the location of the client is indeed only a precondition to determine which services to use. Depending on the service you want to use, policies on which server is appropriate to contact may vary. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Sat Nov 8 01:35:06 2008 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 07 Nov 2008 20:35:06 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226097399.10160.405.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> Message-ID: <4914ECCA.1030003@redhat.com> Simo Sorce wrote: > On Fri, 2008-11-07 at 15:18 -0500, Dmitri Pal wrote: > >> Simo Sorce wrote: >> >>> On Fri, 2008-11-07 at 12:07 -0500, Dmitri Pal wrote: >>> >>> >>>>> If we add some sort of location identifier for automount what >>>>> implications does this have for other features? Do we want to have >>>>> per-location settings for anything else? >>>>> >>>>> My plan is to embed the location into the DN of the automount map and >>>>> key names using cn. >>>>> >>>>> For the UI we'll need some method of selecting/managing this list of >>>>> locations (drop-down box comes to mind). I'm not sure if storing this >>>>> separately is a good idea or not. >>>>> >>>>> I just want to avoid any short-term choices I make don't have >>>>> long-term consequences. >>>>> >>>>> >>>>> >>>> Yes I agree . We need to have a more generic approach to that. >>>> >>>> I am struggling with understanding how client would determine which >>>> location he is in. >>>> I do not think that it should be based on the server the client is >>>> connected to. >>>> This approach would require servers to serve out data different from >>>> data being served by other replica. But what if the client fails over? >>>> Does not seem like a good approach imo. >>>> So may be using IP address range would be a better indication? >>>> May be we have a part of client policy that would say : >>>> >>>> Location A: range1, ragne2, range 3 >>>> Location B: range4, ragne5, range 6 >>>> >>>> This approach works if you can determine the right IP to check. From my >>>> past experience answering the question "which of my IPs is the IP that >>>> server sees" is not a trivial answer. >>>> But let us say we dealt with that. Then as soon as client gets online it >>>> would know its IP so it will know its location so it will be able to >>>> construct a search filter and add it to the search string. >>>> >>>> >>> It is not a simple task, that said I have been working on a document on >>> DNS based location discovery for some time, I will try to publish it as >>> soon as I have some time. >>> >>> >>> >>>> I really like the idea of having location as an attribute. Then for any >>>> object that client needs to fetch and it can be location specific it >>>> would just add to the search filter: >>>> ( | (location=) (!(location))) This will pull down >>>> all entries that are specific to the location or entries that are not >>>> location specific. >>>> >>>> This logic can be then reused across different entities not only mounts. >>>> >>>> >>> It is fundamental for SSSD to be able to discover the closest IPA >>> service and fallback to the next closest in case of problems and so on. >>> It's a fundamental feature we have been discussing for some time months >>> ago, I will publish some docs as soon as possible. >>> >>> Simo. >>> >>> >>> >> Simo I remember and do not discard that. >> But I was talking about the fact that affinity and closest server do not >> mean that the location of the client is the determined correctly. >> >> What if the APAC replica is down and all clients now to use one in >> Europe. But they are still in APAC so they need to use APAC mount shares. >> For the visiting salesman with laptop from Europe his mounts should be >> APAC is he visits APAC right though he is affiliated naturally with >> European server? >> >> These are the scenarios I was talking about . I think determining >> location of the client is different from the affinity and failover logic. >> > > Determining the location of the client is indeed only a precondition to > determine which services to use. Depending on the service you want to > use, policies on which server is appropriate to contact may vary. > > Simo. > > I would say they are two independent things: which server to connect and which policies to get. From ssorce at redhat.com Sat Nov 8 17:18:07 2008 From: ssorce at redhat.com (Simo Sorce) Date: Sat, 08 Nov 2008 12:18:07 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <4914ECCA.1030003@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> Message-ID: <1226164687.10160.420.camel@localhost.localdomain> On Fri, 2008-11-07 at 20:35 -0500, Dmitri Pal wrote: > > Determining the location of the client is indeed only a precondition to > > determine which services to use. Depending on the service you want to > > use, policies on which server is appropriate to contact may vary. > > > > Simo. > > > > > I would say they are two independent things: which server to connect and > which policies to get. I think you misunderstood my comment, I will try to rephrase. Depending on what kind of service (not server) you want to connect to (ldap, nfs, http, ssh, ftp, mysql, voip, imap, smtp, ....) the admins may decide that different servers are the ones you should connect to, depending on which location your machine is located. If you are in Europe and there are servers in Madrid, Berlin, London that offer nfs and imap, admins may have a policy (not in the sense of IPA policies, I mean company policy) for which nfs is always going to be served from the server in Madrid, while imap can be served either from London or Berlin, whichever is closest to you. So for this decision to happen, location is just a precondition, further configuration directives are needed to determine what should be the client preferences. (For some services this will be done through DNS discovery, while in some cases an IPA policy might override DNS mechanisms). (I am willfully completely ignoring the fact that currently most Linux apps cannot dynamically change their configuration this way, but IPA components should move toward this goal). Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Mon Nov 10 14:38:56 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 09:38:56 -0500 Subject: [Freeipa-devel] Please review: Bug 470379 - winsync requires updated dna plugin to handle internal operations In-Reply-To: <4914B437.4090007@redhat.com> References: <4914A2DE.5030604@redhat.com> <4914B437.4090007@redhat.com> Message-ID: <49184780.3020209@redhat.com> Nathan Kinder wrote: > Rich Megginson wrote: >> https://bugzilla.redhat.com/show_bug.cgi?id=470379 >> Resolves: bug 470379 >> Description: winsync requires updated dna plugin to handle internal >> operations >> Fix Description: Nathan already did most of the work, I just made some >> minor tweaks. This allows the DNA plugin in IPA to handle internal >> operations such as those from WinSync. >> Tested on: Fedora 9 with IPA WinSync >> https://bugzilla.redhat.com/attachment.cgi?id=322896&action=diff > ack >> > pushed to master and ipa-1-2 rob From dpal at redhat.com Mon Nov 10 15:01:49 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 10 Nov 2008 10:01:49 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226164687.10160.420.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> Message-ID: <49184CDD.2070007@redhat.com> Simo Sorce wrote: > On Fri, 2008-11-07 at 20:35 -0500, Dmitri Pal wrote: > > >>> Determining the location of the client is indeed only a precondition to >>> determine which services to use. Depending on the service you want to >>> use, policies on which server is appropriate to contact may vary. >>> >>> Simo. >>> >>> >>> >> I would say they are two independent things: which server to connect and >> which policies to get. >> > > I think you misunderstood my comment, I will try to rephrase. > > Depending on what kind of service (not server) you want to connect to > (ldap, nfs, http, ssh, ftp, mysql, voip, imap, smtp, ....) the admins > may decide that different servers are the ones you should connect to, > depending on which location your machine is located. > > If you are in Europe and there are servers in Madrid, Berlin, London > that offer nfs and imap, admins may have a policy (not in the sense of > IPA policies, I mean company policy) for which nfs is always going to be > served from the server in Madrid, while imap can be served either from > London or Berlin, whichever is closest to you. > > So for this decision to happen, location is just a precondition, further > configuration directives are needed to determine what should be the > client preferences. (For some services this will be done through DNS > discovery, while in some cases an IPA policy might override DNS > mechanisms). > > (I am willfully completely ignoring the fact that currently most Linux > apps cannot dynamically change their configuration this way, but IPA > components should move toward this goal). > > Simo. > > Ok I got it. Sounds like a good idea to keep in mind for future. This is definitely not something we would be able to solve in v2. Thanks Dmitri From rcritten at redhat.com Mon Nov 10 15:10:30 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 10:10:30 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49184CDD.2070007@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> Message-ID: <49184EE6.70409@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Fri, 2008-11-07 at 20:35 -0500, Dmitri Pal wrote: >> >> >>>> Determining the location of the client is indeed only a precondition to >>>> determine which services to use. Depending on the service you want to >>>> use, policies on which server is appropriate to contact may vary. >>>> >>>> Simo. >>>> >>>> >>> I would say they are two independent things: which server to connect >>> and which policies to get. >>> >> >> I think you misunderstood my comment, I will try to rephrase. >> >> Depending on what kind of service (not server) you want to connect to >> (ldap, nfs, http, ssh, ftp, mysql, voip, imap, smtp, ....) the admins >> may decide that different servers are the ones you should connect to, >> depending on which location your machine is located. >> >> If you are in Europe and there are servers in Madrid, Berlin, London >> that offer nfs and imap, admins may have a policy (not in the sense of >> IPA policies, I mean company policy) for which nfs is always going to be >> served from the server in Madrid, while imap can be served either from >> London or Berlin, whichever is closest to you. >> >> So for this decision to happen, location is just a precondition, further >> configuration directives are needed to determine what should be the >> client preferences. (For some services this will be done through DNS >> discovery, while in some cases an IPA policy might override DNS >> mechanisms). >> >> (I am willfully completely ignoring the fact that currently most Linux >> apps cannot dynamically change their configuration this way, but IPA >> components should move toward this goal). >> >> Simo. >> >> > Ok I got it. > Sounds like a good idea to keep in mind for future. > This is definitely not something we would be able to solve in v2. And this is what I meant by poor choices now affecting the future :-) Right now I'm sort of waving my hand saying 'location will be in the DN of the automount entry' but I don't yet say where I'm storing location other than in the DN. This will require the UI to fetch all the automount entries and sift thru the names to determine the list of locations to present to a user. On the command-line it would be easier as we'd just pass along the location requested, though this would be prone to typos. We could live with this for now and in the future store location in some central point. It wouldn't affect the UI, just make the processing a bit less intensive. I'm thinking of just setting location as a cn in the DN, so a map DN would look like: dn: automountmapname=auto.direct, cn=Baltimore, cn=automount, dc=example, dc=com or dn: automountmapname=auto.direct, cn=default, cn=automount, dc=example, dc=com I should probably treat the location cn as case-sensitive since that is what the cn attribute defines. rob From sbose at redhat.com Mon Nov 10 15:50:06 2008 From: sbose at redhat.com (Sumit Bose) Date: Mon, 10 Nov 2008 16:50:06 +0100 Subject: [Freeipa-devel] Minor change to host based access control time stamp related logic In-Reply-To: <4914BD99.9000905@redhat.com> References: <4914BD99.9000905@redhat.com> Message-ID: <4918582E.50107@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dmitri Pal schrieb: > Hi, > > After some evaluation and research the original idea of including > timezone into time stamps stored in HBAC rules does not fly well since > different platforms have different level of support for modern time zone > definitions. > > See details on the following page: > http://www.freeipa.org/page/Representing_Objects_in_DS_%28Part_1_-_Base_objects%29#Expressing_Time_in_a_Rule > > Is there already a scheme how 'week-of-the-month' is calculated. The 'week-of-the-year' is defined by ISO8601, but after a short search I haven't found anything similar for 'week-of-the-month'. bye, Sumit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkkYWCsACgkQUDGHpI6P4rpX+ACfaH17YCpbIgtDJ4Yd2Fn0NAkK mN4An3NKeyPeiF7x2eAxGpscR79DeaT6 =v/v2 -----END PGP SIGNATURE----- From ssorce at redhat.com Mon Nov 10 16:14:20 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 10 Nov 2008 11:14:20 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49184EE6.70409@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> Message-ID: <1226333660.10160.446.camel@localhost.localdomain> On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: > And this is what I meant by poor choices now affecting the future :-) > > Right now I'm sort of waving my hand saying 'location will be in the DN > of the automount entry' but I don't yet say where I'm storing location > other than in the DN. This will require the UI to fetch all the > automount entries and sift thru the names to determine the list of > locations to present to a user. Is it necessary to store it in the DN ? Why can't we add it into the entry instead ? > On the command-line it would be easier as we'd just pass along the > location requested, though this would be prone to typos. True, but it is not the end of the world if it is just an attribute. If it is part of the DN in a hierarchy of objects it is much more severe as we still do not have subtree renames with FDS. > We could live with this for now and in the future store location in some > central point. It wouldn't affect the UI, just make the processing a bit > less intensive. I see the location for now just as a filter, if we put it as an additional attribute for the entry you don't need to show anything else but what is already there (and allow to add arbitrary new location strings) > I'm thinking of just setting location as a cn in the DN, so a map DN > would look like: > > dn: automountmapname=auto.direct, cn=Baltimore, cn=automount, > dc=example, dc=com > > or > > dn: automountmapname=auto.direct, cn=default, cn=automount, dc=example, > dc=com > > I should probably treat the location cn as case-sensitive since that is > what the cn attribute defines. IIRC the CN is case-insensitive, anyway I would really prefer not to embed the location in the DN also because, as I said in another mail, we may want to associate an automount map with multiple locations. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Mon Nov 10 17:26:14 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 10 Nov 2008 12:26:14 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226333660.10160.446.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> Message-ID: <49186EB6.9050400@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: > >> And this is what I meant by poor choices now affecting the future :-) >> >> Right now I'm sort of waving my hand saying 'location will be in the DN >> of the automount entry' but I don't yet say where I'm storing location >> other than in the DN. This will require the UI to fetch all the >> automount entries and sift thru the names to determine the list of >> locations to present to a user. >> > > Is it necessary to store it in the DN ? > Why can't we add it into the entry instead ? > > >> On the command-line it would be easier as we'd just pass along the >> location requested, though this would be prone to typos. >> > > True, but it is not the end of the world if it is just an attribute. > If it is part of the DN in a hierarchy of objects it is much more severe > as we still do not have subtree renames with FDS. > > >> We could live with this for now and in the future store location in some >> central point. It wouldn't affect the UI, just make the processing a bit >> less intensive. >> > > I see the location for now just as a filter, if we put it as an > additional attribute for the entry you don't need to show anything else > but what is already there (and allow to add arbitrary new location > strings) > > >> I'm thinking of just setting location as a cn in the DN, so a map DN >> would look like: >> >> dn: automountmapname=auto.direct, cn=Baltimore, cn=automount, >> dc=example, dc=com >> >> or >> >> dn: automountmapname=auto.direct, cn=default, cn=automount, dc=example, >> dc=com >> >> I should probably treat the location cn as case-sensitive since that is >> what the cn attribute defines. >> > > IIRC the CN is case-insensitive, anyway I would really prefer not to > embed the location in the DN also because, as I said in another mail, we > may want to associate an automount map with multiple locations. > > Simo. > > +1 on all Simo's points. :-) From dpal at redhat.com Mon Nov 10 17:28:46 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 10 Nov 2008 12:28:46 -0500 Subject: [Freeipa-devel] Minor change to host based access control time stamp related logic In-Reply-To: <4918582E.50107@redhat.com> References: <4914BD99.9000905@redhat.com> <4918582E.50107@redhat.com> Message-ID: <49186F4E.1020205@redhat.com> Sumit Bose wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Dmitri Pal schrieb: > >> Hi, >> >> After some evaluation and research the original idea of including >> timezone into time stamps stored in HBAC rules does not fly well since >> different platforms have different level of support for modern time zone >> definitions. >> >> See details on the following page: >> http://www.freeipa.org/page/Representing_Objects_in_DS_%28Part_1_-_Base_objects%29#Expressing_Time_in_a_Rule >> >> >> > Is there already a scheme how 'week-of-the-month' is calculated. The > 'week-of-the-year' is defined by ISO8601, but after a short search I > haven't found anything similar for 'week-of-the-month'. > > I am not aware of any. I am planning to create a library that will deal with these time stamps and validate HBAC rules. I will propose the interface probably later this week. Long weekend is looming :-) Thanks Dmitri > bye, > Sumit > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org > > iEYEARECAAYFAkkYWCsACgkQUDGHpI6P4rpX+ACfaH17YCpbIgtDJ4Yd2Fn0NAkK > mN4An3NKeyPeiF7x2eAxGpscR79DeaT6 > =v/v2 > -----END PGP SIGNATURE----- > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > From rcritten at redhat.com Mon Nov 10 20:18:56 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 15:18:56 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226333660.10160.446.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> Message-ID: <49189730.8090104@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: >> And this is what I meant by poor choices now affecting the future :-) >> >> Right now I'm sort of waving my hand saying 'location will be in the DN >> of the automount entry' but I don't yet say where I'm storing location >> other than in the DN. This will require the UI to fetch all the >> automount entries and sift thru the names to determine the list of >> locations to present to a user. > > Is it necessary to store it in the DN ? > Why can't we add it into the entry instead ? Because all shares are rooted in the same place, auto_master. So we need a separate auto_master for each location. Our UI would be clever enough to look at another attribute but the autofs code isn't. >> On the command-line it would be easier as we'd just pass along the >> location requested, though this would be prone to typos. > > True, but it is not the end of the world if it is just an attribute. > If it is part of the DN in a hierarchy of objects it is much more severe > as we still do not have subtree renames with FDS. I don't think that would be a problem. If they want to rename a location we'll just demand that they remove and re-create the maps for now. If we wanted to be super-user-friendly we could probably do this automatically but I think manually would be fine for starters. >> We could live with this for now and in the future store location in some >> central point. It wouldn't affect the UI, just make the processing a bit >> less intensive. > > I see the location for now just as a filter, if we put it as an > additional attribute for the entry you don't need to show anything else > but what is already there (and allow to add arbitrary new location > strings) Granted I'm not an expert at autofs but I don't think the client-side code works that way. >> I'm thinking of just setting location as a cn in the DN, so a map DN >> would look like: >> >> dn: automountmapname=auto.direct, cn=Baltimore, cn=automount, >> dc=example, dc=com >> >> or >> >> dn: automountmapname=auto.direct, cn=default, cn=automount, dc=example, >> dc=com >> >> I should probably treat the location cn as case-sensitive since that is >> what the cn attribute defines. > > IIRC the CN is case-insensitive, anyway I would really prefer not to > embed the location in the DN also because, as I said in another mail, we > may want to associate an automount map with multiple locations. I think you'd have to duplicate the map for each location. If you have the same map everywhere then the argument for having separate locations becomes weaker, IMHO. rob From rcritten at redhat.com Mon Nov 10 22:10:19 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 17:10:19 -0500 Subject: [Freeipa-devel] [PATCH] fix ipa-moduser with multi-valued attribute Message-ID: <4918B14B.9010204@redhat.com> Only the first value of a multi-value attribute was being retrieved so if you updated one with --addattr the all but the first value was dropped. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-101-moduser.patch Type: text/x-patch Size: 4488 bytes Desc: not available URL: From rcritten at redhat.com Mon Nov 10 22:11:25 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 17:11:25 -0500 Subject: [Freeipa-devel] [PATCH] let ipa-replica-manage delete winsync replicas Message-ID: <4918B18D.4020000@redhat.com> There was no way to delete winsync replicas because this assumed that both sides could be contacted. In the case of winsync all of the work is done on the IPA side, there is no equivalent agreement to remove. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-102-winsync.patch Type: text/x-patch Size: 4488 bytes Desc: not available URL: From dpal at redhat.com Mon Nov 10 22:30:30 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 10 Nov 2008 17:30:30 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49189730.8090104@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> <49189730.8090104@redhat.com> Message-ID: <4918B606.9070908@redhat.com> Rob Crittenden wrote: > Simo Sorce wrote: >> On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: >>> And this is what I meant by poor choices now affecting the future :-) >>> >>> Right now I'm sort of waving my hand saying 'location will be in the >>> DN of the automount entry' but I don't yet say where I'm storing >>> location other than in the DN. This will require the UI to fetch all >>> the automount entries and sift thru the names to determine the list >>> of locations to present to a user. >> >> Is it necessary to store it in the DN ? >> Why can't we add it into the entry instead ? > > Because all shares are rooted in the same place, auto_master. So we > need a separate auto_master for each location. Our UI would be clever > enough to look at another attribute but the autofs code isn't. > >>> On the command-line it would be easier as we'd just pass along the >>> location requested, though this would be prone to typos. >> >> True, but it is not the end of the world if it is just an attribute. >> If it is part of the DN in a hierarchy of objects it is much more severe >> as we still do not have subtree renames with FDS. > > I don't think that would be a problem. If they want to rename a > location we'll just demand that they remove and re-create the maps for > now. If we wanted to be super-user-friendly we could probably do this > automatically but I think manually would be fine for starters. > >>> We could live with this for now and in the future store location in >>> some central point. It wouldn't affect the UI, just make the >>> processing a bit less intensive. >> >> I see the location for now just as a filter, if we put it as an >> additional attribute for the entry you don't need to show anything else >> but what is already there (and allow to add arbitrary new location >> strings) > > Granted I'm not an expert at autofs but I don't think the client-side > code works that way. > >>> I'm thinking of just setting location as a cn in the DN, so a map DN >>> would look like: >>> >>> dn: automountmapname=auto.direct, cn=Baltimore, cn=automount, >>> dc=example, dc=com >>> >>> or >>> >>> dn: automountmapname=auto.direct, cn=default, cn=automount, >>> dc=example, dc=com >>> >>> I should probably treat the location cn as case-sensitive since that >>> is what the cn attribute defines. >> >> IIRC the CN is case-insensitive, anyway I would really prefer not to >> embed the location in the DN also because, as I said in another mail, we >> may want to associate an automount map with multiple locations. > > I think you'd have to duplicate the map for each location. If you have > the same map everywhere then the argument for having separate > locations becomes weaker, IMHO. > > rob Is there a requirement to support locations in v2? If not then we do not need to solve the problem now. We can just have a designated default location and have a map for everybody in v2. In v3 we can solve a problem in different ways - create separate containers for separate locations under default place, use "compat" tree and create virtual entries like we do with other things or explore more the use of location attribute and so on. Thanks Dmitri From ssorce at redhat.com Mon Nov 10 22:46:22 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 10 Nov 2008 17:46:22 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <49189730.8090104@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> <49189730.8090104@redhat.com> Message-ID: <1226357182.10160.475.camel@localhost.localdomain> On Mon, 2008-11-10 at 15:18 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: > >> And this is what I meant by poor choices now affecting the future :-) > >> > >> Right now I'm sort of waving my hand saying 'location will be in the DN > >> of the automount entry' but I don't yet say where I'm storing location > >> other than in the DN. This will require the UI to fetch all the > >> automount entries and sift thru the names to determine the list of > >> locations to present to a user. > > > > Is it necessary to store it in the DN ? > > Why can't we add it into the entry instead ? > > Because all shares are rooted in the same place, auto_master. So we need > a separate auto_master for each location. Our UI would be clever enough > to look at another attribute but the autofs code isn't. Ok in this case I would just name these configurations with admin chosen names, and not refer to them as locations, with cn=default being the default. Let's just provide a way to name their configuration so that if admins need more than one for different set of clients they can easily do it, but let's not call this name a "location", as it can be anything the admin want's to call it. We can add the concept of location later on as attributes. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Mon Nov 10 22:49:44 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 10 Nov 2008 17:49:44 -0500 Subject: [Freeipa-devel] [PATCH] let ipa-replica-manage delete winsync replicas In-Reply-To: <4918B18D.4020000@redhat.com> References: <4918B18D.4020000@redhat.com> Message-ID: <1226357384.10160.477.camel@localhost.localdomain> On Mon, 2008-11-10 at 17:11 -0500, Rob Crittenden wrote: > There was no way to delete winsync replicas because this assumed that > both sides could be contacted. In the case of winsync all of the work is > done on the IPA side, there is no equivalent agreement to remove. ack Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Mon Nov 10 22:50:09 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 10 Nov 2008 17:50:09 -0500 Subject: [Freeipa-devel] [PATCH] fix ipa-moduser with multi-valued attribute In-Reply-To: <4918B14B.9010204@redhat.com> References: <4918B14B.9010204@redhat.com> Message-ID: <1226357409.10160.479.camel@localhost.localdomain> On Mon, 2008-11-10 at 17:10 -0500, Rob Crittenden wrote: > Only the first value of a multi-value attribute was being retrieved so > if you updated one with --addattr the all but the first value was dropped. I think you sent the wrong patch Rob. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Mon Nov 10 23:04:46 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 18:04:46 -0500 Subject: [Freeipa-devel] [PATCH] fix ipa-moduser with multi-valued attribute In-Reply-To: <1226357409.10160.479.camel@localhost.localdomain> References: <4918B14B.9010204@redhat.com> <1226357409.10160.479.camel@localhost.localdomain> Message-ID: <4918BE0E.3060303@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 17:10 -0500, Rob Crittenden wrote: >> Only the first value of a multi-value attribute was being retrieved so >> if you updated one with --addattr the all but the first value was dropped. > > I think you sent the wrong patch Rob. > > Simo. > > Oops. Correct one attached. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-101-moduser.patch Type: text/x-patch Size: 887 bytes Desc: not available URL: From rcritten at redhat.com Mon Nov 10 23:08:40 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 10 Nov 2008 18:08:40 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226357182.10160.475.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> <49189730.8090104@redhat.com> <1226357182.10160.475.camel@localhost.localdomain> Message-ID: <4918BEF8.2010302@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 15:18 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Mon, 2008-11-10 at 10:10 -0500, Rob Crittenden wrote: >>>> And this is what I meant by poor choices now affecting the future :-) >>>> >>>> Right now I'm sort of waving my hand saying 'location will be in the DN >>>> of the automount entry' but I don't yet say where I'm storing location >>>> other than in the DN. This will require the UI to fetch all the >>>> automount entries and sift thru the names to determine the list of >>>> locations to present to a user. >>> Is it necessary to store it in the DN ? >>> Why can't we add it into the entry instead ? >> Because all shares are rooted in the same place, auto_master. So we need >> a separate auto_master for each location. Our UI would be clever enough >> to look at another attribute but the autofs code isn't. > > Ok in this case I would just name these configurations with admin chosen > names, and not refer to them as locations, with cn=default being the > default. > Let's just provide a way to name their configuration so that if admins > need more than one for different set of clients they can easily do it, > but let's not call this name a "location", as it can be anything the > admin want's to call it. > We can add the concept of location later on as attributes. > > Simo. > Ok, for clarification, this is why I'm proposing having the location/name/whatever in the DN. You have to supply a base search DN in the autofs config file, on Linux anyway. So if we cleverly supply this baseDN it is easy to support multiple configurations. autofs searches for its base configuration like this: SRCH base="dc=example,dc=com" scope=2 filter="(&(objectClass=automountMap)(automountMapName=auto.master))" attrs="automountMapName" So if we can set that base to something where a unique auto.master will be found we can control what is in that master. It seems like that is the only thing that truly needs to be unique. rob From mnagy at redhat.com Mon Nov 10 23:15:02 2008 From: mnagy at redhat.com (Martin Nagy) Date: Tue, 11 Nov 2008 00:15:02 +0100 Subject: [Freeipa-devel] [PATCH] fix ipa-moduser with multi-valued attribute In-Reply-To: <4918BE0E.3060303@redhat.com> References: <4918B14B.9010204@redhat.com> <1226357409.10160.479.camel@localhost.localdomain> <4918BE0E.3060303@redhat.com> Message-ID: <20081111001502.58cbce56@notas> Rob Crittenden wrote: > Simo Sorce wrote: > > On Mon, 2008-11-10 at 17:10 -0500, Rob Crittenden wrote: > >> Only the first value of a multi-value attribute was being > >> retrieved so if you updated one with --addattr the all but the > >> first value was dropped. > > > > I think you sent the wrong patch Rob. > > > > Simo. > > > > > > Oops. Correct one attached. > > rob ack From jderose at redhat.com Tue Nov 11 01:19:52 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 10 Nov 2008 18:19:52 -0700 Subject: [Freeipa-devel] Introducing the freeIPA v2 source tree Message-ID: <4918DDB8.9000709@redhat.com> The experimental freeIPA v2 branch is now available! NOTE: This is NOT the release of freeIPA 2.0! This is only a highly experimental, incomplete, and unstable work-in-progress! Do not use this code in production yet! Details follow, but for the impatient, you can checkout the code with this command: git clone git://fedorapeople.org/~jderose/freeipa2.git And the new API documentation can be found here: http://jderose.fedorapeople.org/freeipa2-dev-doc/ Some of this initial development was done behind closed doors, for which I personally apologize. The initial plugin work started as my personal doodling and brainstorming, and then ended up being our v2 code base. Between this and the fact that we are all very busy here, I didn't get this work out into the community as quickly as I should have. ====================== Overview of what's new ====================== For v2 we have done a substantial refactoring of the underlying Python code to eliminate code redundancy and to utilize fully the rapid development capabilities of Python. This was rather desperately needed to keep the code manageable as we add the huge increase in functionality planned for v2. These are the goals of the refactoring, all of which have been met: 1. Implement a pluggable architecture. In v2, all major functionality is pluggable, and it's trivial to add new classes of pluggable functionality. There is now a single VCS-style "ipa" script where the first argument is the command name (and where each command corresponds to a command plugin). The pluggable architecture was inspired by Bazaar. 2. Eliminate the code duplication present in v1. This has been fixed by the design of the v2 pluggable architecture. 3. Make it easy to run and to test from within the source tree. All of the Python packages are now located in the root of the source tree, allowing one Python package to import another. For one, this allows you to run freeIPA entirely from the source tree, meaning you no longer need to build the distribution package to experiment with certain changes. For another, this change was required to make it easy (or sometimes just possible) to write unit tests, particularly if the test involved multiple Python packages. Currently the v2 code has more-or-less all the v1 functionality minus the installation script and web-UI. One notable new addition in v2 (at this early stage) is automount support. Obviously many more features will be added before the stable 2.0 release is made. The easiest way to experiment with v2 is to first install v1 so that LDAP and Kerberos get configured correctly. Then v2 can be installed in parallel with v1, or simply run from its source tree. To get a list of the available commands, run the "ipa" script with no command, like this: ./ipa =================== Python coding style =================== For v2, we now officially have a Python coding style. Although Python's syntax enforces quite a lot of stylistic consistency, there is still much wiggle room, so a style guide will help keep us productive and happy. Fortunately, thanks to the "Benevolent Dictator For Life" (Guido van Rossum, the author of Python), there is only one coding style for Python, which is defined in PEP-8 and PEP-257. PEP 8 -- Style Guide for Python Code: http://www.python.org/dev/peps/pep-0008 PEP 257 -- Docstring Conventions: http://www.python.org/dev/peps/pep-0257 ====================== Unit testing with nose ====================== For v2, all the Python unit testing is being done using nose: http://somethingaboutorange.com/mrl/projects/nose/ nose was chosen because it allows unit tests to be written very quickly and in a quite natural and flexible manner. All the unit tests are located in the tests/ directory in the source tree. ======================================= API documentation generated with epydoc ======================================= For v2 we are generating API documentation using the epydoc tool: http://epydoc.sourceforge.net The API documentation is currently available here: http://freeipa.org/developer-docs/ And a tutorial for plugin writers is available here: http://jderose.fedorapeople.org/freeipa2-dev-doc/ipalib-module.html We are using reStructuredText markup in the docstrings, which you can read about here: http://docutils.sourceforge.net/rst.html The epydoc generated documentation is an excellent way to learn the v2 code base, particularly because all the cross-references are hyper-linked. I'm personally making a big effort to include lots of example code in the docstrings for the core v2 library. All these examples are automatically tested using the Python doctest module: http://docs.python.org/library/doctest.html We are relying on nose to automatically discover all the code examples and then to test them with doctest. This is done using the --with-doctest option, like this: nosetests --with-doctest ================ Getting involved ================ We certainly welcome comments, ideas, code, documentation, etc. See the "Contribute" page on freeIPA.org: http://freeipa.org/page/Contribute If you have questions or comments about the v2 plugin architecture, feel free to ping me (jderose) on the #freeipa-devel IRC channel on freenode.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From ssorce at redhat.com Tue Nov 11 05:04:54 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 11 Nov 2008 00:04:54 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <4918BEF8.2010302@redhat.com> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> <49189730.8090104@redhat.com> <1226357182.10160.475.camel@localhost.localdomain> <4918BEF8.2010302@redhat.com> Message-ID: <1226379894.10160.485.camel@localhost.localdomain> On Mon, 2008-11-10 at 18:08 -0500, Rob Crittenden wrote: > > Ok, for clarification, this is why I'm proposing having the > location/name/whatever in the DN. Understood and I second that. > You have to supply a base search DN in the autofs config file, on > Linux > anyway. > > So if we cleverly supply this baseDN it is easy to support multiple > configurations. > > autofs searches for its base configuration like this: > > SRCH base="dc=example,dc=com" scope=2 > filter="(&(objectClass=automountMap)(automountMapName=auto.master))" > attrs="automountMapName" Is this filter hardcoded into the autofs binary? Or is it possible to change via a configuration directive? > So if we can set that base to something where a unique auto.master > will > be found we can control what is in that master. It seems like that is > the only thing that truly needs to be unique. Yes, if the filter cannot be changed we will have "named" configurations where the names will be used to create a container. Like: cn=fooname,cn=automount,dc=example,dc=com And this should probably be the auto.master object. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Tue Nov 11 05:25:46 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 11 Nov 2008 00:25:46 -0500 Subject: [Freeipa-devel] Introducing the freeIPA v2 source tree In-Reply-To: <4918DDB8.9000709@redhat.com> References: <4918DDB8.9000709@redhat.com> Message-ID: <1226381146.10160.490.camel@localhost.localdomain> On Mon, 2008-11-10 at 18:19 -0700, Jason Gerard DeRose wrote: > The experimental freeIPA v2 branch is now available! I can't wait to get it in the master tree soon. Thanks Jason, this is very good stuff! Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Tue Nov 11 14:35:55 2008 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 11 Nov 2008 09:35:55 -0500 Subject: [Freeipa-devel] Introducing the freeIPA v2 source tree In-Reply-To: <1226381146.10160.490.camel@localhost.localdomain> References: <4918DDB8.9000709@redhat.com> <1226381146.10160.490.camel@localhost.localdomain> Message-ID: <4919984B.9070208@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 18:19 -0700, Jason Gerard DeRose wrote: > >> The experimental freeIPA v2 branch is now available! >> > > I can't wait to get it in the master tree soon. > > Thanks Jason, > this is very good stuff! > > Simo. > > Cool! Thanks Jason. From dpal at redhat.com Tue Nov 11 14:38:57 2008 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 11 Nov 2008 09:38:57 -0500 Subject: [Freeipa-devel] automount in LDAP In-Reply-To: <1226379894.10160.485.camel@localhost.localdomain> References: <49111F98.4020806@redhat.com> <20081105172013.GB10119@redhat.com> <49121505.8040501@redhat.com> <20081105223453.GD10119@redhat.com> <4912F694.8090309@redhat.com> <1225982374.10160.117.camel@localhost.localdomain> <49130700.7040909@redhat.com> <1225991919.10160.217.camel@localhost.localdomain> <49144FEB.8000406@redhat.com> <491475B4.3050604@redhat.com> <1226085453.10160.400.camel@localhost.localdomain> <4914A27F.7090109@redhat.com> <1226097399.10160.405.camel@localhost.localdomain> <4914ECCA.1030003@redhat.com> <1226164687.10160.420.camel@localhost.localdomain> <49184CDD.2070007@redhat.com> <49184EE6.70409@redhat.com> <1226333660.10160.446.camel@localhost.localdomain> <49189730.8090104@redhat.com> <1226357182.10160.475.camel@localhost.localdomain> <4918BEF8.2010302@redhat.com> <1226379894.10160.485.camel@localhost.localdomain> Message-ID: <49199901.5050501@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 18:08 -0500, Rob Crittenden wrote: > >> Ok, for clarification, this is why I'm proposing having the >> location/name/whatever in the DN. >> > > Understood and I second that. > > >> You have to supply a base search DN in the autofs config file, on >> Linux >> anyway. >> >> So if we cleverly supply this baseDN it is easy to support multiple >> configurations. >> >> autofs searches for its base configuration like this: >> >> SRCH base="dc=example,dc=com" scope=2 >> filter="(&(objectClass=automountMap)(automountMapName=auto.master))" >> attrs="automountMapName" >> > > Is this filter hardcoded into the autofs binary? Or is it possible to > change via a configuration directive? > > >> So if we can set that base to something where a unique auto.master >> will >> be found we can control what is in that master. It seems like that is >> the only thing that truly needs to be unique. >> > > Yes, if the filter cannot be changed we will have "named" configurations > where the names will be used to create a container. > > Like: cn=fooname,cn=automount,dc=example,dc=com > And this should probably be the auto.master object. > ... but if it can be changed we can use it to find different "master" maps from different clients. Then the master maps can be name like auto.master. Is this the case? Dmitri > Simo. > > From dpal at redhat.com Tue Nov 11 21:57:29 2008 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 11 Nov 2008 16:57:29 -0500 Subject: [Freeipa-devel] Coding style Message-ID: <4919FFC9.7050600@redhat.com> Hello, Coding style has been updated on the freeIPA project site. Based on the discussion we had last week. http://www.freeipa.org/page/Coding_Style Thanks Dmitri From rmeggins at redhat.com Tue Nov 11 22:28:36 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Tue, 11 Nov 2008 15:28:36 -0700 Subject: [Freeipa-devel] Please review: Set IPA WinSync account disable sync default value to "both" Message-ID: <491A0714.6020403@redhat.com> Enable/Disable account sync should work both ways by default. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-set-winsync-account-disable-sync-default-value-to-bo.patch Type: text/x-patch Size: 1147 bytes Desc: not available URL: From ssorce at redhat.com Tue Nov 11 22:38:55 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 11 Nov 2008 17:38:55 -0500 Subject: [Freeipa-devel] Please review: Set IPA WinSync account disable sync default value to "both" In-Reply-To: <491A0714.6020403@redhat.com> References: <491A0714.6020403@redhat.com> Message-ID: <1226443135.10160.522.camel@localhost.localdomain> On Tue, 2008-11-11 at 15:28 -0700, Rich Megginson wrote: > Enable/Disable account sync should work both ways by default. ack -- Simo Sorce * Red Hat, Inc * New York From jderose at redhat.com Tue Nov 11 23:11:44 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 11 Nov 2008 16:11:44 -0700 Subject: [Freeipa-devel] Coding style In-Reply-To: <4919FFC9.7050600@redhat.com> References: <4919FFC9.7050600@redhat.com> Message-ID: <491A1130.5070707@redhat.com> Dmitri Pal wrote: > Hello, > > Coding style has been updated on the freeIPA project site. > Based on the discussion we had last week. > > http://www.freeipa.org/page/Coding_Style > > Thanks > Dmitri Looks good, Dmitri. My only suggestion is that we need to clarify that this is the C coding style for freeIPA. We need a separate Python coding style page, which I'll gladly author. If no one objects, I'll call the new page Python_Coding_Style. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From ssorce at redhat.com Tue Nov 11 23:30:30 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 11 Nov 2008 18:30:30 -0500 Subject: [Freeipa-devel] Coding style In-Reply-To: <491A1130.5070707@redhat.com> References: <4919FFC9.7050600@redhat.com> <491A1130.5070707@redhat.com> Message-ID: <1226446230.10160.525.camel@localhost.localdomain> On Tue, 2008-11-11 at 16:11 -0700, Jason Gerard DeRose wrote: > Dmitri Pal wrote: > > Hello, > > > > Coding style has been updated on the freeIPA project site. > > Based on the discussion we had last week. > > > > http://www.freeipa.org/page/Coding_Style > > > > Thanks > > Dmitri > > Looks good, Dmitri. My only suggestion is that we need to clarify that > this is the C coding style for freeIPA. We need a separate Python coding > style page, which I'll gladly author. > > If no one objects, I'll call the new page Python_Coding_Style. Works for me. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Wed Nov 12 02:45:23 2008 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 11 Nov 2008 21:45:23 -0500 Subject: [Freeipa-devel] Coding style In-Reply-To: <1226446230.10160.525.camel@localhost.localdomain> References: <4919FFC9.7050600@redhat.com> <491A1130.5070707@redhat.com> <1226446230.10160.525.camel@localhost.localdomain> Message-ID: <491A4343.9050700@redhat.com> Simo Sorce wrote: > On Tue, 2008-11-11 at 16:11 -0700, Jason Gerard DeRose wrote: > >> Dmitri Pal wrote: >> >>> Hello, >>> >>> Coding style has been updated on the freeIPA project site. >>> Based on the discussion we had last week. >>> >>> http://www.freeipa.org/page/Coding_Style >>> >>> Thanks >>> Dmitri >>> >> Looks good, Dmitri. My only suggestion is that we need to clarify that >> this is the C coding style for freeIPA. We need a separate Python coding >> style page, which I'll gladly author. >> >> If no one objects, I'll call the new page Python_Coding_Style. >> > > Works for me. > > Simo. > > Yes works for me two. Then you just need to remove couple things that mention Python on that page and add a sentence that this is the page for C coding style. Thanks Dmitri From jonathansteffan at gmail.com Wed Nov 12 02:45:23 2008 From: jonathansteffan at gmail.com (Jonathan Steffan) Date: Tue, 11 Nov 2008 19:45:23 -0700 Subject: [Freeipa-devel] SPEC Patch for 1.2.0 Message-ID: <491A4343.3090702@gmail.com> This is a proposed update for the 1.2.0 specfile (in CVS, which we should really move into git.) It fixes a few Requires issues and updates the %build to allow for a direct git export to actually build. It also removes the unneeded patch set. --- Fedora/Extras/ipa/F-9/ipa.spec 2008-09-10 13:08:14.000000000 -0600 +++ rpmbuild/SPECS/ipa.spec 2008-11-10 17:16:47.000000000 -0700 @@ -5,8 +5,8 @@ %define POLICYCOREUTILSVER 1.33.12-1 Name: ipa -Version: 1.1.0 -Release: 7%{?dist} +Version: 1.2.0 +Release: 1%{?dist} Summary: The Identity, Policy and Audit system Group: System Environment/Base @@ -15,23 +15,6 @@ Source0: freeipa-%{version}.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) -Patch1: freeipa-replica-default.patch -Patch2: freeipa-noinit.patch -Patch3: freeipa-memberofindex.patch -Patch4: freeipa-index.patch -Patch5: 0001-slapi_pw_find_sv-expects-an-array-make-sure-we-ha.patch -Patch6: 0001-Fix-typo-in-inet-type.patch -Patch7: 0001-In-openvz-we-found-out-some-interfaces-may-return-a.patch - -Patch101: freeipa-refresh-mkey.patch -Patch102: freeipa-fix-expire.patch -Patch103: freeipa-fix-key-encryption.patch -Patch104: freeipa-user-input.patch -Patch105: freeipa-encrypt-files.patch -Patch106: freeipa-change-mkey.patch -Patch107: freeipa-cve-2008-3274-fixes.patch -Patch108: freeipa-cve-2008-3274-utility.patch - BuildRequires: fedora-ds-base-devel >= 1.1 BuildRequires: mozldap-devel BuildRequires: openssl-devel @@ -90,6 +73,7 @@ Requires: python-pyasn1 Requires: libcap Requires: selinux-policy +Requires: slapi-nis Requires(post): selinux-policy-base Conflicts: mod_ssl @@ -169,7 +153,7 @@ %package radius-server Summary: IPA authentication server - radius plugin Group: System Environment/Base -Requires: freeradius +Requires: freeradius, freeradius-ldap Requires: %{name}-python = %{version}-%{release} %description radius-server @@ -195,30 +179,12 @@ %prep %setup -n freeipa-%{version} -q -%patch1 -p1 -b replica-default -%patch2 -p1 -b noinit -%patch3 -p1 -b memberofindex -%patch4 -p1 -b index -%patch5 -p1 -b changepw -%patch6 -p1 -b af-inet-fix -%patch7 -p1 -b kpasswd-segfault - -#START CVE-2008-3274 block -%patch101 -p1 -b refresh-mkey -%patch102 -p1 -b fix-expire -%patch103 -p1 -b fix-key-encryption -%patch104 -p1 -b user-input -%patch105 -p1 -b encrypt-files -%patch106 -p1 -b change-mkey -%patch107 -p1 -b cve-2008-3274-fixes -%patch108 -p1 -b cve-2008-3274-utility -#END CVE-2008-3274 block %build +make IPA_VERSION_IS_GIT_SNAPSHOT=no version-update cd ipa-server; ./autogen.sh --prefix=%{_usr} --sysconfdir=%{_sysconfdir} --localstatedir=%{_localstatedir} --libdir=%{_libdir} --mandir=%{_mandir} --with-openldap=yes; cd .. cd ipa-client; ./autogen.sh --prefix=%{_usr} --sysconfdir=%{_sysconfdir} --localstatedir=%{_localstatedir} --libdir=%{_libdir} --mandir=%{_mandir} --with-openldap=yes; cd .. - -make IPA_VERSION_IS_GIT_SNAPSHOT=no %{?_smp_mflags} version-update all +make all cd ipa-server/selinux # This isn't multi-process make capable yet make all @@ -234,6 +200,7 @@ rm %{buildroot}/%{plugin_dir}/libipa_pwd_extop.la rm %{buildroot}/%{plugin_dir}/libipa-memberof-plugin.la rm %{buildroot}/%{plugin_dir}/libipa-dna-plugin.la +rm %{buildroot}/%{plugin_dir}/libipa_winsync.la # Some user-modifiable HTML files are provided. Move these to /etc # and link back. @@ -330,6 +297,8 @@ %{_sbindir}/ipa_webgui %{_sbindir}/ipactl %{_sbindir}/ipa-upgradeconfig +%{_sbindir}/ipa-defaultoptions +%{_sbindir}/ipa-ldap-updater %attr(755,root,root) %{_initrddir}/ipa_kpasswd %attr(755,root,root) %{_initrddir}/ipa_webgui %dir %{_usr}/share/ipa @@ -353,9 +322,12 @@ %{_usr}/share/ipa/ipaserver/* %dir %{_usr}/share/ipa/locales/ %{_usr}/share/ipa/locales/* +%dir %{_usr}/share/ipa/updates/ +%{_usr}/share/ipa/updates/* %attr(755,root,root) %{plugin_dir}/libipa_pwd_extop.so %attr(755,root,root) %{plugin_dir}/libipa-memberof-plugin.so %attr(755,root,root) %{plugin_dir}/libipa-dna-plugin.so +%attr(755,root,root) %{plugin_dir}/libipa_winsync.so %dir %{_localstatedir}/lib/ipa %attr(700,root,root) %dir %{_localstatedir}/lib/ipa/sysrestore %dir %{_localstatedir}/cache/ipa @@ -368,6 +340,8 @@ %{_mandir}/man1/ipa-replica-prepare.1.gz %{_mandir}/man1/ipa-server-certinstall.1.gz %{_mandir}/man1/ipa-server-install.1.gz +%{_mandir}/man1/ipa-defaultoptions.1.gz +%{_mandir}/man1/ipa-ldap-updater.1.gz %{_mandir}/man8/ipa_kpasswd.8.gz %{_mandir}/man8/ipa_webgui.8.gz %{_mandir}/man8/ipactl.8.gz @@ -463,6 +437,12 @@ %{_sbindir}/ipa-modradiusprofile %changelog +* Mon Nov 11 2008 Jonathan Steffan - 1.2.0-1 +- Update to 1.2.0 +- Remove all patches +- Update %%build +- Update Requires + * Wed Sep 10 2008 Simo Sorce - 1.1.0-7 - Fix for CVE-2008-3274 -- Jonathan Steffan daMaestro GPG Fingerprint: 93A2 3E2F DC26 5570 3472 5B16 AD12 6CE7 0D86 AF59 From dpal at redhat.com Wed Nov 12 02:48:09 2008 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 11 Nov 2008 21:48:09 -0500 Subject: [Freeipa-devel] Coding style In-Reply-To: <491A4343.9050700@redhat.com> References: <4919FFC9.7050600@redhat.com> <491A1130.5070707@redhat.com> <1226446230.10160.525.camel@localhost.localdomain> <491A4343.9050700@redhat.com> Message-ID: <491A43E9.1050301@redhat.com> > Yes works for me two. Then you just need to remove couple things that > mention Python on that page and add a sentence that this is the page > for C coding style. Hm, Two? Yes both me :-) From ssorce at redhat.com Wed Nov 12 14:58:48 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 09:58:48 -0500 Subject: [Freeipa-devel] [PATCH]: Add special passSyncManagers support Message-ID: <1226501928.10160.557.camel@localhost.localdomain> This patch adds support to read a list of DNs from the plugin configuration entry. These DNs are considered superusers from the PoV of password changes, so no policies nor immediate expiration are applied to the password being set. This functionality is needed to allow a passsycn plugin on an AD server to synchronize the password to IPA. The patch also cleans up a bit the ipapwd_start function. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-the-list-of-users-that-can-skip-passwrod-polici.patch Type: application/mbox Size: 9091 bytes Desc: not available URL: From ssorce at redhat.com Wed Nov 12 18:28:05 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 13:28:05 -0500 Subject: [Freeipa-devel] [PATCH]: Add special passSyncManagers support In-Reply-To: <1226501928.10160.557.camel@localhost.localdomain> References: <1226501928.10160.557.camel@localhost.localdomain> Message-ID: <1226514485.10160.583.camel@localhost.localdomain> On Wed, 2008-11-12 at 09:58 -0500, Simo Sorce wrote: > This patch adds support to read a list of DNs from the plugin > configuration entry. These DNs are considered superusers from the PoV of > password changes, so no policies nor immediate expiration are applied to > the password being set. > > This functionality is needed to allow a passsycn plugin on an AD server > to synchronize the password to IPA. > > The patch also cleans up a bit the ipapwd_start function. I forgot to test this with ldapmodify and of course as soon as I did (thanks Rob) I relalized I forgot to add checks for passsync managers to mod preop and add preop. This is a rebase patch that adds the missing checks. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Wed Nov 12 18:30:16 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 13:30:16 -0500 Subject: [Freeipa-devel] [PATCH]: Add special passSyncManagers support In-Reply-To: <1226514485.10160.583.camel@localhost.localdomain> References: <1226501928.10160.557.camel@localhost.localdomain> <1226514485.10160.583.camel@localhost.localdomain> Message-ID: <1226514616.10160.585.camel@localhost.localdomain> On Wed, 2008-11-12 at 13:28 -0500, Simo Sorce wrote: > On Wed, 2008-11-12 at 09:58 -0500, Simo Sorce wrote: > > This patch adds support to read a list of DNs from the plugin > > configuration entry. These DNs are considered superusers from the PoV of > > password changes, so no policies nor immediate expiration are applied to > > the password being set. > > > > This functionality is needed to allow a passsycn plugin on an AD server > > to synchronize the password to IPA. > > > > The patch also cleans up a bit the ipapwd_start function. > > I forgot to test this with ldapmodify and of course as soon as I did > (thanks Rob) I relalized I forgot to add checks for passsync managers to > mod preop and add preop. > > This is a rebase patch that adds the missing checks. Meh, with the patch this time :/ -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-the-list-of-users-that-can-skip-passwrod-polici.patch Type: application/mbox Size: 10761 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 12 18:52:37 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 13:52:37 -0500 Subject: [Freeipa-devel] [PATCH]: Add special passSyncManagers support In-Reply-To: <1226514616.10160.585.camel@localhost.localdomain> References: <1226501928.10160.557.camel@localhost.localdomain> <1226514485.10160.583.camel@localhost.localdomain> <1226514616.10160.585.camel@localhost.localdomain> Message-ID: <491B25F5.3090001@redhat.com> Simo Sorce wrote: > On Wed, 2008-11-12 at 13:28 -0500, Simo Sorce wrote: >> On Wed, 2008-11-12 at 09:58 -0500, Simo Sorce wrote: >>> This patch adds support to read a list of DNs from the plugin >>> configuration entry. These DNs are considered superusers from the PoV of >>> password changes, so no policies nor immediate expiration are applied to >>> the password being set. >>> >>> This functionality is needed to allow a passsycn plugin on an AD server >>> to synchronize the password to IPA. >>> >>> The patch also cleans up a bit the ipapwd_start function. >> I forgot to test this with ldapmodify and of course as soon as I did >> (thanks Rob) I relalized I forgot to add checks for passsync managers to >> mod preop and add preop. >> >> This is a rebase patch that adds the missing checks. > > Meh, with the patch this time :/ > It works for me. Ack. rob From ssorce at redhat.com Wed Nov 12 18:55:31 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 13:55:31 -0500 Subject: [Freeipa-devel] [PATCH]: Add special passSyncManagers support In-Reply-To: <491B25F5.3090001@redhat.com> References: <1226501928.10160.557.camel@localhost.localdomain> <1226514485.10160.583.camel@localhost.localdomain> <1226514616.10160.585.camel@localhost.localdomain> <491B25F5.3090001@redhat.com> Message-ID: <1226516131.10160.587.camel@localhost.localdomain> On Wed, 2008-11-12 at 13:52 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Wed, 2008-11-12 at 13:28 -0500, Simo Sorce wrote: > >> On Wed, 2008-11-12 at 09:58 -0500, Simo Sorce wrote: > >>> This patch adds support to read a list of DNs from the plugin > >>> configuration entry. These DNs are considered superusers from the PoV of > >>> password changes, so no policies nor immediate expiration are applied to > >>> the password being set. > >>> > >>> This functionality is needed to allow a passsycn plugin on an AD server > >>> to synchronize the password to IPA. > >>> > >>> The patch also cleans up a bit the ipapwd_start function. > >> I forgot to test this with ldapmodify and of course as soon as I did > >> (thanks Rob) I relalized I forgot to add checks for passsync managers to > >> mod preop and add preop. > >> > >> This is a rebase patch that adds the missing checks. > > > > Meh, with the patch this time :/ > > > > It works for me. Ack. Pushed to ipa-1-2 and master Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Nov 12 19:02:51 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 14:02:51 -0500 Subject: [Freeipa-devel] [PATCH] create windows PassSync user Message-ID: <491B285B.5020301@redhat.com> This works in concert with Simo's password changing patch. Create a user for Windows PassSync and grant password changing permissions This does 3 things: 1. Create a user for the Windows PassSync service 2. Add this use to the list of users that can skip password policies 3. Add an aci that grants permission to write the password attributes rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-103-passsync.patch Type: text/x-patch Size: 5004 bytes Desc: not available URL: From rcritten at redhat.com Wed Nov 12 19:04:16 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 14:04:16 -0500 Subject: [Freeipa-devel] [PATCH] provide less-cryptic error message Message-ID: <491B28B0.4030605@redhat.com> If you try to delete a sync agreement that doesn't exist you'll get a cryptic message. This fixes that. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-104-replica.patch Type: text/x-patch Size: 1166 bytes Desc: not available URL: From ssorce at redhat.com Wed Nov 12 19:36:23 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 14:36:23 -0500 Subject: [Freeipa-devel] [PATCH] provide less-cryptic error message In-Reply-To: <491B28B0.4030605@redhat.com> References: <491B28B0.4030605@redhat.com> Message-ID: <1226518583.10160.589.camel@localhost.localdomain> On Wed, 2008-11-12 at 14:04 -0500, Rob Crittenden wrote: > If you try to delete a sync agreement that doesn't exist you'll get a > cryptic message. This fixes that. ack -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Wed Nov 12 19:37:06 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 14:37:06 -0500 Subject: [Freeipa-devel] [PATCH] create windows PassSync user In-Reply-To: <491B285B.5020301@redhat.com> References: <491B285B.5020301@redhat.com> Message-ID: <1226518626.10160.591.camel@localhost.localdomain> On Wed, 2008-11-12 at 14:02 -0500, Rob Crittenden wrote: > This works in concert with Simo's password changing patch. > > Create a user for Windows PassSync and grant password changing > permissions > > This does 3 things: > 1. Create a user for the Windows PassSync service > 2. Add this use to the list of users that can skip password policies > 3. Add an aci that grants permission to write the password attributes Looks good, but shouldn't we test if the user already exists and skip the add operation if it is there? Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Wed Nov 12 19:38:21 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 14:38:21 -0500 Subject: [Freeipa-devel] [PATCH] create windows PassSync user In-Reply-To: <1226518626.10160.591.camel@localhost.localdomain> References: <491B285B.5020301@redhat.com> <1226518626.10160.591.camel@localhost.localdomain> Message-ID: <1226518701.10160.593.camel@localhost.localdomain> On Wed, 2008-11-12 at 14:37 -0500, Simo Sorce wrote: > On Wed, 2008-11-12 at 14:02 -0500, Rob Crittenden wrote: > > This works in concert with Simo's password changing patch. > > > > Create a user for Windows PassSync and grant password changing > > permissions > > > > This does 3 things: > > 1. Create a user for the Windows PassSync service > > 2. Add this use to the list of users that can skip password policies > > 3. Add an aci that grants permission to write the password attributes > > Looks good, > but shouldn't we test if the user already exists and skip the add > operation if it is there? Of course I saw that there was indeed a check *after* sending this email :/ Full Ack. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Nov 12 19:56:27 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 14:56:27 -0500 Subject: [Freeipa-devel] [PATCH] remove width from td in tables Message-ID: <491B34EB.2030106@redhat.com> A 60% width was specified which resulted in the user results page to be very squished, particularly the Job Title column. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-105-userfind.patch Type: text/x-patch Size: 858 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From ssorce at redhat.com Wed Nov 12 20:24:47 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 15:24:47 -0500 Subject: [Freeipa-devel] [PATCH] remove width from td in tables In-Reply-To: <491B34EB.2030106@redhat.com> References: <491B34EB.2030106@redhat.com> Message-ID: <1226521487.10160.595.camel@localhost.localdomain> On Wed, 2008-11-12 at 14:56 -0500, Rob Crittenden wrote: > A 60% width was specified which resulted in the user results page to > be > very squished, particularly the Job Title column. Have you checked all other users of that css directive still render fine? If so ack. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Nov 12 20:55:17 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 15:55:17 -0500 Subject: [Freeipa-devel] [PATCH] let ipa-replica-manage delete winsync replicas In-Reply-To: <1226357384.10160.477.camel@localhost.localdomain> References: <4918B18D.4020000@redhat.com> <1226357384.10160.477.camel@localhost.localdomain> Message-ID: <491B42B5.40005@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-10 at 17:11 -0500, Rob Crittenden wrote: >> There was no way to delete winsync replicas because this assumed that >> both sides could be contacted. In the case of winsync all of the work is >> done on the IPA side, there is no equivalent agreement to remove. > > ack > > Simo. > pushed to ipa-1-2 and master From rcritten at redhat.com Wed Nov 12 20:56:20 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 15:56:20 -0500 Subject: [Freeipa-devel] [PATCH] fix ipa-moduser with multi-valued attribute In-Reply-To: <20081111001502.58cbce56@notas> References: <4918B14B.9010204@redhat.com> <1226357409.10160.479.camel@localhost.localdomain> <4918BE0E.3060303@redhat.com> <20081111001502.58cbce56@notas> Message-ID: <491B42F4.9030205@redhat.com> Martin Nagy wrote: > Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Mon, 2008-11-10 at 17:10 -0500, Rob Crittenden wrote: >>>> Only the first value of a multi-value attribute was being >>>> retrieved so if you updated one with --addattr the all but the >>>> first value was dropped. >>> I think you sent the wrong patch Rob. >>> >>> Simo. >>> >>> >> Oops. Correct one attached. >> >> rob > ack pushed to ipa-1-2 and master From rcritten at redhat.com Wed Nov 12 20:56:49 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 15:56:49 -0500 Subject: [Freeipa-devel] [PATCH] create windows PassSync user In-Reply-To: <1226518701.10160.593.camel@localhost.localdomain> References: <491B285B.5020301@redhat.com> <1226518626.10160.591.camel@localhost.localdomain> <1226518701.10160.593.camel@localhost.localdomain> Message-ID: <491B4311.5020003@redhat.com> Simo Sorce wrote: > On Wed, 2008-11-12 at 14:37 -0500, Simo Sorce wrote: >> On Wed, 2008-11-12 at 14:02 -0500, Rob Crittenden wrote: >>> This works in concert with Simo's password changing patch. >>> >>> Create a user for Windows PassSync and grant password changing >>> permissions >>> >>> This does 3 things: >>> 1. Create a user for the Windows PassSync service >>> 2. Add this use to the list of users that can skip password policies >>> 3. Add an aci that grants permission to write the password attributes >> Looks good, >> but shouldn't we test if the user already exists and skip the add >> operation if it is there? > > Of course I saw that there was indeed a check *after* sending this > email :/ > > Full Ack. > pushed to ipa-1-2 and master From rcritten at redhat.com Wed Nov 12 20:57:08 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 15:57:08 -0500 Subject: [Freeipa-devel] [PATCH] remove width from td in tables In-Reply-To: <1226521487.10160.595.camel@localhost.localdomain> References: <491B34EB.2030106@redhat.com> <1226521487.10160.595.camel@localhost.localdomain> Message-ID: <491B4324.3070501@redhat.com> Simo Sorce wrote: > On Wed, 2008-11-12 at 14:56 -0500, Rob Crittenden wrote: >> A 60% width was specified which resulted in the user results page to >> be >> very squished, particularly the Job Title column. > > Have you checked all other users of that css directive still render > fine? > If so ack. > Yes, seems ok to me. Pushed to ipa-1-2 and master From rcritten at redhat.com Wed Nov 12 20:58:13 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 12 Nov 2008 15:58:13 -0500 Subject: [Freeipa-devel] [PATCH] provide less-cryptic error message In-Reply-To: <1226518583.10160.589.camel@localhost.localdomain> References: <491B28B0.4030605@redhat.com> <1226518583.10160.589.camel@localhost.localdomain> Message-ID: <491B4365.3090303@redhat.com> Simo Sorce wrote: > On Wed, 2008-11-12 at 14:04 -0500, Rob Crittenden wrote: >> If you try to delete a sync agreement that doesn't exist you'll get a >> cryptic message. This fixes that. > > ack > Pushed to ipa-1-2 and master From ssorce at redhat.com Wed Nov 12 21:50:57 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 12 Nov 2008 16:50:57 -0500 Subject: [Freeipa-devel] SPEC Patch for 1.2.0 In-Reply-To: <491A4343.3090702@gmail.com> References: <491A4343.3090702@gmail.com> Message-ID: <1226526657.10160.607.camel@localhost.localdomain> On Tue, 2008-11-11 at 19:45 -0700, Jonathan Steffan wrote: > This is a proposed update for the 1.2.0 specfile (in CVS, which we > should really move into git.) It fixes a few Requires issues and > updates > the %build to allow for a direct git export to actually build. It also > removes the unneeded patch set. Thanks, actually it turns out Rob already had some uncommitted changes in the same direction. He will integrate your patch in for the release. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Wed Nov 12 22:13:22 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 12 Nov 2008 15:13:22 -0700 Subject: [Freeipa-devel] Please review: wait for winsync agreement to become ready before starting Message-ID: <491B5502.9030606@redhat.com> There is a race condition in the directory server. When creating a new sync agreement, it takes some time before it is ready to be started. This is because adding the agreement also makes it perform an incremental update. If the script attempts to do an initialization before it is ready, the script will appear to hang for 20 minutes. This fix allows the script to check the status of the incremental update, and only start if the incremental update is complete. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-wait-for-sync-agreement-to-be-ready-before-starting.patch Type: text/x-patch Size: 3007 bytes Desc: not available URL: From ssorce at redhat.com Thu Nov 13 13:23:02 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 13 Nov 2008 08:23:02 -0500 Subject: [Freeipa-devel] Please review: wait for winsync agreement to become ready before starting In-Reply-To: <491B5502.9030606@redhat.com> References: <491B5502.9030606@redhat.com> Message-ID: <1226582582.10160.613.camel@localhost.localdomain> On Wed, 2008-11-12 at 15:13 -0700, Rich Megginson wrote: > There is a race condition in the directory server. When creating a > new > sync agreement, it takes some time before it is ready to be started. > This is because adding the agreement also makes it perform an > incremental update. If the script attempts to do an initialization > before it is ready, the script will appear to hang for 20 minutes. > This > fix allows the script to check the status of the incremental update, > and > only start if the incremental update is complete. Patch is ok, but I'd like to see a maximum number of retries being implemented so that the wait routing cannot possibly wait forever in case something is wrong in DS. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Thu Nov 13 15:33:54 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 13 Nov 2008 08:33:54 -0700 Subject: [Freeipa-devel] Please review: wait for winsync agreement to become ready before starting In-Reply-To: <1226582582.10160.613.camel@localhost.localdomain> References: <491B5502.9030606@redhat.com> <1226582582.10160.613.camel@localhost.localdomain> Message-ID: <491C48E2.80007@redhat.com> Simo Sorce wrote: > On Wed, 2008-11-12 at 15:13 -0700, Rich Megginson wrote: > >> There is a race condition in the directory server. When creating a >> new >> sync agreement, it takes some time before it is ready to be started. >> This is because adding the agreement also makes it perform an >> incremental update. If the script attempts to do an initialization >> before it is ready, the script will appear to hang for 20 minutes. >> This >> fix allows the script to check the status of the incremental update, >> and >> only start if the incremental update is complete. >> > > Patch is ok, but I'd like to see a maximum number of retries being > implemented so that the wait routing cannot possibly wait forever in > case something is wrong in DS. > Ok. New patch attached. > Simo. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-wait-for-sync-agreement-to-be-ready-before-starting.patch Type: text/x-patch Size: 3734 bytes Desc: not available URL: From ssorce at redhat.com Thu Nov 13 15:47:16 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 13 Nov 2008 10:47:16 -0500 Subject: [Freeipa-devel] Please review: wait for winsync agreement to become ready before starting In-Reply-To: <491C48E2.80007@redhat.com> References: <491B5502.9030606@redhat.com> <1226582582.10160.613.camel@localhost.localdomain> <491C48E2.80007@redhat.com> Message-ID: <1226591236.10160.621.camel@localhost.localdomain> On Thu, 2008-11-13 at 08:33 -0700, Rich Megginson wrote: > > > Patch is ok, but I'd like to see a maximum number of retries being > > implemented so that the wait routing cannot possibly wait forever in > > case something is wrong in DS. > > > Ok. New patch attached. Thanks, ack! Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Fri Nov 14 15:02:19 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 14 Nov 2008 10:02:19 -0500 Subject: [Freeipa-devel] [PATCH] free before use probelm Message-ID: <1226674939.32715.62.camel@localhost.localdomain> In one of the recent patches we got into a free-before-use issue. Patch attached. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-a-free-before-use-bug-it-may-lead-to-crashes-bu.patch Type: application/mbox Size: 1707 bytes Desc: not available URL: From mnagy at redhat.com Fri Nov 14 15:16:58 2008 From: mnagy at redhat.com (Martin Nagy) Date: Fri, 14 Nov 2008 16:16:58 +0100 Subject: [Freeipa-devel] [PATCH] free before use probelm In-Reply-To: <1226674939.32715.62.camel@localhost.localdomain> References: <1226674939.32715.62.camel@localhost.localdomain> Message-ID: <20081114161658.5db8680e@wolverine.englab.brq.redhat.com> On Fri, 14 Nov 2008 10:02:19 -0500, Simo Sorce wrote: > In one of the recent patches we got into a free-before-use issue. > > Patch attached. > > Simo. > ack From ssorce at redhat.com Fri Nov 14 15:26:40 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 14 Nov 2008 10:26:40 -0500 Subject: [Freeipa-devel] [PATCH] free before use probelm In-Reply-To: <20081114161658.5db8680e@wolverine.englab.brq.redhat.com> References: <1226674939.32715.62.camel@localhost.localdomain> <20081114161658.5db8680e@wolverine.englab.brq.redhat.com> Message-ID: <1226676400.32715.64.camel@localhost.localdomain> On Fri, 2008-11-14 at 16:16 +0100, Martin Nagy wrote: > On Fri, 14 Nov 2008 10:02:19 -0500, Simo Sorce > wrote: > > > In one of the recent patches we got into a free-before-use issue. > > > > Patch attached. > > > > Simo. > > > ack Thanks. This bug was really serious so I am pulling the half-published freeipa-1.2.0 tarball and replace the tag in git too. Then re-release. Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Fri Nov 14 17:15:24 2008 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 14 Nov 2008 12:15:24 -0500 Subject: [Freeipa-devel] FreeIPA 1.2 Released Message-ID: <1226682924.32715.105.camel@localhost.localdomain> The FreeIPA Project (http://freeipa.org) is proud to present FreeIPA version 1.2. FreeIPA is an integrated security information management solution combining Linux (Fedora), Fedora Directory Server, MIT Kerberos and NTP. FreeIPA binds together a number of technologies and adds a web interface and command-line administration tools. Currently it supports identity management with plans to support policy and auditing management. This is primarily a bugfix release and contains many improvements especially in the area of replication. With version 1.2 we finally introduced the ability to do some basic synchronization with a Windows Active Directory domain. However, for now, only users and passwords can be synchronized. A serious bug in the setup scripts generated incorrect replication agreements. This bug caused replication issues in setups with more than 2 IPA servers. This bug is now fixed, if you intend to deploy more than 2 IPA servers, you should use this release. The complete source code is available for download here: http://www.freeipa.org/page/Downloads FreeIPA 1.2 is available in Fedora 9 and will be available in Fedora 10. Have Fun! The FreeIPA Project Team. -- Simo Sorce * Red Hat, Inc * New York From jderose at redhat.com Mon Nov 17 03:57:47 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Sun, 16 Nov 2008 20:57:47 -0700 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP Message-ID: <4920EBBB.20102@redhat.com> Say we have some operation that should be subject to access control, but does not involve something stored in LDAP... how should we enforce the access control? Here's the situation I'm thinking of: v2 has an "env" command that returns a list of (key, value) pairs describing the configuration state (run-time configuration, not the configuration stored in LDAP). In a client context, the command returns the client environment. But with the --server option, it is forwarded to the server and returns the configuration state on the server. Although this configuration data does not contain anything sensitive, we might want to restrict who can retrieve the server configuration. I know our general paradigm is to make LDAP responsible for enforcing access control. So should IPA be in the business of enforcing access control in special cases like the "env" command, or should we just avoid capabilities like this altogether? Does v1 have any special cases like this? That's the end of what I have to say about access control, but while I'm on the subject of the "env" command, I'll update everyone on a few new features. You can now specify particular environment variables to look up by including them as positional arguments, like this: [root at fedora freeipa2]# ./ipa env conf conf = '/root/.ipa/cli.conf' If you include multiple arguments, you get back multiple variables, like this: [root at fedora freeipa2]# ./ipa env conf in_server ---- env: ---- conf = '/root/.ipa/cli.conf' in_server = False ----------- 2 variables ----------- Lastly, you can now do wild-card matching, like this: [root at fedora freeipa2]# ./ipa env container* ---- env: ---- container_accounts = 'cn=accounts' container_automount = 'cn=automount' container_group = 'cn=groups,cn=accounts' container_host = 'cn=computers,cn=accounts' container_hostgroup = 'cn=hostgroups,cn=accounts' container_service = 'cn=services,cn=accounts' container_user = 'cn=users,cn=accounts' ----------- 7 variables ----------- Cheers, Jason DeRose -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From jderose at redhat.com Mon Nov 17 04:41:06 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Sun, 16 Nov 2008 21:41:06 -0700 Subject: [Freeipa-devel] freeIPA v2: Changes to output_for_cli() signature Message-ID: <4920F5E2.9080204@redhat.com> The output_for_cli() method signature has changed from: output_for_cli(self, result) to: output_for_cli(self, textui, result, *args, **options) Rob had the excellent idea of passing the args and options the command was called with (thus the new *args, **options in the signature). I've been thinking about how output should be formatted when a command is called through the CLI, and my conclusion was that we really don't want commands using the print statement directly... we want the commands to provide a consistent experience to the user, and that's too difficult to do if every command does its own output totally willy-nilly. So I started work on a new "textui" backend plugin. The goal is to provide maybe a dozen or so high-level methods for common output scenarios, and then have each command's output_for_cli() method use only this textui API. This way we can tweak the CLI output by just modifying the textui plugin, but have it affect all the commands simultaneously. This textui plugin will also do things like determine the tty width and word wrap appropriately. The textui plugin is in ipalib/cli.py The best examples so far of using textui are probably the "env" and "plugins" commands in ipalib/plugins/f_misc.py I also added two related sections in the tutorial. I still haven't figured out how to make epydoc include anchors, but the two new sections are named: "Allowed return values from your command" and "How your command should print to stdout" They are about half way down the page. As always, the latest build of the tutorial can be found here: http://freeipa.org/developer-docs/ipalib-module.html Cheers, Jason DeRose -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From ssorce at redhat.com Mon Nov 17 14:23:29 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 17 Nov 2008 09:23:29 -0500 Subject: [Freeipa-devel] freeIPA v2: Changes to output_for_cli() signature In-Reply-To: <4920F5E2.9080204@redhat.com> References: <4920F5E2.9080204@redhat.com> Message-ID: <1226931809.32715.172.camel@localhost.localdomain> On Sun, 2008-11-16 at 21:41 -0700, Jason Gerard DeRose wrote: > The output_for_cli() method signature has changed from: > > output_for_cli(self, result) > > to: > > output_for_cli(self, textui, result, *args, **options) > > Rob had the excellent idea of passing the args and options the command > was called with (thus the new *args, **options in the signature). > > I've been thinking about how output should be formatted when a command > is called through the CLI, and my conclusion was that we really don't > want commands using the print statement directly... we want the commands > to provide a consistent experience to the user, and that's too difficult > to do if every command does its own output totally willy-nilly. > > So I started work on a new "textui" backend plugin. The goal is to > provide maybe a dozen or so high-level methods for common output > scenarios, and then have each command's output_for_cli() method use only > this textui API. This way we can tweak the CLI output by just modifying > the textui plugin, but have it affect all the commands simultaneously. > This textui plugin will also do things like determine the tty width and > word wrap appropriately. > > The textui plugin is in ipalib/cli.py > > The best examples so far of using textui are probably the "env" and > "plugins" commands in ipalib/plugins/f_misc.py > > I also added two related sections in the tutorial. I still haven't > figured out how to make epydoc include anchors, but the two new sections > are named: > > "Allowed return values from your command" > > and > > "How your command should print to stdout" > > They are about half way down the page. As always, the latest build of > the tutorial can be found here: > > http://freeipa.org/developer-docs/ipalib-module.html > This is an excellent idea! It is indeed fundamental that we do not do any printing directly, but let application developers choose how to mange the output. Thanks, Simo. -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Mon Nov 17 14:33:59 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 17 Nov 2008 14:33:59 +0000 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <4920EBBB.20102@redhat.com> References: <4920EBBB.20102@redhat.com> Message-ID: <1226932439.32715.184.camel@localhost.localdomain> On Sun, 2008-11-16 at 20:57 -0700, Jason Gerard DeRose wrote: > Say we have some operation that should be subject to access control, > but > does not involve something stored in LDAP... how should we enforce the > access control? This is a very good question. > Here's the situation I'm thinking of: v2 has an "env" command that > returns a list of (key, value) pairs describing the configuration > state > (run-time configuration, not the configuration stored in LDAP). In a > client context, the command returns the client environment. But with > the --server option, it is forwarded to the server and returns the > configuration state on the server. > > Although this configuration data does not contain anything sensitive, > we > might want to restrict who can retrieve the server configuration. Yes we may want to restrict it indeed. > I know our general paradigm is to make LDAP responsible for enforcing > access control. So should IPA be in the business of enforcing access > control in special cases like the "env" command, or should we just > avoid > capabilities like this altogether? Does v1 have any special cases like > this? I think we should probably implement ACIs for each function call, and attach default ones at installation time. We should probably think if it make sense to store them in LDAP in this case, although it may also be useful to have different ACIs depending on which server you contact. I am not sure we really want to store them centrally and have them all identical, probably not, it would be too inflexible. Besides they will probably rarely be changed, if we use groups the right way I am sure in most cases people will just need to change group memberships. We could mimic the LDAP server format so that they will be familiar and easily comparable to LDAP ACIs, we want them to use the same ID used on the LDAP side too. Right now we use DNs in ldap, but we might switchy to UUIDs some day, we should be able to support both (means we probably need a look aside cache for the XML-RPC ACI code that maps kerberos principals to the entry UUID and DN). I would also make the ACIs inheritable, so that they can apply to child objects. To make it easier to apply ACIs it would be nice therefore to group commands in logical units. Make this "group of commands" a parent of the specific commands, and apply the ACI to it. How difficult is it to change the core code to add ACIs and a generic enforcement code that operates before anythng else is called on the server side and returns "permission denied" if the user connecting does not have rights on that command ? Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Mon Nov 17 15:34:15 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 17 Nov 2008 10:34:15 -0500 Subject: [Freeipa-devel] freeIPA v2: Changes to output_for_cli() signature In-Reply-To: <1226931809.32715.172.camel@localhost.localdomain> References: <4920F5E2.9080204@redhat.com> <1226931809.32715.172.camel@localhost.localdomain> Message-ID: <49218EF7.7070403@redhat.com> Simo Sorce wrote: > On Sun, 2008-11-16 at 21:41 -0700, Jason Gerard DeRose wrote: > >> The output_for_cli() method signature has changed from: >> >> output_for_cli(self, result) >> >> to: >> >> output_for_cli(self, textui, result, *args, **options) >> >> Rob had the excellent idea of passing the args and options the command >> was called with (thus the new *args, **options in the signature). >> >> I've been thinking about how output should be formatted when a command >> is called through the CLI, and my conclusion was that we really don't >> want commands using the print statement directly... we want the commands >> to provide a consistent experience to the user, and that's too difficult >> to do if every command does its own output totally willy-nilly. >> >> So I started work on a new "textui" backend plugin. The goal is to >> provide maybe a dozen or so high-level methods for common output >> scenarios, and then have each command's output_for_cli() method use only >> this textui API. This way we can tweak the CLI output by just modifying >> the textui plugin, but have it affect all the commands simultaneously. >> This textui plugin will also do things like determine the tty width and >> word wrap appropriately. >> >> The textui plugin is in ipalib/cli.py >> >> The best examples so far of using textui are probably the "env" and >> "plugins" commands in ipalib/plugins/f_misc.py >> >> I also added two related sections in the tutorial. I still haven't >> figured out how to make epydoc include anchors, but the two new sections >> are named: >> >> "Allowed return values from your command" >> >> and >> >> "How your command should print to stdout" >> >> They are about half way down the page. As always, the latest build of >> the tutorial can be found here: >> >> http://freeipa.org/developer-docs/ipalib-module.html >> >> > > This is an excellent idea! > It is indeed fundamental that we do not do any printing directly, but > let application developers choose how to mange the output. > > Thanks, > Simo. > > Yes makes sense. +1. From dpal at redhat.com Mon Nov 17 18:29:44 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 17 Nov 2008 13:29:44 -0500 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <1226932439.32715.184.camel@localhost.localdomain> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> Message-ID: <4921B818.6010909@redhat.com> Simo Sorce wrote: > On Sun, 2008-11-16 at 20:57 -0700, Jason Gerard DeRose wrote: > >> Say we have some operation that should be subject to access control, >> but >> does not involve something stored in LDAP... how should we enforce the >> access control? >> > > This is a very good question. > > >> Here's the situation I'm thinking of: v2 has an "env" command that >> returns a list of (key, value) pairs describing the configuration >> state >> (run-time configuration, not the configuration stored in LDAP). In a >> client context, the command returns the client environment. But with >> the --server option, it is forwarded to the server and returns the >> configuration state on the server. >> >> Although this configuration data does not contain anything sensitive, >> we >> might want to restrict who can retrieve the server configuration. >> > > Yes we may want to restrict it indeed. > > >> I know our general paradigm is to make LDAP responsible for enforcing >> access control. So should IPA be in the business of enforcing access >> control in special cases like the "env" command, or should we just >> avoid >> capabilities like this altogether? Does v1 have any special cases like >> this? >> > > I think we should probably implement ACIs for each function call, and > attach default ones at installation time. We should probably think if it > make sense to store them in LDAP in this case, although it may also be > useful to have different ACIs depending on which server you contact. > I am not sure we really want to store them centrally and have them all > identical, probably not, it would be too inflexible. Besides they will > probably rarely be changed, if we use groups the right way I am sure in > most cases people will just need to change group memberships. > > The ACI approach though consistent with other commands would be too much work in v2. Is there any simpler way of solving the same problem? > We could mimic the LDAP server format so that they will be familiar and > easily comparable to LDAP ACIs, we want them to use the same ID used on > the LDAP side too. Right now we use DNs in ldap, but we might switchy to > UUIDs some day, we should be able to support both (means we probably > need a look aside cache for the XML-RPC ACI code that maps kerberos > principals to the entry UUID and DN). > > I would also make the ACIs inheritable, so that they can apply to child > objects. To make it easier to apply ACIs it would be nice therefore to > group commands in logical units. Make this "group of commands" a parent > of the specific commands, and apply the ACI to it. > > How difficult is it to change the core code to add ACIs and a generic > enforcement code that operates before anythng else is called on the > server side and returns "permission denied" if the user connecting does > not have rights on that command ? > > Too complex for v2 and a lot of work... > Simo. > > From jderose at redhat.com Mon Nov 17 18:42:36 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Mon, 17 Nov 2008 11:42:36 -0700 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <4921B818.6010909@redhat.com> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> <4921B818.6010909@redhat.com> Message-ID: <4921BB1C.4080008@redhat.com> Dmitri Pal wrote: > Simo Sorce wrote: >> On Sun, 2008-11-16 at 20:57 -0700, Jason Gerard DeRose wrote: >> >>> Say we have some operation that should be subject to access control, >>> but >>> does not involve something stored in LDAP... how should we enforce the >>> access control? >>> >> >> This is a very good question. >> >> >>> Here's the situation I'm thinking of: v2 has an "env" command that >>> returns a list of (key, value) pairs describing the configuration >>> state >>> (run-time configuration, not the configuration stored in LDAP). In a >>> client context, the command returns the client environment. But with >>> the --server option, it is forwarded to the server and returns the >>> configuration state on the server. >>> >>> Although this configuration data does not contain anything sensitive, >>> we >>> might want to restrict who can retrieve the server configuration. >>> >> >> Yes we may want to restrict it indeed. >> >> >>> I know our general paradigm is to make LDAP responsible for enforcing >>> access control. So should IPA be in the business of enforcing access >>> control in special cases like the "env" command, or should we just >>> avoid >>> capabilities like this altogether? Does v1 have any special cases like >>> this? >>> >> >> I think we should probably implement ACIs for each function call, and >> attach default ones at installation time. We should probably think if it >> make sense to store them in LDAP in this case, although it may also be >> useful to have different ACIs depending on which server you contact. >> I am not sure we really want to store them centrally and have them all >> identical, probably not, it would be too inflexible. Besides they will >> probably rarely be changed, if we use groups the right way I am sure in >> most cases people will just need to change group memberships. >> >> > The ACI approach though consistent with other commands would be too much > work in v2. > Is there any simpler way of solving the same problem? As I don't myself understand ACI's that well yet, my thought was to do it using groups, something like this: Each command plugin has an optional "requires_group" attribute. If this attribute is None (the default in the base class), it means that the command can be executed by any authenticated user. Otherwise the attribute is a group name... if the user is a member of this group, they are allowed to executed the command. So when a command request comes in over XML-RPC, we do the LDAP bind, locate the command and check the command's "requires_group" attribute. If "requires_group" is a and the user is not a member of this group, we return a 403 Forbidden error. This group approach would be a small change (I could do it in a day). Plus, I think we might already need this very feature for the CA integration plugin (Andrew Wnuk: if you're reading this, can you comment?) >> We could mimic the LDAP server format so that they will be familiar and >> easily comparable to LDAP ACIs, we want them to use the same ID used on >> the LDAP side too. Right now we use DNs in ldap, but we might switchy to >> UUIDs some day, we should be able to support both (means we probably >> need a look aside cache for the XML-RPC ACI code that maps kerberos >> principals to the entry UUID and DN). >> >> I would also make the ACIs inheritable, so that they can apply to child >> objects. To make it easier to apply ACIs it would be nice therefore to >> group commands in logical units. Make this "group of commands" a parent >> of the specific commands, and apply the ACI to it. >> >> How difficult is it to change the core code to add ACIs and a generic >> enforcement code that operates before anythng else is called on the >> server side and returns "permission denied" if the user connecting does >> not have rights on that command ? >> >> > Too complex for v2 and a lot of work... > >> Simo. >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From ssorce at redhat.com Mon Nov 17 19:05:02 2008 From: ssorce at redhat.com (Simo Sorce) Date: Mon, 17 Nov 2008 14:05:02 -0500 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <4921BB1C.4080008@redhat.com> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> <4921B818.6010909@redhat.com> <4921BB1C.4080008@redhat.com> Message-ID: <1226948702.32715.201.camel@localhost.localdomain> On Mon, 2008-11-17 at 11:42 -0700, Jason Gerard DeRose wrote: > > As I don't myself understand ACI's that well yet, my thought was to do > it using groups, something like this: > > Each command plugin has an optional "requires_group" attribute. If > this > attribute is None (the default in the base class), it means that the > command can be executed by any authenticated user. Otherwise the > attribute is a group name... if the user is a member of this group, > they > are allowed to executed the command. > > So when a command request comes in over XML-RPC, we do the LDAP bind, > locate the command and check the command's "requires_group" attribute. > If "requires_group" is a and the user is not a member of > this group, we return a 403 Forbidden error. IIRC we already have some ACI parser code available in python, so I would rather have an "ACI" attribute and put an ACI in there. To make things simpler to manage in v2 without having to implement the full meaning of an ACI we might then restricted the accepted syntax for this version to the rule "read" and to the targets being either a groupdn="ldap:///cn=foobar,cn=..." or "userdn = ldap:///anyone" We can later on add a more comprehensive management of the ACI, including multiple rules, etc... once we have more time, but that will allow us to keep the format unchanged and backward compatible. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Mon Nov 17 22:18:07 2008 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 17 Nov 2008 17:18:07 -0500 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <1226948702.32715.201.camel@localhost.localdomain> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> <4921B818.6010909@redhat.com> <4921BB1C.4080008@redhat.com> <1226948702.32715.201.camel@localhost.localdomain> Message-ID: <4921ED9F.8000001@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-17 at 11:42 -0700, Jason Gerard DeRose wrote: > >> As I don't myself understand ACI's that well yet, my thought was to do >> it using groups, something like this: >> >> Each command plugin has an optional "requires_group" attribute. If >> this >> attribute is None (the default in the base class), it means that the >> command can be executed by any authenticated user. Otherwise the >> attribute is a group name... if the user is a member of this group, >> they >> are allowed to executed the command. >> >> So when a command request comes in over XML-RPC, we do the LDAP bind, >> locate the command and check the command's "requires_group" attribute. >> If "requires_group" is a and the user is not a member of >> this group, we return a 403 Forbidden error. >> > > IIRC we already have some ACI parser code available in python, so I > would rather have an "ACI" attribute and put an ACI in there. > > I did not get a feeling from Rob that the ACI parser is 100% prime time ready. But may be I am missing something. > To make things simpler to manage in v2 without having to implement the > full meaning of an ACI we might then restricted the accepted syntax for > this version to the rule "read" and to the targets being either a > groupdn="ldap:///cn=foobar,cn=..." or "userdn = ldap:///anyone" > > You lost me there. Can you explain it in more details? > We can later on add a more comprehensive management of the ACI, > including multiple rules, etc... once we have more time, but that will > allow us to keep the format unchanged and backward compatible. > > Simo. > > From rcritten at redhat.com Tue Nov 18 02:49:50 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 17 Nov 2008 21:49:50 -0500 Subject: [Freeipa-devel] [PATCH] fix group edit Message-ID: <49222D4E.4070700@redhat.com> Ignore unknown attributes during validation. This is identical to the fix when adding a new group. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-106-group.patch Type: text/x-patch Size: 893 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From ssorce at redhat.com Tue Nov 18 13:26:48 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 18 Nov 2008 13:26:48 +0000 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <4921ED9F.8000001@redhat.com> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> <4921B818.6010909@redhat.com> <4921BB1C.4080008@redhat.com> <1226948702.32715.201.camel@localhost.localdomain> <4921ED9F.8000001@redhat.com> Message-ID: <1227014808.32715.213.camel@localhost.localdomain> On Mon, 2008-11-17 at 17:18 -0500, Dmitri Pal wrote: > Simo Sorce wrote: > > On Mon, 2008-11-17 at 11:42 -0700, Jason Gerard DeRose wrote: > > > >> As I don't myself understand ACI's that well yet, my thought was to do > >> it using groups, something like this: > >> > >> Each command plugin has an optional "requires_group" attribute. If > >> this > >> attribute is None (the default in the base class), it means that the > >> command can be executed by any authenticated user. Otherwise the > >> attribute is a group name... if the user is a member of this group, > >> they > >> are allowed to executed the command. > >> > >> So when a command request comes in over XML-RPC, we do the LDAP bind, > >> locate the command and check the command's "requires_group" attribute. > >> If "requires_group" is a and the user is not a member of > >> this group, we return a 403 Forbidden error. > >> > > > > IIRC we already have some ACI parser code available in python, so I > > would rather have an "ACI" attribute and put an ACI in there. > > > > > I did not get a feeling from Rob that the ACI parser is 100% prime time > ready. But may be I am missing something. For v2 we do not need full parsing, just enough to determine what dn is referenced. > > To make things simpler to manage in v2 without having to implement the > > full meaning of an ACI we might then restricted the accepted syntax for > > this version to the rule "read" and to the targets being either a > > groupdn="ldap:///cn=foobar,cn=..." or "userdn = ldap:///anyone" > > > > > You lost me there. Can you explain it in more details? The proposal would be to use a subset of ACI capabilities so that in fact we just check some group membership for now (or check for everybody), but we already use the right syntax so that going forward we do not have to change ACI rules, but just improve the parsing and validation code to support a full fledged ACI. Simo. -- Simo Sorce * Red Hat, Inc * New York From jderose at redhat.com Tue Nov 18 15:53:37 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Tue, 18 Nov 2008 08:53:37 -0700 Subject: [Freeipa-devel] freeIPA v2: Access control for things not stored in LDAP In-Reply-To: <1227014808.32715.213.camel@localhost.localdomain> References: <4920EBBB.20102@redhat.com> <1226932439.32715.184.camel@localhost.localdomain> <4921B818.6010909@redhat.com> <4921BB1C.4080008@redhat.com> <1226948702.32715.201.camel@localhost.localdomain> <4921ED9F.8000001@redhat.com> <1227014808.32715.213.camel@localhost.localdomain> Message-ID: <4922E501.7050202@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-17 at 17:18 -0500, Dmitri Pal wrote: >> Simo Sorce wrote: >>> On Mon, 2008-11-17 at 11:42 -0700, Jason Gerard DeRose wrote: >>> >>>> As I don't myself understand ACI's that well yet, my thought was to do >>>> it using groups, something like this: >>>> >>>> Each command plugin has an optional "requires_group" attribute. If >>>> this >>>> attribute is None (the default in the base class), it means that the >>>> command can be executed by any authenticated user. Otherwise the >>>> attribute is a group name... if the user is a member of this group, >>>> they >>>> are allowed to executed the command. >>>> >>>> So when a command request comes in over XML-RPC, we do the LDAP bind, >>>> locate the command and check the command's "requires_group" attribute. >>>> If "requires_group" is a and the user is not a member of >>>> this group, we return a 403 Forbidden error. >>>> >>> IIRC we already have some ACI parser code available in python, so I >>> would rather have an "ACI" attribute and put an ACI in there. >>> >>> >> I did not get a feeling from Rob that the ACI parser is 100% prime time >> ready. But may be I am missing something. > > For v2 we do not need full parsing, just enough to determine what dn is > referenced. > >>> To make things simpler to manage in v2 without having to implement the >>> full meaning of an ACI we might then restricted the accepted syntax for >>> this version to the rule "read" and to the targets being either a >>> groupdn="ldap:///cn=foobar,cn=..." or "userdn = ldap:///anyone" >>> >>> >> You lost me there. Can you explain it in more details? > > The proposal would be to use a subset of ACI capabilities so that in > fact we just check some group membership for now (or check for > everybody), but we already use the right syntax so that going forward we > do not have to change ACI rules, but just improve the parsing and > validation code to support a full fledged ACI. > > Simo. > +1. This sounds simple enough that it will be quick to implement and I definitely like that we can extend it later while remaining backward compatible. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From ssorce at redhat.com Tue Nov 18 16:01:30 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 18 Nov 2008 11:01:30 -0500 Subject: [Freeipa-devel] [PATCH] fix group edit In-Reply-To: <49222D4E.4070700@redhat.com> References: <49222D4E.4070700@redhat.com> Message-ID: <1227024090.32715.246.camel@localhost.localdomain> On Mon, 2008-11-17 at 21:49 -0500, Rob Crittenden wrote: > Ignore unknown attributes during validation. This is identical to the > fix when adding a new group. ack -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Wed Nov 19 17:55:56 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 19 Nov 2008 17:55:56 +0000 Subject: [Freeipa-devel] [PATCH] Fix segfault grade errors in the password plugin Message-ID: <1227117356.32715.298.camel@localhost.localdomain> This should fix problems when invalid DNs are provided to the server when changing userPassword or when other errors happens. Some segfaults depend on improper freeing memory in some error paths. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Avoid-potential-crashbug-on-invalid-DNs-not-in-the.patch Type: application/mbox Size: 7907 bytes Desc: not available URL: From nkinder at redhat.com Wed Nov 19 18:07:03 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 19 Nov 2008 10:07:03 -0800 Subject: [Freeipa-devel] [PATCH] Fix segfault grade errors in the password plugin In-Reply-To: <1227117356.32715.298.camel@localhost.localdomain> References: <1227117356.32715.298.camel@localhost.localdomain> Message-ID: <492455C7.9030607@redhat.com> Simo Sorce wrote: > This should fix problems when invalid DNs are provided to the server > when changing userPassword or when other errors happens. Some segfaults > depend on improper freeing memory in some error paths. > ack > Simo. > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From rcritten at redhat.com Wed Nov 19 18:44:32 2008 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 19 Nov 2008 13:44:32 -0500 Subject: [Freeipa-devel] [PATCH] fix group edit In-Reply-To: <1227024090.32715.246.camel@localhost.localdomain> References: <49222D4E.4070700@redhat.com> <1227024090.32715.246.camel@localhost.localdomain> Message-ID: <49245E90.7020900@redhat.com> Simo Sorce wrote: > On Mon, 2008-11-17 at 21:49 -0500, Rob Crittenden wrote: >> Ignore unknown attributes during validation. This is identical to the >> fix when adding a new group. > > ack > pushed to master and ipa-1-2 From ssorce at redhat.com Wed Nov 19 19:03:14 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 19 Nov 2008 14:03:14 -0500 Subject: [Freeipa-devel] [PATCH] Fix segfault grade errors in the password plugin In-Reply-To: <492455C7.9030607@redhat.com> References: <1227117356.32715.298.camel@localhost.localdomain> <492455C7.9030607@redhat.com> Message-ID: <1227121394.32715.304.camel@localhost.localdomain> On Wed, 2008-11-19 at 10:07 -0800, Nathan Kinder wrote: > Simo Sorce wrote: > > This should fix problems when invalid DNs are provided to the server > > when changing userPassword or when other errors happens. Some segfaults > > depend on improper freeing memory in some error paths. > > > ack Pushed to master and ipa-1-2 -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Thu Nov 20 00:17:30 2008 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 19 Nov 2008 19:17:30 -0500 Subject: [Freeipa-devel] [PATCH] yet another bug in an error path Message-ID: <1227140250.32715.307.camel@localhost.localdomain> The target entry was not cleared out and then freed twice in the destructor. Do not rely on the destructor, always handle it in the same function. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-We-must-always-zero-out-the-target-ientry-unconditio.patch Type: application/mbox Size: 3281 bytes Desc: not available URL: From nkinder at redhat.com Thu Nov 20 00:24:30 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 19 Nov 2008 16:24:30 -0800 Subject: [Freeipa-devel] [PATCH] yet another bug in an error path In-Reply-To: <1227140250.32715.307.camel@localhost.localdomain> References: <1227140250.32715.307.camel@localhost.localdomain> Message-ID: <4924AE3E.50109@redhat.com> Simo Sorce wrote: > The target entry was not cleared out and then freed twice in the > destructor. > > Do not rely on the destructor, always handle it in the same function. > ack > Simo. > > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ssorce at redhat.com Thu Nov 20 19:43:52 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 20 Nov 2008 14:43:52 -0500 Subject: [Freeipa-devel] [PATCH] yet another bug in an error path In-Reply-To: <4924AE3E.50109@redhat.com> References: <1227140250.32715.307.camel@localhost.localdomain> <4924AE3E.50109@redhat.com> Message-ID: <1227210232.32715.329.camel@localhost.localdomain> On Wed, 2008-11-19 at 16:24 -0800, Nathan Kinder wrote: > Simo Sorce wrote: > > The target entry was not cleared out and then freed twice in the > > destructor. > > > > Do not rely on the destructor, always handle it in the same function. > > > ack pushed to master and ipa-1-2 Simo -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Thu Nov 20 22:34:57 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 20 Nov 2008 17:34:57 -0500 Subject: [Freeipa-devel] [PATCH] Fix mem leaks in password plugin Message-ID: <1227220497.32715.337.camel@localhost.localdomain> $subject -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-memleaks-found-by-valgrind.patch Type: application/mbox Size: 3202 bytes Desc: not available URL: From nkinder at redhat.com Thu Nov 20 23:01:05 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Thu, 20 Nov 2008 15:01:05 -0800 Subject: [Freeipa-devel] [PATCH] Fix mem leaks in password plugin In-Reply-To: <1227220497.32715.337.camel@localhost.localdomain> References: <1227220497.32715.337.camel@localhost.localdomain> Message-ID: <4925EC31.3070807@redhat.com> Simo Sorce wrote: > $subject > Use slapi_ch_free_string() for freeing char * types instead of slapi_ch_free() as in your patch. This lets you avoid an ugly cast to void** (which you are missing, and should be getting warning about). void slapi_ch_free(void **ptr); void slapi_ch_free_string(char **s); Other than the above, it looks good. > > ------------------------------------------------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From ssorce at redhat.com Fri Nov 21 00:56:39 2008 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 20 Nov 2008 19:56:39 -0500 Subject: [Freeipa-devel] [PATCH] Fix mem leaks in password plugin In-Reply-To: <4925EC31.3070807@redhat.com> References: <1227220497.32715.337.camel@localhost.localdomain> <4925EC31.3070807@redhat.com> Message-ID: <1227228999.32715.340.camel@localhost.localdomain> On Thu, 2008-11-20 at 15:01 -0800, Nathan Kinder wrote: > Simo Sorce wrote: > > $subject > > > Use slapi_ch_free_string() for freeing char * types instead of > slapi_ch_free() as in your patch. This lets you avoid an ugly cast to > void** (which you are missing, and should be getting warning about). > > void slapi_ch_free(void **ptr); > void slapi_ch_free_string(char **s); > > Other than the above, it looks good. Ok changed to slapi_ch_free_string() and pushed to master and ipa-1-2 Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Mon Nov 24 22:24:32 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 24 Nov 2008 15:24:32 -0700 Subject: [Freeipa-devel] Please review: ipa-replica-manage remove same winsync agreement second time reports error msg Message-ID: <492B29A0.90009@redhat.com> The script was using the ipaerror class directly. It seems that none of the scripts in ipa-server/ipa-install use ipaerror directly. Instead, they just handle the exception using the ldap.NAME exception. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-do-not-use-ipaerror-directly-in-ipa-replica-manage.patch Type: text/x-patch Size: 947 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3258 bytes Desc: S/MIME Cryptographic Signature URL: From rmeggins at redhat.com Mon Nov 24 22:39:59 2008 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 24 Nov 2008 15:39:59 -0700 Subject: [Freeipa-devel] Please review: add passsync to ipa-replica-manage web page Message-ID: <492B2D3F.2070902@redhat.com> Need to add the passsync option to the man page -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-add-passsync-to-ipa-replica-manage-man-page.patch Type: text/x-patch Size: 1042 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3258 bytes Desc: S/MIME Cryptographic Signature URL: From ssorce at redhat.com Tue Nov 25 23:05:32 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 25 Nov 2008 18:05:32 -0500 Subject: [Freeipa-devel] Please review: ipa-replica-manage remove same winsync agreement second time reports error msg In-Reply-To: <492B29A0.90009@redhat.com> References: <492B29A0.90009@redhat.com> Message-ID: <1227654332.3838.5.camel@localhost.localdomain> On Mon, 2008-11-24 at 15:24 -0700, Rich Megginson wrote: > The script was using the ipaerror class directly. It seems that none > of > the scripts in ipa-server/ipa-install use ipaerror directly. > Instead, > they just handle the exception using the ldap.NAME exception. ack -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Tue Nov 25 23:05:54 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 25 Nov 2008 18:05:54 -0500 Subject: [Freeipa-devel] Please review: add passsync to ipa-replica-manage web page In-Reply-To: <492B2D3F.2070902@redhat.com> References: <492B2D3F.2070902@redhat.com> Message-ID: <1227654354.3838.7.camel@localhost.localdomain> On Mon, 2008-11-24 at 15:39 -0700, Rich Megginson wrote: > Need to add the passsync option to the man page ack -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Tue Nov 25 23:37:35 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 25 Nov 2008 18:37:35 -0500 Subject: [Freeipa-devel] Please review: ipa-replica-manage remove same winsync agreement second time reports error msg In-Reply-To: <1227654332.3838.5.camel@localhost.localdomain> References: <492B29A0.90009@redhat.com> <1227654332.3838.5.camel@localhost.localdomain> Message-ID: <1227656255.3838.10.camel@localhost.localdomain> On Tue, 2008-11-25 at 18:05 -0500, Simo Sorce wrote: > On Mon, 2008-11-24 at 15:24 -0700, Rich Megginson wrote: > > The script was using the ipaerror class directly. It seems that none > > of > > the scripts in ipa-server/ipa-install use ipaerror directly. > > Instead, > > they just handle the exception using the ldap.NAME exception. > > ack pushed to master and ipa-1-2 -- Simo Sorce * Red Hat, Inc * New York From ssorce at redhat.com Tue Nov 25 23:37:52 2008 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 25 Nov 2008 18:37:52 -0500 Subject: [Freeipa-devel] Please review: add passsync to ipa-replica-manage web page In-Reply-To: <1227654354.3838.7.camel@localhost.localdomain> References: <492B2D3F.2070902@redhat.com> <1227654354.3838.7.camel@localhost.localdomain> Message-ID: <1227656272.3838.12.camel@localhost.localdomain> On Tue, 2008-11-25 at 18:05 -0500, Simo Sorce wrote: > On Mon, 2008-11-24 at 15:39 -0700, Rich Megginson wrote: > > Need to add the passsync option to the man page > > ack pushed to master and ipa-1-2 -- Simo Sorce * Red Hat, Inc * New York From nkinder at redhat.com Wed Nov 26 00:33:16 2008 From: nkinder at redhat.com (Nathan Kinder) Date: Tue, 25 Nov 2008 16:33:16 -0800 Subject: [Freeipa-devel] Make ipa-replica-manage init description match command logic Message-ID: <492C994C.6030301@redhat.com> The usage message and the manpage for ipa-replica-manage init had the logic reversed from the actual code with regards to which server the tool is initializing. This just fixes the manpage and usage message. -NGK -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-Fix-ipa-replica-manage-init-description.patch URL: From sgayda2 at uiuc.edu Sat Nov 29 02:44:00 2008 From: sgayda2 at uiuc.edu (Stoyan Gaydarov) Date: Fri, 28 Nov 2008 20:44:00 -0600 Subject: [Freeipa-devel] Freeipa and Kerberos Message-ID: <4930AC70.7030607@uiuc.edu> Hi my name is Stoyan and I am working on a project that involves python, xmlrpc, and Kerberos, similarly like Freeipa does, and i wanted to see if someone could help me in understanding how Freeipa does their authentication so that i can do something similar for our project. I have looked at the code and saw that the client side uses the xmlrpclib that is part of python and it extends the transport layer. This seems perfectly reasonable and i understand most of it. However the server side is a little more complex. I would like some help understanding what is going on. Currently I just use the SimpleXMLRPCServer that is part of python and I just extend it. I don't need the server do to anything special other then Kerberos authentication so it works well for me. Any information about how it works would be greatly appreciated. -Stoyan From jderose at redhat.com Sun Nov 30 03:29:00 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Sat, 29 Nov 2008 20:29:00 -0700 Subject: [Freeipa-devel] Freeipa and Kerberos In-Reply-To: <4930AC70.7030607@uiuc.edu> References: <4930AC70.7030607@uiuc.edu> Message-ID: <4932087C.3020201@redhat.com> Stoyan Gaydarov wrote: > Hi my name is Stoyan and I am working on a project that involves python, > xmlrpc, and Kerberos, similarly like Freeipa does, and i wanted to see > if someone could help me in understanding how Freeipa does their > authentication so that i can do something similar for our project. I > have looked at the code and saw that the client side uses the xmlrpclib > that is part of python and it extends the transport layer. This seems > perfectly reasonable and i understand most of it. However the server > side is a little more complex. I would like some help understanding what > is going on. Currently I just use the SimpleXMLRPCServer that is part of > python and I just extend it. I don't need the server do to anything > special other then Kerberos authentication so it works well for me. Any > information about how it works would be greatly appreciated. > > -Stoyan Stoyan, In a production deployment, freeIPA runs under Apache2 and we use mod_auth_kerb as our first layer of authentication. For information on mod_auth_kerb, see: http://modauthkerb.sourceforge.net/ Our second layer of authentication is to do an LDAP bind using the user's Kerberos credentials. We rely on LDAP to determine what the user can and can't do because (at least in v1) all the things a user might do involve reading from or writing to LDAP. So in freeIPA itself all we really do is make sure no anonymous access is allow (users always need a valid Kerberos ticket). I don't know v1 very well (I pretty much just work on v2), so other people on the list might be able to fill in more v1 details. However, authentication in v2 is more or less the same except we also have development XML-RPC and web-UI servers designed to run from within the source tree, so these development server don't have the mod_auth_kerb layer (because they don't run under Apache2). I hope this helps. Best of luck on your project! And if you get an itch to work on another Python/Kerberos/XML-RPC project, we *always* welcome new freeIPA developers! Cheers, Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: From sgayda2 at uiuc.edu Sun Nov 30 07:13:59 2008 From: sgayda2 at uiuc.edu (Stoyan Gaydarov) Date: Sun, 30 Nov 2008 01:13:59 -0600 Subject: [Freeipa-devel] Freeipa and Kerberos In-Reply-To: <4932087C.3020201@redhat.com> References: <4930AC70.7030607@uiuc.edu> <4932087C.3020201@redhat.com> Message-ID: <49323D37.4000208@uiuc.edu> Jason Gerard DeRose wrote: > Stoyan Gaydarov wrote: >> Hi my name is Stoyan and I am working on a project that involves python, >> xmlrpc, and Kerberos, similarly like Freeipa does, and i wanted to see >> if someone could help me in understanding how Freeipa does their >> authentication so that i can do something similar for our project. I >> have looked at the code and saw that the client side uses the xmlrpclib >> that is part of python and it extends the transport layer. This seems >> perfectly reasonable and i understand most of it. However the server >> side is a little more complex. I would like some help understanding what >> is going on. Currently I just use the SimpleXMLRPCServer that is part of >> python and I just extend it. I don't need the server do to anything >> special other then Kerberos authentication so it works well for me. Any >> information about how it works would be greatly appreciated. >> >> -Stoyan > > Stoyan, > > In a production deployment, freeIPA runs under Apache2 and we use > mod_auth_kerb as our first layer of authentication. For information on > mod_auth_kerb, see: > > http://modauthkerb.sourceforge.net/ > > Our second layer of authentication is to do an LDAP bind using the > user's Kerberos credentials. We rely on LDAP to determine what the user > can and can't do because (at least in v1) all the things a user might do > involve reading from or writing to LDAP. So in freeIPA itself all we > really do is make sure no anonymous access is allow (users always need a > valid Kerberos ticket). > > I don't know v1 very well (I pretty much just work on v2), so other > people on the list might be able to fill in more v1 details. However, > authentication in v2 is more or less the same except we also have > development XML-RPC and web-UI servers designed to run from within the > source tree, so these development server don't have the mod_auth_kerb > layer (because they don't run under Apache2). > > I hope this helps. Best of luck on your project! And if you get an itch > to work on another Python/Kerberos/XML-RPC project, we *always* welcome > new freeIPA developers! > > Cheers, > Jason > > Thank you so much for the information. As I had suspected, you do rely on apache do the authentication. I am however interested in v2 now because as you say there will be components that run outside of apache, and still need the kerberos authentication. This is similar to what I am working on. Do you know anything about it or do you know where I can get some information on this. Any help would be great. -Stoyan From jderose at redhat.com Sun Nov 30 19:48:36 2008 From: jderose at redhat.com (Jason Gerard DeRose) Date: Sun, 30 Nov 2008 12:48:36 -0700 Subject: [Freeipa-devel] Freeipa and Kerberos In-Reply-To: <49323D37.4000208@uiuc.edu> References: <4930AC70.7030607@uiuc.edu> <4932087C.3020201@redhat.com> <49323D37.4000208@uiuc.edu> Message-ID: <4932EE14.4020805@redhat.com> Stoyan Gaydarov wrote: > Jason Gerard DeRose wrote: >> Stoyan Gaydarov wrote: >>> Hi my name is Stoyan and I am working on a project that involves python, >>> xmlrpc, and Kerberos, similarly like Freeipa does, and i wanted to see >>> if someone could help me in understanding how Freeipa does their >>> authentication so that i can do something similar for our project. I >>> have looked at the code and saw that the client side uses the xmlrpclib >>> that is part of python and it extends the transport layer. This seems >>> perfectly reasonable and i understand most of it. However the server >>> side is a little more complex. I would like some help understanding what >>> is going on. Currently I just use the SimpleXMLRPCServer that is part of >>> python and I just extend it. I don't need the server do to anything >>> special other then Kerberos authentication so it works well for me. Any >>> information about how it works would be greatly appreciated. >>> >>> -Stoyan >> >> Stoyan, >> >> In a production deployment, freeIPA runs under Apache2 and we use >> mod_auth_kerb as our first layer of authentication. For information on >> mod_auth_kerb, see: >> >> http://modauthkerb.sourceforge.net/ >> >> Our second layer of authentication is to do an LDAP bind using the >> user's Kerberos credentials. We rely on LDAP to determine what the user >> can and can't do because (at least in v1) all the things a user might do >> involve reading from or writing to LDAP. So in freeIPA itself all we >> really do is make sure no anonymous access is allow (users always need a >> valid Kerberos ticket). >> >> I don't know v1 very well (I pretty much just work on v2), so other >> people on the list might be able to fill in more v1 details. However, >> authentication in v2 is more or less the same except we also have >> development XML-RPC and web-UI servers designed to run from within the >> source tree, so these development server don't have the mod_auth_kerb >> layer (because they don't run under Apache2). >> >> I hope this helps. Best of luck on your project! And if you get an itch >> to work on another Python/Kerberos/XML-RPC project, we *always* welcome >> new freeIPA developers! >> >> Cheers, >> Jason >> >> > > Thank you so much for the information. As I had suspected, you do rely > on apache do the authentication. I am however interested in v2 now > because as you say there will be components that run outside of apache, > and still need the kerberos authentication. This is similar to what I am > working on. Do you know anything about it or do you know where I can get > some information on this. Any help would be great. > > -Stoyan First, to make sure we're on the same page, these v2 components that run outside of Apache are special servers that are *only* used for development within the source tree, are never used in a production deployment. They're simply a convenience to the developers. Anyway, outside Apache mod_auth_kerb, the only authentication we do is an LDAP bind using the user's Kerberos credentials. Unless your application uses LDAP this wont help you because we totally rely on LDAP to do the authentication... all we do is forward the user's Kerberos ticket. So based on what you've told me, I don't think freeIPA (v1 or v2) does any Kerberos authentication the way you are planing to. However, there are two Kerberos-related Python libraries that we use, and I'm sure one of them can do what you need (although I don't personally know how to do it). We use the Apple Kerberos/GSSAPI python library: http://trac.calendarserver.org/browser/PyKerberos And we use the Red Hat python-krbV binding to the Kerberos C library: http://people.redhat.com/mikeb/python-krbV/ One other thing: because our production version always runs behind Apache, we send the Kerberos ticket in the HTTP headers (which is what mod_auth_kerb expects). But if you aren't planning to run behind Apache, it will probably be more convenient for you (and for the consumers of your XML-RPC API) to send the Kerberos ticket as an XML-RPC argument (say the first argument). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: OpenPGP digital signature URL: