[Avocado-devel] Avocado 0.31.0 multi user problems

Cleber Rosa crosa at redhat.com
Thu Feb 4 14:09:09 UTC 2016 

----- Olav Philipp Henschel <olavph at linux.vnet.ibm.com> wrote:
> Thanks Cleber,
> 
> I've been able to locate and delete all the files you mentioned.
> 
> Then I proceeded to "make rpm", but it exits with an error:
> Error: No Package found for aexpect
> Even though it is already install by pip.
> 
> I proceeded anyway with "python setup.py develop --user". I don't need 
> to use a path that all users can access yet.
> I did the same for avocado-vt.
> 
> 
> Avocado is still getting .conf files from my other user, however 
> (powerkvm). I've tried with the absolute paths with the same results.
> The paths that it shows for the subcommand "config" and "vt-bootstrap" 
> differ:
> 
> $ sudo ~/.local/bin/avocado --config 
> /home/olavph/ibm-kvm-tests/avocado/etc/avocado/conf.d/pkvm.conf config
> [sudo] password for olavph:
> Config files read (in order):
>      /home/powerkvm/ibm-kvm-tests/avocado/etc/avocado/avocado.conf
>      /home/powerkvm/ibm-kvm-tests/avocado/etc/avocado/conf.d/gdb.conf
> /home/powerkvm/ibm-kvm-tests/avocado/etc/avocado/conf.d/pkvm.conf
>      /home/olavph/ibm-kvm-tests/avocado/etc/avocado/conf.d/pkvm.conf
> 
>      Section.Key                             Value
>      datadir.paths.base_dir                  /home/olavph/avocado
>      datadir.paths.test_dir /home/olavph/avocado/tests
>      datadir.paths.data_dir /home/olavph/avocado/data
>      datadir.paths.logs_dir /home/olavph/avocado/job-results
>      ...
> 
> $ sudo ~/.local/bin/avocado --config 
> /home/olavph/ibm-kvm-tests/avocado/etc/avocado/conf.d/pkvm.conf 
> vt-bootstrap --vt-type libvirt --vt-guest-os PowerKVM --y
> es-to-all
> ...
> 10:44:58 INFO | 4 - Verifying directories
> 10:44:58 DEBUG| Dir /var/avocado/data/avocado-vt/images exists, not creating
> 10:44:58 DEBUG| Dir /var/avocado/data/avocado-vt/isos exists, not creating
> 10:44:58 DEBUG| Dir /var/avocado/data/avocado-vt/steps_data exists, not 
> creating
> 10:44:58 DEBUG| Dir /var/avocado/data/avocado-vt/gpg exists, not creating
> ...
> 
> 
> Any other hints?
> 
> Olav
> 

Olav,

Sorry for taking that long to get back on this. Hopefully this is
still relevant. So, better pinpoint the issue, first I tried to check
if avocado is using the "right" configuration files:

[cleber at localhost ~]$ sudo avocado config 
Config files read (in order):
    /etc/avocado/avocado.conf
    /etc/avocado/conf.d/gdb.conf
    /root/.config/avocado/avocado.conf
...

So, both examples look fine, besides the system wide configuration,
avocado looks for the local user (in this case, the "sudoed" user,
root). Then I tried to check how it behaves with additional config
files:

[cleber at localhost ~]$ sudo avocado --config /tmp/external_avocado_config.conf config 
Config files read (in order):
    /etc/avocado/avocado.conf
    /etc/avocado/conf.d/gdb.conf
    /root/.config/avocado/avocado.conf
    /tmp/external_avocado_config.conf
...

And it also looks OK. Next step was to check that one can set
*avocado's* data_dir using an (external) configuration file:


[cleber at localhost ~]$ echo -e "[datadir.paths]\ndata_dir = /tmp/custom/data_dir" > /tmp/external_avocado_config.conf
[cleber at localhost ~]$ sudo avocado --config /tmp/external_avocado_config.conf config 
Config files read (in order):
    /etc/avocado/avocado.conf
    /etc/avocado/conf.d/gdb.conf
    /root/.config/avocado/avocado.conf
    /tmp/external_avocado_config.conf

    Section.Key                             Value
    datadir.paths.base_dir                  /usr/share/avocado
    datadir.paths.test_dir                  /usr/share/avocado/tests
    datadir.paths.data_dir                  /tmp/custom/data_dir
    ^^^^^^^^^^^^^^^^^^^^^^                  ^^^^^^^^^^^^^^^^^^^^

Also looks OK. But then I believe I finally found the bug. From
avocado/core/data_dir.py(47):

   SETTINGS_DATA_DIR = os.path.expanduser(settings.get_value('datadir.paths', 'data_dir'))

This is set when the module is evaluated, that is, when it's imported
by other modules. Then, much later, the extra configuration files
given on the command line are read, as per avocado/core/parser.py(73):

    # Load settings from file, if user provides one
    if self.args.config is not None:
        settings.settings.process_config_path(self.args.config)

The big issue is that avocado/core/data_dir.py always uses
SETTINGS_DATA_DIR, which is already set and doesn't reflect the
configuration files that were added later. This looks like a simple
bug, but I believe it can open a can of worms, so I'm opening a proper
issue to deal with that:

https://trello.com/c/QSNlIbP6/574-bug-data-dir-module-ignores-settings

Sorry for the trouble and delay, and thanks for reporting that.
Cleber Rosa.



> 
> On 07-01-2016 19:15, Cleber Rosa wrote:
> >
> > ----- Original Message -----
> >> From: "Olav Philipp Henschel" <olavph at linux.vnet.ibm.com>
> >> To: avocado-devel at redhat.com
> >> Sent: Thursday, January 7, 2016 4:43:27 PM
> >> Subject: [Avocado-devel] Avocado 0.31.0 multi user problems
> >>
> >> Hi,
> >>
> >> I've updated avocado to version 0.31.0 and got it to work using "python
> >> setup.py develop". Everything worked fine until I tried to run it from
> >> another user (with another local git clone).
> > That may be expected. Setuptools "develop" will create a link file (not a
> > POSIX symlink or hardlink) that points to the original location. The second
> > user may not have permission to access those.
> >
> >> I've ran "python setup.py develop" from that user too to try to
> >> overwrite whatever configurations were made. It keeps using the files
> >> from the first user avocado-vt plugin folder however, getting its tests
> >> provider and custom .cfgs.
> >> At some point I've ran "python setup.py install", so I'm not sure which
> >> command made this mess.
> >> I've also tried using "python setup.py install" and "python setup.py
> >> develop --user" from the second user.
> >>
> > Given the different commands ran as different users, it's pretty hard
> > to give any thoughts about your current installation state.
> >
> >> Is there any way I can make a clean install? Which folders does the
> >> setup.py write to?
> > It depends on how you ran it. What I'd recommend is:
> >
> >   1) Look for egg link files (avocado*.egg-link), and then look at their
> >      contents to check where they point to
> >   2) At the link targets, (the locations given in the .egg-link files)
> >      check if they have avocado*.egg-info directories
> >   3) Look for the "avocado" script in your system.
> >
> > Cleaning all of those, and then starting fresh, should give you pretty
> > good confidence about where you're running Avocado from.
> >
> > Now, for some advice on how to install Avocado:
> >
> >   1) Even if you're tracking Avocado from master (or your local branch)
> >      you could depend on RPM packages (if your system supports it). The
> >      `make rpm` command should give you pretty reliable versions. And,
> >      because of the packaging system itself, you don't have to go looking
> >      for scattered files if you want to remove it.
> >
> >   2) If you want to run directly from git, I'd recommend using:
> >
> >      $ python setup.py develop --prefix=/path/that/all/users/can/access
> >
> >      Setuptools itself will tell you that the given prefix must be on the
> >      Python search path for your users.
> >
> > I hope that helps.
> >
> >>
> >> Regards,
> >> Olav P. Henschel
> >>
> >> _______________________________________________
> >> Avocado-devel mailing list
> >> Avocado-devel at redhat.com
> >> https://www.redhat.com/mailman/listinfo/avocado-devel
> >>
> > _______________________________________________
> > Avocado-devel mailing list
> > Avocado-devel at redhat.com
> > https://www.redhat.com/mailman/listinfo/avocado-devel
> >
> 
> _______________________________________________
> Avocado-devel mailing list
> Avocado-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/avocado-devel

More information about the Avocado-devel mailing list