From crobinso at redhat.com Mon Jul 1 22:56:16 2019 From: crobinso at redhat.com (Cole Robinson) Date: Mon, 1 Jul 2019 18:56:16 -0400 Subject: [Libosinfo] [libosinfo PATCH v2 0/7] Add osinfo_db_identify_tree() API In-Reply-To: <20190626112710.18532-1-fidencio@redhat.com> References: <20190626112710.18532-1-fidencio@redhat.com> Message-ID: On 6/26/19 7:27 AM, Fabiano Fid?ncio wrote: > Similarly to osinfo_db_identify_media(), let's add > osinfo_db_identify_tree() API and be more consistent on how the apps > should check whether a media/tree is known. > > More than that, this simplifies a little bit the code on the apps side. > > Together with this change, let's also deprecate > osinfo_db_guess_os_from_tree() and "force" the apps to use the new and > simpler API for tree detection. > > Changes since v1: > - A few patches touching tree, in order to: > - Remove some useless properties; > - Set "has-treeinfo" as FALSE by default; > - Add treeinfo properties; > - Fixing treeinfo APIs' documentation; > - A test has been added for osinfo_db_identify_tree() > - The osinfo_db_guess_os_from_tree() test has been removed, as the > function is now deprecated > > Fabiano Fid?ncio (7): > tree: Remove useless properties > tree: Don't set "has-treeinfo" property's default to TRUE > tree: Add treeinfo-* properties > tree: Fix treeinfo APIs' documentation > db: Add identify_tree() API > tools,detect: Use osinfo_db_identify_tree() > db: deprecate osinfo_db_guess_os_from_tree() > > osinfo/libosinfo.syms | 2 + > osinfo/osinfo_db.c | 140 ++++++++++++++++++++++++++++++++++++---- > osinfo/osinfo_db.h | 3 + > osinfo/osinfo_tree.c | 146 ++++++++++++++++++++---------------------- > tests/test-db.c | 21 ++---- > tools/osinfo-detect.c | 21 +++--- > 6 files changed, 219 insertions(+), 114 deletions(-) > Reviewed-by: Cole Robinson Tested-by: Cole Robinson - Cole From crobinso at redhat.com Mon Jul 1 23:51:52 2019 From: crobinso at redhat.com (Cole Robinson) Date: Mon, 1 Jul 2019 19:51:52 -0400 Subject: [Libosinfo] [libosinfo PATCH 0/7] Drop GVFS dependency in favour of libsoup In-Reply-To: <20190627125613.28852-1-fidencio@redhat.com> References: <20190627125613.28852-1-fidencio@redhat.com> Message-ID: On 6/27/19 8:56 AM, Fabiano Fid?ncio wrote: > Let's rely on libsoup in order to to http://, https://, and ftp:// > requests. > > By doing this, GVFS (more specifically, gvfs-http) won't be dragged as > a dependency, which caused errors for apps running as root (as GVFS > relies on being run in the user's session). > > https://gitlab.com/libosinfo/libosinfo/issues/30 > > Fabiano Fid?ncio (7): > media: Fix indentation in create_from_location_asunc_data_free() > tree: Remove not used Tree from _CreateFromLocationAsyncData struct > media: Improve on_location_read error message readability > media: Use GInpuStream directly in on_location_read() > media: Use libsoup for http://, https://, and ftp:// requests > tree: Use libsoup for http://, https://, and ftp:// requests > spec: Drop gvfs dependency > > configure.ac | 5 +- > libosinfo.spec.in | 2 +- > osinfo/Makefile.am | 5 +- > osinfo/libosinfo.syms | 2 + > osinfo/osinfo_media.c | 86 ++++++++++++++++++++++++--------- > osinfo/osinfo_tree.c | 108 +++++++++++++++++++++++++++++------------- > osinfo/osinfo_tree.h | 11 +++++ > 7 files changed, 159 insertions(+), 60 deletions(-) > Does libsoup actually handle FTP? Googling doesn't turn up any obvious hits. That said I don't know if GVFS handles it either. With the virt-manager test suite and libosinfo git master, I see: $ ./setup.py test_urls --debug --force-libosinfo --only fedora-ftp ... DEBUG Error creating libosinfo tree object for location=ftp://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/30/Server/x86_64/os/ : g-io-error-quark: Failed to load .treeinfo|treeinfo file: The specified location is not mounted (16) ... So it doesn't seem to work anyway. But with your patch series it hangs until it hits a two minute socket timeout. Which makes me wonder if libsoup is trying to talk http to an ftp URL. FWIW I don't think that should be an argument for keeping a gvfs dependency. It's getting hard to even find public ftp mirrors these days - Cole From berrange at redhat.com Tue Jul 2 08:24:16 2019 From: berrange at redhat.com (Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?=) Date: Tue, 2 Jul 2019 09:24:16 +0100 Subject: [Libosinfo] [libosinfo PATCH 0/7] Drop GVFS dependency in favour of libsoup In-Reply-To: References: <20190627125613.28852-1-fidencio@redhat.com> Message-ID: <20190702082416.GB21569@redhat.com> On Mon, Jul 01, 2019 at 07:51:52PM -0400, Cole Robinson wrote: > On 6/27/19 8:56 AM, Fabiano Fid?ncio wrote: > > Let's rely on libsoup in order to to http://, https://, and ftp:// > > requests. > > > > By doing this, GVFS (more specifically, gvfs-http) won't be dragged as > > a dependency, which caused errors for apps running as root (as GVFS > > relies on being run in the user's session). > > > > https://gitlab.com/libosinfo/libosinfo/issues/30 > > > > Fabiano Fid?ncio (7): > > media: Fix indentation in create_from_location_asunc_data_free() > > tree: Remove not used Tree from _CreateFromLocationAsyncData struct > > media: Improve on_location_read error message readability > > media: Use GInpuStream directly in on_location_read() > > media: Use libsoup for http://, https://, and ftp:// requests > > tree: Use libsoup for http://, https://, and ftp:// requests > > spec: Drop gvfs dependency > > > > configure.ac | 5 +- > > libosinfo.spec.in | 2 +- > > osinfo/Makefile.am | 5 +- > > osinfo/libosinfo.syms | 2 + > > osinfo/osinfo_media.c | 86 ++++++++++++++++++++++++--------- > > osinfo/osinfo_tree.c | 108 +++++++++++++++++++++++++++++------------- > > osinfo/osinfo_tree.h | 11 +++++ > > 7 files changed, 159 insertions(+), 60 deletions(-) > > > > Does libsoup actually handle FTP? Googling doesn't turn up any obvious > hits. That said I don't know if GVFS handles it either. With the > virt-manager test suite and libosinfo git master, I see: > > $ ./setup.py test_urls --debug --force-libosinfo --only fedora-ftp > ... > DEBUG Error creating libosinfo tree object for > location=ftp://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/30/Server/x86_64/os/ > : g-io-error-quark: Failed to load .treeinfo|treeinfo file: The > specified location is not mounted (16) > ... > > So it doesn't seem to work anyway. But with your patch series it hangs > until it hits a two minute socket timeout. Which makes me wonder if > libsoup is trying to talk http to an ftp URL. > > FWIW I don't think that should be an argument for keeping a gvfs > dependency. It's getting hard to even find public ftp mirrors these days Yes, I think we should be explicit that FTP is not supported. There's no compelling reason to use FTP over HTTP(S) these days, even the kernel has disabled it https://www.kernel.org/shutting-down-ftp-services.html Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From fidencio at redhat.com Tue Jul 2 18:07:56 2019 From: fidencio at redhat.com (=?UTF-8?Q?Fabiano_Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 20:07:56 +0200 Subject: [Libosinfo] [libosinfo PATCH 0/7] Drop GVFS dependency in favour of libsoup In-Reply-To: References: <20190627125613.28852-1-fidencio@redhat.com> Message-ID: On Tue, Jul 2, 2019 at 1:51 AM Cole Robinson wrote: > > On 6/27/19 8:56 AM, Fabiano Fid?ncio wrote: > > Let's rely on libsoup in order to to http://, https://, and ftp:// > > requests. > > > > By doing this, GVFS (more specifically, gvfs-http) won't be dragged as > > a dependency, which caused errors for apps running as root (as GVFS > > relies on being run in the user's session). > > > > https://gitlab.com/libosinfo/libosinfo/issues/30 > > > > Fabiano Fid?ncio (7): > > media: Fix indentation in create_from_location_asunc_data_free() > > tree: Remove not used Tree from _CreateFromLocationAsyncData struct > > media: Improve on_location_read error message readability > > media: Use GInpuStream directly in on_location_read() > > media: Use libsoup for http://, https://, and ftp:// requests > > tree: Use libsoup for http://, https://, and ftp:// requests > > spec: Drop gvfs dependency > > > > configure.ac | 5 +- > > libosinfo.spec.in | 2 +- > > osinfo/Makefile.am | 5 +- > > osinfo/libosinfo.syms | 2 + > > osinfo/osinfo_media.c | 86 ++++++++++++++++++++++++--------- > > osinfo/osinfo_tree.c | 108 +++++++++++++++++++++++++++++------------- > > osinfo/osinfo_tree.h | 11 +++++ > > 7 files changed, 159 insertions(+), 60 deletions(-) > > > > Does libsoup actually handle FTP? Theoretically, yes: https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-uri.h#L35 > Googling doesn't turn up any obvious > hits. That said I don't know if GVFS handles it either. It also should, through gvfs-ftp, which seems to just use GSocket under the hood. > With the > virt-manager test suite and libosinfo git master, I see: > > $ ./setup.py test_urls --debug --force-libosinfo --only fedora-ftp > ... > DEBUG Error creating libosinfo tree object for > location=ftp://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/30/Server/x86_64/os/ > : g-io-error-quark: Failed to load .treeinfo|treeinfo file: The > specified location is not mounted (16) > ... > > So it doesn't seem to work anyway. But with your patch series it hangs > until it hits a two minute socket timeout. Which makes me wonder if > libsoup is trying to talk http to an ftp URL. May be the case and may be a bug in libsoup. > > FWIW I don't think that should be an argument for keeping a gvfs > dependency. It's getting hard to even find public ftp mirrors these days Indeed. I'm wondering about the check to do. Hardcoding "https://", "http://" seems quite bad. Maybe I should just create the file and then check for g_file_is_native() as done in osinfo-db-tools? I'm also proposing a patch for libsoup in order to get from them which are the supported schemas. > > - Cole From crobinso at redhat.com Tue Jul 2 19:29:07 2019 From: crobinso at redhat.com (Cole Robinson) Date: Tue, 2 Jul 2019 15:29:07 -0400 Subject: [Libosinfo] [libosinfo PATCH 0/7] Drop GVFS dependency in favour of libsoup In-Reply-To: References: <20190627125613.28852-1-fidencio@redhat.com> Message-ID: <0b2519f2-201c-6070-ab0a-97486d044add@redhat.com> On 7/2/19 2:07 PM, Fabiano Fid?ncio wrote: > On Tue, Jul 2, 2019 at 1:51 AM Cole Robinson wrote: >> >> On 6/27/19 8:56 AM, Fabiano Fid?ncio wrote: >>> Let's rely on libsoup in order to to http://, https://, and ftp:// >>> requests. >>> >>> By doing this, GVFS (more specifically, gvfs-http) won't be dragged as >>> a dependency, which caused errors for apps running as root (as GVFS >>> relies on being run in the user's session). >>> >>> https://gitlab.com/libosinfo/libosinfo/issues/30 >>> >>> Fabiano Fid?ncio (7): >>> media: Fix indentation in create_from_location_asunc_data_free() >>> tree: Remove not used Tree from _CreateFromLocationAsyncData struct >>> media: Improve on_location_read error message readability >>> media: Use GInpuStream directly in on_location_read() >>> media: Use libsoup for http://, https://, and ftp:// requests >>> tree: Use libsoup for http://, https://, and ftp:// requests >>> spec: Drop gvfs dependency >>> >>> configure.ac | 5 +- >>> libosinfo.spec.in | 2 +- >>> osinfo/Makefile.am | 5 +- >>> osinfo/libosinfo.syms | 2 + >>> osinfo/osinfo_media.c | 86 ++++++++++++++++++++++++--------- >>> osinfo/osinfo_tree.c | 108 +++++++++++++++++++++++++++++------------- >>> osinfo/osinfo_tree.h | 11 +++++ >>> 7 files changed, 159 insertions(+), 60 deletions(-) >>> >> >> Does libsoup actually handle FTP? > > Theoretically, yes: > https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-uri.h#L35 > I just grepped through the sources and git log, I don't see any evidence that it supports actual use of the ftp protocol, unless it's using some extra library for that. There's mention of http-aliases which seems to indicate that libsoup can be taught to treat ftp:// URLs as just an alias for http://, and the git log seems to suggest that was the historical default. >> Googling doesn't turn up any obvious >> hits. That said I don't know if GVFS handles it either. > > It also should, through gvfs-ftp, which seems to just use GSocket > under the hood. > >> With the >> virt-manager test suite and libosinfo git master, I see: >> >> $ ./setup.py test_urls --debug --force-libosinfo --only fedora-ftp >> ... >> DEBUG Error creating libosinfo tree object for >> location=ftp://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/30/Server/x86_64/os/ >> : g-io-error-quark: Failed to load .treeinfo|treeinfo file: The >> specified location is not mounted (16) >> ... >> >> So it doesn't seem to work anyway. But with your patch series it hangs >> until it hits a two minute socket timeout. Which makes me wonder if >> libsoup is trying to talk http to an ftp URL. > > May be the case and may be a bug in libsoup. > >> >> FWIW I don't think that should be an argument for keeping a gvfs >> dependency. It's getting hard to even find public ftp mirrors these days > > Indeed. > I'm wondering about the check to do. Hardcoding "https://", "http://" > seems quite bad. FWIW that's what we've done in virt-install for years and it's never really been a problem. Maybe I should just create the file and then check > for g_file_is_native() as done in osinfo-db-tools? I'd need to study up on this approach more to give an opinion. I think the important thing is to try and limit accidental access to gvfs, unless it's for an explicit case we really don't care much about like ftp:// - Cole From fidencio at redhat.com Tue Jul 2 19:38:59 2019 From: fidencio at redhat.com (=?UTF-8?Q?Fabiano_Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:38:59 +0200 Subject: [Libosinfo] [libosinfo PATCH 0/7] Drop GVFS dependency in favour of libsoup In-Reply-To: <0b2519f2-201c-6070-ab0a-97486d044add@redhat.com> References: <20190627125613.28852-1-fidencio@redhat.com> <0b2519f2-201c-6070-ab0a-97486d044add@redhat.com> Message-ID: On Tue, Jul 2, 2019 at 9:29 PM Cole Robinson wrote: > > On 7/2/19 2:07 PM, Fabiano Fid?ncio wrote: > > On Tue, Jul 2, 2019 at 1:51 AM Cole Robinson wrote: > >> > >> On 6/27/19 8:56 AM, Fabiano Fid?ncio wrote: > >>> Let's rely on libsoup in order to to http://, https://, and ftp:// > >>> requests. > >>> > >>> By doing this, GVFS (more specifically, gvfs-http) won't be dragged as > >>> a dependency, which caused errors for apps running as root (as GVFS > >>> relies on being run in the user's session). > >>> > >>> https://gitlab.com/libosinfo/libosinfo/issues/30 > >>> > >>> Fabiano Fid?ncio (7): > >>> media: Fix indentation in create_from_location_asunc_data_free() > >>> tree: Remove not used Tree from _CreateFromLocationAsyncData struct > >>> media: Improve on_location_read error message readability > >>> media: Use GInpuStream directly in on_location_read() > >>> media: Use libsoup for http://, https://, and ftp:// requests > >>> tree: Use libsoup for http://, https://, and ftp:// requests > >>> spec: Drop gvfs dependency > >>> > >>> configure.ac | 5 +- > >>> libosinfo.spec.in | 2 +- > >>> osinfo/Makefile.am | 5 +- > >>> osinfo/libosinfo.syms | 2 + > >>> osinfo/osinfo_media.c | 86 ++++++++++++++++++++++++--------- > >>> osinfo/osinfo_tree.c | 108 +++++++++++++++++++++++++++++------------- > >>> osinfo/osinfo_tree.h | 11 +++++ > >>> 7 files changed, 159 insertions(+), 60 deletions(-) > >>> > >> > >> Does libsoup actually handle FTP? > > > > Theoretically, yes: > > https://gitlab.gnome.org/GNOME/libsoup/blob/master/libsoup/soup-uri.h#L35 > > > > I just grepped through the sources and git log, I don't see any evidence > that it supports actual use of the ftp protocol, unless it's using some > extra library for that. There's mention of http-aliases which seems to > indicate that libsoup can be taught to treat ftp:// URLs as just an > alias for http://, and the git log seems to suggest that was the > historical default. > > >> Googling doesn't turn up any obvious > >> hits. That said I don't know if GVFS handles it either. > > > > It also should, through gvfs-ftp, which seems to just use GSocket > > under the hood. > > >> With the > >> virt-manager test suite and libosinfo git master, I see: > >> > >> $ ./setup.py test_urls --debug --force-libosinfo --only fedora-ftp > >> ... > >> DEBUG Error creating libosinfo tree object for > >> location=ftp://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/30/Server/x86_64/os/ > >> : g-io-error-quark: Failed to load .treeinfo|treeinfo file: The > >> specified location is not mounted (16) > >> ... > >> > >> So it doesn't seem to work anyway. But with your patch series it hangs > >> until it hits a two minute socket timeout. Which makes me wonder if > >> libsoup is trying to talk http to an ftp URL. > > > > May be the case and may be a bug in libsoup. > > > >> > >> FWIW I don't think that should be an argument for keeping a gvfs > >> dependency. It's getting hard to even find public ftp mirrors these days > > > > Indeed. > > I'm wondering about the check to do. Hardcoding "https://", "http://" > > seems quite bad. > > FWIW that's what we've done in virt-install for years and it's never > really been a problem. > > Maybe I should just create the file and then check > > for g_file_is_native() as done in osinfo-db-tools? > > I'd need to study up on this approach more to give an opinion. I think > the important thing is to try and limit accidental access to gvfs, > unless it's for an explicit case we really don't care much about like ftp:// Based on this and on the answer above, I'll just have the harcoded check for "http://" and "https://" and be happy about this. I'll change the docs to mention explicitly that we support local, http://, and https:// files and submit a v2. Thanks a lot for the input! > > - Cole From fidencio at redhat.com Tue Jul 2 19:52:16 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:16 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 0/8] Drop GVFS dependency in favour of libsoup Message-ID: <20190702195224.12291-1-fidencio@redhat.com> Let's rely on libsoup in order to to http://, https://, and ftp:// requests. By doing this, GVFS (more specifically, gvfs-http) won't be dragged as a dependency, which caused errors for apps running as root (as GVFS relies on being run in the user's session). https://gitlab.com/libosinfo/libosinfo/issues/30 Changes since v1: https://www.redhat.com/archives/libosinfo/2019-June/msg00121.html - Explicitly mention that we support only http:// and https:// as "non local"; - One patch was added fixing a doc issue that was noticed during v2 Fabiano Fid?ncio (8): media: Fix indentation in create_from_location_asunc_data_free() tree: Remove not used Tree from _CreateFromLocationAsyncData struct media: Improve on_location_read error message readability media: Use GInpuStream directly in on_location_read() media: Use libsoup for http:// & https:// requests media: Fix _create_from_location_with_flags_async() docs tree: Use libsoup for http:// & https:// requests spec: Drop gvfs dependency configure.ac | 5 +- libosinfo.spec.in | 2 +- osinfo/Makefile.am | 5 +- osinfo/libosinfo.syms | 2 + osinfo/osinfo_media.c | 92 +++++++++++++++++++++++++---------- osinfo/osinfo_tree.c | 110 ++++++++++++++++++++++++++++-------------- osinfo/osinfo_tree.h | 11 +++++ 7 files changed, 163 insertions(+), 64 deletions(-) -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:17 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:17 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 1/8] media: Fix indentation in create_from_location_asunc_data_free() In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-2-fidencio@redhat.com> Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_media.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/osinfo/osinfo_media.c b/osinfo/osinfo_media.c index 3bedeb4..040584f 100644 --- a/osinfo/osinfo_media.c +++ b/osinfo/osinfo_media.c @@ -150,14 +150,14 @@ struct _CreateFromLocationAsyncData { static void create_from_location_async_data_free (CreateFromLocationAsyncData *data) { - g_object_unref(data->file); - g_object_unref(data->res); - g_free(data->volume); - g_free(data->system); - g_free(data->application); - g_free(data->publisher); + g_object_unref(data->file); + g_object_unref(data->res); + g_free(data->volume); + g_free(data->system); + g_free(data->application); + g_free(data->publisher); - g_slice_free(CreateFromLocationAsyncData, data); + g_slice_free(CreateFromLocationAsyncData, data); } typedef struct _CreateFromLocationData CreateFromLocationData; -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:18 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:18 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 2/8] tree: Remove not used Tree from _CreateFromLocationAsyncData struct In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-3-fidencio@redhat.com> As the OsinfoTree is not used anywhere in our code, let's just remove it. Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_tree.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/osinfo/osinfo_tree.c b/osinfo/osinfo_tree.c index b797cb4..c0ec7e4 100644 --- a/osinfo/osinfo_tree.c +++ b/osinfo/osinfo_tree.c @@ -38,14 +38,10 @@ struct _CreateFromLocationAsyncData { gchar *treeinfo; GTask *res; - - OsinfoTree *tree; }; static void create_from_location_async_data_free(CreateFromLocationAsyncData *data) { - if (data->tree) - g_object_unref(data->tree); g_object_unref(data->file); g_object_unref(data->res); -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:19 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:19 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 3/8] media: Improve on_location_read error message readability In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-4-fidencio@redhat.com> Let's add a ": " by the end of the error message, so the prefix doesn't get concatenated with error message, increasing its readability. Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_media.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/osinfo/osinfo_media.c b/osinfo/osinfo_media.c index 040584f..8cd4929 100644 --- a/osinfo/osinfo_media.c +++ b/osinfo/osinfo_media.c @@ -1301,7 +1301,7 @@ static void on_location_read(GObject *source, stream = g_file_read_finish(G_FILE(source), res, &error); if (error != NULL) { - g_prefix_error(&error, _("Failed to open file")); + g_prefix_error(&error, _("Failed to open file: ")); g_task_return_error(data->res, error); create_from_location_async_data_free(data); -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:20 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:20 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 4/8] media: Use GInpuStream directly in on_location_read() In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-5-fidencio@redhat.com> Let's do this in order to always deal with a GInputStream directly and avoid dealing with a GFileInputStream or something else when we start using libsoup to deal with non-local files. Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_media.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/osinfo/osinfo_media.c b/osinfo/osinfo_media.c index 8cd4929..aba3e20 100644 --- a/osinfo/osinfo_media.c +++ b/osinfo/osinfo_media.c @@ -1293,13 +1293,13 @@ static void on_location_read(GObject *source, GAsyncResult *res, gpointer user_data) { - GFileInputStream *stream; + GInputStream *stream; CreateFromLocationAsyncData *data; GError *error = NULL; data = (CreateFromLocationAsyncData *)user_data; - stream = g_file_read_finish(G_FILE(source), res, &error); + stream = G_INPUT_STREAM(g_file_read_finish(G_FILE(source), res, &error)); if (error != NULL) { g_prefix_error(&error, _("Failed to open file: ")); g_task_return_error(data->res, error); @@ -1308,7 +1308,7 @@ static void on_location_read(GObject *source, return; } - g_input_stream_skip_async(G_INPUT_STREAM(stream), + g_input_stream_skip_async(stream, PVD_OFFSET, g_task_get_priority(data->res), g_task_get_cancellable(data->res), -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:21 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:21 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 5/8] media: Use libsoup for http:// & https:// requests In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-6-fidencio@redhat.com> As osinfo_media_create_from_location_with_flags_async() can handle non-local cases, they'd end up relying on GVFS under the hood, which would cause those APIs to not work when called from an app running as root. In order to avoid this situation, let's rely on libsoup for these cases. https://gitlab.com/libosinfo/libosinfo/issues/30 Signed-off-by: Fabiano Fid?ncio --- configure.ac | 1 + libosinfo.spec.in | 1 + osinfo/Makefile.am | 5 ++- osinfo/osinfo_media.c | 72 ++++++++++++++++++++++++++++++++++--------- 4 files changed, 63 insertions(+), 16 deletions(-) diff --git a/configure.ac b/configure.ac index 59c701d..02ea1df 100644 --- a/configure.ac +++ b/configure.ac @@ -42,6 +42,7 @@ GLIB_ENCODED_VERSION="GLIB_VERSION_2_38" PKG_CHECK_MODULES([LIBXML], [libxml-2.0 >= 2.6.0]) PKG_CHECK_MODULES([LIBXSLT], [libxslt >= 1.0.0]) +PKG_CHECK_MODULES([LIBSOUP], [libsoup-2.4]) PKG_CHECK_MODULES([GLIB], [glib-2.0 >= $GLIB_MINIMUM_VERSION gobject-2.0 gio-2.0]) GLIB_CFLAGS="$GLIB_CFLAGS -DGLIB_VERSION_MIN_REQUIRED=$GLIB_ENCODED_VERSION" diff --git a/libosinfo.spec.in b/libosinfo.spec.in index 8118755..e5a16aa 100644 --- a/libosinfo.spec.in +++ b/libosinfo.spec.in @@ -13,6 +13,7 @@ BuildRequires: gettext-devel BuildRequires: glib2-devel BuildRequires: libxml2-devel >= 2.6.0 BuildRequires: libxslt-devel >= 1.0.0 +BuildRequires: libsoup-devel BuildRequires: vala BuildRequires: /usr/bin/pod2man BuildRequires: hwdata diff --git a/osinfo/Makefile.am b/osinfo/Makefile.am index cb1df8f..33e9c66 100644 --- a/osinfo/Makefile.am +++ b/osinfo/Makefile.am @@ -50,6 +50,7 @@ libosinfo_impl_la_CFLAGS = \ $(GOBJECT_CFLAGS) \ $(GLIB_CFLAGS) \ $(GIO_CFLAGS) \ + $(LIBSOUP_CFLAGS) \ -DDATA_DIR='"$(datadir)"' \ -DPKG_DATA_DIR='"$(pkgdatadir)"' \ -DSYS_CONF_DIR='"$(sysconfdir)"' \ @@ -61,7 +62,9 @@ libosinfo_impl_la_LIBADD = \ $(LIBXSLT_LIBS) \ $(GOBJECT_LIBS) \ $(GLIB_LIBS) \ - $(GIO_LIBS) + $(GIO_LIBS) \ + $(LIBSOUP_LIBS) \ + $(NULL) libosinfo_impl_includedir = $(includedir)/libosinfo-1.0/osinfo diff --git a/osinfo/osinfo_media.c b/osinfo/osinfo_media.c index aba3e20..65b47e6 100644 --- a/osinfo/osinfo_media.c +++ b/osinfo/osinfo_media.c @@ -31,6 +31,7 @@ #include #include #include +#include #define MAX_VOLUME 32 #define MAX_SYSTEM 32 @@ -129,6 +130,9 @@ static void search_ppc_bootinfo_async_data_free(SearchPPCBootinfoAsyncData *data typedef struct _CreateFromLocationAsyncData CreateFromLocationAsyncData; struct _CreateFromLocationAsyncData { GFile *file; + SoupSession *session; + SoupMessage *message; + gchar *uri; GTask *res; @@ -150,7 +154,12 @@ struct _CreateFromLocationAsyncData { static void create_from_location_async_data_free (CreateFromLocationAsyncData *data) { - g_object_unref(data->file); + if (data->file != NULL) + g_object_unref(data->file); + if (data->session != NULL) + g_object_unref(data->session); + if (data->message != NULL) + g_object_unref(data->message); g_object_unref(data->res); g_free(data->volume); g_free(data->system); @@ -730,6 +739,19 @@ OsinfoMedia *osinfo_media_new(const gchar *id, return media; } +static gboolean requires_soup(const gchar *location) +{ + const gchar *prefixes[] = { "http://", "https://", NULL }; + gsize i; + + for (i = 0; prefixes[i] != NULL; i++) { + if (g_str_has_prefix(location, prefixes[i])) + return TRUE; + } + + return FALSE; +} + static void on_media_create_from_location_ready(GObject *source_object, GAsyncResult *res, gpointer user_data) @@ -748,7 +770,7 @@ static void on_media_create_from_location_ready(GObject *source_object, * @error: The location where to store any error, or %NULL * * Creates a new #OsinfoMedia for installation media at @location. The @location - * could be any URI that GIO can handle or a local path. + * could be a http:// or a https:// URI or a local path. * * NOTE: Currently this only works for ISO images/devices. * @@ -772,7 +794,7 @@ OsinfoMedia *osinfo_media_create_from_location(const gchar *location, * @flags: An #OsinfoMediaDetectFlag, or 0. * * Creates a new #OsinfoMedia for installation media at @location. The @location - * could be any URI that GIO can handle or a local path. + * could be a http:// or a https:// URI or a local path. * * NOTE: Currently this only works for ISO images/devices. * @@ -835,18 +857,15 @@ static OsinfoMedia * create_from_location_async_data(CreateFromLocationAsyncData *data) { OsinfoMedia *media; - gchar *uri; guint64 vol_size; guint8 index; - uri = g_file_get_uri(data->file); media = g_object_new(OSINFO_TYPE_MEDIA, - "id", uri, + "id", data->uri, NULL); osinfo_entity_set_param(OSINFO_ENTITY(media), OSINFO_MEDIA_PROP_URL, - uri); - g_free(uri); + data->uri); if (!is_str_empty(data->volume)) osinfo_entity_set_param(OSINFO_ENTITY(media), OSINFO_MEDIA_PROP_VOLUME_ID, @@ -1299,7 +1318,17 @@ static void on_location_read(GObject *source, data = (CreateFromLocationAsyncData *)user_data; - stream = G_INPUT_STREAM(g_file_read_finish(G_FILE(source), res, &error)); + if (data->file != NULL) { + stream = G_INPUT_STREAM(g_file_read_finish(G_FILE(source), res, &error)); + } else { + stream = soup_session_send_finish(SOUP_SESSION(source), res, &error); + if (!SOUP_STATUS_IS_SUCCESSFUL(data->message->status_code) && error == NULL) { + g_set_error_literal(&error, + OSINFO_MEDIA_ERROR, + OSINFO_MEDIA_ERROR_NO_DESCRIPTORS, + soup_status_get_phrase(data->message->status_code)); + } + } if (error != NULL) { g_prefix_error(&error, _("Failed to open file: ")); g_task_return_error(data->res, error); @@ -1386,12 +1415,25 @@ void osinfo_media_create_from_location_with_flags_async(const gchar *location, g_task_set_priority(data->res, priority); data->flags = flags; - data->file = g_file_new_for_commandline_arg(location); - g_file_read_async(data->file, - priority, - cancellable, - on_location_read, - data); + data->uri = g_strdup(location); + + if (requires_soup(location)) { + data->session = soup_session_new(); + data->message = soup_message_new("GET", location); + + soup_session_send_async(data->session, + data->message, + cancellable, + on_location_read, + data); + } else { + data->file = g_file_new_for_commandline_arg(location); + g_file_read_async(data->file, + priority, + cancellable, + on_location_read, + data); + } } /** -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:22 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:22 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 6/8] media: Fix _create_from_location_with_flags_async() docs In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-7-fidencio@redhat.com> The function is the async version of osinfo_media_create_from_location_with_flags(). Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_media.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/osinfo/osinfo_media.c b/osinfo/osinfo_media.c index 65b47e6..b5a30c9 100644 --- a/osinfo/osinfo_media.c +++ b/osinfo/osinfo_media.c @@ -1394,7 +1394,7 @@ OsinfoMedia *osinfo_media_create_from_location_finish(GAsyncResult *res, * @flags: An #OsinfoMediaDetectFlag, or 0. * @user_data: The user data to pass to @callback, or %NULL * - * Asynchronous variant of #osinfo_media_create_from_location. + * Asynchronous variant of #osinfo_media_create_from_location_with_flags. */ void osinfo_media_create_from_location_with_flags_async(const gchar *location, gint priority, -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:23 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:23 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 7/8] tree: Use libsoup for http:// & https:// requests In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-8-fidencio@redhat.com> As osinfo_tree_create_from_location() handles non-local requests, it'd end up relying on GVFS under the hood, which would cause this API to not work when called from an app running as root. In order to avoid this situation, let's rely on libsoup for this cases. It's important to mention that glib minimum version has been bumped to 2.44 as that's the version where g_input_stream_read_all_async() has been introuced. https://gitlab.com/libosinfo/libosinfo/issues/30 Signed-off-by: Fabiano Fid?ncio --- configure.ac | 4 +- osinfo/libosinfo.syms | 2 + osinfo/osinfo_tree.c | 106 ++++++++++++++++++++++++++++++------------ osinfo/osinfo_tree.h | 11 +++++ 4 files changed, 90 insertions(+), 33 deletions(-) diff --git a/configure.ac b/configure.ac index 02ea1df..0fde62c 100644 --- a/configure.ac +++ b/configure.ac @@ -37,8 +37,8 @@ m4_if(m4_version_compare([2.61a.100], m4_ifdef([AM_SILENT_RULES],[AM_SILENT_RULES([yes])]) # Keep these two definitions in agreement. -GLIB_MINIMUM_VERSION="2.38" -GLIB_ENCODED_VERSION="GLIB_VERSION_2_38" +GLIB_MINIMUM_VERSION="2.44" +GLIB_ENCODED_VERSION="GLIB_VERSION_2_44" PKG_CHECK_MODULES([LIBXML], [libxml-2.0 >= 2.6.0]) PKG_CHECK_MODULES([LIBXSLT], [libxslt >= 1.0.0]) diff --git a/osinfo/libosinfo.syms b/osinfo/libosinfo.syms index 016d167..6d81e48 100644 --- a/osinfo/libosinfo.syms +++ b/osinfo/libosinfo.syms @@ -595,6 +595,8 @@ LIBOSINFO_1.6.0 { osinfo_media_is_bootable; osinfo_os_get_kernel_url_argument; + + osinfo_tree_error_get_type; } LIBOSINFO_1.5.0; /* Symbols in next release... diff --git a/osinfo/osinfo_tree.c b/osinfo/osinfo_tree.c index c0ec7e4..2fe8b4c 100644 --- a/osinfo/osinfo_tree.c +++ b/osinfo/osinfo_tree.c @@ -30,10 +30,14 @@ #include #include #include +#include typedef struct _CreateFromLocationAsyncData CreateFromLocationAsyncData; struct _CreateFromLocationAsyncData { - GFile *file; + SoupSession *session; + SoupMessage *message; + + gchar *content; gchar *location; gchar *treeinfo; @@ -42,7 +46,8 @@ struct _CreateFromLocationAsyncData { static void create_from_location_async_data_free(CreateFromLocationAsyncData *data) { - g_object_unref(data->file); + g_object_unref(data->session); + g_object_unref(data->message); g_object_unref(data->res); g_slice_free(CreateFromLocationAsyncData, data); @@ -455,7 +460,7 @@ static void on_tree_create_from_location_ready(GObject *source_object, * @error: The location where to store any error, or %NULL * * Creates a new #OsinfoTree for installation tree at @location. The @location - * could be any URI that GIO can handle or a local path. + * could be a http:// or a https:// URI. * * NOTE: Currently this only works for trees with a .treeinfo file * @@ -630,24 +635,57 @@ static void osinfo_tree_create_from_location_async_helper(CreateFromLocationAsyncData *data, const gchar *treeinfo); +static void on_content_read(GObject *source, + GAsyncResult *res, + gpointer user_data) +{ + CreateFromLocationAsyncData *data; + gsize length = 0; + GError *error = NULL; + OsinfoTree *ret; + + data = (CreateFromLocationAsyncData *)user_data; + + if (!g_input_stream_read_all_finish(G_INPUT_STREAM(source), + res, + &length, + &error)) { + g_prefix_error(&error, _("Failed to load .treeinfo|treeinfo content: ")); + g_task_return_error(data->res, error); + create_from_location_async_data_free(data); + } + + if (!(ret = load_keyinfo(data->location, + data->content, + length, + &error))) { + g_prefix_error(&error, _("Failed to process keyinfo file: ")); + g_task_return_error(data->res, error); + goto cleanup; + } + + g_task_return_pointer(data->res, ret, g_object_unref); + + cleanup: + create_from_location_async_data_free(data); +} + static void on_location_read(GObject *source, GAsyncResult *res, gpointer user_data) { CreateFromLocationAsyncData *data; GError *error = NULL; - gchar *content = NULL; - gsize length = 0; - OsinfoTree *ret = NULL; + GInputStream *stream; + goffset content_size; data = (CreateFromLocationAsyncData *)user_data; - if (!g_file_load_contents_finish(G_FILE(source), - res, - &content, - &length, - NULL, - &error)) { + stream = soup_session_send_finish(SOUP_SESSION(source), + res, + &error); + if (stream == NULL || + !SOUP_STATUS_IS_SUCCESSFUL(data->message->status_code)) { /* It means no ".treeinfo" file has been found. Try again, this time * looking for a "treeinfo" file. */ if (g_str_equal(data->treeinfo, ".treeinfo")) { @@ -655,26 +693,28 @@ static void on_location_read(GObject *source, return; } + if (error == NULL) { + g_set_error_literal(&error, + OSINFO_TREE_ERROR, + OSINFO_TREE_ERROR_NO_TREEINFO, + soup_status_get_phrase(data->message->status_code)); + } g_prefix_error(&error, _("Failed to load .treeinfo|treeinfo file: ")); g_task_return_error(data->res, error); create_from_location_async_data_free(data); return; } - if (!(ret = load_keyinfo(data->location, - content, - length, - &error))) { - g_prefix_error(&error, _("Failed to process keyinfo file: ")); - g_task_return_error(data->res, error); - goto cleanup; - } - - g_task_return_pointer(data->res, ret, g_object_unref); + content_size = soup_message_headers_get_content_length(data->message->response_headers); + data->content = g_malloc0(content_size); - cleanup: - create_from_location_async_data_free(data); - g_free(content); + g_input_stream_read_all_async(stream, + data->content, + content_size, + g_task_get_priority(data->res), + g_task_get_cancellable(data->res), + on_content_read, + data); } static void @@ -687,16 +727,20 @@ osinfo_tree_create_from_location_async_helper(CreateFromLocationAsyncData *data, location = g_strdup_printf("%s/%s", data->location, treeinfo); - g_clear_object(&data->file); - data->file = g_file_new_for_uri(location); + if (data->session == NULL) + data->session = soup_session_new(); + + g_clear_object(&data->message); + data->message = soup_message_new("GET", location); g_free(data->treeinfo); data->treeinfo = g_strdup(treeinfo); - g_file_load_contents_async(data->file, - g_task_get_cancellable(data->res), - on_location_read, - data); + soup_session_send_async(data->session, + data->message, + g_task_get_cancellable(data->res), + on_location_read, + data); g_free(location); } diff --git a/osinfo/osinfo_tree.h b/osinfo/osinfo_tree.h index 661d06b..711c244 100644 --- a/osinfo/osinfo_tree.h +++ b/osinfo/osinfo_tree.h @@ -35,6 +35,17 @@ osinfo_tree_error_quark (void) G_GNUC_CONST; #define OSINFO_TREE_ERROR (osinfo_tree_error_quark ()) +/** + * OsinfoTreeError: + * @OSINFO_TREE_ERROR_NO_TREEINFO: No treeinfo found; + * + * #GError codes used for errors in the #OSINFO_TREE_ERROR domain, during + * reading the treeinfo from a URI. + */ +typedef enum { + OSINFO_TREE_ERROR_NO_TREEINFO +} OsinfoTreeError; + /* * Type macros. -- 2.21.0 From fidencio at redhat.com Tue Jul 2 19:52:24 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 2 Jul 2019 21:52:24 +0200 Subject: [Libosinfo] [libosinfo PATCH v2 8/8] spec: Drop gvfs dependency In-Reply-To: <20190702195224.12291-1-fidencio@redhat.com> References: <20190702195224.12291-1-fidencio@redhat.com> Message-ID: <20190702195224.12291-9-fidencio@redhat.com> As we rely on libsoup to handle http://, https://, and ftp:// requests there's no need to depend on gvfs anymore. https://gitlab.com/libosinfo/libosinfo/issues/30 Signed-off-by: Fabiano Fid?ncio --- libosinfo.spec.in | 1 - 1 file changed, 1 deletion(-) diff --git a/libosinfo.spec.in b/libosinfo.spec.in index e5a16aa..2378853 100644 --- a/libosinfo.spec.in +++ b/libosinfo.spec.in @@ -22,7 +22,6 @@ BuildRequires: osinfo-db Requires: hwdata Requires: osinfo-db Requires: osinfo-db-tools -Requires: gvfs %description libosinfo is a library that allows virtualization provisioning tools to -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:01 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:01 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 0/9] Drop GVFS dependency in favour of libsoup (+ some cleanups) Message-ID: <20190703101410.15567-1-fidencio@redhat.com> Let's rely on libsoup in order to to http://, https://, and ftp:// requests. By doing this, GVFS (more specifically, gvfs-http) won't be dragged as a dependency, which caused errors for running osinfo-db-import as root (as GVFS relies on being run in the user's session). https://gitlab.com/libosinfo/libosinfo/issues/30 The cleanups are basically in order to use g_clear_*() functions. Changes since v1: https://www.redhat.com/archives/libosinfo/2019-June/msg00129.html - Adjusted commit message to only mention support for http:// and https://; - Added to the manpage that only http:// and https:// protocols are supported for non-local files; - Used the same requires_soup() logic used in libosinfo here, just for the sake of consistency; Fabiano Fid?ncio (9): import: Use libsoup for http:// & https:// requests spec: Drop gvfs dependency import: Use g_clear_*() functions export: Simplify osinfo_db_export_create_dir() export: use g_clear_*() functions path: Use g_clear_*() functions validate: Simplify validate_file_directory() validate: Use g_clear_*() functions util: Use g_clear_*() functions configure.ac | 5 +- osinfo-db-tools.spec.in | 2 +- tools/Makefile.am | 2 + tools/osinfo-db-export.c | 38 ++++----- tools/osinfo-db-import.c | 163 ++++++++++++++++++++++++++----------- tools/osinfo-db-path.c | 4 +- tools/osinfo-db-util.c | 3 +- tools/osinfo-db-validate.c | 20 ++--- 8 files changed, 148 insertions(+), 89 deletions(-) -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:02 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:02 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 1/9] import: Use libsoup for http:// & https:// requests In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-2-fidencio@redhat.com> As osinfo-db-import handles non-local requests, it'd end up relying on GVFS under the hood, which would cause the app to not work when running as root. In order to avoid this situation, let's rely on libsoup for this case. It's important to mention that glib minimum version has been bumped to 2.44 as that's the version where g_input_stream_read_all() has been introduced. https://gitlab.com/libosinfo/libosinfo/issues/30 Signed-off-by: Fabiano Fid?ncio --- configure.ac | 5 +- osinfo-db-tools.spec.in | 1 + tools/Makefile.am | 2 + tools/osinfo-db-import.c | 137 ++++++++++++++++++++++++++++++--------- 4 files changed, 113 insertions(+), 32 deletions(-) diff --git a/configure.ac b/configure.ac index 4df1139..d019941 100644 --- a/configure.ac +++ b/configure.ac @@ -63,13 +63,14 @@ m4_if(m4_version_compare([2.61a.100], m4_ifdef([AM_SILENT_RULES],[AM_SILENT_RULES([yes])]) # Keep these two definitions in agreement. -GLIB_MINIMUM_VERSION="2.36" -GLIB_ENCODED_VERSION="GLIB_VERSION_2_36" +GLIB_MINIMUM_VERSION="2.44" +GLIB_ENCODED_VERSION="GLIB_VERSION_2_44" PKG_CHECK_MODULES([LIBXML], [libxml-2.0 >= 2.6.0]) PKG_CHECK_MODULES([LIBXSLT], [libxslt >= 1.0.0]) PKG_CHECK_MODULES([LIBARCHIVE], [libarchive >= 3.0.0]) PKG_CHECK_MODULES([JSON_GLIB], [json-glib-1.0]) +PKG_CHECK_MODULES([LIBSOUP], [libsoup-2.4]) PKG_CHECK_MODULES([GLIB], [glib-2.0 >= $GLIB_MINIMUM_VERSION gobject-2.0 gio-2.0]) GLIB_CFLAGS="$GLIB_CFLAGS -DGLIB_VERSION_MIN_REQUIRED=$GLIB_ENCODED_VERSION" diff --git a/osinfo-db-tools.spec.in b/osinfo-db-tools.spec.in index 459dde9..8a82c56 100644 --- a/osinfo-db-tools.spec.in +++ b/osinfo-db-tools.spec.in @@ -18,6 +18,7 @@ BuildRequires: gettext-devel BuildRequires: glib2-devel BuildRequires: libxml2-devel >= 2.6.0 BuildRequires: libxslt-devel >= 1.0.0 +BuildRequires: libsoup-devel BuildRequires: libarchive-devel BuildRequires: json-glib-devel BuildRequires: /usr/bin/pod2man diff --git a/tools/Makefile.am b/tools/Makefile.am index e43e85d..f124d4f 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -2,6 +2,7 @@ AM_CFLAGS = $(GOBJECT_CFLAGS) \ $(GIO_CFLAGS) \ $(GLIB_CFLAGS) \ $(LIBXML_CFLAGS) \ + $(LIBSOUP_CFLAGS) \ -DPKGDATADIR="\"$(pkgdatadir)\"" \ -DDATA_DIR="\"$(datadir)\"" \ -DSYSCONFDIR="\"$(sysconfdir)\"" \ @@ -36,6 +37,7 @@ osinfo_db_import_SOURCES = osinfo-db-import.c $(COMMON_SOURCES) osinfo_db_import_LDADD = $(GOBJECT_LIBS) \ $(GIO_LIBS) \ $(GLIB_LIBS) \ + $(LIBSOUP_LIBS) \ $(JSON_GLIB_LIBS) \ $(LIBARCHIVE_LIBS) osinfo_db_import_CFLAGS = $(AM_CFLAGS) \ diff --git a/tools/osinfo-db-import.c b/tools/osinfo-db-import.c index 675961d..830af3c 100644 --- a/tools/osinfo-db-import.c +++ b/tools/osinfo-db-import.c @@ -29,6 +29,7 @@ #include #include #include +#include #include "osinfo-db-util.h" @@ -36,6 +37,7 @@ #define VERSION_FILE "VERSION" const char *argv0; +static SoupSession *session = NULL; static int osinfo_db_import_create_reg(GFile *file, struct archive *arc, @@ -139,33 +141,64 @@ static GFile *osinfo_db_import_get_file(GFile *target, return g_file_resolve_relative_path(target, tmp); } -static int -osinfo_db_import_download_file(GFile *file, - gchar **source_file) +static gchar * +osinfo_db_import_download_file(const gchar *source) { + GInputStream *stream = NULL; GFile *out = NULL; GError *err = NULL; + SoupMessage *message = NULL; gchar *filename = NULL; - GFileCopyFlags flags = G_FILE_COPY_OVERWRITE | G_FILE_COPY_BACKUP; + gchar *source_file = NULL; + gchar *content = NULL; + goffset content_size = 0; + GFileCreateFlags flags = G_FILE_CREATE_REPLACE_DESTINATION; int ret = -1; - filename = g_file_get_basename(file); + filename = g_path_get_basename(source); if (filename == NULL) goto cleanup; - *source_file = g_build_filename(g_get_user_cache_dir(), filename, NULL); - if (*source_file == NULL) + source_file = g_build_filename(g_get_user_cache_dir(), filename, NULL); + if (source_file == NULL) goto cleanup; - out = g_file_new_for_path(*source_file); + out = g_file_new_for_path(source_file); if (out == NULL) goto cleanup; - if (!g_file_copy(file, out, flags, NULL, NULL, NULL, &err)) { - gchar *path = g_file_get_path(file); + if (session == NULL) + session = soup_session_new(); + + if (session == NULL) + goto cleanup; + + message = soup_message_new("GET", source); + if (message == NULL) + goto cleanup; + + stream = soup_session_send(session, message, NULL, &err); + if (stream == NULL || + !SOUP_STATUS_IS_SUCCESSFUL(message->status_code)) { + g_printerr("Could not access %s: %s\n", + source, + err != NULL ? err->message : + soup_status_get_phrase(message->status_code)); + goto cleanup; + } + + content_size = soup_message_headers_get_content_length(message->response_headers); + content = g_malloc0(content_size); + + if (!g_input_stream_read_all(stream, content, content_size, NULL, NULL, &err)) { + g_printerr("Could not load the content of %s: %s\n", + source, err->message); + goto cleanup; + } + + if (!g_file_replace_contents(out, content, content_size, NULL, TRUE, flags, NULL, NULL, &err)) { g_printerr("Could not download file \"%s\": %s\n", - path, err->message); - g_free(path); + source, err->message); goto cleanup; } @@ -173,14 +206,20 @@ osinfo_db_import_download_file(GFile *file, cleanup: g_free(filename); + g_free(content); + g_clear_object(&message); + g_clear_object(&stream); if (out != NULL) g_object_unref(out); if (err != NULL) g_error_free(err); - if (ret != 0 && *source_file != NULL) - unlink(*source_file); + if (ret != 0 && source_file != NULL) { + unlink(source_file); + g_free(source_file); + source_file = NULL; + } - return ret; + return source_file; } static gboolean osinfo_db_get_installed_version(GFile *dir, @@ -214,18 +253,39 @@ static gboolean osinfo_db_get_installed_version(GFile *dir, static gboolean osinfo_db_get_latest_info(gchar **version, gchar **url) { + SoupMessage *message = NULL; + GInputStream *stream = NULL; JsonParser *parser = NULL; JsonReader *reader = NULL; - GFile *uri = NULL; GError *err = NULL; gchar *content = NULL; + goffset content_size = 0; gboolean ret = FALSE; - uri = g_file_new_for_uri(LATEST_URI); - if (uri == NULL) - return FALSE; + if (session == NULL) + session = soup_session_new(); - if (!g_file_load_contents(uri, NULL, &content, NULL, NULL, &err)) { + if (session == NULL) + goto cleanup; + + message = soup_message_new("GET", LATEST_URI); + if (message == NULL) + goto cleanup; + + stream = soup_session_send(session, message, NULL, &err); + if (stream == NULL || + !SOUP_STATUS_IS_SUCCESSFUL(message->status_code)) { + g_printerr("Could not access %s: %s\n", + LATEST_URI, + err != NULL ? err->message : + soup_status_get_phrase(message->status_code)); + goto cleanup; + } + + content_size = soup_message_headers_get_content_length(message->response_headers); + content = g_malloc0(content_size); + + if (!g_input_stream_read_all(stream, content, content_size, NULL, NULL, &err)) { g_printerr("Could not load the content of %s: %s\n", LATEST_URI, err->message); goto cleanup; @@ -286,7 +346,7 @@ static gboolean osinfo_db_get_latest_info(gchar **version, ret = TRUE; cleanup: - g_object_unref(uri); + g_clear_object(&message); if (parser != NULL) g_object_unref(parser); if (reader != NULL) @@ -297,6 +357,19 @@ static gboolean osinfo_db_get_latest_info(gchar **version, return ret; } +static gboolean requires_soup(const gchar *source) +{ + const gchar *prefixes[] = { "http://", "https://", NULL }; + gsize i; + + for (i = 0; prefixes[i] != NULL; i++) { + if (g_str_has_prefix(source, prefixes[i])) + return TRUE; + } + + return FALSE; +} + static int osinfo_db_import_extract(GFile *target, const char *source, gboolean verbose) @@ -318,21 +391,21 @@ static int osinfo_db_import_extract(GFile *target, source = NULL; if (source != NULL) { - file = g_file_new_for_commandline_arg(source); - if (file == NULL) - goto cleanup; + file_is_native = !requires_soup(source); - file_is_native = g_file_is_native(file); - if (!file_is_native) { - if (osinfo_db_import_download_file(file, &source_file) < 0) + if (file_is_native) { + file = g_file_new_for_commandline_arg(source); + if (file == NULL) goto cleanup; - } else { + source_file = g_file_get_path(file); + g_clear_object(&file); + } else { + source_file = osinfo_db_import_download_file(source); } + if (source_file == NULL) goto cleanup; - - g_clear_object(&file); } if ((r = archive_read_open_filename(arc, source_file, 10240)) != ARCHIVE_OK) { @@ -478,6 +551,7 @@ gint main(gint argc, gchar **argv) g_free(installed_version); g_free(latest_version); g_free(latest_url); + g_clear_object(&session); g_clear_error(&error); g_option_context_free(context); @@ -528,6 +602,9 @@ If run by a privileged account (ie root), the B database location will be used by default, otherwise the B location will be used. +When passing a non local ARCHIVE-FILE, only http:// and https:// +protocols are supported. + With no ARCHIVE-FILE, or when ARCHIVE-FILE is -, read standard input. -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:03 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:03 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 2/9] spec: Drop gvfs dependency In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-3-fidencio@redhat.com> As we rely on libsoup to handle http://, https://, and ftp:// requests there's no need to depend on gvfs anymore. https://gitlab.com/libosinfo/libosinfo/issues/30 Signed-off-by: Fabiano Fid?ncio --- osinfo-db-tools.spec.in | 1 - 1 file changed, 1 deletion(-) diff --git a/osinfo-db-tools.spec.in b/osinfo-db-tools.spec.in index 8a82c56..358321a 100644 --- a/osinfo-db-tools.spec.in +++ b/osinfo-db-tools.spec.in @@ -22,7 +22,6 @@ BuildRequires: libsoup-devel BuildRequires: libarchive-devel BuildRequires: json-glib-devel BuildRequires: /usr/bin/pod2man -Requires: gvfs %if %{supports_python3} #Required for testing purposes -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:04 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:04 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 3/9] import: Use g_clear_*() functions In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-4-fidencio@redhat.com> Let's use g_clear_object() & g_clear_error() as those two functions avoid us checking whether the object is NULL or not and also sets the object to NULL after unrefing it. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-import.c | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/tools/osinfo-db-import.c b/tools/osinfo-db-import.c index 830af3c..47e04b8 100644 --- a/tools/osinfo-db-import.c +++ b/tools/osinfo-db-import.c @@ -82,7 +82,7 @@ static int osinfo_db_import_create_reg(GFile *file, } ret = 0; cleanup: - g_object_unref(os); + g_clear_object(&os); return ret; } @@ -209,10 +209,8 @@ osinfo_db_import_download_file(const gchar *source) g_free(content); g_clear_object(&message); g_clear_object(&stream); - if (out != NULL) - g_object_unref(out); - if (err != NULL) - g_error_free(err); + g_clear_object(&out); + g_clear_error(&err); if (ret != 0 && source_file != NULL) { unlink(source_file); g_free(source_file); @@ -245,7 +243,7 @@ static gboolean osinfo_db_get_installed_version(GFile *dir, cleanup: g_clear_error(&err); - g_object_unref(file); + g_clear_object(&file); return ret; } @@ -347,10 +345,8 @@ static gboolean osinfo_db_get_latest_info(gchar **version, cleanup: g_clear_object(&message); - if (parser != NULL) - g_object_unref(parser); - if (reader != NULL) - g_object_unref(reader); + g_clear_object(&parser); + g_clear_object(&reader); g_free(content); g_clear_error(&err); @@ -428,8 +424,7 @@ static int osinfo_db_import_extract(GFile *target, if (osinfo_db_import_create(file, arc, entry, verbose) < 0) { goto cleanup; } - g_object_unref(file); - file = NULL; + g_clear_object(&file); } if (archive_read_close(arc) != ARCHIVE_OK) { @@ -441,8 +436,7 @@ static int osinfo_db_import_extract(GFile *target, ret = 0; cleanup: archive_read_free(arc); - if (file) - g_object_unref(file); + g_clear_object(&file); if (!file_is_native && source_file != NULL) unlink(source_file); g_free(source_file); @@ -545,9 +539,7 @@ gint main(gint argc, gchar **argv) ret = EXIT_SUCCESS; error: - if (dir) { - g_object_unref(dir); - } + g_clear_object(&dir); g_free(installed_version); g_free(latest_version); g_free(latest_url); -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:05 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:05 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 4/9] export: Simplify osinfo_db_export_create_dir() In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-5-fidencio@redhat.com> Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-export.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/osinfo-db-export.c b/tools/osinfo-db-export.c index eef6688..5bf61fe 100644 --- a/tools/osinfo-db-export.c +++ b/tools/osinfo-db-export.c @@ -104,6 +104,8 @@ static int osinfo_db_export_create_dir(const gchar *prefix, gboolean verbose) { GFileEnumerator *children; + GFileInfo *childinfo = NULL; + GFile *child = NULL; GError *err = NULL; int ret = -1; @@ -121,8 +123,7 @@ static int osinfo_db_export_create_dir(const gchar *prefix, } while (1) { - GFileInfo *childinfo = NULL; - GFile *child = NULL; + int export_create_ret; childinfo = g_file_enumerator_next_file(children, NULL, &err); if (!childinfo) { @@ -137,14 +138,12 @@ static int osinfo_db_export_create_dir(const gchar *prefix, child = g_file_enumerator_get_child(children, childinfo); - if (osinfo_db_export_create_file(prefix, child, childinfo, base, target, arc, verbose) < 0) { - g_object_unref(child); - g_object_unref(childinfo); - goto cleanup; - } + export_create_ret = osinfo_db_export_create_file(prefix, child, childinfo, base, target, arc, verbose); + g_clear_object(&child); + g_clear_object(&childinfo); - g_object_unref(child); - g_object_unref(childinfo); + if (export_create_ret < 0) + goto cleanup; } ret = 0; -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:06 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:06 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 5/9] export: use g_clear_*() functions In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-6-fidencio@redhat.com> Let's use g_clear_object() & g_clear_error() as those two functions avoid us checking whether the object is NULL or not and also sets the object to NULL after unrefing it. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-export.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/tools/osinfo-db-export.c b/tools/osinfo-db-export.c index 5bf61fe..f6e18ec 100644 --- a/tools/osinfo-db-export.c +++ b/tools/osinfo-db-export.c @@ -88,9 +88,8 @@ static int osinfo_db_export_create_reg(GFile *file, ret = 0; cleanup: - if (err) - g_error_free(err); - g_object_unref(is); + g_clear_error(&err); + g_clear_object(&is); g_free(buf); return ret; } @@ -148,8 +147,7 @@ static int osinfo_db_export_create_dir(const gchar *prefix, ret = 0; cleanup: - if (err) - g_error_free(err); + g_clear_error(&err); return ret; } @@ -284,10 +282,8 @@ static int osinfo_db_export_create_file(const gchar *prefix, g_free(abspath); g_free(relpath); g_free(entpath); - if (info) - g_object_unref(info); - if (err) - g_error_free(err); + g_clear_object(&info); + g_clear_error(&err); return ret; } @@ -391,8 +387,7 @@ static int osinfo_db_export_create_license(const gchar *prefix, ret = 0; cleanup: archive_entry_free(entry); - if (err) - g_error_free(err); + g_clear_error(&err); return ret; } @@ -559,9 +554,7 @@ gint main(gint argc, gchar **argv) g_free(archive); g_free(autoversion); g_free(prefix); - if (dir) { - g_object_unref(dir); - } + g_clear_object(&dir); g_clear_error(&error); g_option_context_free(context); -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:07 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:07 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 6/9] path: Use g_clear_*() functions In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-7-fidencio@redhat.com> Let's use g_clear_object() & g_clear_error() as those two functions avoid us checking whether the object is NULL or not and also sets the object to NULL after unrefing it. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-path.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tools/osinfo-db-path.c b/tools/osinfo-db-path.c index 8ef7e55..cd133a3 100644 --- a/tools/osinfo-db-path.c +++ b/tools/osinfo-db-path.c @@ -102,9 +102,7 @@ gint main(gint argc, gchar **argv) ret = EXIT_SUCCESS; error: - if (dir) { - g_object_unref(dir); - } + g_clear_object(&dir); g_clear_error(&error); g_option_context_free(context); -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:08 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:08 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 7/9] validate: Simplify validate_file_directory() In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-8-fidencio@redhat.com> Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-validate.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/osinfo-db-validate.c b/tools/osinfo-db-validate.c index 7fd87d2..3a1717f 100644 --- a/tools/osinfo-db-validate.c +++ b/tools/osinfo-db-validate.c @@ -123,11 +123,12 @@ static gboolean validate_file_directory(xmlRelaxNGValidCtxtPtr rngValid, GFile * while ((info = g_file_enumerator_next_file(children, NULL, error))) { GFile *child = g_file_get_child(file, g_file_info_get_name(info)); - if (!validate_file(rngValid, child, info, error)) { - g_object_unref(child); + gboolean ret_validate; + ret_validate = validate_file(rngValid, child, info, error); + g_clear_object(&child); + + if (!ret_validate) goto cleanup; - } - g_object_unref(child); } if (*error) -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:09 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:09 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 8/9] validate: Use g_clear_*() functions In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-9-fidencio@redhat.com> Let's use g_clear_object() & g_clear_error() as those two functions avoid us checking whether the object is NULL or not and also sets the object to NULL after unrefing it. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-validate.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tools/osinfo-db-validate.c b/tools/osinfo-db-validate.c index 3a1717f..ea9508e 100644 --- a/tools/osinfo-db-validate.c +++ b/tools/osinfo-db-validate.c @@ -137,7 +137,7 @@ static gboolean validate_file_directory(xmlRelaxNGValidCtxtPtr rngValid, GFile * ret = TRUE; cleanup: - g_object_unref(children); + g_clear_object(&children); return ret; } @@ -176,8 +176,7 @@ static gboolean validate_file(xmlRelaxNGValidCtxtPtr rngValid, GFile *file, GFil cleanup: g_free(uri); - if (thisinfo) - g_object_unref(thisinfo); + g_clear_object(&thisinfo); return ret; } @@ -324,10 +323,8 @@ gint main(gint argc, gchar **argv) ret = EXIT_SUCCESS; error: - if (schema) - g_object_unref(schema); - if (dir) - g_object_unref(dir); + g_clear_object(&schema); + g_clear_object(&dir); g_free(files); g_clear_error(&error); g_option_context_free(context); -- 2.21.0 From fidencio at redhat.com Wed Jul 3 10:14:10 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Wed, 3 Jul 2019 12:14:10 +0200 Subject: [Libosinfo] [osinfo-db-tools PATCH v2 9/9] util: Use g_clear_*() functions In-Reply-To: <20190703101410.15567-1-fidencio@redhat.com> References: <20190703101410.15567-1-fidencio@redhat.com> Message-ID: <20190703101410.15567-10-fidencio@redhat.com> Let's use g_clear_object() & g_clear_error() as those two functions avoid us checking whether the object is NULL or not and also sets the object to NULL after unrefing it. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-db-util.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tools/osinfo-db-util.c b/tools/osinfo-db-util.c index 1030b06..45d8b58 100644 --- a/tools/osinfo-db-util.c +++ b/tools/osinfo-db-util.c @@ -148,8 +148,7 @@ GFile *osinfo_db_get_file(const char *root, ret = g_file_resolve_relative_path(paths[i], file); if (g_file_query_exists(ret, NULL)) break; - g_object_unref(ret); - ret = NULL; + g_clear_object(&ret); } if (!ret) { -- 2.21.0 From fidencio at redhat.com Fri Jul 5 07:31:50 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 5 Jul 2019 09:31:50 +0200 Subject: [Libosinfo] [libosinfo PATCH] tree: Add "os" property Message-ID: <20190705073150.11678-1-fidencio@redhat.com> Similarly to what we have for media, let's add the "os" property to the trees. osinfo-detect code is already relying on having an "os" property and, since commit 722e7d1d4, it errors out due to the missing property: (osinfo-detect:7808): GLib-GObject-WARNING **: 09:27:58.656: g_object_get_is_valid_property: object class 'OsinfoTree' has no property named 'os Signed-off-by: Fabiano Fid?ncio --- osinfo/osinfo_tree.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/osinfo/osinfo_tree.c b/osinfo/osinfo_tree.c index b797cb4..a11eee6 100644 --- a/osinfo/osinfo_tree.c +++ b/osinfo/osinfo_tree.c @@ -112,6 +112,7 @@ enum { PROP_INITRD_PATH, PROP_BOOT_ISO_PATH, PROP_HAS_TREEINFO, + PROP_OS, }; static void @@ -173,6 +174,10 @@ osinfo_tree_get_property(GObject *object, osinfo_tree_has_treeinfo(tree)); break; + case PROP_OS: + g_value_take_object(value, osinfo_tree_get_os(tree)); + break; + default: /* We don't have any other property... */ G_OBJECT_WARN_INVALID_PROPERTY_ID(object, property_id, pspec); @@ -250,6 +255,10 @@ osinfo_tree_set_property(GObject *object, g_value_get_boolean(value)); break; + case PROP_OS: + osinfo_tree_set_os(tree, g_value_get_object(value)); + break; + default: /* We don't have any other property... */ G_OBJECT_WARN_INVALID_PROPERTY_ID(object, property_id, pspec); @@ -415,6 +424,22 @@ osinfo_tree_class_init(OsinfoTreeClass *klass) G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS); g_object_class_install_property(g_klass, PROP_TREEINFO_FAMILY, pspec); + + /** + * OsinfoTree:os: + * + * Os information for the current tree. For tree stored in an + * #OsinfoDB, it will be filled when the database is loaded, otherwise + * the property will be filled after a successful call to + * osinfo_db_identify_tree(). + */ + pspec = g_param_spec_object("os", + "Os", + _("Information about the operating system on this tree"), + OSINFO_TYPE_OS, + G_PARAM_READWRITE | + G_PARAM_STATIC_STRINGS); + g_object_class_install_property(g_klass, PROP_OS, pspec); } static void -- 2.21.0 From fidencio at redhat.com Fri Jul 5 08:27:18 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 5 Jul 2019 10:27:18 +0200 Subject: [Libosinfo] [libosinfo PATCH 0/2] Do not expose user & admin password in the command line Message-ID: <20190705082720.26774-1-fidencio@redhat.com> Those two patches introduce a fix for a low impact CVE where both user & admin password would be passed to the osinfo-install-script via command line. In order to avoid doing so, let's introduce a --config-file and error out whenever a password is passed via --config. Fabiano Fid?ncio (2): tools,install-script: Add --config-file (-f) option tools,install-script: Do not accept user & admin password via --config tools/osinfo-install-script.c | 111 +++++++++++++++++++++++++++++++++- 1 file changed, 108 insertions(+), 3 deletions(-) -- 2.21.0 From fidencio at redhat.com Fri Jul 5 08:27:19 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 5 Jul 2019 10:27:19 +0200 Subject: [Libosinfo] [libosinfo PATCH 1/2] tools, install-script: Add --config-file (-f) option In-Reply-To: <20190705082720.26774-1-fidencio@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> Message-ID: <20190705082720.26774-2-fidencio@redhat.com> Let's add a new option so users can set their config from a file, instead of directly passing the values via command-line. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-install-script.c | 100 +++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 3 deletions(-) diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c index 15af48d..efa96ee 100644 --- a/tools/osinfo-install-script.c +++ b/tools/osinfo-install-script.c @@ -37,6 +37,34 @@ static gboolean list_profile = FALSE; static gboolean list_inj_method = FALSE; static gboolean quiet = FALSE; +static const gchar *configs[] = { + OSINFO_INSTALL_CONFIG_PROP_HARDWARE_ARCH, + OSINFO_INSTALL_CONFIG_PROP_L10N_TIMEZONE, + OSINFO_INSTALL_CONFIG_PROP_L10N_LANGUAGE, + OSINFO_INSTALL_CONFIG_PROP_L10N_KEYBOARD, + OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD, + OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD, + OSINFO_INSTALL_CONFIG_PROP_USER_LOGIN, + OSINFO_INSTALL_CONFIG_PROP_USER_REALNAME, + OSINFO_INSTALL_CONFIG_PROP_USER_AUTOLOGIN, + OSINFO_INSTALL_CONFIG_PROP_USER_ADMIN, + OSINFO_INSTALL_CONFIG_PROP_REG_LOGIN, + OSINFO_INSTALL_CONFIG_PROP_REG_PASSWORD, + OSINFO_INSTALL_CONFIG_PROP_REG_PRODUCTKEY, + OSINFO_INSTALL_CONFIG_PROP_HOSTNAME, + OSINFO_INSTALL_CONFIG_PROP_TARGET_DISK, + OSINFO_INSTALL_CONFIG_PROP_SCRIPT_DISK, + OSINFO_INSTALL_CONFIG_PROP_AVATAR_LOCATION, + OSINFO_INSTALL_CONFIG_PROP_AVATAR_DISK, + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_DISK, + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_LOCATION, + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_DISK, + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_LOCATION, + OSINFO_INSTALL_CONFIG_PROP_DRIVER_SIGNING, + OSINFO_INSTALL_CONFIG_PROP_INSTALLATION_URL, + NULL +}; + static OsinfoInstallConfig *config; static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, @@ -65,6 +93,47 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, } +static gboolean handle_config_file(const gchar *option_name G_GNUC_UNUSED, + const gchar *value, + gpointer data G_GNUC_UNUSED, + GError **error) +{ + GKeyFile *key_file = NULL; + gchar *val = NULL; + gsize i; + gboolean ret = FALSE; + + key_file = g_key_file_new(); + if (!g_key_file_load_from_file(key_file, value, G_KEY_FILE_NONE, error)) + goto error; + + for (i = 0; configs[i] != NULL; i++) { + val = g_key_file_get_string(key_file, "install-script", configs[i], error); + if (val == NULL) { + if (g_error_matches(*error, G_KEY_FILE_ERROR, + G_KEY_FILE_ERROR_KEY_NOT_FOUND)) { + g_clear_error(error); + continue; + } + + goto error; + } + + osinfo_entity_set_param(OSINFO_ENTITY(config), + configs[i], + val); + g_free(val); + } + + ret = TRUE; + +error: + g_key_file_unref(key_file); + + return ret; +} + + static GOptionEntry entries[] = { { "profile", 'p', 0, G_OPTION_ARG_STRING, (void*)&profile, @@ -78,6 +147,9 @@ static GOptionEntry entries[] = { "config", 'c', 0, G_OPTION_ARG_CALLBACK, handle_config, N_("Set configuration parameter"), "key=value" }, + { "config-file", 'f', 0, G_OPTION_ARG_CALLBACK, + handle_config_file, + N_("Set configuration parameters"), "file:///path/to/config/file" }, { "list-config", '\0', 0, G_OPTION_ARG_NONE, (void*)&list_config, N_("List configuration parameters"), NULL }, { "list-profiles", '\0', 0, G_OPTION_ARG_NONE, (void*)&list_profile, @@ -448,6 +520,10 @@ script. Defaults to C, but can also be C. Set the configuration parameter C to C. +=item B<--config-file=config-file> + +Set the configurations parameters according to the config-file passed. + =back =head1 CONFIGURATION KEYS @@ -510,18 +586,36 @@ The software registration user password =back +=head1 CONFIGURATION FILE FORMAT + +The configuration file must consist in a file which contains a +`install-script` group and, under this group, C=C +pairs, as shown below: + +[install-script] +l10n-timezone=GMT +l10n-keyboard=uk +l10n-language=en_GB +admin-password=123456 +user-login=berrange +user-password=123456 +user-realname="Daniel P Berrange" + =head1 EXAMPLE USAGE -The following usage generates a Fedora 16 kickstart script +The following usages generates a Fedora 16 kickstart script + + # osinfo-install-script \ + --profile jeos \ + --config-file /path/to/the/config/file \ + fedora16 # osinfo-install-script \ --profile jeos \ --config l10n-timezone=GMT \ --config l10n-keyboard=uk \ --config l10n-language=en_GB \ - --config admin-password=123456 \ --config user-login=berrange \ - --config user-password=123456 \ --config user-realname="Daniel P Berrange" \ fedora16 -- 2.21.0 From fidencio at redhat.com Fri Jul 5 08:27:20 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 5 Jul 2019 10:27:20 +0200 Subject: [Libosinfo] [libosinfo PATCH 2/2] tools, install-script: Do not accept user & admin password via --config In-Reply-To: <20190705082720.26774-1-fidencio@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> Message-ID: <20190705082720.26774-3-fidencio@redhat.com> As passing user & admin password via command line is a low impact CVE, let's error out when it's done and advertise the users to use --config-file instead. Signed-off-by: Fabiano Fid?ncio --- tools/osinfo-install-script.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c index efa96ee..3da4a69 100644 --- a/tools/osinfo-install-script.c +++ b/tools/osinfo-install-script.c @@ -85,6 +85,15 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, val++; key = g_strndup(value, len); + if (g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD) || + g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD)) { + g_set_error(error, OSINFO_ERROR, 0, + _("When setting user or admin password, use " + "--config-file instead.\n")); + g_free(key); + return FALSE; + } + osinfo_entity_set_param(OSINFO_ENTITY(config), key, val); @@ -520,6 +529,8 @@ script. Defaults to C, but can also be C. Set the configuration parameter C to C. +Note: this option has been deprecated, use B<--config-file=> instead. + =item B<--config-file=config-file> Set the configurations parameters according to the config-file passed. -- 2.21.0 From fidencio at redhat.com Fri Jul 5 09:15:48 2019 From: fidencio at redhat.com (=?UTF-8?Q?Fabiano_Fid=C3=AAncio?=) Date: Fri, 5 Jul 2019 11:15:48 +0200 Subject: [Libosinfo] [libosinfo PATCH 2/2] tools, install-script: Do not accept user & admin password via --config In-Reply-To: <20190705082720.26774-3-fidencio@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> <20190705082720.26774-3-fidencio@redhat.com> Message-ID: On Fri, Jul 5, 2019 at 10:27 AM Fabiano Fid?ncio wrote: > > As passing user & admin password via command line is a low impact CVE, > let's error out when it's done and advertise the users to use > --config-file instead. > > Signed-off-by: Fabiano Fid?ncio > --- > tools/osinfo-install-script.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c > index efa96ee..3da4a69 100644 > --- a/tools/osinfo-install-script.c > +++ b/tools/osinfo-install-script.c > @@ -85,6 +85,15 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > val++; > key = g_strndup(value, len); > > + if (g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD) || > + g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD)) { > + g_set_error(error, OSINFO_ERROR, 0, > + _("When setting user or admin password, use " > + "--config-file instead.\n")); > + g_free(key); > + return FALSE; > + } > + > osinfo_entity_set_param(OSINFO_ENTITY(config), > key, > val); > @@ -520,6 +529,8 @@ script. Defaults to C, but can also be C. > > Set the configuration parameter C to C. > > +Note: this option has been deprecated, use B<--config-file=> instead. > + > =item B<--config-file=config-file> > > Set the configurations parameters according to the config-file passed. > -- > 2.21.0 > Ops, with this diff included: fidencio at laerte ~/src/upstream/libosinfo $ git diff diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c index 3da4a69..e641008 100644 --- a/tools/osinfo-install-script.c +++ b/tools/osinfo-install-script.c @@ -529,8 +529,6 @@ script. Defaults to C, but can also be C. Set the configuration parameter C to C. -Note: this option has been deprecated, use B<--config-file=> instead. - =item B<--config-file=config-file> Set the configurations parameters according to the config-file passed. @@ -561,11 +559,11 @@ The local language =item C -The administrator password +The administrator password (only supported via --config-file) =item C -The user password +The user password (only supported via --config-file) =item C From berrange at redhat.com Fri Jul 5 15:51:16 2019 From: berrange at redhat.com (Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?=) Date: Fri, 5 Jul 2019 16:51:16 +0100 Subject: [Libosinfo] [libosinfo PATCH 1/2] tools, install-script: Add --config-file (-f) option In-Reply-To: <20190705082720.26774-2-fidencio@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> <20190705082720.26774-2-fidencio@redhat.com> Message-ID: <20190705155116.GN32473@redhat.com> On Fri, Jul 05, 2019 at 10:27:19AM +0200, Fabiano Fid?ncio wrote: > Let's add a new option so users can set their config from a file, > instead of directly passing the values via command-line. > > Signed-off-by: Fabiano Fid?ncio > --- > tools/osinfo-install-script.c | 100 +++++++++++++++++++++++++++++++++- > 1 file changed, 97 insertions(+), 3 deletions(-) > > diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c > index 15af48d..efa96ee 100644 > --- a/tools/osinfo-install-script.c > +++ b/tools/osinfo-install-script.c > @@ -37,6 +37,34 @@ static gboolean list_profile = FALSE; > static gboolean list_inj_method = FALSE; > static gboolean quiet = FALSE; > > +static const gchar *configs[] = { > + OSINFO_INSTALL_CONFIG_PROP_HARDWARE_ARCH, > + OSINFO_INSTALL_CONFIG_PROP_L10N_TIMEZONE, > + OSINFO_INSTALL_CONFIG_PROP_L10N_LANGUAGE, > + OSINFO_INSTALL_CONFIG_PROP_L10N_KEYBOARD, > + OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD, > + OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD, > + OSINFO_INSTALL_CONFIG_PROP_USER_LOGIN, > + OSINFO_INSTALL_CONFIG_PROP_USER_REALNAME, > + OSINFO_INSTALL_CONFIG_PROP_USER_AUTOLOGIN, > + OSINFO_INSTALL_CONFIG_PROP_USER_ADMIN, > + OSINFO_INSTALL_CONFIG_PROP_REG_LOGIN, > + OSINFO_INSTALL_CONFIG_PROP_REG_PASSWORD, > + OSINFO_INSTALL_CONFIG_PROP_REG_PRODUCTKEY, > + OSINFO_INSTALL_CONFIG_PROP_HOSTNAME, > + OSINFO_INSTALL_CONFIG_PROP_TARGET_DISK, > + OSINFO_INSTALL_CONFIG_PROP_SCRIPT_DISK, > + OSINFO_INSTALL_CONFIG_PROP_AVATAR_LOCATION, > + OSINFO_INSTALL_CONFIG_PROP_AVATAR_DISK, > + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_DISK, > + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_LOCATION, > + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_DISK, > + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_LOCATION, > + OSINFO_INSTALL_CONFIG_PROP_DRIVER_SIGNING, > + OSINFO_INSTALL_CONFIG_PROP_INSTALLATION_URL, > + NULL > +}; I wonder if its better to keep this valid list of keys in the OsinfoInstallConfig class. eg have an osinfo_install_config_is_valid_parameter() function we can call > + > static OsinfoInstallConfig *config; > > static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > @@ -65,6 +93,47 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > } > > > +static gboolean handle_config_file(const gchar *option_name G_GNUC_UNUSED, > + const gchar *value, > + gpointer data G_GNUC_UNUSED, > + GError **error) > +{ > + GKeyFile *key_file = NULL; > + gchar *val = NULL; > + gsize i; > + gboolean ret = FALSE; > + > + key_file = g_key_file_new(); > + if (!g_key_file_load_from_file(key_file, value, G_KEY_FILE_NONE, error)) > + goto error; > + > + for (i = 0; configs[i] != NULL; i++) { > + val = g_key_file_get_string(key_file, "install-script", configs[i], error); > + if (val == NULL) { > + if (g_error_matches(*error, G_KEY_FILE_ERROR, > + G_KEY_FILE_ERROR_KEY_NOT_FOUND)) { > + g_clear_error(error); > + continue; > + } > + > + goto error; > + } > + > + osinfo_entity_set_param(OSINFO_ENTITY(config), > + configs[i], > + val); > + g_free(val); > + } Then here we can just iterate over g_key_file_get_keys() checking if each is a valid parameter. > + > + ret = TRUE; > + > +error: > + g_key_file_unref(key_file); > + > + return ret; > +} > +=head1 CONFIGURATION FILE FORMAT > + > +The configuration file must consist in a file which contains a > +`install-script` group and, under this group, C=C > +pairs, as shown below: > + > +[install-script] > +l10n-timezone=GMT > +l10n-keyboard=uk > +l10n-language=en_GB > +admin-password=123456 > +user-login=berrange > +user-password=123456 > +user-realname="Daniel P Berrange" Heh, my name is not quite correct > + > =head1 EXAMPLE USAGE > > -The following usage generates a Fedora 16 kickstart script > +The following usages generates a Fedora 16 kickstart script > + > + # osinfo-install-script \ > + --profile jeos \ > + --config-file /path/to/the/config/file \ > + fedora16 > > # osinfo-install-script \ > --profile jeos \ > --config l10n-timezone=GMT \ > --config l10n-keyboard=uk \ > --config l10n-language=en_GB \ > - --config admin-password=123456 \ > --config user-login=berrange \ > - --config user-password=123456 \ > --config user-realname="Daniel P Berrange" \ but then it wasn't correct originally either :-) We should have a note here "Note that use of --config-file is strongly recommended if the user or admin passwords need to be set. Providing passwords directly using --config is insecure as the password is visible to all processes and users o nthe same host." Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From berrange at redhat.com Fri Jul 5 15:54:00 2019 From: berrange at redhat.com (Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?=) Date: Fri, 5 Jul 2019 16:54:00 +0100 Subject: [Libosinfo] [libosinfo PATCH 2/2] tools, install-script: Do not accept user & admin password via --config In-Reply-To: <20190705082720.26774-3-fidencio@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> <20190705082720.26774-3-fidencio@redhat.com> Message-ID: <20190705155400.GO32473@redhat.com> On Fri, Jul 05, 2019 at 10:27:20AM +0200, Fabiano Fid?ncio wrote: > As passing user & admin password via command line is a low impact CVE, > let's error out when it's done and advertise the users to use > --config-file instead. > > Signed-off-by: Fabiano Fid?ncio > --- > tools/osinfo-install-script.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c > index efa96ee..3da4a69 100644 > --- a/tools/osinfo-install-script.c > +++ b/tools/osinfo-install-script.c > @@ -85,6 +85,15 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > val++; > key = g_strndup(value, len); > > + if (g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD) || > + g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD)) { > + g_set_error(error, OSINFO_ERROR, 0, > + _("When setting user or admin password, use " > + "--config-file instead.\n")); > + g_free(key); > + return FALSE; > + } If we make this a hard error then we are pretty much forcing everyone to use the --config-file arg, because with this CLI tool there is no way to see the auto-generated password. I think perhaps we should do 1. Deprecate --config for all uses 2. Print a warning mesage if used for admin/user password 3. 1 year later delete --config entirely > + > osinfo_entity_set_param(OSINFO_ENTITY(config), > key, > val); > @@ -520,6 +529,8 @@ script. Defaults to C, but can also be C. > > Set the configuration parameter C to C. > > +Note: this option has been deprecated, use B<--config-file=> instead. > + > =item B<--config-file=config-file> > > Set the configurations parameters according to the config-file passed. > -- > 2.21.0 > > _______________________________________________ > Libosinfo mailing list > Libosinfo at redhat.com > https://www.redhat.com/mailman/listinfo/libosinfo Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From fidencio at redhat.com Mon Jul 8 06:35:45 2019 From: fidencio at redhat.com (=?UTF-8?Q?Fabiano_Fid=C3=AAncio?=) Date: Mon, 8 Jul 2019 08:35:45 +0200 Subject: [Libosinfo] [libosinfo PATCH 1/2] tools, install-script: Add --config-file (-f) option In-Reply-To: <20190705155116.GN32473@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> <20190705082720.26774-2-fidencio@redhat.com> <20190705155116.GN32473@redhat.com> Message-ID: On Fri, Jul 5, 2019 at 5:51 PM Daniel P. Berrang? wrote: > > On Fri, Jul 05, 2019 at 10:27:19AM +0200, Fabiano Fid?ncio wrote: > > Let's add a new option so users can set their config from a file, > > instead of directly passing the values via command-line. > > > > Signed-off-by: Fabiano Fid?ncio > > --- > > tools/osinfo-install-script.c | 100 +++++++++++++++++++++++++++++++++- > > 1 file changed, 97 insertions(+), 3 deletions(-) > > > > diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c > > index 15af48d..efa96ee 100644 > > --- a/tools/osinfo-install-script.c > > +++ b/tools/osinfo-install-script.c > > @@ -37,6 +37,34 @@ static gboolean list_profile = FALSE; > > static gboolean list_inj_method = FALSE; > > static gboolean quiet = FALSE; > > > > +static const gchar *configs[] = { > > + OSINFO_INSTALL_CONFIG_PROP_HARDWARE_ARCH, > > + OSINFO_INSTALL_CONFIG_PROP_L10N_TIMEZONE, > > + OSINFO_INSTALL_CONFIG_PROP_L10N_LANGUAGE, > > + OSINFO_INSTALL_CONFIG_PROP_L10N_KEYBOARD, > > + OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD, > > + OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD, > > + OSINFO_INSTALL_CONFIG_PROP_USER_LOGIN, > > + OSINFO_INSTALL_CONFIG_PROP_USER_REALNAME, > > + OSINFO_INSTALL_CONFIG_PROP_USER_AUTOLOGIN, > > + OSINFO_INSTALL_CONFIG_PROP_USER_ADMIN, > > + OSINFO_INSTALL_CONFIG_PROP_REG_LOGIN, > > + OSINFO_INSTALL_CONFIG_PROP_REG_PASSWORD, > > + OSINFO_INSTALL_CONFIG_PROP_REG_PRODUCTKEY, > > + OSINFO_INSTALL_CONFIG_PROP_HOSTNAME, > > + OSINFO_INSTALL_CONFIG_PROP_TARGET_DISK, > > + OSINFO_INSTALL_CONFIG_PROP_SCRIPT_DISK, > > + OSINFO_INSTALL_CONFIG_PROP_AVATAR_LOCATION, > > + OSINFO_INSTALL_CONFIG_PROP_AVATAR_DISK, > > + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_DISK, > > + OSINFO_INSTALL_CONFIG_PROP_PRE_INSTALL_DRIVERS_LOCATION, > > + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_DISK, > > + OSINFO_INSTALL_CONFIG_PROP_POST_INSTALL_DRIVERS_LOCATION, > > + OSINFO_INSTALL_CONFIG_PROP_DRIVER_SIGNING, > > + OSINFO_INSTALL_CONFIG_PROP_INSTALLATION_URL, > > + NULL > > +}; > > I wonder if its better to keep this valid list of keys in the > OsinfoInstallConfig class. eg have an > > osinfo_install_config_is_valid_parameter() > > function we can call I thought about that, but then a few other things came to my mind: - the correct place to have it would be in OsinfoInstallConfigParam class, something like: osinfo_install_config_param_is_valid(); - but then it'd require the user to have an OsinfoInstallConfigParam object at the first place; - and the check should be done when creating the OsinfoInstallConfigParam; Having a function in OsinfoInstallConfig seems weird to be, but if you really prefer it I can go for it. (I'm assuming we don't want to backport this patch, as introduce a new function and depending on that would make the backport not viable). > > > + > > static OsinfoInstallConfig *config; > > > > static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > > @@ -65,6 +93,47 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > > } > > > > > > +static gboolean handle_config_file(const gchar *option_name G_GNUC_UNUSED, > > + const gchar *value, > > + gpointer data G_GNUC_UNUSED, > > + GError **error) > > +{ > > + GKeyFile *key_file = NULL; > > + gchar *val = NULL; > > + gsize i; > > + gboolean ret = FALSE; > > + > > + key_file = g_key_file_new(); > > + if (!g_key_file_load_from_file(key_file, value, G_KEY_FILE_NONE, error)) > > + goto error; > > + > > + for (i = 0; configs[i] != NULL; i++) { > > + val = g_key_file_get_string(key_file, "install-script", configs[i], error); > > + if (val == NULL) { > > + if (g_error_matches(*error, G_KEY_FILE_ERROR, > > + G_KEY_FILE_ERROR_KEY_NOT_FOUND)) { > > + g_clear_error(error); > > + continue; > > + } > > + > > + goto error; > > + } > > + > > + osinfo_entity_set_param(OSINFO_ENTITY(config), > > + configs[i], > > + val); > > + g_free(val); > > + } > > Then here we can just iterate over g_key_file_get_keys() checking if > each is a valid parameter. > > > + > > + ret = TRUE; > > + > > +error: > > + g_key_file_unref(key_file); > > + > > + return ret; > > +} > > > +=head1 CONFIGURATION FILE FORMAT > > + > > +The configuration file must consist in a file which contains a > > +`install-script` group and, under this group, C=C > > +pairs, as shown below: > > + > > +[install-script] > > +l10n-timezone=GMT > > +l10n-keyboard=uk > > +l10n-language=en_GB > > +admin-password=123456 > > +user-login=berrange > > +user-password=123456 > > +user-realname="Daniel P Berrange" > > Heh, my name is not quite correct It's consistently wrong. It's better than correct in one place and wrong in another, no? :-) > > > + > > =head1 EXAMPLE USAGE > > > > -The following usage generates a Fedora 16 kickstart script > > +The following usages generates a Fedora 16 kickstart script > > + > > + # osinfo-install-script \ > > + --profile jeos \ > > + --config-file /path/to/the/config/file \ > > + fedora16 > > > > # osinfo-install-script \ > > --profile jeos \ > > --config l10n-timezone=GMT \ > > --config l10n-keyboard=uk \ > > --config l10n-language=en_GB \ > > - --config admin-password=123456 \ > > --config user-login=berrange \ > > - --config user-password=123456 \ > > --config user-realname="Daniel P Berrange" \ > > but then it wasn't correct originally either :-) > > We should have a note here > > > "Note that use of --config-file is strongly recommended if the > user or admin passwords need to be set. Providing passwords > directly using --config is insecure as the password is visible > to all processes and users o nthe same host." Indeed. > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From fidencio at redhat.com Mon Jul 8 06:36:23 2019 From: fidencio at redhat.com (=?UTF-8?Q?Fabiano_Fid=C3=AAncio?=) Date: Mon, 8 Jul 2019 08:36:23 +0200 Subject: [Libosinfo] [libosinfo PATCH 2/2] tools, install-script: Do not accept user & admin password via --config In-Reply-To: <20190705155400.GO32473@redhat.com> References: <20190705082720.26774-1-fidencio@redhat.com> <20190705082720.26774-3-fidencio@redhat.com> <20190705155400.GO32473@redhat.com> Message-ID: On Fri, Jul 5, 2019 at 5:54 PM Daniel P. Berrang? wrote: > > On Fri, Jul 05, 2019 at 10:27:20AM +0200, Fabiano Fid?ncio wrote: > > As passing user & admin password via command line is a low impact CVE, > > let's error out when it's done and advertise the users to use > > --config-file instead. > > > > Signed-off-by: Fabiano Fid?ncio > > --- > > tools/osinfo-install-script.c | 11 +++++++++++ > > 1 file changed, 11 insertions(+) > > > > diff --git a/tools/osinfo-install-script.c b/tools/osinfo-install-script.c > > index efa96ee..3da4a69 100644 > > --- a/tools/osinfo-install-script.c > > +++ b/tools/osinfo-install-script.c > > @@ -85,6 +85,15 @@ static gboolean handle_config(const gchar *option_name G_GNUC_UNUSED, > > val++; > > key = g_strndup(value, len); > > > > + if (g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_USER_PASSWORD) || > > + g_str_equal(key, OSINFO_INSTALL_CONFIG_PROP_ADMIN_PASSWORD)) { > > + g_set_error(error, OSINFO_ERROR, 0, > > + _("When setting user or admin password, use " > > + "--config-file instead.\n")); > > + g_free(key); > > + return FALSE; > > + } > > If we make this a hard error then we are pretty much forcing everyone > to use the --config-file arg, because with this CLI tool there is no > way to see the auto-generated password. > > I think perhaps we should do > > 1. Deprecate --config for all uses > 2. Print a warning mesage if used for admin/user password > 3. 1 year later delete --config entirely > Agreed. > > > + > > osinfo_entity_set_param(OSINFO_ENTITY(config), > > key, > > val); > > @@ -520,6 +529,8 @@ script. Defaults to C, but can also be C. > > > > Set the configuration parameter C to C. > > > > +Note: this option has been deprecated, use B<--config-file=> instead. > > + > > =item B<--config-file=config-file> > > > > Set the configurations parameters according to the config-file passed. > > -- > > 2.21.0 > > > > _______________________________________________ > > Libosinfo mailing list > > Libosinfo at redhat.com > > https://www.redhat.com/mailman/listinfo/libosinfo > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From fidencio at redhat.com Tue Jul 9 07:58:51 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 9 Jul 2019 09:58:51 +0200 Subject: [Libosinfo] [osinfo-db PATCH 0/2] debian: Add Debian 10 info & update Debian 9 URLs Message-ID: <20190709075853.29657-1-fidencio@redhat.com> In this series let's: - Update the deadlinks from Debian 9 as, due to the Debian 10 release, its ISOs have been moved to the archive; - Add Debian 10 info; * Unattended installation on both jeos/desktop have been tested using Debian tree; Fabiano Fid?ncio (2): debian9: Update dead links debian10: Add info data/os/debian.org/debian-10.xml.in | 203 ++++++++++++++++++ data/os/debian.org/debian-9.xml.in | 24 +-- .../debian-10.0.0-amd64-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-amd64-netinst.iso.txt | 29 +++ .../debian-10.0.0-arm64-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-arm64-netinst.iso.txt | 29 +++ .../debian-10.0.0-armhf-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-armhf-netinst.iso.txt | 29 +++ .../debian10/debian-10.0.0-i386-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-i386-netinst.iso.txt | 29 +++ .../debian-10.0.0-ppc64el-DVD-1.iso.txt | 16 ++ .../debian-10.0.0-ppc64el-netinst.iso.txt | 16 ++ .../debian-10.0.0-s390x-DVD-1.iso.txt | 16 ++ .../debian-10.0.0-s390x-netinst.iso.txt | 16 ++ 14 files changed, 511 insertions(+), 12 deletions(-) create mode 100644 data/os/debian.org/debian-10.xml.in create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-amd64-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-amd64-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-arm64-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-arm64-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-armhf-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-armhf-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-i386-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-i386-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-ppc64el-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-ppc64el-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-s390x-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-s390x-netinst.iso.txt -- 2.21.0 From fidencio at redhat.com Tue Jul 9 07:58:52 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 9 Jul 2019 09:58:52 +0200 Subject: [Libosinfo] [osinfo-db PATCH 1/2] debian9: Update dead links In-Reply-To: <20190709075853.29657-1-fidencio@redhat.com> References: <20190709075853.29657-1-fidencio@redhat.com> Message-ID: <20190709075853.29657-2-fidencio@redhat.com> As Debian 10 has been released, Debian 9 ISOs have been moved to the archive. Signed-off-by: Fabiano Fid?ncio --- data/os/debian.org/debian-9.xml.in | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/data/os/debian.org/debian-9.xml.in b/data/os/debian.org/debian-9.xml.in index 859c979..3eee3ce 100644 --- a/data/os/debian.org/debian-9.xml.in +++ b/data/os/debian.org/debian-9.xml.in @@ -53,7 +53,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/i386/iso-cd/debian-9.9.0-i386-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/i386/iso-cd/debian-9.9.0-i386-netinst.iso Debian 9.(\d)+.(\d)+ i386 n @@ -62,7 +62,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/amd64/iso-cd/debian-9.9.0-amd64-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/amd64/iso-cd/debian-9.9.0-amd64-netinst.iso Debian 9.(\d)+.(\d)+ amd64 n @@ -71,7 +71,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/arm64/iso-cd/debian-9.9.0-arm64-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/arm64/iso-cd/debian-9.9.0-arm64-netinst.iso Debian 9.(\d)+.(\d)+ arm64 n @@ -80,7 +80,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/armhf/iso-cd/debian-9.9.0-armhf-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/armhf/iso-cd/debian-9.9.0-armhf-netinst.iso Debian 9.(\d)+.(\d)+ armhf n @@ -89,7 +89,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/ppc64el/iso-cd/debian-9.9.0-ppc64el-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/ppc64el/iso-cd/debian-9.9.0-ppc64el-netinst.iso Debian 9.(\d)+.(\d)+ ppc64el n @@ -98,7 +98,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/s390x/iso-cd/debian-9.9.0-s390x-netinst.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/s390x/iso-cd/debian-9.9.0-s390x-netinst.iso Debian 9.(\d)+.(\d)+ s390x n @@ -109,7 +109,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/i386/iso-dvd/debian-9.9.0-i386-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/i386/iso-dvd/debian-9.9.0-i386-DVD-1.iso Debian 9.(\d)+.(\d)+ i386 (\d)+ @@ -118,7 +118,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/amd64/iso-dvd/debian-9.9.0-amd64-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/amd64/iso-dvd/debian-9.9.0-amd64-DVD-1.iso Debian 9.(\d)+.(\d)+ amd64 (\d)+ @@ -127,7 +127,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/arm64/iso-dvd/debian-9.9.0-arm64-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/arm64/iso-dvd/debian-9.9.0-arm64-DVD-1.iso Debian 9.(\d)+.(\d)+ arm64 (\d)+ @@ -136,7 +136,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/armhf/iso-dvd/debian-9.9.0-armhf-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/armhf/iso-dvd/debian-9.9.0-armhf-DVD-1.iso Debian 9.(\d)+.(\d)+ armhf (\d)+ @@ -145,7 +145,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/ppc64el/iso-dvd/debian-9.9.0-ppc64el-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/ppc64el/iso-dvd/debian-9.9.0-ppc64el-DVD-1.iso Debian 9.(\d)+.(\d)+ ppc64el (\d)+ @@ -154,7 +154,7 @@ - http://cdimage.debian.org/cdimage/release/9.9.0/s390x/iso-dvd/debian-9.9.0-s390x-DVD-1.iso + http://cdimage.debian.org/mirror/cdimage/archive/9.9.0/s390x/iso-dvd/debian-9.9.0-s390x-DVD-1.iso Debian 9.(\d)+.(\d)+ s390x (\d)+ -- 2.21.0 From fidencio at redhat.com Tue Jul 9 07:58:53 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Tue, 9 Jul 2019 09:58:53 +0200 Subject: [Libosinfo] [osinfo-db PATCH 2/2] debian10: Add info In-Reply-To: <20190709075853.29657-1-fidencio@redhat.com> References: <20190709075853.29657-1-fidencio@redhat.com> Message-ID: <20190709075853.29657-3-fidencio@redhat.com> Debian 10 has been released on July 6th, 2019. Signed-off-by: Fabiano Fid?ncio --- data/os/debian.org/debian-10.xml.in | 203 ++++++++++++++++++ .../debian-10.0.0-amd64-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-amd64-netinst.iso.txt | 29 +++ .../debian-10.0.0-arm64-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-arm64-netinst.iso.txt | 29 +++ .../debian-10.0.0-armhf-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-armhf-netinst.iso.txt | 29 +++ .../debian10/debian-10.0.0-i386-DVD-1.iso.txt | 29 +++ .../debian-10.0.0-i386-netinst.iso.txt | 29 +++ .../debian-10.0.0-ppc64el-DVD-1.iso.txt | 16 ++ .../debian-10.0.0-ppc64el-netinst.iso.txt | 16 ++ .../debian-10.0.0-s390x-DVD-1.iso.txt | 16 ++ .../debian-10.0.0-s390x-netinst.iso.txt | 16 ++ 13 files changed, 499 insertions(+) create mode 100644 data/os/debian.org/debian-10.xml.in create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-amd64-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-amd64-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-arm64-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-arm64-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-armhf-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-armhf-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-i386-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-i386-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-ppc64el-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-ppc64el-netinst.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-s390x-DVD-1.iso.txt create mode 100644 tests/isodata/debian/debian10/debian-10.0.0-s390x-netinst.iso.txt diff --git a/data/os/debian.org/debian-10.xml.in b/data/os/debian.org/debian-10.xml.in new file mode 100644 index 0000000..47537e3 --- /dev/null +++ b/data/os/debian.org/debian-10.xml.in @@ -0,0 +1,203 @@ + + + + debian10 + debianbuster + <_name>Debian 10 + 10 + <_vendor>Debian Project + linux + debian + buster + + + + 2018-07-06 + + + + 1000000000 + 1 + 1073741824 + 10737418240 + + + 1000000000 + 1073741824 + 21474836480 + + + + + <_name>Debian 10 + + + <_name>Debian 10 + + + + + + http://cdimage.debian.org/cdimage/release/10.0.0/i386/iso-cd/debian-10.0.0-i386-netinst.iso + + Debian 10.(\d)+.(\d)+ i386 n + + install.386/vmlinuz + install.386/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/amd64/iso-cd/debian-10.0.0-amd64-netinst.iso + + Debian 10.(\d)+.(\d)+ amd64 n + + install.amd/vmlinuz + install.amd/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/arm64/iso-cd/debian-10.0.0-arm64-netinst.iso + + Debian 10.(\d)+.(\d)+ arm64 n + + install.a64/vmlinuz + install.a64/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/armhf/iso-cd/debian-10.0.0-armhf-netinst.iso + + Debian 10.(\d)+.(\d)+ armhf n + + install.armhf/vmlinuz + install.armhf/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/ppc64el/iso-cd/debian-10.0.0-ppc64el-netinst.iso + + Debian 10.(\d)+.(\d)+ ppc64el n + + install/vmlinux + install/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/s390x/iso-cd/debian-10.0.0-s390x-netinst.iso + + Debian 10.(\d)+.(\d)+ s390x n + + boot/linux_vm + boot/root.bin + + + + + + http://cdimage.debian.org/cdimage/release/10.0.0/i386/iso-dvd/debian-10.0.0-i386-DVD-1.iso + + Debian 10.(\d)+.(\d)+ i386 (\d)+ + + install.386/vmlinuz + install.386/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/amd64/iso-dvd/debian-10.0.0-amd64-DVD-1.iso + + Debian 10.(\d)+.(\d)+ amd64 (\d)+ + + install.amd/vmlinuz + install.amd/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/arm64/iso-dvd/debian-10.0.0-arm64-DVD-1.iso + + Debian 10.(\d)+.(\d)+ arm64 (\d)+ + + install.a64/vmlinuz + install.a64/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/armhf/iso-dvd/debian-10.0.0-armhf-DVD-1.iso + + Debian 10.(\d)+.(\d)+ armhf (\d)+ + + install.armhf/vmlinuz + install.armhf/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/ppc64el/iso-dvd/debian-10.0.0-ppc64el-DVD-1.iso + + Debian 10.(\d)+.(\d)+ ppc64el (\d)+ + + install/vmlinux + install/initrd.gz + + + + http://cdimage.debian.org/cdimage/release/10.0.0/s390x/iso-dvd/debian-10.0.0-s390x-DVD-1.iso + + Debian 10.(\d)+.(\d)+ s390x (\d)+ + + boot/linux_vm + boot/root.bin + + + + http://deb.debian.org/debian/dists/buster/main/installer-i386 + current/images/netboot/debian-installer/i386/linux + current/images/netboot/debian-installer/i386/initrd.gz + + + http://deb.debian.org/debian/dists/buster/main/installer-amd64 + current/images/netboot/debian-installer/amd64/linux + current/images/netboot/debian-installer/amd64/initrd.gz + + + http://deb.debian.org/debian/dists/buster/main/installer-arm64 + current/images/netboot/debian-installer/arm64/linux + current/images/netboot/debian-installer/arm64/initrd.gz + + + http://deb.debian.org/debian/dists/buster/main/installer-armhf + current/images/netboot/debian-installer/armhf/linux + current/images/netboot/debian-installer/armhf/initrd.gz + + + http://deb.debian.org/debian/dists/buster/main/installer-s390x + current/images/generic/kernel.debian + current/images/generic/initrd.debian + + + http://deb.debian.org/debian/dists/buster/main/installer-ppc64el + current/images/netboot/debian-installer/ppc64el/vmlinux + current/images/netboot/debian-installer/ppc64el/initrd.gz + + + + https://cloud.debian.org/images/cloud/OpenStack/current-10/debian-10-openstack-amd64.qcow2 + + + + https://cloud.debian.org/images/cloud/OpenStack/current-10/debian-10-openstack-amd64.raw + + + + https://cloud.debian.org/images/cloud/OpenStack/current-10/debian-10-openstack-arm64.qcow2 + + + + https://cloud.debian.org/images/cloud/OpenStack/current-10/debian-10-openstack-arm64.raw + + + + diff --git a/data/install-script/opensuse.org/opensuse-autoyast-jeos.xml.in b/data/install-script/opensuse.org/opensuse-autoyast-jeos.xml.in index 9f7af9f..94d35a7 100644 --- a/data/install-script/opensuse.org/opensuse-autoyast-jeos.xml.in +++ b/data/install-script/opensuse.org/opensuse-autoyast-jeos.xml.in @@ -253,6 +253,9 @@ zypper ar -f repo-update-oss zypper ar -f repo-update-non-oss + +# Create an osinfo-install-successful file in /root +touch /root/osinfo-install-successful -- 2.21.0 From fidencio at redhat.com Fri Jul 19 14:22:10 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 19 Jul 2019 16:22:10 +0200 Subject: [Libosinfo] [osinfo-db PATCH 5/6] install-script, rhel, centos: Add osinfo-install-successful file In-Reply-To: <20190719142211.11108-1-fidencio@redhat.com> References: <20190719142211.11108-1-fidencio@redhat.com> Message-ID: <20190719142211.11108-6-fidencio@redhat.com> Let's create an `osinfo-install-successful` file in /root after we finish the installation. Although this may not be totally accurate, it'll help up in the future to test whether the installation was successful in an automated manner. Signed-off-by: Fabiano Fid?ncio --- data/install-script/redhat.com/rhel-kickstart-desktop.xml.in | 3 +++ data/install-script/redhat.com/rhel-kickstart-jeos.xml.in | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/data/install-script/redhat.com/rhel-kickstart-desktop.xml.in b/data/install-script/redhat.com/rhel-kickstart-desktop.xml.in index 7bfaf6e..3eee257 100644 --- a/data/install-script/redhat.com/rhel-kickstart-desktop.xml.in +++ b/data/install-script/redhat.com/rhel-kickstart-desktop.xml.in @@ -248,6 +248,9 @@ AutomaticLogin= [debug] " > /etc/gdm/custom.conf +# Create an osinfo-install-successful file in /root +touch /root/osinfo-install-successful + %end diff --git a/data/install-script/redhat.com/rhel-kickstart-jeos.xml.in b/data/install-script/redhat.com/rhel-kickstart-jeos.xml.in index fdef836..e664168 100644 --- a/data/install-script/redhat.com/rhel-kickstart-jeos.xml.in +++ b/data/install-script/redhat.com/rhel-kickstart-jeos.xml.in @@ -171,6 +171,11 @@ url --url= @core +%post --erroronfail + +# Create an osinfo-install-successful file in /root +touch /root/osinfo-install-successful + %end -- 2.21.0 From fidencio at redhat.com Fri Jul 19 14:22:11 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 19 Jul 2019 16:22:11 +0200 Subject: [Libosinfo] [osinfo-db PATCH 6/6] install-script, ubuntu: Add osinfo-install-successful file In-Reply-To: <20190719142211.11108-1-fidencio@redhat.com> References: <20190719142211.11108-1-fidencio@redhat.com> Message-ID: <20190719142211.11108-7-fidencio@redhat.com> Let's create an `osinfo-install-successful` file in /root after we finish the installation. Although this may not be totally accurate, it'll help up in the future to test whether the installation was successful in an automated manner. Signed-off-by: Fabiano Fid?ncio --- data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in | 4 ++++ data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in b/data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in index 9fd8bcf..f7484f9 100644 --- a/data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in +++ b/data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in @@ -91,6 +91,10 @@ d-i finish-install/reboot_in_progress note d-i preseed/early_command string chmod 666 /dev/ttyS0 ubiquity ubiquity/summary note ubiquity ubiquity/reboot boolean true + +# Create an osinfo-install-successful file in /root +d-i preseed/late_command string in-target \ + touch /root/osinfo-install-successful diff --git a/data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in b/data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in index af9eb4c..b1edba3 100644 --- a/data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in +++ b/data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in @@ -117,6 +117,10 @@ d-i preseed/late_command string \ ubiquity ubiquity/success_command string sed -i 's/eject -p -m.*/& [ "$prompt" ] || return 0/' /etc/init.d/casper + +# Create an osinfo-install-successful file in /root +d-i preseed/late_command string in-target \ + touch /root/osinfo-install-successful -- 2.21.0 From crobinso at redhat.com Tue Jul 23 19:43:12 2019 From: crobinso at redhat.com (Cole Robinson) Date: Tue, 23 Jul 2019 15:43:12 -0400 Subject: [Libosinfo] [osinfo-db PATCH] install-script, rhel, centos, desktop: Fix gnome-desktop & multimedia group install In-Reply-To: <20190719140845.9700-1-fidencio@redhat.com> References: <20190719140845.9700-1-fidencio@redhat.com> Message-ID: <3c97406e-2c2d-f97c-834c-3ed0f859da0a@redhat.com> On 7/19/19 10:08 AM, Fabiano Fid?ncio wrote: > Commit 0a74335fe32ebe moved the x11 group installation to only be done > for RHEL < 8. However, the condition was wrong and ended up with > gnome-desktop and multimedia groups not being installed on RHEL 7. > > Let's fix the check and have those two groups installed for RHEL > 6. > > Signed-off-by: Fabiano Fid?ncio > --- > .../install-script/redhat.com/rhel-kickstart-desktop.xml.in | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Reviewed-by: Cole Robinson - Cole From crobinso at redhat.com Tue Jul 23 19:49:27 2019 From: crobinso at redhat.com (Cole Robinson) Date: Tue, 23 Jul 2019 15:49:27 -0400 Subject: [Libosinfo] [libosinfo PATCH] Update NEWS for 1.6.0 release In-Reply-To: <20190719122020.28047-1-fidencio@redhat.com> References: <20190719122020.28047-1-fidencio@redhat.com> Message-ID: <9251715c-dfce-7fbe-fe1c-c3f4f38e8144@redhat.com> On 7/19/19 8:20 AM, Fabiano Fid?ncio wrote: > Signed-off-by: Fabiano Fid?ncio > --- > I'd like to roll a libosinfo release next Friday (July 26th) and here > you can find the proposed update for the NEWS. > --- > NEWS | 22 ++++++++++++++++++++++ > 1 file changed, 22 insertions(+) > > diff --git a/NEWS b/NEWS > index 54c525a..6a305c3 100644 > --- a/NEWS > +++ b/NEWS > @@ -1,6 +1,28 @@ > libosinfo News > ============== > > +1.6.0 - Jul 26, 2019 > +==================== > + > +- Add API to get the kernel URL argument for an OsinfoOs > +- Improve detection of PPC ISOs > +- Add API to create an OsinfoMedia from location using > + OsinfoMediaDetectFlags > +- Add API to get whether a media is bootable or not > +- Add OsinfoTree treeinfo properties > +- Add API to identify an OsinfoTree > +- Deprecate API to guess an OsinfoOs from OsinfoTree > +- Add --config-file to osinfo-install-script tool > +- Deprecate --config usage for user & admin passwords on > + osinfo-install-script tool > +- Add API to generate an install-script and its command-line > + for OsinfoTree > +- Use libsoup for "http://" & "https://" requests for creating > + both OsinfoMedia & OsinfoTree > +- Drop GVFS dependency > +- Add OsinfoOs property to OsinfoTree > + > + Looks good to me Reviewed-by: Cole Robinson - Cole From crobinso at redhat.com Tue Jul 23 19:54:33 2019 From: crobinso at redhat.com (Cole Robinson) Date: Tue, 23 Jul 2019 15:54:33 -0400 Subject: [Libosinfo] [osinfo-db-tools PATCH] Update NEWS for 1.6.0 release In-Reply-To: <20190719122105.28258-1-fidencio@redhat.com> References: <20190719122105.28258-1-fidencio@redhat.com> Message-ID: <27f44073-e816-8db5-bf09-762f906f7f7b@redhat.com> On 7/19/19 8:21 AM, Fabiano Fid?ncio wrote: > Signed-off-by: Fabiano Fid?ncio > --- > I'd like to roll an osinfo-db-tools release next Friday (July 26th) and > here you can find the proposed update for the NEWS. > --- > NEWS | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/NEWS b/NEWS > index 7bdf55d..949d340 100644 > --- a/NEWS > +++ b/NEWS > @@ -1,3 +1,10 @@ > +1.6.0 > +===== > + > +- Use libsoup for "http://" & "https://" requests for > + osinfo-db-import > +- Drop GVFS dependency > + > 1.5.0 > ===== > > Looks good to me Reviewed-by: Cole Robinson - Cole From crobinso at redhat.com Wed Jul 24 18:50:45 2019 From: crobinso at redhat.com (Cole Robinson) Date: Wed, 24 Jul 2019 14:50:45 -0400 Subject: [Libosinfo] [osinfo-db PATCH 0/6] Add osinfo-installation-successful file on unattended installations In-Reply-To: <20190719142211.11108-1-fidencio@redhat.com> References: <20190719142211.11108-1-fidencio@redhat.com> Message-ID: On 7/19/19 10:22 AM, Fabiano Fid?ncio wrote: > Although this is not the most error-proof way to ensure an installation > was successfully finished, this is already something that we can use to > (try to) check whether the system was installed in an automated manner. > > Or, in other words, take those patches as the first step to be able to > try to have some kind of CI around the unattended installations. > > This series does *not* touch Windows related stuff, but the plan is to > also have the same done for Windows at some point in the future. > > Fabiano Fid?ncio (6): > install-script,debian: Add osinfo-install-successful file > install-script,fedora: Add osinfo-install-successful file > install-script,silverblue: Add osinfo-install-successful file > install-script,opensuse: Add osinfo-install-successful file > install-script,rhel,centos: Add osinfo-install-successful file > install-script,ubuntu: Add osinfo-install-successful file > > data/install-script/debian.org/debian-preseed-desktop.xml.in | 4 ++++ > data/install-script/debian.org/debian-preseed-jeos.xml.in | 4 ++++ > .../fedoraproject.org/fedora-kickstart-desktop.xml.in | 3 +++ > .../fedoraproject.org/fedora-kickstart-jeos.xml.in | 5 +++++ > .../fedoraproject.org/silverblue-kickstart-desktop.xml.in | 3 +++ > .../opensuse.org/opensuse-autoyast-desktop.xml.in | 3 +++ > .../opensuse.org/opensuse-autoyast-jeos.xml.in | 3 +++ > data/install-script/redhat.com/rhel-kickstart-desktop.xml.in | 3 +++ > data/install-script/redhat.com/rhel-kickstart-jeos.xml.in | 5 +++++ > data/install-script/ubuntu.com/ubuntu-preseed-desktop.xml.in | 4 ++++ > data/install-script/ubuntu.com/ubuntu-preseed-jeos.xml.in | 4 ++++ > 11 files changed, 41 insertions(+) > Reviewed-by: Cole Robinson I tested with a Fedora install and it seemed to work as intended. Seems a reasonable idea to verify install 'success', I can't think of anything better offhand - Cole From fidencio at redhat.com Fri Jul 26 12:07:49 2019 From: fidencio at redhat.com (=?UTF-8?q?Fabiano=20Fid=C3=AAncio?=) Date: Fri, 26 Jul 2019 14:07:49 +0200 Subject: [Libosinfo] [osinfo-db PATCH] ubuntu-18.10: Remove dead links Message-ID: <20190726120749.28523-1-fidencio@redhat.com> Cosmic cloud images where removed from cloud-images.ubuntu.com. Signed-off-by: Fabiano Fid?ncio --- Patch already pushed under "tests breakage" rules --- data/os/ubuntu.com/ubuntu-18.10.xml.in | 28 -------------------------- 1 file changed, 28 deletions(-) diff --git a/data/os/ubuntu.com/ubuntu-18.10.xml.in b/data/os/ubuntu.com/ubuntu-18.10.xml.in index 79ffb8b..e8d22c0 100644 --- a/data/os/ubuntu.com/ubuntu-18.10.xml.in +++ b/data/os/ubuntu.com/ubuntu-18.10.xml.in @@ -100,34 +100,6 @@ current/images/netboot/ubuntu-installer/amd64/initrd.gz - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-amd64.img - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-amd64.vmdk - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-arm64.img - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-armhf.img - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-i386.img - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-ppc64el.img - - - - https://cloud-images.ubuntu.com/cosmic/current/cosmic-server-cloudimg-s390x.img - -