[Libguestfs] [PATCH 5/5] builder: ignore repositories with download failures

[Richard W.M. Jones]

On Wed, Jan 16, 2019 at 03:17:35PM +0100, Pino Toscano wrote:
> If any URL of a repository (the index itself, or any of the other
> resources associated except the templates themselves) cannot be
> downloaded, ignore the repository with a warning message.
> 
> This way, if a repository is temporary unavailable, the rest of the
> repositories can still be used.

It's worth noting that DNF has a special flag ‘skip_if_unavailable=
True|False’ to indicate repositories which are skipped with a warning
versus an error.  Well it's not _quite_ the same thing because AIUI
DNF will disable repositories (permanently I think?) if they are
unavailable, which we don't want, but adding some sort of skippable
flag might be worth considering for virt-builder anyway.

   skip_if_unavailable
          boolean

          If enabled, DNF will continue running and disable the repository
          that couldn't be contacted for any reason when downloading meta‐
          data.  This  option doesn't affect skipping of unavailable pack‐
          ages after dependency resolution. To  check  inaccessibility  of
          repository  use  it  in  combination  with  refresh command line
          option. The default is True.

Rich.

>  builder/builder.ml | 30 +++++++++++++++++++-----------
>  1 file changed, 19 insertions(+), 11 deletions(-)
> 
> diff --git a/builder/builder.ml b/builder/builder.ml
> index 651db83f0..b8d36f6fa 100644
> --- a/builder/builder.ml
> +++ b/builder/builder.ml
> @@ -199,18 +199,26 @@ let main () =
>    let sources = List.append sources repos in
>    let index : Index.index =
>      List.concat (
> -      List.map (
> +      List.filter_map (
>          fun source ->
> -          let sigchecker =
> -            Sigchecker.create ~gpg:cmdline.gpg
> -                              ~check_signature:cmdline.check_signature
> -                              ~gpgkey:source.Sources.gpgkey
> -                              ~tmpdir in
> -          match source.Sources.format with
> -          | Sources.FormatNative ->
> -            Index_parser.get_index ~downloader ~sigchecker source
> -          | Sources.FormatSimpleStreams ->
> -            Simplestreams_parser.get_index ~downloader ~sigchecker source
> +          try
> +            let sigchecker =
> +              Sigchecker.create ~gpg:cmdline.gpg
> +                                ~check_signature:cmdline.check_signature
> +                                ~gpgkey:source.Sources.gpgkey
> +                                ~tmpdir in
> +            let parsed_index =
> +              match source.Sources.format with
> +              | Sources.FormatNative ->
> +                Index_parser.get_index ~downloader ~sigchecker source
> +              | Sources.FormatSimpleStreams ->
> +                Simplestreams_parser.get_index ~downloader ~sigchecker source in
> +            Some parsed_index
> +          with
> +          | Curl.Curl_failed (code, url) ->
> +            warning (f_"failed to download ‘%s’ (code %d), ignoring repository ‘%s’")
> +              url code source.Sources.name;
> +            None
>        ) sources
>      ) in
>    let index = remove_duplicates index in
> -- 
> 2.20.1
> 
> _______________________________________________
> Libguestfs mailing list
> Libguestfs at redhat.com
> https://www.redhat.com/mailman/listinfo/libguestfs

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/