wiki.libvirt.org replacement

Wed Feb 15 11:43:00 UTC 2023

On Wed, Feb 15, 2023 at 12:22:04PM +0100, Peter Krempa wrote:
> On Wed, Feb 15, 2023 at 10:55:42 +0000, Daniel P. Berrangé wrote:
> > On Tue, Feb 14, 2023 at 11:03:20PM +0100, Peter Krempa wrote:
> > > https://gitlab.com/libvirt/libvirt-wiki/-/merge_requests/1
> > > 
> > > Hi,
> > > 
> > > in order to ease editing of the libvirt wiki, remove the need for
> > > registering users and remove the need to run PHP in an openshift instance
> > > I've decided to contents of the libvirt wiki into a staticaly generated
> > > page using (almost) the same approach we use to generate the libvirt web
> > > pages from rST documents in the repo.
> > 
> > Excellent !
> > 
> > > All articles were converted into rST files [1] and images linked from
> > > currently existing articles were dowloaded.
> > > 
> > > Advantages:
> > > 
> > >  - same workflow as with editing the libvirt pages
> > >  - gitlab still provides a web editor
> > >  - local editing for users who hate web
> > >  - no need to deal with user registration
> > >  - no need to run PHP in openshift
> > >  - we still keep separate space for docs which don't really belong into the
> > >    main repo
> > >  - even if we decide to kill-off the wiki eventually the valuable content
> > >    will be easier to port to the kbase as it'll be in rST
> > >  - the conversion fixed many orphaned pages
> > 
> > You missed the single most important benefit
> > 
> >  * Remove me as a single point of failure for hosting of the wiki
> > 
> > The wiki is attached to a personal openshift account and there's no
> > way for me to grant other users access to co-admin it :-( Removing
> > me as a failure point outweighs all the disadvantages !
> > 
> > > 
> > > Disadvantages:
> > > 
> > >  - all links will be broken [2]
> > 
> > IIUC, the problem is that gitlab needs to have a file extension  to make
> > it correctly serve with HTML content type. All our wiki pages lack a file
> > extension
> > 
> > It might be possible to achieve this by using sub-directories.
> > 
> > eg instead of converting
> > 
> >  TroubleshootMacvtapHostFail.rst -> TroubleshootMacvtapHostFail.html
> > 
> > do
> > 
> >  TroubleshootMacvtapHostFail.rst -> TroubleshootMacvtapHostFail/index.html
> 
> I think I'd rather keep the conversion flat so that linking is not
> confusing. If we want to keep links we can then keep a directory and do
> a bunch of symlinks for all the pages. Eventually we'd be able to get
> rid of them.
> 
> In certain cases the coversion removed special characters from the
> filenames as xsltproc didn't like having ' and ". I've also purged :
> ()_and few other which I don't really want to keep part of filenames.
> 
> I obviously vote for breaking all links.

I'm not so bothered about the wierdly named pages as probably no
one links to them. A few pages though are fairly widely linked,
so it'd be a shame to break them.

If we want to keep the main pages flat, perhaps generate the
subdirs with the gross <meta> redirect trick ?

> > In theory then, if someone accesses
> > 
> >   /TroubleshootMacvtapHostFail
> > 
> > they should get a redirect to
> > 
> >   /TroubleshootMacvtapHostFail/
> > 
> > which should then serve
> > 
> >   /TroubleshootMacvtapHostFail/index.html
> > 
> > >  - changes will need to be reviewed/approved
> > 
> > We could turn off required for merge request approval if really
> > needed, which would allow any libvirt committer to merge without
> > waiting, 3rd parties wold still need help. I don't think it is
> > a big deal though, as the frequency of edits is very small.
> > 
> > >  - low quality/obsolete content is forward-ported as I didn't review
> > >     anything
> > 
> > Not an issue since we're keeping it separate from main libvirt
> > docs.
> > 
> > >  - the build script is in bash (This obviously can be changed if somebody
> > >    cares more than I do.)
> > > 
> > > The generator is based on a cleaned up page.xsl and other assets from
> > > libvirt's repo and the check-html-references script [3] to validate linking.
> > > 
> > > The new wiki can for now be browsed from artifacts of the pipeline job that
> > > I've used to test it:
> > > 
> > >  https://pipo.sk.gitlab.io/-/libvirt-wiki/-/jobs/3771076975/artifacts/website/index.html
> > 
> > That's a 404, the right URL seems to be this one (at least this is the
> > only pipeline I see that exists in your fork)
> > 
> >   https://pipo.sk.gitlab.io/-/libvirt-wiki/-/jobs/3771270309/artifacts/website/index.html
> 
> Oops, I linked to an older job and then I've purged everything besides
> the last one :)
> 
> > > [1] The conversion was done using a collection of ugly scripts and pandoc.
> > > Unfortunately markdown-eque formats have the issue of not really having
> > > strict rules and in certain cases the tools used to process them not
> > > implementing them correctly. This meant that the least painful was actually
> > > the coversion from HTML!!! and not the internal mediawiki format.
> > > 
> > > [2] If we really want to preserve links I can modify the script to generate
> > > a list of redirects for the webserver, but I really doubt that preserving
> > > links has value. Additionally some content was moved to the knowledge base,
> > > so I really want to just delete it from the wiki, so breaking links is a
> > > feature.
> > 
> > AFAIK, you can't do redirects in gitlab pages, except the hacky way by
> > actually creating a page with the desired name, and using the <meta>
> > tag to redirect when the browser loads it. If you go that far you might
> > as well just output the content in that page to start with.
> 
> The original idea was that it'd be served the same way as libvirt's web
> through DV's server and thus didn't think originally about pages. There
> we could generate a .htaccess or whatever to keep the redirects.

I'd really like to avoid adding stuff to the physical libvirt.org
server. It is an alarming single point of failure and even though
DV has backups captured, I'm not confident in correctly bringing
everything back on a new server without a bunch of manual
intervention.

I would have already moved the main website to gitlab, except that
the apache webroot has got content merged from multiple different
sources into one, and its difficult to untangle it.

> Gitlab pages is obviously also a good idea here, but thus will require
> changing the ci definition to generate into proper directory and name
> the job properly.

Essentially there needs to be job called 'pages' and it needs to put
all the content in a subdir called 'public/' under the source root,
and should have 'rules' to limit publishing to a push event on the
default branch.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|