[Fedora-packaging] Java (jpackage) naming scheme rehash -- part 1 Goals

Fernando Nasser fnasser at redhat.com
Fri Jan 12 16:42:55 UTC 2007


Hi Toshio,

Toshio Kuratomi wrote:
> On Thu, 2007-01-11 at 17:49 -0500, Fernando Nasser wrote:
>> 1) Problem Space
>>
>> "Many Java programs are first packaged by the [WWW] jpackage project. In 
>> essence, the Fedora packages have two upstreams: The upstream project 
>> itself and the jpackage project's spec file and patches. Fedora imports 
>> a package from jpackage and makes changes to enable native AOT 
>> compilation via gcj and bugfixes that have not been merged into either 
>> of the upstreams."
>>
>>  > Well, The last sentence is no longer true.  The AOT compilation bits 
>> were propagated upstream, so all the Java team has to do now is to 
>> import the upstram package as is and build.  We still need to make any 
>> changes to the code to circumvent GCJ shortcomes, but these are so rare 
>> nowadays that almost all packages build without any changes at all.
>>
> Sounds good.
> 
>>  > As a consequence, fixes have been made on the usptream version, which 
>> is them re-imported, and we've been also reimporting when fixes are 
>> applied upstream comming from the other JPackage-based distros or from 
>> direct JPackage users feedback.
>>
> Sounds like a good deal.
>  
>>
>> "Presently, Core's jpackage-based packages have reflected the package 
>> origin within the package Release: 1jpp_2fc In this scheme, the 1jpp is 
>> the upstream jpackage release, _ is a separator, and 2fc is the Fedora 
>> release. (Note, this scheme is not currently as defined as the Fedora 
>> Naming Guidelines. So there are some practices above the base standard 
>> that would need to change.) This scheme has two expressed goals:"
>>
>>  > OK, here we have made much progress:
>>
>> a. The "_" character has been removed and replaced by a '.'
>>
>> b. The "fc" string has been removed
>>
>> c. All FC6 packages have been rebuilt to comply with a and b above
>>
> Reading this it looks like the packages are now namedSo
> name-numericversion-1jpp upstream and name-numericversion-1jpp.2 by
> Fedora.  Is that correct?
> 

Yes, we were able to do this without breaking any of our scripts or 
processes and it is much more clean (no extraneous or superfluous 
characters).


>> d. An initial agreement has been reached to also compatibilize the 
>> pre-release cases, with JPackage adopting a Fedora-like order.
>>
> Do you have a link to this documentation?  I found this:
> 
> http://www.jpackage.org/cgi-bin/viewvc.cgi/*checkout*/src/jpackage-utils/doc/jpackage-1.5-policy.xhtml?root=jpackage&content-type=text%2Fhtml
> Which doesn't cover versioning and this:
>   http://www.jpackage.org/jpppolicy.php
> which says it's the old version and indeed, has the old versioning
> information.
> 

Yes, you are right.  As you see, it says 1.5 there.  That was for JPP 
1.5, and JPackage is releasing JPP 1.7 soon (it is release candidate now).

The idea is that as soon as we figure out how we are going to handle the 
imports into Fedora a "Guidelines" link will be added with updated 
recommendations.  There is a couple of things that must be known first: 
(a) what is the decision on that thread on Epoch and how that affects 
the Java packages (they currently always specify the Epoch)  (b) the 
exact syntax for the pre-release tags so that is compatible with Fedora 
and still incorporates the 'jpp' string.

One "de facto" procedure that is already used is that Jpackage 
maintainers refer people to the Fedora guidelines for any case that is 
not explicitly covered by some local rule.  And the main reason for any 
local rule is to make sure the other distros that are based on JPackage 
are also OK with it (Mandriva, Suse...).


>>  > Summary: It all comes down now to the remaining ".NNjpp" in the 
>> release which indicates the upstream version that was last imported 
>> (sync point you could call it).
>>
>>
>> In the case of the goals, I think the original ones in the week were not 
>>   correct in the first place.  Lets try again:
>>
>> "   1.  Allow for upgrading between the Fedora and JPackage repositories 
>> so that upgrade paths similar to the following works:
>>            *
>>
>>                  java-foo-1.0-1jpp_2fc [Fedora Release]
>>                  java-foo-1.0-2jpp     [Updated JPackage Release]
>>                  java-foo-1.0-2jpp_3fc [Updated Fedora Release rebased 
>> against the new JPackage]
>>
>>     3.      A third expressed goal that conflicts with 1. is to upgrade 
>> Fedora packages only with Fedora packages and jpackage packages with 
>> jpackage packages."
>>
>>
>>  > These two goals can be made into a single one, lets say: "flexible 
>> upgrade paths".
>>
>>  > There is no conflict whatsoever -- we currently do all these:
>>
>> a. Both Fedora and JPackage (what is stated in 1, i.e., newest release 
>> globally) by leaving all repos enabled (both fedora-core and jpackage)
>>
>> b. Fedora only by disabling the jpackage.repo
>>
>> c. JPackage only by using --exclude=*jpp* --excluce=eclipse* when 
>> upgrading from fedora-core.repo and enabling only the jpackage.repo for 
>> the Java packages update (people play with --enablerepo for that management)
>>
> You're not understanding Goal #3.  The goal is to have a mixture of
> Fedora and JPackage packages installed on the system.  Running "yum
> update" will update to the latest version of the software from the
> repository which the current version comes from.  Your (b) and (c) are
> for the case where you want to draw your java packages from either
> Fedora or JPackage, not both.
> 

You mean, if a package is from Fedora, you update it with only another 
Fedora package and if it came from JPackage only with another JPackage 
one?  Yes, perhaps someone may want that (I never encountered that 
situation, but I think it may be useful).  That is a letter (d) though. 
  (a) has no restriction where a package comes from, the newest one is 
what is wanted, no matter from where.

BTW, I am sure it can be scripted currently with "*jpp$" used to 
identify what is JPackage.  We must be careful with what we define for 
pre-release tags for that to continue to work though.


> Goal #3 could probably be realized with commandline switches if Fedora
> did not include jpp in the name: yum update --disablerepo=jpackage
> --exclude='*jpp*'; yum update --enablerepo=jpackage '*jpp*'
> 

Yes, that would work.  It is only my letter (d) case though.

>>
>>     "2.      Allow users and packagers to tell what JPackage release the 
>> java package was based against."
>>
>>
>>  > This is very important to us (Java team).  We can quickly identify if 
>> the version we have already has a fix or not, among other things.
>>
> Can you give a developer use case?  We tried to imagine a use case for
> developers and what we came up with was looking in cvs for the package.
> At that point, there is no rpm package so there is no release tag.
> Giving us a few use cases will help to see what will and won't work.
> 

I'd like the other Java team members to comment on this.  I only have a 
share of the packages to maintain.  That is how I control my packages 
but I'd rather other people give testmony.

Deepak?  Vivek?  Matthew?  Permaine? ...


>> I will have to add a goal that was not stated in the initial Wiki, lets 
>> call it 4:
>>
>>    "4.  Allow the switching of Java stacks as a whole"
>>
>>  > This is currently accomplished by using regular expressions based on 
>> the *jpp* and eclipse* patterns.
>>
> Again, a use case would be nice.  I'm imagining switching between the
> jpackage java stack and the Fedora java stack and don't see how *jpp*
> would help here.
> 
> Unless you mean it makes it easy for you to select all the java packages
> on the system and replace them with something else?  That strikes me as
> a misuse of the release tag -- 1) Not all java packages will have the
> jpp name as not all java packages will come to us through jpackage. 2)
> If we do this for java, why not for python, perl, php, etc?
> 

Several things in this paragraph.

1) Except for the eclipse package, which removed the 'jpp' tag to 
conform to the guidelines and is now maintained (the version we use) 
locally, all Java packages have the 'jpp' string and all come from JPackage.

As the system of packaging Java we use is the JPackage system, we 
contribute all our packages upstream and import them.  As long as 
someone is willing to maintain a package there is no restrictions.  This 
helps us with solving any issues with other Java libraries need by that 
piece, Java-related packaging issues and so one on a community of Java 
developers and packagers.

2) Our Java packages (Fedora has a subset, we actually deal with almost 
300) have a very complex graph-like relationship among them, creating 
what people have bee calling a "ecosystem" that is mostly independent, 
just requiring a JVM to be present.  I don't know about python, perl, 
php etc., I can only comment on Java.

0) There is also the question of using the release tag for that 
identification purpose.  There are several considerations:  (i) there 
are other uses for that string, so as if it is there why not use it 
(ii) Some other strings in the release identify things as well, for 
instance, anything marked with %{dist} tells a package has a variant for 
that distribution; that would also be a similar abuse  (iii) some of the 
proposals suggested even the Provides tag; that would be a serious 
abuse, it is used by one of the most basic mechanisms of RPM; the 
release tag markings like %{dist} or jpp are innocuous  (iv) RPM and you 
have no adequate mechanisms at the moment that are so handy, visible and 
easy to use at the moment.

P.S.: We should try and suggest enhancements to the new RPM and to yum 
(perhaps even contributing patches) to allow this kind of functionality 
in the future without the need of using the release tag.  For instance, 
I'd like to see easy ways to retrieve, filter on, etc. on tags like 
Group, both for RPM and Yum.


>>  > This is useful for development so we can try sets of packages built 
>> with a different compiler for instance, without having to change all 
>> spec files to bump releases, and to revert the system to the original 
>> state afterwards.  We use this a lot.
>>
>>  > There are also users that want to switch the collection of Java 
>> packages  to use a set compiled with a proprietary JDK for instance 
>> (maybe a different version of Java), or something provided by the 
>> software vendor of some major component they bought.
>>
> You're going to have to post the commands you use to achieve these
> things as I'm having problems understanding how you're doing this based
> on the release tag.
> 

I've already posted the yum ones.  The above is just a piping of rpm -qa 
thorugh grep and things like xargs.   The regular expresiions given to 
grep are the key here.


>>
>> OK, now the Wiki page states:
>>
>> "Which naming guideline is chosen depends in large part on whether these 
>> are judged to be worthwhile and attainable goals."
>>
>>  > Which I think is correct.
>>
>>
>>
>>
>> Discussion of the goals
>>
>> "Goal #1
>>
>> If JPackage were truly an upstream then we wouldn't want users to 
>> upgrade Fedora Packages with JPackage. We have bugfixes, local changes, 
>> and other patches that we add to packages to make them work better on 
>> Fedora. Having users upgrade between upstream and our packages is not a 
>> goal with any other upstream. gbenson states this [WWW] here although he 
>> phrases it in the specifics of Fedora/JPackage instead of the general 
>> Fedora/Upstream."
>>
>>
>>  > We don't do anything to make packages run better on Fedora.  We do 
>> pre-compilation, which does help in the startup time.  But there is no 
>> real harm in revert to a bytecode-only package to get an upstream fix 
>> until it shows up in pre-compiled form in a Fedora update.
>>
> This may be true in the RH-Fedora java group but it won't be true in the
> Fedora-community java space unless the mandate from the FTC is to get
> packages approved in jpackage before getting them into Fedora.  Not
> everything happens upstream.  Not every upstream agrees with the
> downstream.  Not every upstream has the same time frame for updates as
> downstream.  We will carry local patches and changes at some point.
> 

IMO that should be a recommendation (no mandate -- too many mandates 
already).  As the system that allows the proper Java interoperation of 
all these things is done upstream, people should try and get their 
contributions there first.

This is the same as wanting a new Perl module and not going through that 
  Perl modules community first.

Remember, there may be (we hope there will be) a large Fedora Java 
community in the future.  But a Fedora+other distros comunnity will 
always be larger than that (just set theory), and the larger the 
community the better the software/package produced.

>>
>> "There could be times when the jpackage naming is not compliant in the 
>> name, version, or release fields. When this happens, we would not be 
>> able to attain this goal. For instance: 
>> cryptix-asn1-20011119-4jpp_2fc.1.1.src.rpm. The version field must be 
>> changed before importing otherwise the final release of cryptix-asn1-1.0 
>> will not upgrade the package."
>>
>>  > That was a wrongly versioned package from an older JPackage release. 
>>   The maintainer was informed and the problem was fixed.  It is now 
>> cryptix-3.2.0-9jpp.  In general, we have had no problems in getting this 
>> kind of things fixed.  I think this comment can be removed.
>>
> No.  cryptix-asn1 is an example.  It doesn't matter if the specific
> package has been fixed.

I disagree. That is not representative and was a unique mistake made by 
an individual on a certain date.  That would never be propagated into 
Fedora as it would be detected during the import.  The upstream 
community would be notified and would have fixed it quickly (which was 
exactly what happened).


> The question's contained here are:
> 1) What are jpackage's naming guidelines so we can see how they mesh
> with our names?

No different from Fedora, RHEL or any other package distribution. 
Actually, people have been referred to the Fedora guidelines lately.


> 2) If there's a bug with the jpackage naming what will we do until it's
> fixed?

The time to fix that is negligible,  As the Java team is maintaining the 
packages upstream, they have commit access so it is very easy to fix 
these things.


> 3) Do we want to force our packagers and reviewers to learn two sets of
> guidelines, one for jpackage, and one for Fedora?
> 

There is no conflicting guidelines at the moment that we are aware of, 
except for the release tag and the pre-release tags that is being worked 
out.  Maybe we will have something else depending on that Epoch thread, 
but  not at the moment.

As I said before, except for some Java-specific things that are not 
covered in generic guidelines, people have been refered to the Fedora 
guidelines ad so far this has not raised any complaints from the other 
distros.


>> "Goal #3
>>
>> This is not truly implementable via the package release tag.
>>
>> Separating the native libraries into a subpackage might have some 
>> bearing on this."
>>
>>  > Yes it is implementable, as shown above.
> 
> You're right.  Remove the jpp from Fedora package release tags and we
> should be able to do this :-)  Seriously, the goal is to have "yum
> update" just work, so even with jpp removed, we can't achieve this.
> 

We were talking about two different goals.  This one yes, but the (a) in 
my list is not.  I deam that more important than this one (my letter (d) 
now) that I haven't see anyone use yet (although I recognize it may be 
useful).


>> "Goal #2
>>
>> Users shouldn't need this information. The Fedora Package may have more 
>> fixes than the JPackage one. It may be synced with more upstream 
>> bugfixes. It may contain snapshots of JPackage work that haven't hit the 
>> repositories yet because JPackage is working on a new major release. If 
>> the package works without bugs then the end user is happy."
>>
>>  > Putting a set of compatible packages together is a major endeavor. 
>> The Java team, as well as the folks from Mandriva and Suse have been 
>> basically polishing the sets that have been created upstream.
>>
> Ain't that the truth!  We have to do this for the entire distro.  That's
> a bit besides the point.
> 

It is not you who are having to deal with all these Java packages, so 
please don't speak for us.  We have already a heavy load of work and 
can't handle the results of a fork.


>>  > Also, we make the fixes/changes upstream first, then import.  This 
>> way we benefit from feedback from people (both packages and Java users) 
>> who know deeply some of these Java software libraries.
>>
> This is where we have divergence.  You claim that all fixes go upstream
> first.  I claim that 1) this won't continue to be the case (all fixes)
> and 2) it may not be desirable.   Example: There's a showstopper bug in
> a java package just before F7 is spun.  The jpackage maintainer doesn't
> think it's a showstopper or would rather wait for upstream to fix it
> before committing a patch.  We aren't going to hold F7 for it so we have
> to fix it locally.  Example: jpackage upgrades to a beta version of a
> component while we want to stay on the stable release or vice versa.
> 

We do what we do for any software: we add a local change in urgent mode 
and immediately contact upstream so the fix is incorporated in their 
next release.  If the change gets incorporated into another form that is 
  better than our stop gag fix we correct that in an errata.

We've been doing this for things like debuggers, DBMSs, Internet Servers 
of all sorts for 10+ years.


>>  > Again: updating paths are optional, and it is up to the user to 
>> choose.  The current naming only allows that to be done if so desired, 
>> which would be otherwise not possible.
>>
> Not true.  Jesse Keating proposed a naming scheme which would work
> equally well.
> 

Lets discuss the goals first.  How can we discuss solutions we we havent 
agreed on the goals?

We will examine all the proposed solutions based on the goals we agree upon.


>>
>> "Developers can use this information. But they aren't going to be 
>> staring at the package name most of the time. When they checkout from 
>> cvs, they'll have a spec file, not an RPM so the filename is no help. 
>> They have to look inside the spec file at which point it's just as easy 
>> to put the information in the %description, a Provides, or a comment. 
>> Using a Provides makes it queryable from the rpm command line as well."
>>
>>
>>  > The way we do now is by "rpm -qa | grep jpp" and variations.  Or 
>> similar pattern usage in the repos.  We don't need to look inside the 
>> spec file to know that the base for that release was a specific upstream 
>> release.
>>
> What are you looking for inside the package repos?  What is your rpm -qa
> |grep jpp used for?  The comment above was from the projected usage case
> of a developer needing to look at a package in the cvs repository to
> decide what version a package was built against.  In that case, you
> would need to look inside the spec.  Help us understand what you're
> doing so we can see where the other proposals fail.
> 

If we know what is the base point by looking at the release tag we don't 
have to look into a file inside cvs just to see that.  And it can be 
done at large with diffs we do between lists of package sets.


>>  > Using things like "Provides" for things that are not their original 
>> goal would be abusing that feature, and the information for that is used 
>> by depsolvers and such.  We should stay away from that.  In a lesser 
>> degree, maintaining a release in the description is not convenient and 
>> error prone, and does not address the other goals anyway.
> 
> There's all manner of odd things in Provides.
>  Provides: jpackage(foo) = 1.0-1jpp
> Would fit right in :-)
> 

So, you rather "abuse" (in you words) the Provides tag?  That makes no 
sense and it does not attain most of the goals above.

With changes to future RPM and Yum releases we could do things based on 
other tags like Group and so one.  It is just not available yet.


Regards,
Fernando




More information about the Fedora-packaging mailing list