Summary of the 2009-01-06 Packaging Committee meeting

Thu Jan 8 00:51:36 UTC 2009

Doug Ledford wrote:
> On Wed, 2009-01-07 at 14:48 -0800, Toshio Kuratomi wrote:
>> It depends on how you interpret the FHS, I suppose.  In the old
>> packages, the config files are in /etc, the arch independent data (help
>> files) are in a subdir or /usr/share/openmpi/, and most of the
>> arch-specific files are under /usr/lib/openmpi/.  This satisfies the
>> overarching goal of the FHS, separation of sharable and unsharable data.
>>  it also satisfies the goal of separating arch specific and arch
>> independent files.
>>
>> The question is whether the binaries can go there or have to go in
>> /usr/bin and whether the libraries can go there or must go directly in
>> /usr/lib.  For the libraries, we often put private libraries in a
>> subdirectory of /usr/lib.  These differ in that they're public
>> libraries.  I lean towards this being okay.  The binaries being in the
>> subdirectory of %{_libdir} doesn't have as much precedent.  Perhaps we
>> need to make that usage explicit in the Guidelines just like %{_libexecdir}?
>>
>> Looking at the new package I see that there's config files under
>> %{_libdir}/openmpi.  I think these need to go in %{_sysconfdir} instead.
>>  This is more important than binaries and libraries for several reasons:
>>
>> 1) Having them in %{_libdir} breaks the sharable/unsharable boundary
> 
> Not really, but that's due to typical usage of these specific files.  I
> would tend to agree that files normally in /etc are something that are
> intended to be edited on a per machine basis.  These files, even though
> they are in %{_libdir}/%{mpidir}/etc, are not something that you would
> edit on a per machine basis.  If anything, things like the
> openmpi-default-hostfile would be edited on a per version basis (and
> with this layout they have a per version etc directory to be contained
> in).  This is because on a large cluster, you are likely to either allow
> all the machines in the cluster to participate and would put all the
> machines in the cluster in this config file, or you would have a segment
> of the cluster that is dedicated to running this version of openmpi and
> only those machines would be in this file.  Either way, for all the
> machines you want running this version of openmpi by default, the file
> would be the same (this assumes that a person might start the openmpi
> job from any machine in the cluster that's part of the appropriate
> group, you may have a control machine doing things instead, in which
> case you really only have to edit the file on that one machine and all
> the others will be passive clients and not care about the contents of
> this file).
> 
Okay.. but then you preclude the possibility of running multiple
instances of one mpi version within a cluster.  It sounds like that's
not typical in your experience but it doesn't sound like a necessary
limitation.

> Now, the even more common scenario is that you have multiple different
> MPI apps.  The admins typically would do a login per app so that the
> default login environment for a given app is already pre-configured.
> Amongst that would be things like selection of the right mpi, and host
> files specific to what machines that app is allowed to run on.  Those
> would all be in the home directory for the login and wouldn't require
> editing the system wide etc files in here.
> 
Despite the environment being somewhat different than normal this kind
of configuration is normal for any apps.

>> 2) They are files edited by system admins and looked at by the user.
>> They should be in a predictable place for this reason.
> 
> In truth, they aren't edited much at all, and relying upon them is
> frowned upon.  But, as I noted above, even if they are edited, they are
> still generally shareable due to the nature of MPI clusters.
> 

This is true of other applications as well, though....

So even if we don't care about people having multiple different openmpi
instances within their cluster, this still doesn't answer what breaks by
putting the config files in /etc.  Which is important because deviating
breaks other sysadmin assumptions.  For instance, if I was backing up
all configuration files on these machines by backing up /etc, this would
miss the openmpi configs.  If I was mounting the /usr filesystem
read-only, this would prevent me from updating the config file on-the-fly.

>> As you noted, there's also some FHS regressions compared to the current
>> package:
>>
>> - include files are under %{_libdir} instead of under %{_includedir} --
>> If these are arch specific include files then this makes sense.  If not,
>> they belong in %{_includedir}.  What things were broken by doing that?
> 
> Two things here.  Remember that we allow simultaneous installs of
> different versions of OpenMPI (you can't get it out of the yum channel
> this way, and you can't do upgrades of OpenMPI or it wipes older
> versions out, but you can download anything after the openmpi-1.2.5 I
> think and install different copies of different versions, although that
> does not include multiple releases of the same version, I only use n-v
> in the naming, not full n-v-r, so for instance you couldn't have 1.2.7-5
> and 1.2.7-6 installed, but you can have 1.1.8 and 1.2.7 installed at the
> same time) in order to meet user requirements.  Differing versions can
> have differing header files, so we can't just use %{_includedir}/%{name}
> or they might conflict.  Putting the includes alongside the libs works
> for just about any devel package that needs to use it because you can
> just use --prefix to configure it to the right place.  Of course, the
> gcc wrappers also know about where the right include files are, so it
> works with mpicc without doing anything.  The second reason is that for
> fortran use in particular, the header file produced during build is
> different for different arches.

The correct way to do this is by having the version in the includedir:
  %{_includedir}/openmpi-1.2.7/*.h

>  So aside from the multi-install issue,
> there is an arch specific component to the headers that can't be worked
> around due to limitations in the fortran language (or that's my
> understanding, I haven't touched fortran since 1991 or so).
> 
So is it only the fortran headers that are arch specific or all all of
them arch specific and only fortran doesn't have a way to workaround that?

arch specific headers do belong in a subdir of %{_libdir}.  But most of
the times just that file goes into %{_libdir}.  If you take a look at
glib-devel, for instance, you have:
  /usr/lib/glib/include/glibconfig.h
  /usr/include/glib-1.2
  /usr/include/glib-1.2/glib.h
  /usr/include/glib-1.2/gmodule.h

>> - man dirs are now under %{_libdir} instead of under %{_datadir}.  What
>> broke by having these under %{_datadir}?
> 
> Multiple installs

This shouldn't be the case.  Once again, the correct solution to this
problem is including the version in the directory name.

> and also if we put it under datadir, then we have to
> fiddle with manpath when we set up the environment.  With them where
> they are, the presence of %{_libdir}/%{mpidir}/bin in the exec path is
> enough for man to track down the right man page automatically.
> 
And this should be something that environment modules takes care of.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20090107/8ccb8fa1/attachment.sig>