[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Default Groups, Default Packages, CD Sets and DVD Size



Hi there,

there's a number of issues which you may or may not agree with, and are very real concerns. I'll first try to explain what the issue is;

The CD sets in Fedora 10 Alpha, but also Fedora 8 and 9 Re-Spins, require all discs in order to be installed. I say installed, but what I really mean is next, next, next, finish. Eg., this includes the "Office and Productivity" shortcut to the office, games and sound-and-video comps groups of packages.

This problem may need a little more detailed description of 1) CD sets are composed, and 2) how the installation procedures selects groups/packages to install -in a default install.

FWIW, when I use the term "default install" in this email, it defines "next, next, finish" without checking or unchecking any package or group related settings.

Also, FWIW, I'm not putting in all the details in composing- and installing-logic, so that only the bare flow of logic related to the problem set forth remains (or at least I'm attempting to do so).

== The compose ==

Compose tools get input via a kickstart package manifest, which basically describes what groups and packages should be available during the installation. It selects all the relevant groups and packages, resolves the dependencies (inclusive or exclusive -see PS.), puts these in a tree and may or may not create a DVD from that tree. In the case of CD sets however, the packages need to be split over multiple discs and here's how that works (the process is called package ordering for those of you who didn't know that already):

From a list of groups, the compose tools first select @core and @base since these are most commonly (e.g. "always") installed. It resolves dependencies for the packages selected (e.g. the mandatory and default packages in the groups @core and @base). The transaction is ordered and the order in which yum will want to install the packages is spit out.

Then, another set of groups is selected, depsolved, ordered is if it were for a real transaction, and spit out in the correct order.

This continues until all groups the compose tool knows about are selected and spit out, and the package ordering process will then spit out "the remainder of packages".

Note that the next generation of compose tools is going to change this package ordering process to match the behavior of the installation procedure, but I'll continue with more about that later.

== The installation procedure ==

By the time the installation procedure has gained the necessary input wrt. which packages to install, it has also selected a number of default groups (as well as "Office and Productivity" when performing a default installation). For this purpose, it uses the "default" (True/False) attribute in the available -possibly aggregated from multiple repositories, but not in this case- comps.xml. Given no additional input during the default installation, these are the groups selected to be installed[1] (from today's x86_64 rawhide):

 -> office
 -> admin-tools
 -> editors
 -> input-methods
 -> fonts
 -> text-internet
 -> gnome-desktop
 -> core
 -> base
 -> hardware-support
 -> games
 -> java
 -> base-x
 -> graphics
 -> dial-up
 -> printing
 -> sound-and-video
 -> graphical-internet

Resulting in a required RPM payload (on the media) of 2.88 GB, using exclusive dependency resolving.

2.88 GB in RPMs spans 5 CDs. This means, that a default installation of Fedora Rawhide today, when using CDs, would require 5 discs minimum.

== However ==

However, the default installation requires all 7 CDs, because of how the compose tools resolve dependencies (inclusive) during the compose of the media, and pull in more then is minimally required to complete the actual transaction of a default installation. The compose tools do so for good reason:

- one cannot know what package is the user-preferred package for any given required capability (eg. for fictive capability 'web-client', there's firefox, iceweasel, elinks, wget, curl, emacs, emacs, emacs, foo, bar and baz)

- one cannot predict on what installed system one is performing an upgrade, and to be able to close the transaction certain considerations must be met justifying the need for inclusive dependency resolving when composing the media (set) or installation tree.

- it makes the released media apply to N+X use-cases where the package set or transaction payload during the installation is controlled in a more granular fashion then the selection dialogs allow (by means of a kickstart package manifest maybe?), which I guess applies more to businesses or advanced users using Fedora then it does to Joe Average users.

Basically what I'm saying is that the 2.88 GB is spread over more then the minimal amount of discs it would fit on, since the complete package payload when using inclusive dependency resolving grows to 3.66 GB, or 7 discs. So, the compose process spits out 7 discs, each of which contain a part of the 2.88 GB sized RPM payload needed for a default installation.

== Next Generation of Package Ordering ==

So, what package ordering is going to do -instead of having a static list of groups to add to a transaction, resolve, spit out the packages- is use the "default" parameter to groups in comps.xml as well (and then instead of exclusive dependency resolving like it does now, move to inclusive dependency resolving as well). This makes the "which packages and groups are in a default installation" a little less hard to maintain and the package ordering will almost automagically match up with comps.xml (of which the installation procedure also uses the default parameter to groups!)

== DVD and Sizes ==

This leads me to another concern which may or may not be an immediate issue but requires attention from those in the decision making chain as well as generally interested people; the size of the DVD ISO is getting to it's maximum allowed size (just under 4GB for those who use FAT systems as their downloaded data partition), providing just a default (eg. not including anything in addition to the default).

== Advise needed ==

There's several ways this can be solved, but I'm not sure what is the most advisable (some are not feasible I'm sure, I'm just brainstorming here):

1) reduce the number of mandatory and default packages per group in comps.xml

2) reduce the number of groups in comps.xml that have "default" set to True

3) Revisit how comps is formatted; Example: Keep the "default" for compose decisions, but add an "install" attribute for installation decision making. Install set to True may require default set to True as well for the group to even be included on the media.

4) Split the packages that are mandatory or default in comps groups, into smaller packages providing what the group needs and another set of smaller packages belonging to the group as to reduce the number of dependencies needing to be met when the compose or installation procedure selects a group. See also PS2.

5) Or, compared to 4, revisit the Requires in mandatory and default packages and the Provides in the packages that provide the required capabilities so that it abstracts from the requires/provides matching with too many other packages (related to the inclusive dependency resolving which will then make for a thinner RPM payload on the composed media)

6) Have the compose tools as well as the installation procedures not depend on the default attribute to groups anymore, at all.

Thank you very much for reading this message so far, and please don't hesitate to ask any questions if I wasn't clear enough in how this works (though I also hope I didn't overdo it for those who already knew) or make a remark or two when you think I'm wrong about something ;-)

Ideas and insight on the topic very much appreciated,

Kind regards,

Jeroen van Meeuwen
-kanarip

PS. Inclusive dependency resolving is grabbing _all_ packages that provide a required capability. Exclusive dependency resolving is grabbing the one best fit (this is YUM dependency resolving). I'm taking a few shortcuts here but I hope that's OK.

PS2. Selecting all 18 default groups in rawhide results in 79 mandatory packages, 302 default packages, and (after depsolving) 1386 packages in total, being a 3.66 GB payload

[1] the find-default-groups.py run against a rawhide yum configuration
[jmeeuwen ghandalf scripts]$ ./find-default-groups.py -c ../unity/conf/conf.d/revisor-rawhide-x86_64-respin.conf -r
Default groups:
 -> office
 -> admin-tools
 -> editors
 -> input-methods
 -> fonts
 -> text-internet
 -> gnome-desktop
 -> core
 -> base
 -> hardware-support
 -> games
 -> java
 -> base-x
 -> graphics
 -> dial-up
 -> printing
 -> sound-and-video
 -> graphical-internet
3.66 GB (3934874324 archive_size)
79 mandatory packages, 302 default packages, 1386 packages in total (after depsolving)

The actual script can be found at https://fedorahosted.org/revisor/browser/scripts/find-default-groups.py


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]