are you using Fedora in a production environment?

[Dmitry Butskoy]

"Are you using Fedora in a production or live enviornment?" -- Yes!

We use it at "united dispatching control board of the power supply 
systems". And yes, it sounds like it is more than just production 
environment, it is even some kind of critical environments.

Use Google Maps and go to Saint-Petersburg (it is second big town of 
Russia, it is placed near Finland, where Linus is from :) ). Do you see 
that lanterns light somewhere in the streets? If yes, it means that our 
"control board" works, which means that our Fedora systems work properly. :)
(Surely I know that these Maps are yet not online ;) )


The Using of Fedora in the critical environment might seem strange, but 
there are some circumstances which allow it.

Such kind of environment implies an existence of high skilled engineers, 
who do the support and who work full day immediately near the 
corresponding hardware. These engineers must not trust any external 
opinions, they must check out each update to be installed etc. And they 
must be capable to fix (and hack) software when it is needed. Besides 
they must create/support/etc. a lot of specific programs on the 
supported platform too.
If so, it seems that they are capable to do the whole support 
themselves, therefore they just not need any external support (like RHEL 
support).

Such kind of environments implies high availability, which mean that 
there are several servers for each critical application, i.e. atleast 
two mail-servers, two dns etc. This allow to do the changes/updates 
gradually, step-by-step. First, change something on one server. If 
something failed, you have the second working server anyway. If all is 
OK, do the changes on the second server, etc.etc.etc. "Several servers 
for each application" allow to do changes that way faster and more 
bravely. It means that we can forget about worries of too often changed 
things in Fedora.

These two circumstances can help to explain, why we use Fedora, but "the 
light is shining" :)


Why do we use Fedora?

First of all, it is for historical reasons. We used RH-5.2 initially, 
then RH-6.0-6.2-7.1-7.3.
Even using latest release, we often actually needed some features of the 
next upstream versions. Therefore we've rejected idea to switch to 
CentOS and friends (and even to RHEL).
When RH line became Fedora, we've estimated our forces and have decided 
to continue with this line.


What do we use exactly?

Application servers (postgresql, mysql, httpd, php, perl), domain 
control (OpenLdap + Samba), mail (postfix based), routers/firewalls, 
dns/dhcp, proxy, web-sites, ntp (gps), vlc (multicast) etc. And a lot of 
Fedora servers with specific software (either bought or own (including 
GPLed)). Several desktops. Certainly I cannot report the exact amount of 
servers, but using "whois" you can see that we have at least 512 extern 
IP addresses... :)


What do we do to provide local Fedora support ourselves?

We have a local yum repository. (Surely with some packages fixed :) ). 
Updates go to corresponding machines automatically, but downloaded only. 
All installs/updates are manual, after the all needed checks. Normally 
we wait a little after an update is shipped, for possible "update for 
the latest update" :)
Sometimes we are compelled to change standard package, sometimes go back 
to the previous version, but most often just forward to the next version 
from upstream (even under latest Fedora).

We do upgrades just one time per year. Yep, we jump for it through one 
Fedora release, i.e. FC1->FC3->FC5. The "6 month" schedule is too fast 
for us. Moreover, we try to skip releases with some changes which seem 
too global for us to be used at once, i.e. FC2 (kernel 2.4->2.6 switch), 
FC4 (gcc 3->4) etc.


What problems we feel recently?

The main problem is that in the context of production environment, the 
Fedora community seems to be unpresentable now.

When there was "RedHat Linux-->Fedora Core" switching, "production" 
people have read the words "enthusiasts, developers, not cricial" etc., 
and have read the schedules (one release per half of year). All this has 
frightened them a lot. Therefore as soon as some free RHEL-clones have 
appeared, they were switched to them. Now they "enjoy stability", 
degrading as specialists. :(

People who have remained with Fedora, are normally busy (as production 
environment implies that there are enough issues besides writing 
bugzilla/maillist report). It decreases presentability too.


Another problem is conservatives.
Doing good changes forward, the old features/details should be saved 
(whenever it is possible). I know several people who install new RH or 
Fedora version, see some changes, try to adapt new version for their 
habits, have failed, become angry and switch to another distro.

The solution to just ignore such kind of people is not good.
Let consider, who are this people actually? "Conservatives" implies that 
they are not too young, they have some skill and they have some habits 
which they cannot refuse. Now, let try to guess, what a role they play 
in their companies?.. You have guessed -- they are managers/experts 
etc., i.e. "make decision men".
Including decision whether to use Fedora or not...


What changes we would like?

- RH people should stop consider Fedora as some kind of testing 
environment (i.e., try in Fedora, if no reports, put in RHELbeta). FC 
maintainers (mostly the same for FC and RHEL) should feel Fedora as same 
stable as well as RHEL, just the changes and updates are faster than in 
RHEL.

- Update policy should be consistent. Do not throttle useful stable 
changes in rawhide, ship the update immediately. Ship the update if 
upstream ships new release and you do not see anything dangerous in the 
"diff -Nrbu" output.
On the other hand, do not trust upstream and don't ship an update 
automatically, if you see a lot of unclear or too complex changes.

- IMHO it would be better to change EOL policy.  For example, instead of 
EOL'ing FC5 at FC7test2 time, we would prefer FC5 EOL at least 1 month 
later after FC7 is released. This way will help people who skips one 
Fedora release. (The current speed of Fedora Legacy work is too low to 
consider it as a solution for production environments).

- Do not offend conservatives. Do not consider them "too rare". People, 
making decisions for million dollars, is "too rare" too.



Regards,
Dmitry Butskoy,

Leading expert,
United dispatch control board of the power supply systems of the Northwest,
Saint-Petersburg, Russia
http://www.fedoraproject.org/wiki/DmitryButskoy