iptables won't start today

[Andrew Kelly]

On Fri, 2004-04-23 at 00:11, Matthew Galgoci wrote:
> On Thu, 22 Apr 2004, Chuck Campbell wrote:
> 
<snip>

> Always reboot immediately after a kernel upgrade. That way you are sure to catch
> stuff like this. 

Amen, brother.

> One of the biggest problems I have seen in my career are changes that people make 
> on a running system that don't follow through and update the corresponding config 
> files, regardless if the system is a linux box or a cisco router. :)
> 
> Whenever you do major surery on a machine, it is always good to make sure it passes
> the reboot test and comes up configured as you would expect. Of course this is not
> always possible :\

It still surprises me how often experienced admins can get caught
flat-footed by something like this. It's necessarily humbling.
In fact, I'll admit that I had to eat one myself just the other day.

A script kiddie on a windows box tried to get at two of my servers a few
weeks ago. A quick check showed me that the only thing that got in were
the log entries of the attempt and, after losing his trail in a pool of
addresses held by a Korean ISP, I gave up and forgot the whole thing.
Weeks later I rebooted one of the servers and when it was back up, I
couldn't get in. NONE of my accounts were getting access.
I flashed on the probe from before and started kicking myself about
letting a root kit get past me or something equally stupid. 

Luckily, although it's a remote server, I have serial access to it
through another server, and the ability to force it to reboot regardless
of its condition via yet another server. (great farm, that)

So, I figured I'd attach to serial, force a reboot to single user and
reset the root password, push my logs off for later intense scrutiny and
then rebuild the machine. I was grousing about the work and the delay to
a project for which this particular server was rather important and
pretty much whining "why me" to the world at large when my serial
connection was established and I was presented with a prompt that I
recognized but wasn't expecting to see. 
And then it clicked.

That server was delivered with SuSE on it originally. I needed RH 9 on
that particular box and when I'd installed it, I'd left the SuSE intact
and bootable on a 10 Gig partition. It didn't get in the way and I
didn't see any reason to kill it. Yes, well... during the build of the
RH box I'd kept booting by talking to the loader during the boot, and
had left the SuSE OS as default. I'd figured, if I'm interrupted or
whatever, at least an unstable or unsafe system won't go up unattended.

So, ages later, I'd forgotten all about not having changed the default
boot image and was feeling a bit freaky when I couldn't get at my
server.

Humble is good.

Andy