Quoting John Haxby <jch scalix com>:
Yup, this is usually due to somebody being too helpful, knowing too little, or
both. The erase character will usually be set to correct value (based on
termcap, terminfo, or serial line settings), and if you are using ssh to
connect to remote machine, this setting will be transfered too.
Yeah that was it, thanks. There was the following in /etc/bashrc: