UTF-8 & imap folder name handling
Nicolas Mailhot
nicolas.mailhot at laposte.net
Thu Oct 27 12:38:01 UTC 2005
Hi,
The Fedora Linux project and Red Hat are pushing very hard for good
internationalisation support. That means UTF-8 support in every single
app they ship even if it may cause interoperability problems with old
applications (the common feeling is if we wait for everyone to jump on
the unicode boat before using unicode, we'll still be waiting in ten
years).
Squirrelmail by default does not use UTF-8 for many of its locales,
causing character losses when you reply to a message that contains
characters not available in the encoding declared for your locale.
To workaround this all the locales in the fedora squirrelmail version
are converted to UTF-8 at build time (cf attached build script). This
fixes interoperability between locales, since all characters in a
received message now fit inside the locale encoding, so quoting people
when replying to them now work.
Unfortunately this fix has unearthed a problem with squirrelmail
handling of imap folders. squirrelmail seems to assume 7bit imap folder
names, as soon as you try do display a folder name that use >7bit
characters in an UTF-8 locale things break (it doesn't with 8bit iso
locales, probably because so many bits assume iso-8859-1 transcoding
problems cancel one another).
With an UTF-8 locale SM won't display properly non-ascii folder names
when those have been created by another mail client, if you
recreate/rename the folders in squirrelmail display is now fine inside
SM but broken for everyone else.
Many common default folders use >7bit encodings in many locales (Sent ->
"Éléments envoyés" in French) so you're almost certain to hit this bug
as soon as you use an UTF-8 locale that actually needs >7bit for naming
folders. It's not a problem that can be ignored if one wants to ship and
i18n SM.
Can someone familiar with squirrelmail fix the imap folder encoding code
when local is UTF-8-enabled ? This is a bit too much to do at the Fedora
level (especially since Fedora tries to stay close to upstream). If not
Fedora will have to accept SM won't do UTF-8 sanely any time soon, which
will mean SM removal according to Fedora internal rules.
Various bug reports related to the problem :
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=162852
https://sourceforge.net/tracker/?func=detail&aid=1339393&group_id=311&atid=100311
https://sourceforge.net/tracker/?func=detail&atid=423691&aid=1235345&group_id=311
Regards,
--
Nicolas Mailhot
-------------- next part --------------
# OPTION: Fedora (1) or RHEL (0) Splash
%define fedora_splash 1
%define contentdir /var/www
Summary: SquirrelMail webmail client
Name: squirrelmail
Version: 1.4.6
Release: 0.cvs20050812.2.fc5.1.nim
License: GPL
URL: http://www.squirrelmail.org/
Group: Applications/Internet
#Source: %{name}-%{version}.tar.bz2
Source0: %{name}-1.4.5.tar.bz2
Source1: squirrelmail.conf
Source2: squirrelmail-splash-fedora.png
Source3: squirrelmail-splash-rhel.png
Source4: squirrelmail-20050812_1242-CVS.locales.tar.bz2
Patch2: squirrelmail-1.4.3-config.patch
Patch3: squirrelmail-1.4.6-cvs20050812.patch
Patch4: squirrelmail-1.4.5-charset.patch
BuildRoot: %{_tmppath}/%{name}-%{version}-root
BuildArch: noarch
Requires: httpd, php >= 4.0.4, perl, tmpwatch >= 2.8, aspell
Requires: /usr/sbin/sendmail
Prereq: httpd, perl
Provides: squirrelmail-i18n
%description
SquirrelMail is a standards-based webmail package written in PHP4. It
includes built-in pure PHP support for the IMAP and SMTP protocols, and
all pages render in pure HTML 4.0 (with no Javascript) for maximum
compatibility across browsers. It has very few requirements and is very
easy to configure and install. SquirrelMail has all the functionality
you would want from an email client, including strong MIME support,
address books, and folder manipulation.
%prep
#%setup -q
%setup -q -n squirrelmail-1.4.5
%patch2 -p0
%patch3 -p1
%patch4 -p1
rm -f plugins/make_archive.pl
# Rearrange the documentation
mv AUTHORS ChangeLog COPYING INSTALL README UPGRADE doc/
mv ReleaseNotes doc/ReleaseNotes.txt
mv themes/README.themes doc/
for f in `find plugins -name "README*" -or -name INSTALL \
-or -name CHANGES -or -name HISTORY`; do
mkdir -p doc/`dirname $f`
mv $f $_
done
mv doc/plugins/squirrelspell/doc/README doc/plugins/squirrelspell
rmdir doc/plugins/squirrelspell/doc
mv plugins/squirrelspell/doc/* doc/plugins/squirrelspell
rm -f doc/plugins/squirrelspell/index.php
rmdir plugins/squirrelspell/doc
perl -pi -e "s/\.\.//g" doc/index.html
# Fixup various files
echo "left_refresh=300" >> data/default_pref
for f in contrib/RPM/squirrelmail.cron contrib/RPM/config.php.redhat; do
perl -pi -e "s|__ATTDIR__|%{_localstatedir}/spool/squirrelmail/attach/|g;"\
-e "s|__PREFSDIR__|%{_localstatedir}/lib/squirrelmail/prefs/|g;" $f
done
# Fix the version
%{__perl} -pi -e "s|^(\s*\\\$version\s*=\s*'[^']+)'|\1-%{release}'|g"\
functions/strings.php
# replace splash screen
%if %{fedora_splash}
cp %{SOURCE2} images/sm_logo.png
%else
cp %{SOURCE3} images/sm_logo.png
%endif
%install
rm -rf $RPM_BUILD_ROOT
mkdir -p -m0755 $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail
mkdir -p -m0755 $RPM_BUILD_ROOT%{_localstatedir}/lib/squirrelmail/prefs
mkdir -p -m0755 $RPM_BUILD_ROOT%{_localstatedir}/spool/squirrelmail/attach
mkdir -p -m0755 $RPM_BUILD_ROOT%{_datadir}/squirrelmail
mkdir -p -m0755 $RPM_BUILD_ROOT%{contentdir}/html
mkdir -p -m0755 $RPM_BUILD_ROOT%{_sysconfdir}/cron.daily
# install default_pref
install -m 0644 data/default_pref \
$RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/
ln -s ../../../..%{_sysconfdir}/squirrelmail/default_pref \
$RPM_BUILD_ROOT%{_localstatedir}/lib/squirrelmail/prefs/default_pref
# install the config files
mkdir -p -m0755 $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config
install -m 0644 config/*.php $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/
rm -f $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config_local.php
install -m 0644 config/config_local.php \
$RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config_local.php
ln -s ../../../..%{_sysconfdir}/squirrelmail/config_local.php \
$RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config_local.php
install -m 0644 contrib/RPM/config.php.redhat \
$RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config.php
ln -s ../../../..%{_sysconfdir}/squirrelmail/config.php \
$RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config.php
install -m 0755 config/*.pl $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/
# set default_folder_prefix to be dovecot compatible
echo -e "\n\$default_folder_prefix = '';" >> $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config_local.php
# install index.php
install -m 0644 index.php $RPM_BUILD_ROOT%{_datadir}/squirrelmail/
# copy over the rest
for d in class functions help images include locale plugins src themes; do
cp -rp $d $RPM_BUILD_ROOT%{_datadir}/squirrelmail/
done
# install the cron script
install -m 0755 contrib/RPM/squirrelmail.cron \
$RPM_BUILD_ROOT/%{_sysconfdir}/cron.daily/
# install the config file
mkdir -p $RPM_BUILD_ROOT%{_sysconfdir}/httpd/conf.d
install -m 644 $RPM_SOURCE_DIR/squirrelmail.conf \
$RPM_BUILD_ROOT%{_sysconfdir}/httpd/conf.d/
# install locales
mkdir locale_tempdir
cd locale_tempdir
tar xfj %SOURCE4
cd squirrelmail.locales
# Convert all locales to utf-8. Not only is this probably the right thing
# to do anyway, but SquirrelMail will corrupt charsets unless the charset
# of the user's locale is a superset of the charset of any mail they reply to
# https://sf.net/tracker/?func=detail&atid=423691&aid=1235345&group_id=311
for LOCALE in `ls locale/` ; do
case $LOCALE in
ja_JP)
# ja_JP uses iso2022-jp for email but euc-jp in its interface.
CHARSET=euc-jp
;;
ko_KR)
# Not really, but I can't work out what charset the ko_KR help
# files are in, so we'll just leave it alone for now.
CHARSET=utf-8
;;
*)
CHARSET=`grep CHARSET locale/$LOCALE/setup.php | cut -f6 -d\'`
;;
esac
# Check for locales where CHARSET isn't in LOCALE.
grep LOCALE locale/$LOCALE/setup.php | grep -vi $CHARSET || :
if [ "$CHARSET" != "utf-8" -a "$CHARSET" != "UTF-8" ]; then
for a in `ls help/$LOCALE/ 2>/dev/null` ; do
iconv -f $CHARSET -t utf-8 help/$LOCALE/$a > $a.new && mv $a.new help/$LOCALE/$a
done
sed -e "s/CHARSET..[ ]*= [^;]*;/CHARSET'] = 'utf-8';/" \
-e "s/LOCALE..[ ]*= [^;]*;/LOCALE'] = '$LOCALE.UTF-8';/" \
locale/$LOCALE/setup.php > setup.php.new ; mv setup.php.new locale/$LOCALE/setup.php
fi
done
# do the pofiles separately since they each specify their own charset
for POFILE in `find locale -name \*.po` ; do
CHARSET=`grep charset= $POFILE | cut -f2 -d= | cut -f1 -d\\\\`
if [ "$CHARSET" != "utf-8" -a "$CHARSET" != "UTF-8" ]; then
sed s/charset=$CHARSET/charset=utf-8/ $POFILE | iconv -f $CHARSET -t utf-8 > $POFILE.new && mv $POFILE.new $POFILE
fi
done
./compilelocales
#find -name '*.mo' |xargs rm
cp -r locale/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/locale/
cp -r images/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/images/
cp -r help/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/help/
cd ../..
rm $RPM_BUILD_ROOT%{_datadir}/squirrelmail/locale/README.locales
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root)
%config %dir %{_sysconfdir}/squirrelmail
%attr(640,root,apache) %config(noreplace) %{_sysconfdir}/squirrelmail/*.php
%attr(640,root,apache) %config(noreplace) %{_sysconfdir}/squirrelmail/default_pref
%config(noreplace) %{_sysconfdir}/httpd/conf.d/*.conf
%doc doc/*
%dir %{_datadir}/squirrelmail
%dir %{_localstatedir}/lib/squirrelmail
%dir %{_localstatedir}/spool/squirrelmail
%{_datadir}/squirrelmail/class
%{_datadir}/squirrelmail/config
%{_datadir}/squirrelmail/functions
%{_datadir}/squirrelmail/help
%{_datadir}/squirrelmail/images
%{_datadir}/squirrelmail/include
%{_datadir}/squirrelmail/locale
%{_datadir}/squirrelmail/plugins
%{_datadir}/squirrelmail/src
%{_datadir}/squirrelmail/themes
%{_datadir}/squirrelmail/index.php
%attr(0700, apache, apache) %dir %{_localstatedir}/lib/squirrelmail/prefs
%attr(0700, apache, apache) %dir %{_localstatedir}/spool/squirrelmail/attach
%{_localstatedir}/lib/squirrelmail/prefs/default_pref
%{_sysconfdir}/cron.daily/squirrelmail.cron
%changelog
* Mon Sep 12 2005 David Woodhouse <dwmw2 at redhat.com> 1.4.6-0.cvs20050812.2
- Convert all locales to UTF-8 instead of legacy character sets to
work around bug #162852. Except for ko_KR, because iconv doesn't
believe its help files are actually in EUC-KR as claimed.
* Sun Aug 14 2005 Warren Togami <wtogami at redhat.com> 1.4.6-0.cvs20050812.1
- snapshot of 1.4.6 because 1.4.5 upstream was a bad release
this hopefully will also work on PHP5 too...
* Mon Jun 20 2005 Warren Togami <wtogami at redhat.com> 1.4.5-0.rc1
- 1.4.5-0.rc1
* Thu Jan 27 2005 Warren Togami <wtogami at redhat.com> 1.4.4-2
- 1.4.4
- re-include translations and Provide squirrelmail-i18n
better compatible with upstream, but we cannot split sub-package
due to support of existing distributions
- remove unnecessary .po files
* Fri Nov 19 2004 Warren Togami <wtogami at redhat.com> 1.4.3a-7
- CAN-2004-1036 Cross Site Scripting in encoded text
- #112769 updated splash screens
* Thu Oct 14 2004 Warren Togami <wtogami at redhat.com> 1.4.3a-5
- default_folder_prefix dovecot compatible by default
/etc/squirrelmail/config_local.php if you must change it
* Wed Oct 13 2004 Warren Togami <wtogami at redhat.com> 1.4.3a-4
- HIGASHIYAMA Masato's patch to improve Japanese support
(coordinated by Scott A. Hughes).
- real 1.4.3a tarball
* Tue Sep 21 2004 Gary Benson <gbenson at redhat.com> 1.4.3-3
- rebuilt.
* Tue Aug 31 2004 Warren Togami <wtogami at redhat.com> 1.4.3-2
- #125638 config_local.php and default_pref in /etc/squirrelmail/
to match upstream RPM. This should allow smoother drop-in
replacements and upgrades.
- other spec cleanup.
* Mon Jun 7 2004 Gary Benson <gbenson at redhat.com> 1.4.3-1
- upgrade to 1.4.3a.
- retain stuff after version when adding release to it.
* Wed Jun 2 2004 Gary Benson <gbenson at redhat.com>
- upgrade to 1.4.3.
* Fri Feb 13 2004 Elliot Lee <sopwith at redhat.com>
- rebuilt.
* Wed Jan 21 2004 Gary Benson <gbenson at redhat.com> 1.4.2-2
- fix calendar plugin breakage (#113902).
* Thu Jan 8 2004 Gary Benson <gbenson at redhat.com> 1.4.2-1
- upgrade to 1.4.2.
- tighten up permissions on /etc/squirrelmail/config.php (#112774).
* Mon May 12 2003 Gary Benson <gbenson at redhat.com> 1.4.0-1
- upgrade to 1.4.0.
- fix links in /usr/share/doc/squirrelmail-X.Y.Z/index.html (#90269).
* Mon Mar 24 2003 Gary Benson <gbenson at redhat.com> 1.2.11-1
- upgrade to 1.2.11 to fix CAN-2003-0160.
* Mon Feb 10 2003 Gary Benson <gbenson at redhat.com> 1.2.10-4
- fix syntax error in download.php (#82600).
- resized splash screen to be the same size as the one it replaces
(#82790)
- remove piece of squirrelmail-1.2.10-xss.patch that changed the
version from '1.2.10' to '1.2.11 [cvs]'
* Wed Jan 22 2003 Tim Powers <timp at redhat.com> 1.2.10-3
- rebuilt
* Wed Jan 15 2003 Tim Powers <timp at redhat.com> 1.2.10-2
- bump and rebuild
* Mon Dec 9 2002 Gary Benson <gbenson at redhat.com> 1.2.10-1
- patch to fix CAN-2002-1341 (#78982) and CAN-2002-1276 (#79147).
* Tue Dec 03 2002 Elliot Lee <sopwith at redhat.com> 1.2.8-2
- fix prep macro in changelog
* Fri Sep 20 2002 Gary Benson <gbenson at redhat.com> 1.2.8-1
- upgrade to 1.2.8 to fix CAN-2002-1131 and CAN-2002-1132 (#74313)
* Tue Aug 6 2002 Preston Brown <pbrown at redhat.com> 1.2.7-4
- replacement splash screen.
* Mon Jul 22 2002 Gary Benson <gbenson at redhat.com> 1.2.7-3
- get rid of long lines in the specfile.
- remove symlink in docroot and use an alias in conf.d instead.
- work with register_globals off (#68669)
* Tue Jul 09 2002 Gary Benson <gbenson at redhat.com> 1.2.7-2
- hardwire the hostname (well, localhost) into the config file (#67635)
* Mon Jun 24 2002 Gary Benson <gbenson at redhat.com> 1.2.7-1
- hardwire the locations into the config file and cron file.
- install squirrelmail-cleanup.cron as squirrelmail.cron.
- make symlinks relative.
- upgrade to 1.2.7.
- more dependency fixes.
* Fri Jun 21 2002 Gary Benson <gbenson at redhat.com>
- summarize the summary, fix deps, and remove some redundant stuff.
- tidy up the prep section.
- replace directory definitions with standard RHL ones.
* Fri Jun 21 2002 Tim Powers <timp at redhat.com> 1.2.6-3
- automated rebuild
* Wed Jun 19 2002 Preston Brown <pbrown at redhat.com> 1.2.6-2
- adopted Konstantin Riabitsev <icon at duke.edu>'s package for Red Hat
Linux. Nice job Konstantin!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20051027/188421ab/attachment.sig>
More information about the fedora-devel-list
mailing list