SeaMonkey Mail and text encoding

Post by Alexandre Yudenitsch
Maybe someone reading this will understand and recognize the problems I
describe below, and know enough about SM/Mail to comment on them; I
realize that, probably, most will never have encountered something like
it, because it happens when you write and receive messages in different
text encoding options, and languages different from English...
First, a quick round-up of what I've managed to find involving text
...
Now, the problems: I send and receive e-mails in several languages (but
mostly English), ...

So do I. In my case, it's Russian (русский язык) and Korean (한국어).
Both have local encoding specifications such as KOI8-R and EUC-KR, but
are also amenable to Unicode.

Post by Alexandre Yudenitsch
First, when a message is received, the "auto-detect" is mostly worse
than useless, so I usually start it as "Unicode" or "Western", depending
on history ...

For incoming messages, this is usually the fault of the sender. If the
message is well-formed (specifies the encoding actually used, and that
specification is in the standard format), SM will recognize and
understand the specification and apply it. However, some senders live in
countries where everyone assumes that all messages are in the same local
encoding, so their email programs don't even bother to state the
encoding. If your receiving program doesn't make the same assumption, it
has to guess. And sometimes the sending program "lies" -- it says it
uses one encoding but actually uses another.

There are also things you can do at your end to sabotage the display of
well-formed messages.

The most common error on the receiving side is to specify a font that
doesn't work for the incoming encoding. For example, if you have
SeaMonkey set to display Thai text in a font that does not contain Thai
glyphs, you'll get garbage when you receive a well-formed Thai message.
Well, by garbage I mean you'll probably get boxes, or stuff like this: .

To see if this is the case, go to Edit | Preferences | Appearance |
Fonts, where you will see a pull-down list:
Fonts: [Western]
Below that, you'll see a list of six fonts for various conditions:
proportional, serif, sans-serif, etc.

Instead of "Western" at the top, pull down the encoding in question.
Let's suppose you choose "Cyrillic." In the list of six fonts that
follows, do all six fonts support Cyrillic? In other words, do those
fonts contain Cyrillic glyphs? If not, change to fonts that do.

Do this for all the encodings you typically receive. You can specify
different "sans-serif" fonts for different encodings -- e.g., Arial for
Cyrillic but DotumChe for Korean.

* * *

On the other hand, if SeaMonkey is using the wrong encoding to decipher
a message, instead of boxes you'll typically get "mojibake," which the
Russians call "крякозябры" and the French call "hiéroglyphes." Is that
what you're seeing?
<https://en.wikipedia.org/wiki/Mojibake>

A common reason that SeaMonkey could choose the wrong encoding (other
than a sender whose computer lies about the encoding it used) is if the
user gives it improper guidance. For example, if you tell it "no matter
what you see, display it in Western encoding," messages with foreign
characters will be corrupted.

One place you should look is under Edit | Preferences | Appearance |
Fonts -- have you checked the box, "Allow documents to use other fonts"?
That's fine if you have lots of fonts installed, but if an incoming
message specifies a font you don't have, your system will have to
substitute something it does have, and that doesn't always work. I have
it checked, and it hasn't caused problems. Things may be different for
your correspondents.

For mixed-content messages, I recommend specifying Unicode when you
begin drafting (in the composition window, Options | Text Encoding |
Unicode). My implementation of SM does that on its own for all outgoing
messages (both plain text and HTML), which is very convenient. After
all, the definition of Unicode is that it supports all languages. You're
right that the encoding of an incoming message must be selected
correctly before you reply.

I have noticed that sometimes SM will guess wrong when I select a
message in a mail folder, but if I navigate away and then return it'll
guess right. I don't know why that is -- it seems to be sticking to the
encoding it used for the previous message that I just deleted.

HTH a little.

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-10 04:15:17 UTC

Hi, Paul B. Gallagher! Thanks for your quick reply, on 09 Oct 16 21:39!

A happy coincidence that you decided to do so, since I already had the
intention of writing you, to ask for details about your migration of SM
from XP to W7 -- but let's leave that for a little later, OK?

I send and receive e-mails in several languages (but mostly English), ...

So do I. In my case, it's Russian (русский язык) and Korean (한국어).
Both have local encoding specifications such as KOI8-R and EUC-KR, but
are also amenable to Unicode.

That's part of the problem, and why I took the time to list "what I've
managed to find involving text encoding in SeaMonkey Mail": There are
no explanations or specifications for most of these, so one has to
guess... I do speak/write Russian, but so far have had no need to use
it in e-mails, and have no correspondents who use anything except
Unicode and what I THOUGHT could be "Western" characters -- but what's
that: Is that an euphemism for "English"? Portuguese, Spanish, French,
all use "Western characters", IMHO; but I suspect the former (ie,
"Western") actually means "English"...

Assuming that's true, then in my case there would be 3 types of text
encoding: Western/English, Unicode (and I don't know if 'all Unicodes
are created equal'), and "other Western codes", like Portuguese,
Spanish, French, etc.

First, when a message is received, the "auto-detect" is mostly worse
than useless, so I usually start it as "Unicode" or "Western", depending
on history ...

For incoming messages, this is usually the fault of the sender.

Paraphrasing your standard 'footer', "War doesn't determine who's right,
just who's left", I don't care whose 'fault' it is (something like those
who hold SM as 'right' because it follows standards better than MS IE:
Since most sites still code 'for IE', I prefer not to be "dead right",
but instead to be able to read the texts I want with a minimum fuss --
and, in such cases, ignoring what the majority does is self-defeating.

Post by Paul B. Gallagher
If the message is well-formed (specifies the encoding actually used,
and that specification is in the standard format), SM will recognize
and understand the specification and apply it. However, some senders
live in countries where everyone assumes that all messages are in the
same local encoding, so their email programs don't even bother to
state the encoding. If your receiving program doesn't make the same
assumption, it has to guess. And sometimes the sending program "lies"
-- it says it uses one encoding but actually uses another.
I have noticed that sometimes SM will guess wrong when I select a
message in a mail folder, but if I navigate away and then return
it'll guess right. I don't know why that is -- it seems to be
sticking to the encoding it used for the previous message that I just
deleted.

If SM guessed right 98% of the time, I'd think it's worth it; but, since
it's frequently wrong (in over 30% of the cases, in my experience, I
think), I'd prefer if there were a 'preference' to turn the 'guessing' off!

Post by Paul B. Gallagher
There are also things you can do at your end to sabotage the display of
well-formed messages.
The most common error on the receiving side is to specify a font that
doesn't work for the incoming encoding. For example, if you have
SeaMonkey set to display Thai text in a font that does not contain Thai
glyphs, you'll get garbage when you receive a well-formed Thai message.
Well, by garbage I mean you'll probably get boxes, or stuff like this: .
To see if this is the case, go to Edit | Preferences | Appearance |
Fonts: [Western]
proportional, serif, sans-serif, etc.
Instead of "Western" at the top, pull down the encoding in question.
Let's suppose you choose "Cyrillic." In the list of six fonts that
follows, do all six fonts support Cyrillic? In other words, do those
fonts contain Cyrillic glyphs? If not, change to fonts that do.
Do this for all the encodings you typically receive. You can specify
different "sans-serif" fonts for different encodings -- e.g., Arial for
Cyrillic but DotumChe for Korean.
On the other hand, if SeaMonkey is using the wrong encoding to decipher
a message, instead of boxes you'll typically get "mojibake," which the
Russians call "крякозябры" and the French call "hiéroglyphes." Is that
what you're seeing?

Your comments about fonts make me question if that might have something
to do with the problem: Notice that I didn't list the "Fonts" options
in "Preferences / Appearance" so, even though I did mark "Allow
documents to use other fonts", and I have hundreds of installed fonts,
that might still be a problem... But, since I only send 'text-only'
messages (even in reply to HTML), and use the 'standard' "Courier New"
font for that, AND when I write messages all the chgaracters are
displayed in the fonts I use, I don't see how that might be the cause --
but, once more, maye that's just my ignorance talking...

Post by Paul B. Gallagher
A common reason that SeaMonkey could choose the wrong encoding (other
than a sender whose computer lies about the encoding it used) is if the
user gives it improper guidance. For example, if you tell it "no matter
what you see, display it in Western encoding," messages with foreign
characters will be corrupted.
One place you should look is under Edit | Preferences | Appearance |
Fonts -- have you checked the box, "Allow documents to use other fonts"?
That's fine if you have lots of fonts installed, but if an incoming
message specifies a font you don't have, your system will have to
substitute something it does have, and that doesn't always work. I have
it checked, and it hasn't caused problems. Things may be different for
your correspondents.

As i said, guessing the correct coding to display an incoming e-mail
isn't a big problem, for me: If I guess wrong, I just have to re-guess
and start over (or anm I missing something here, also? Since I'm using
text-only for display and composing, and only have 3 types of text
encoding, (Western/English, Unicode, and "other Western codes" like
Portuguese), I thought that the first 2 options (Western and Unicode)
would cover 100 of my received e-mails.

Post by Paul B. Gallagher
For mixed-content messages, I recommend specifying Unicode when you
begin drafting (in the composition window, Options | Text Encoding |
Unicode). My implementation of SM does that on its own for all outgoing
messages (both plain text and HTML), which is very convenient. After
all, the definition of Unicode is that it supports all languages. You're
right that the encoding of an incoming message must be selected
correctly before you reply.
when I compose a large message with parts in English and parts in
another language, and make several drafts: Each time the draft is
re-opened for further editing, parts of it have been changed by SM,
usually changing some non-English characters into gibberish (Not
Unicode!); what's particularly puzzling is that the same character
may be changed in some parts of the message and not in others, and
where/when this happens seems almost random.
I assume there must be some 'automatic processing' going on (like the
"auto-detect" above), but see no way to turn it off; I have tried
changing all the above options to "unicode", or to "English", but
nothing seems to help...

This seems to ba relatively recent problem: In the past, I rarely
noticed this; but, since these messages have gotten longer, and are very
frequently re-drafted, parts of them (and only PARTS, which I find even
more mysterious) turn into gibberish (and not boxes, or stuff like ,
or even "крякозябры" -- a mouthful!). An example:

"ç" might become "Р“В§", or even (with repeated drafts) "Р“С“Р’С’
Р“С“Р’СћР“СћРІР‚С™Р’В¬Р“вЂ¦РІР‚СљР“С“Р’С’Р“СћРІвЂљВ¬РІвЂћСћР“С“РІР‚С™Р“вЂљР’В§"
(I'm not kidding!!)

I mean: If I can write "ç" when composing (like I did right now), and
just save the draft and re-open it, the reason for the gibberish
shouldn't be the font, right? I normally write and send messages with
these characters without any problem (as far as I know: Usually, no-one
complains), it seems to happen only with very long messages which mix
English and another language, which is why I suspect that SM 'guesses'
each time a draft is saved -- and, oveerr dozens of 'saves', it gets it
wrong part of the time.

For a relatively long time, I was able to stop this behaviour by always
choosing the "View | Text Encoding" before opening these drafts (and,
many times, when I forgot to do that, gibberish appeared) but, lately,
even when doing that, the gibberish still appears, and gets worse with
each 'save': Perhaps there have been some 'tweaks' to SM's "encoding
guessing" algorithms?

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Ralph Fox

2016-10-10 08:23:08 UTC

Post by Paul B. Gallagher
when I compose a large message with parts in English and parts in
another language, and make several drafts: Each time the draft is
re-opened for further editing, parts of it have been changed by SM,
usually changing some non-English characters into gibberish (Not
Unicode!); what's particularly puzzling is that the same character
may be changed in some parts of the message and not in others, and
where/when this happens seems almost random.
I assume there must be some 'automatic processing' going on (like the
"auto-detect" above), but see no way to turn it off; I have tried
changing all the above options to "unicode", or to "English", but
nothing seems to help...

This seems to ba relatively recent problem: In the past, I rarely
noticed this; but, since these messages have gotten longer, and are very
frequently re-drafted, parts of them (and only PARTS, which I find even
more mysterious) turn into gibberish (and not boxes, or stuff like ,
"ç" might become "Р“В§", or even (with repeated drafts) "Р“С“Р’С’
Р“С“Р’СћР“СћРІР‚С™Р’В¬Р“вЂ¦РІР‚СљР“С“Р’С’Р“СћРІвЂљВ¬РІвЂћСћР“С“РІР‚С™Р“вЂљР’В§"
(I'm not kidding!!)
I mean: If I can write "ç" when composing (like I did right now), and
just save the draft and re-open it, the reason for the gibberish
shouldn't be the font, right? I normally write and send messages with
these characters without any problem (as far as I know: Usually, no-one
complains), it seems to happen only with very long messages which mix
English and another language, which is why I suspect that SM 'guesses'
each time a draft is saved -- and, oveerr dozens of 'saves', it gets it
wrong part of the time.
For a relatively long time, I was able to stop this behaviour by always
choosing the "View | Text Encoding" before opening these drafts (and,
many times, when I forgot to do that, gibberish appeared) but, lately,
even when doing that, the gibberish still appears, and gets worse with
each 'save': Perhaps there have been some 'tweaks' to SM's "encoding
guessing" algorithms?

Go to about:config [^1][^2] and make sure that the preference
mailnews.force_charset_override is set to false.

What you describe will happen with your own drafts when
mailnews.force_charset_override is set to true.

SeaMonkey does not need to guess the encoding of its own drafts.
SeaMonkey writes the encoding in the "Content-Type" header of the draft.
When you re-open the draft, SeaMonkey will recognize the encoding from
the "Content-Type" header UNLESS the preference
mailnews.force_charset_override is set to true.

The only way in which I can make SeaMonkey not recognize the encoding of
its own drafts is to set mailnews.force_charset_override to true.

REFERENCES
[^1] http://seamonkey.ilias.ca/customizing/
[^2] http://kb.mozillazine.org/about:config

--
Kind regards
Ralph

Alexander Yudenitsch

2016-10-10 14:51:06 UTC

Post by Ralph Fox

This seems to ba relatively recent problem: In the past, I rarely
noticed this; but, since these messages have gotten longer, and are very
frequently re-drafted, parts of them (and only PARTS, which I find even
more mysterious) turn into gibberish (and not boxes, or stuff like ,
"ç" might become "Р“В§", or even (with repeated drafts) "Р“С“Р’С’
Р“С“Р’СћР“СћРІР‚С™Р’В¬Р“вЂ¦РІР‚СљР“С“Р’С’Р“СћРІвЂљВ¬РІвЂћСћР“С“РІР‚С™Р“вЂљР’В§"
(I'm not kidding!!)
I mean: If I can write "ç" when composing (like I did right now), and
just save the draft and re-open it, the reason for the gibberish
shouldn't be the font, right? I normally write and send messages with
these characters without any problem (as far as I know: Usually, no-one
complains), it seems to happen only with very long messages which mix
English and another language, which is why I suspect that SM 'guesses'
each time a draft is saved -- and, oveerr dozens of 'saves', it gets it
wrong part of the time.
For a relatively long time, I was able to stop this behaviour by always
choosing the "View | Text Encoding" before opening these drafts (and,
many times, when I forgot to do that, gibberish appeared) but, lately,
even when doing that, the gibberish still appears, and gets worse with
each 'save': Perhaps there have been some 'tweaks' to SM's "encoding
guessing" algorithms?

Go to about:config [^1][^2] and make sure that the preference
mailnews.force_charset_override is set to false.
What you describe will happen with your own drafts when
mailnews.force_charset_override is set to true.
SeaMonkey does not need to guess the encoding of its own drafts.
SeaMonkey writes the encoding in the "Content-Type" header of the draft.
When you re-open the draft, SeaMonkey will recognize the encoding from
the "Content-Type" header UNLESS the preference
mailnews.force_charset_override is set to true.
The only way in which I can make SeaMonkey not recognize the encoding of
its own drafts is to set mailnews.force_charset_override to true.
REFERENCES
[^1] http://seamonkey.ilias.ca/customizing/
[^2] http://kb.mozillazine.org/about:config

Thanks for the post! My heart skipped a beat when I read it, since it
indicated a simple, specific reason for the problem, and an easy
solution... Unfortunately, it was a false hope, since that preference
IS set to 'false' on my system (and I don't know why it would deviate
from the default, since I never changed it).

So, it seems that there must be some OTHER "way [to] make SeaMonkey not
recognize the encoding of its own drafts"... Let's hope someone can
read this and offer another suggestion -- and thanks for your help.

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-10 20:49:30 UTC

Post by Alexander Yudenitsch
Hi, Paul B. Gallagher! Thanks for your quick reply, on 09 Oct 16 21:39!
A happy coincidence that you decided to do so, since I already had
the intention of writing you, to ask for details about your migration
of SM from XP to W7 -- but let's leave that for a little later, OK?

I can't promise to remember the details of what I did so long ago...

IIRC, I upgraded the OS from XP to Win2000 while SM was already
installed, and it just ran without further action on my part. And then I
did the same thing again when going from Win2000 to Win7.

But don't quote me -- it's been over a decade, and I can't swear that
I'm reembering correctly.

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-10 22:20:41 UTC

Post by Alexander Yudenitsch
A happy coincidence that you decided to do so, since I already had
the intention of writing you, to ask for details about your migration
of SM from XP to W7 -- but let's leave that for a little later, OK?

I can't promise to remember the details of what I did so long ago...
IIRC, I upgraded the OS from XP to Win2000 while SM was already
installed, and it just ran without further action on my part. And then I
did the same thing again when going from Win2000 to Win7.
But don't quote me -- it's been over a decade, and I can't swear that
I'm reembering correctly.

I distinctly remember someone on this forum commenting on how he (yes,
it was a 'he', AFAIK) had migrated a profile from XP to W7, and I
thought that was you; well, apparently not... I did try to find that
message among the hundreds of the 'current' batch but, not remembering
what was the 'nominal' subject, I guess its hopeless.

Don't you mean that you "upgraded the OS from W2K to WXP while SM was
already installed, and it just ran without further action on [your]
part"? After all, XP came after 2k (which came after W95, all of which
I used in turn)... And, yes, going from W2K to WXP was nearly
effortless (as I remember, the main difference was that instead of a
WINNT directory, we had a WINDOWS one), but I think you misremember:
The directory structure in W7 is quite different from the WXP one
(although they did add 'link-like directories' so you can use old-style
addresses.

I made a 'trial install' of SM in W7, just to see where it'd put its
files, and they seem to be 'all over the place' -- but I'm still
learning the ropes concerning this, which is why I'd like to know
exactly what others did, migrating SM from XP to W7.

BTW, I think my SM profile has accumulated several glitches over the
years (it's basically the same one since Netscape..): Lately, I've been
having problems (apparently involving javascript) on some sites, which
work fine on a copy of FireFox I installed for exactly such cases -- so,
I was thinking about creating a fresh new profile in W7, and then
migrating what I need, and reinstalling the extensions which work better
that way, but I'm afraid I'll never catch all the 'ramifications' (eg,
there are extensions which place their data in "about:config", which I
only recently discovered), so I"m undecided...

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-11 03:21:16 UTC

Post by Alexander Yudenitsch
Don't you mean that you "upgraded the OS from W2K to WXP while SM was
already installed, and it just ran without further action on [your]
part"? After all, XP came after 2k (which came after W95, all of
which I used in turn)... And, yes, going from W2K to WXP was nearly
effortless (as I remember, the main difference was that instead of a
The directory structure in W7 is quite different from the WXP one
(although they did add 'link-like directories' so you can use
old-style addresses.

Just goes to show how poor my memory of ancient history is. Of course
you're right.

Windows 2000 was released February 17, 2000, and Windows XP was released
October 25, 2001. So XP is approaching its 15th birthday. Computer years
are like dog years (7:1), so that means it's like a 105-year-old human
being. Even my Win7 at six (42) is starting to look long in the tooth.
It still runs, but...

When I buy computer hardware, I do so with the expectation that it will
be obsolete in five years (like a 35-year-old human athlete).

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-11 03:31:40 UTC

Post by Paul B. Gallagher
Windows 2000 was released February 17, 2000, and Windows XP was released
October 25, 2001. So XP is approaching its 15th birthday. Computer years
are like dog years (7:1), so that means it's like a 105-year-old human
being. Even my Win7 at six (42) is starting to look long in the tooth.
It still runs, but...
When I buy computer hardware, I do so with the expectation that it will
be obsolete in five years (like a 35-year-old human athlete).

Well, XP may be an old dog by now, but actually it's still quite
satisfactory for my needs. I'm only migrating to W7 because, nowadays,
many programs don't support (or even work with) XP, so "the writing is
on the wall"...

I got and installed a valid copy of W10 JIC, while the getting was good
(and free), but have no intention of using it in the foreseeable future:
I hope W7 will 'live' for many dog years more...

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

m***@spamgourmet.com

2016-10-11 20:30:31 UTC

I can't promise to remember the details of what I did so long ago...
IIRC, I upgraded the OS from XP to Win2000 while SM was already
installed, and it just ran without further action on my part. And then I
did the same thing again when going from Win2000 to Win7.
But don't quote me -- it's been over a decade, and I can't swear that
I'm reembering correctly.

You can search the archives from this list via Google Groups at
<http://groups.google.com/group/mozilla.support.seamonkey>. A couple of
recent threads involving moving profiles around are titled "If I
remplace the directory C:\Users\RZ\AppData\Roaming\Mozilla\SeaMonkey"
and "Installing SM 2.40 and Relations To Different Drive". You may also
want to take a look at
<http://kb.mozillazine.org/Moving_your_profile_folder>.

If you have queries about migrating your SeaMonkey profile from one OS
to another, the best thing is to create a new message addressed to this
list / newsgroup with an appropriate subject line. Don't reply to a
message from this thread about a different problem, as some people
ignore threads about things they can't help with or aren't interested
in, so those who might be able to help with that different problem might
not see a reply to a message in this thread.

--
Mark.

Alexander Yudenitsch

2016-10-11 22:54:59 UTC

You can search the archives from this list via Google Groups at
<http://groups.google.com/group/mozilla.support.seamonkey>.

Thanks, but I avoid Google Groups if I can; I have signed into them out
of necessity, but I imagine I'd have to sign onto this group in order to
do a search, and I prefer not to do that.

Post by m***@spamgourmet.com
A couple of recent threads involving moving profiles around are
titled "If I remplace the directory
C:\Users\RZ\AppData\Roaming\Mozilla\SeaMonkey" and "Installing SM
2.40 and Relations To Different Drive". You may also want to take a
look at <http://kb.mozillazine.org/Moving_your_profile_folder>.

Thanks, again! I did look around a little, and had seen those threads
also (and one of them was the one I was referring to, in the text quoted
above)

Post by m***@spamgourmet.com
If you have queries about migrating your SeaMonkey profile from one OS
to another, the best thing is to create a new message addressed to this
list / newsgroup with an appropriate subject line. Don't reply to a
message from this thread about a different problem, as some people
ignore threads about things they can't help with or aren't interested
in, so those who might be able to help with that different problem might
not see a reply to a message in this thread.

Of course! Notice that I did say to Paul: "let's leave that for a
little later", because I do have the intention of opening another thread
for that. I just commented on that because I thought he was the one who
had written the messages I had seen about the migration, and thought it
was a coincidence that he was the first one to reply.

In fact, my query has two 'prongs': Migrating SM from WinXP to Win7,
and saving information from a previous profile to a new one (which I'm
sure is a devious and tricky one, since I'm after specific details, and
not generic advice, and realize there may not be anyone with such advice
available here when I write).

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

m***@spamgourmet.com

2016-10-10 18:48:09 UTC

Post by Paul B. Gallagher
I have noticed that sometimes SM will guess wrong when I select a
message in a mail folder, but if I navigate away and then return it'll
guess right. I don't know why that is -- it seems to be sticking to the
encoding it used for the previous message that I just deleted.

I've noticed that too, and it may be the root of Alexandre's problem. It
seems that SeaMonkey correctly detects the encoding when opening
messages in the "message pane", but doesn't actually use that encoding
until the next message is opened, and doesn't detect the encoding at all
when opening them in a separate window (just using the last-detected
encoding from the message pane).

That also affects opening previously saved drafts. So, for example:
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£")
- Save as draft and close that message.
- View, in the message pane (View > Layout > Message Pane or press F8),
another email which uses a different encoding (e.g. us-ascii)
- Re-open the draft
- The "£" now appears as "Â£" (it just happens that in this case the
second character is the same as the original single character, but
that's not necessarily the case for all characters)

No amount of closing and re-opening the draft seems to fix that. However:
- Select the draft in the thread pane
- Open the message pane (View > Layout > Message Pane or press F8)
- It looks wrong there too, and saving the draft again at this point
will compound the problem. However...
- Close the message pane
- Open the message pane
- It now looks correct, and double-clicking to open the message for
further editing is now also correct.

To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed properly.

I think this has come up on this list before, but not sure it's reported
on Bugzilla.

--
Mark.

Alexander Yudenitsch

2016-10-10 19:54:46 UTC

I've noticed that too, and it may be the root of Alexandre's problem. It
seems that SeaMonkey correctly detects the encoding when opening
messages in the "message pane", but doesn't actually use that encoding
until the next message is opened, and doesn't detect the encoding at all
when opening them in a separate window (just using the last-detected
encoding from the message pane).
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£")
- Save as draft and close that message.
- View, in the message pane (View > Layout > Message Pane or press F8),
another email which uses a different encoding (e.g. us-ascii)
- Re-open the draft
- The "£" now appears as "Â£" (it just happens that in this case the
second character is the same as the original single character, but
that's not necessarily the case for all characters)
- Select the draft in the thread pane
- Open the message pane (View > Layout > Message Pane or press F8)
- It looks wrong there too, and saving the draft again at this point
will compound the problem. However...
- Close the message pane
- Open the message pane
- It now looks correct, and double-clicking to open the message for
further editing is now also correct.
To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.
I think this has come up on this list before, but not sure it's reported
on Bugzilla.

Sounds like potentially good news, and thanks!

Since I never use the 'message pane', but always open e-mails (including
drafts) in their own windows, I think the problem may be a little more
'widespread' than that -- but the main point would be that SM seems to
open or show messages using the 'last-used' text encoding, not the one
specified in the e-mail's properties (which, IMHO, should be considered
a bug).

For a relatively long time, I was able to stop this behavior by
always choosing the "View | Text Encoding" before opening these
drafts (and, many times, when I forgot to do that, gibberish
appeared) but, lately, even when doing that, the gibberish still
appears, and gets worse with each 'save': Perhaps there have
been some 'tweaks' to SM's "encoding guessing" algorithms?

I'll 'clean up' the offending draft, and try once again to choose the
correct "View | Text Encoding" before opening it, and see if that stops
the gibberish. Alternatively, on another draft, I'll try the "F8 way"
which mbourne described (since I also never used F8, that'll be a new
one for me, too).

But, is it possible that the matter with "Edit | Preferences |
Appearance | Fonts" which Paul Gallagher described is also a factor in
this problem?

BTW, before I forget: It's been a few years since SM started opening
that 'message pane' when you classify a message as "Junk"; it wasn't
that way earlier, and I don't see any way to turn that behavior off!
Does anyone know some way to do that?

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-10 20:46:10 UTC

Post by Alexander Yudenitsch
I'll 'clean up' the offending draft, and try once again to choose the
correct "View | Text Encoding" before opening it, and see if that
stops the gibberish. Alternatively, on another draft, I'll try the
"F8 way" which mbourne described (since I also never used F8, that'll
be a new one for me, too).
But, is it possible that the matter with "Edit | Preferences |
Appearance | Fonts" which Paul Gallagher described is also a factor
in this problem?

Could be -- a lot of older fonts have very limited character sets.

Post by Alexander Yudenitsch
BTW, before I forget: It's been a few years since

Careful -- this English phrasing normally implies that the stated
phenomenon has NOT occurred during the stated period. For the positive
version, we'd say "For a few years, SM has been...," or else "Several
years ago, SM started..." It doesn't work the same as "depuis quelques
années."

Post by Alexander Yudenitsch
SM started opening that 'message pane' when you classify a message as
"Junk"; it wasn't that way earlier, and I don't see any way to turn
that behavior off! Does anyone know some way to do that?

Haven't seen this. If I select a message in the list and press "J" to
classify it as junk, SM just puts it in the Junk folder without
displaying it. Of course, if I open it to diagnose it and /then/ mark it
as junk, well, it's already open, isn't it? ;-)

Can you precisely describe the circumstances and steps you take so
someone else can reproduce the problem?

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-10 22:40:48 UTC

Post by Alexander Yudenitsch
But, is it possible that the matter with "Edit | Preferences |
Appearance | Fonts" which Paul Gallagher described is also a factor
in this problem?

Could be -- a lot of older fonts have very limited character sets.

Yes, could be... But, as I said, I've been using these same fonts
(they're the standard SM ones!) for many years, and didn't have this
problem until recently.

Post by Alexander Yudenitsch
BTW, before I forget: It's been a few years since

Yes, you're right; I was in a hurry (to go to my POBox and get the
bottles of wine waiting there for me since Friday)...

Haven't seen this. If I select a message in the list and press "J" to
classify it as junk, SM just puts it in the Junk folder without
displaying it. Of course, if I open it to diagnose it and /then/ mark it
as junk, well, it's already open, isn't it? ;-)
Can you precisely describe the circumstances and steps you take so
someone else can reproduce the problem?

Well, first of all, I don't allow SM to move messages around on its own:
I like and value very much its mail filter (as I'm fond of telling
friends, it even manages to distinguish 'ad-only editions' of a
newsletter from 'normal' ones!), so I let it mark all messages IT
'thinks' are junk, and then I go through my Inbox checking -- and,
usually, some of its Junk/Spam aren't ones I so consider, several ARE
junk but I still want to keep them, and several others are ones I
consider junk/spam, so I mark them as such with a right-click, and
THAT's when SM opens the message pane!

Also, as I commented in another message, I never use the 'message pane'
to read e-mails, always opening them in new windows -- but I'm talking
about cases (the majority, actually) where I decide the message should
classified as junk/spam based only on the subject, sender and recipient,
so it's NOT opened (in case you're wondering, when I'm suspicious about
some message, I just look at its source, thus avoiding triggering any
unknown 'nasties').

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-11 03:35:27 UTC

Post by Paul B. Gallagher
Haven't seen this. If I select a message in the list and press "J" to
classify it as junk, SM just puts it in the Junk folder without
displaying it. Of course, if I open it to diagnose it and /then/ mark it
as junk, well, it's already open, isn't it? ;-)
Can you precisely describe the circumstances and steps you take so
someone else can reproduce the problem?

I like and value very much its mail filter (as I'm fond of telling
friends, it even manages to distinguish 'ad-only editions' of a
newsletter from 'normal' ones!), so I let it mark all messages IT
'thinks' are junk, and then I go through my Inbox checking -- and,
usually, some of its Junk/Spam aren't ones I so consider, several ARE
junk but I still want to keep them, and several others are ones I
consider junk/spam, so I mark them as such with a right-click, and
THAT's when SM opens the message pane!

The usual point of filters (not the same as Junk Mail Control) is to
move incoming messages to appropriate folders. So messages from my bank
go in my bank's folder, messages from my accountant go in my
accountant's folder, messages from my brother go in my Personal folder,
etc. I don't clog up my Inbox by letting everything stay there and
moving it manually. And I can see at a glance where the new messages are
because any folder name with unread messages is shown in bold.

In the same way, I do want SM to move junk messages to the Junk folder,
which then lights up in bold to remind me to check them. In most cases,
I can take one look at the subject line and agree that it's junk -- if
I've just inherited $23 million from some Nigerian prince, or gained the
opportunity to improve my size and performance, I don't have to open it.
But for those that I do want to inspect, I open them with one crucial
setting:
Edit | Preferences | Mail & Newsgroups | Message Display
...
[x] Block images and other content from remote sources.

This way, the message doesn't "phone home" for a web beacon to notify
the spammer that I've opened the message.

Of course, you'll also want to set SM not to run scripts and plugins in
mail messages:
Edit | Preferences | Advanced | Scripts & Plugins
Enable plugins for [ ] Mail & Newsgroups
(JavaScript is disabled by default for the "Mail & Newsgroups"
component, for security reason.)

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-11 06:02:54 UTC

The usual point of filters (not the same as Junk Mail Control) is...

I used "filtering' in a general sense: After all, analyzing a message
and deciding it's junk/spam and/or from your brother is, basically the

... to move incoming messages to appropriate folders. So messages
from my bank go in my bank's folder, messages from my accountant go
in my accountant's folder, messages from my brother go in my Personal
folder, etc. I don't clog up my Inbox by letting everything stay
there and moving it manually. And I can see at a glance where the new
messages are because any folder name with unread messages is shown in
bold.
In the same way, I do want SM to move junk messages to the Junk folder,
which then lights up in bold to remind me to check them. In most cases,
I can take one look at the subject line and agree that it's junk -- if
I've just inherited $23 million from some Nigerian prince, or gained the
opportunity to improve my size and performance, I don't have to open it.

Once more, I think our intents are the same, but we go about them in
different ways: The "one look" you described is the same I take to
decide if something should be (there is no "is", here, just a "should
be", IMHO) classified as junk/spam, bank/personal/etc. and so on -- only
I prefer to do that, several times a day, in the Inbox (which, with this
procedure, never has more than 20 items in it at any given time), and
you let SM place the messages in folders, and then look at them there
(we both use the 'bold=unread' feature, too.

But for those that I do want to inspect, I open them with one crucial
Edit | Preferences | Mail & Newsgroups | Message Display
...
[x] Block images and other content from remote sources.
This way, the message doesn't "phone home" for a web beacon to notify
the spammer that I've opened the message.

I do have that set, but I still prefer to examine the 5% or so
'suspicious' messages by looking at the source, since I imagine here
might be some new technique whereby a hacker can circumvent this setting
-- and learning about it after some dastardly deed has been done is
small comfort.

Of course, you'll also want to set SM not to run scripts and plugins in
Edit | Preferences | Advanced | Scripts & Plugins
Enable plugins for [ ] Mail & Newsgroups
(JavaScript is disabled by default for the "Mail & Newsgroups"
component, for security reason.)

Yes, that's set too. As I said, I think we have similar strategies, but
differ in some tactics...

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-11 07:10:29 UTC

Post by Paul B. Gallagher
In the same way, I do want SM to move junk messages to the Junk
folder, which then lights up in bold to remind me to check them. In
most cases, I can take one look at the subject line and agree that
it's junk -- if I've just inherited $23 million from some Nigerian
prince, or gained the opportunity to improve my size and
performance, I don't have to open it.

Once more, I think our intents are the same, but we go about them in
different ways: The "one look" you described is the same I take to
decide if something should be (there is no "is", here, just a "should
be", IMHO) classified as junk/spam, bank/personal/etc. and so on --
only I prefer to do that, several times a day, in the Inbox (which,
with this procedure, never has more than 20 items in it at any given
time), and you let SM place the messages in folders, and then look at
them there (we both use the 'bold=unread' feature, too.

If I were only getting 20 messages a day, I might try your way. As it
happens, I routinely get over a hundred, sometimes 200, so filtering to
appropriate folders is a great time saver. If messages 3, 28, 47, and 96
relate to the same topic, it's much easier to see all four in the same
folder and read them together than to try to keep several dozen threads
in my head all at the same time as I work through an unsorted inbox.

I also get client requests referring to old jobs, some of them a year or
two old, and I have to be able to find the old correspondence quickly
and easily. I probably have a hundred thousand old messages going back
to the late 90s, and the only way I can cope is with a good filing system.

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-11 18:14:07 UTC

Post by Paul B. Gallagher
If I were only getting 20 messages a day, I might try your way. As it
happens, I routinely get over a hundred, sometimes 200, so filtering to
appropriate folders is a great time saver. If messages 3, 28, 47, and 96
relate to the same topic, it's much easier to see all four in the same
folder and read them together than to try to keep several dozen threads
in my head all at the same time as I work through an unsorted inbox.
I also get client requests referring to old jobs, some of them a year or
two old, and I have to be able to find the old correspondence quickly
and easily. I probably have a hundred thousand old messages going back
to the late 90s, and the only way I can cope is with a good filing system.

Yeah, "You can't really understand another person's experience until
you've walked a mile in their shoes" (and, by then, you're a mile away,
and you have their shoes... :-))

I guess that, on average, I must get 50 messages a day, but most of them
are very quickly and easily taken care of every day using an e-mails
folders structure adequate to my needs and SM's spam/junk filters and
Boolean search (which, BTW, is unparalleled, at least for me: I've
never seen any e-mail client with search capabilities even close to
these!); I imagine that, if I got 100-200/day, maybe I'd start using
folder filtering, too...

BTW, I just remembered that I do use folder filtering, but on my e-mail
service (fastmail.fm): Many years ago, I wrote a filtering script so
that e-mails sent to that address would be sent to my SM client, with a
copy in their 'junk' files, and those sent to an alternate address, on
another service, would be automatically retrieved and go to a specific
folder there, keeping the original on that other service -- but it has
been so long that that's all I remember, and probably I wouldn't be able
to even understand that script if I wanted to change it, these days.

Finally, a 'kindred soul': I also keep many messages since the '90s
(starting with a few from CompuServe!), and am only able to do that
because of the aforementioned e-mails folders structure and SM's Boolean
search capabilities. Most people I know delete 99% of their e-mails
soon after reading them (or even before that :-)); besides sometimes
having a need to get back to earlier info and texts, one of my intents
is being able to avoid 'reinventing the wheel': If I already wrote
extensively about something I have to write again today, why not re-use
the previous text, with adequate changes?

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

m***@spamgourmet.com

2016-10-11 20:33:06 UTC

Post by Alexander Yudenitsch
BTW, before I forget: It's been a few years since SM started opening
that 'message pane' when you classify a message as "Junk"; it wasn't
that way earlier, and I don't see any way to turn that behavior off!
Does anyone know some way to do that?

That one looks like bug 986874
<https://bugzilla.mozilla.org/show_bug.cgi?id=986874>. I don't think
it's an intentional behaviour which would be turned off by an option.

--
Mark.

m***@spamgourmet.com

2016-10-11 21:52:10 UTC

I've noticed that too, and it may be the root of Alexandre's problem. It
seems that SeaMonkey correctly detects the encoding when opening
messages in the "message pane", but doesn't actually use that encoding
until the next message is opened, and doesn't detect the encoding at all
when opening them in a separate window (just using the last-detected
encoding from the message pane).
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£")
- Save as draft and close that message.
- View, in the message pane (View > Layout > Message Pane or press F8),
another email which uses a different encoding (e.g. us-ascii)
- Re-open the draft
- The "£" now appears as "Â£" (it just happens that in this case the
second character is the same as the original single character, but
that's not necessarily the case for all characters)
- Select the draft in the thread pane
- Open the message pane (View > Layout > Message Pane or press F8)
- It looks wrong there too, and saving the draft again at this point
will compound the problem. However...
- Close the message pane
- Open the message pane
- It now looks correct, and double-clicking to open the message for
further editing is now also correct.
To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.
I think this has come up on this list before, but not sure it's reported
on Bugzilla.

That's possible. I've been able to reliably reproduce by toggling the
message pane, but I can believe there may be other ways leading to the
wrong encodings being used. Maybe when messages are deleted, or perhaps
when a new message is displayed in the same window (using next/previous)...

Post by Alexander Yudenitsch
-- but the main point would be that SM seems to
open or show messages using the 'last-used' text encoding, not the one
specified in the e-mail's properties (which, IMHO, should be considered
a bug).

I agree this doesn't seem right. It's been a minor annoyance for me for
a while, and I keep meaning to look into it a bit more and file a bug
report. I've managed to produce a couple of example messages and
reproduce the use of incorrect encodings when opening them, including
corrupting a draft message.

I'm out of time tonight, but hopefully will be able to file a bug report
tomorrow...

--
Mark.

m***@spamgourmet.com

2016-10-12 21:20:37 UTC

I've noticed that too, and it may be the root of Alexandre's problem. It
seems that SeaMonkey correctly detects the encoding when opening
messages in the "message pane", but doesn't actually use that encoding
until the next message is opened, and doesn't detect the encoding at all
when opening them in a separate window (just using the last-detected
encoding from the message pane).
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£")
- Save as draft and close that message.
- View, in the message pane (View > Layout > Message Pane or press F8),
another email which uses a different encoding (e.g. us-ascii)
- Re-open the draft
- The "£" now appears as "Â£" (it just happens that in this case the
second character is the same as the original single character, but
that's not necessarily the case for all characters)
- Select the draft in the thread pane
- Open the message pane (View > Layout > Message Pane or press F8)
- It looks wrong there too, and saving the draft again at this point
will compound the problem. However...
- Close the message pane
- Open the message pane
- It now looks correct, and double-clicking to open the message for
further editing is now also correct.
To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.
I think this has come up on this list before, but not sure it's reported
on Bugzilla.

I agree this doesn't seem right. It's been a minor annoyance for me for
a while, and I keep meaning to look into it a bit more and file a bug
report. I've managed to produce a couple of example messages and
reproduce the use of incorrect encodings when opening them, including
corrupting a draft message.
I'm out of time tonight, but hopefully will be able to file a bug report
tomorrow...

Bug 1309711 raised:
<https://bugzilla.mozilla.org/show_bug.cgi?id=1309711>

I've posted a separate thread asking others to reproduce on other
operating systems / newer versions, in case anyone who's stopped
following this thread is able to test.

--
Mark.

Alexander Yudenitsch

2016-10-13 17:43:21 UTC

Post by m***@spamgourmet.com
<https://bugzilla.mozilla.org/show_bug.cgi?id=1309711>
I've posted a separate thread asking others to reproduce on other
operating systems / newer versions, in case anyone who's stopped
following this thread is able to test.
Yay, Mark! Thanks.

--
Thanks for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Alexander Yudenitsch

2016-10-11 22:43:19 UTC

... the main point would be that SM seems to open or show messages
using the 'last-used' text encoding, not the one specified in the
e-mail's properties (which, IMHO, should be considered a bug).

I agree this doesn't seem right. It's been a minor annoyance for me for
a while, and I keep meaning to look into it a bit more and file a bug
report. I've managed to produce a couple of example messages and
reproduce the use of incorrect encodings when opening them, including
corrupting a draft message.
I'm out of time tonight, but hopefully will be able to file a bug report
tomorrow...

From what I've seen in this forum, filing a bug report may be useful
for SM development, but is unlikely to bring quick results, since the
devs have so much more, and more important, things to attend to (and I
do appreciate and thank them all for their efforts and helpfulness).

If that's true, filing a bug report would definitely be a good thing,
but personally I'll have to concentrate on workarounds (or process
steps) which will allow me to no longer have this problem in the
immediate future.

As I said, I'm currently migrating to a XP/7 dual-boot system, so I
can't concentrate on that, but all the comments and suggestions given
here so far have indicated several things I can try out without too much
effort/time; what I'm still unsure of is how I should define all those
variables in SM/Mail involving text encoding and fonts in a coherent
manner, adequate to a multi-language mail 'system'...

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Paul B. Gallagher

2016-10-10 20:36:46 UTC

Interesting. What you call "standard ASCII" is what we used to call
"high ASCII." The first 128 characters (up to ... xyz{|}~€ in Windows
Character Map) are the most basic, and the last 127 (beginning with
‚ƒ„…†‡ˆ‰Š‹ŒŽ) sometimes cause trouble. The high ASCII set also
includes the accented Western characters mentioned by the OP, such as
àáâãäåæçèéê..., but not Eastern European characters such as ăĺčďů.

At any rate, if high ASCII is causing problems, that suggests that SM is
trying for some reason to make do with a seven-bit encoding instead of
eight bits.

Post by m***@spamgourmet.com
To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.

That F8's a good tip, thanks.

--
War doesn't determine who's right, just who's left.
--
Paul B. Gallagher

Alexander Yudenitsch

2016-10-10 22:57:07 UTC

Interesting. What you call "standard ASCII" is what we used to call
"high ASCII." The first 128 characters (up to ... xyz{|}~€ in Windows
Character Map) are the most basic, and the last 127 (beginning with
‚ƒ„…†‡ˆ‰Š‹ŒŽ) sometimes cause trouble. The high ASCII set also
includes the accented Western characters mentioned by the OP, such as
àáâãäåæçèéê..., but not Eastern European characters such as ăĺčďů.
At any rate, if high ASCII is causing problems, that suggests that SM is
trying for some reason to make do with a seven-bit encoding instead of
eight bits.

And what are implications of that?

(Sorry, but I'm rather lost in all these details: I've worked with
computers since the '70s, but always tried to avoid the technical
parts... still do, but that's the only way to get some things I want
done, done)

I (...) have no correspondents who use anything except Unicode
and what I THOUGHT could be "Western" characters -- but what's
that: Is that an euphemism for "English"? Portuguese, Spanish,
French, all use "Western characters", IMHO; but I suspect the
former (ie, "Western") actually means "English"...
Assuming that's true, then in my case there would be 3 types of
text encoding: Western/English, Unicode (and I don't know if 'all
Unicodes are created equal'), and "other Western codes", like
Portuguese, Spanish, French, etc.

not to mention things like "MIME Encoding", which I don't see any option
for anywhere in SM except that "Preferences | Mail & Newsgroups | Text
Encoding" option: "For messages that contain 8-bit characters, use
'quoted printable' MIME Encoding" (yes, I know it's in Windows itself;
but I thought SM was referring to something one could change inside it).

To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.

That F8's a good tip, thanks.

As I said in another message, I'll try that out after I clean up my
draft, and see if it only works with the message pane, or also for new
windows...

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

m***@spamgourmet.com

2016-10-11 20:14:43 UTC

Interesting. What you call "standard ASCII" is what we used to call
"high ASCII."

By "standard ASCII", I meant the first 128 characters. "£" is one of the
extended characters in some charsets. In UTF-8, it's encoded as a
two-byte sequence. Hence when an email containing "£" encoded in UTF-8
is decoded using an extended ASCII charset, it appears as two separate
characters.

Post by Paul B. Gallagher
The first 128 characters (up to ... xyz{|}~€ in Windows
Character Map) are the most basic, and the last 127 (beginning with
‚ƒ„…†‡ˆ‰Š‹ŒŽ) sometimes cause trouble. The high ASCII set also
includes the accented Western characters mentioned by the OP, such as
àáâãäåæçèéê..., but not Eastern European characters such as ăĺčďů.

The exact characters included in the high/extended ASCII range depend on
the particular character set being used. The one you use may include /
not include those characters, but another might. There's not one single
extended ASCII charset but several, each with an emphasis on particular
languages or regions, including the characters needed for those
languages in its extended range.

Post by Paul B. Gallagher
At any rate, if high ASCII is causing problems, that suggests that SM is
trying for some reason to make do with a seven-bit encoding instead of
eight bits.

It's not extended characters as such that causes a problem, but emails
being decoded using the wrong character set. The 128 standard ASCII
characters are encoded the same in all extended ASCII-based charsets and
also in UTF-8, so for those of us in Western countries, using the wrong
encoding generally isn't noticeable except for the odd character outside
the standard ASCII ranges.

--
Mark.

Alexander Yudenitsch

2016-10-11 22:32:06 UTC

Post by m***@spamgourmet.com
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£") ...

Interesting. What you call "standard ASCII" is what we used to call
"high ASCII."

Post by Paul B. Gallagher
At any rate, if high ASCII is causing problems, that suggests that SM is
trying for some reason to make do with a seven-bit encoding instead of
eight bits.

That was educational (or memory-stirring, since I had read about all
that a long time ago) for me: It seems that the problem might lie in
the extended ASCII characters used for pt-BR text, which (since SM
doesn't have any place to specify that particular language) might mean
it tries to use an inappropriate one (if that's what you meant by
"There's not one single extended ASCII charset but several, each with an
emphasis on particular languages or regions, including the characters
needed for those languages in its extended range.")

Post by Paul B. Gallagher
"Western" characters -- but what's that: Is that an euphemism for
"English"? Portuguese, Spanish, French, all use "Western
characters", IMHO; but I suspect the former (ie, "Western")
actually means "English"...

I had thought that, in SM/Mail, there were basically 3 text encoding
options: English (called "Western"); Unicode; and 'other languages'.
For many years, I used 'Western' but, as it seemed more and more
messages used Unicode, switched to it (this was at the same time as the
'drafts gibberish' problem started to become more annoying), thinking
that Unicode (is that the same as UTF-8?), at least, should be
"universal"...

Discounting the cases where one opens a message with the wrong encoding
(which is easy to identify since, in such cases, re-opening the message
with the correct encoding makes the problem go away), it seems this
problem (which I described with the Drafts) only appears when a message
mixes extended ASCII characters with lots of standard ASCII ones (would
you say that's a correct way of putting it?).

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Alexander Yudenitsch

2016-10-12 16:08:45 UTC

It seems that SeaMonkey correctly detects the encoding when opening
messages in the "message pane", but doesn't actually use that encoding
until the next message is opened, and doesn't detect the encoding at all
when opening them in a separate window (just using the last-detected
encoding from the message pane).
- Start composing a message with UTF-8 encoding (like Paul, that's my
default) and include a character not covered by the standard ASCII set
(in my case, usually "£")
- Save as draft and close that message.
- View, in the message pane (View > Layout > Message Pane or press F8),
another email which uses a different encoding (e.g. us-ascii)
- Re-open the draft
- The "£" now appears as "Â£" (it just happens that in this case the
second character is the same as the original single character, but
that's not necessarily the case for all characters)
- Select the draft in the thread pane
- Open the message pane (View > Layout > Message Pane or press F8)
- It looks wrong there too, and saving the draft again at this point
will compound the problem. However...
- Close the message pane
- Open the message pane
- It now looks correct, and double-clicking to open the message for
further editing is now also correct.
To be honest, I've got to the point where I barely notice this and just
automatically tap F8 a couple of times when a message isn't displayed
correctly. That it's become an automatic reaction isn't a particularly
good sign, but that may be a workaround until the problem is fixed
properly.

Hooray! That works on my system too, exactly as you described it!!

I edited the one of the 'offending' drafts (a long one, mixing standard
and extended ASCII), correcting all the gibberish I could find (*) to
the correct symbols, and saved it, hen opened another message; after
that,I selected the 'target draft' and, with F8, opened it in the
message pane: Bingo, it showed gibberish again! Another F8 closed it,
and a third one re-opened the pane -- and the gibberish was gone! I then
opened the draft for editing, saved it, and the same "3xF8" got the same
results: Now, it's edited, and showing no gibberish. I hope I'll be able
to remember the "3xF8" routine from now on, despite years of an
ingrained different habit...

As I said, I don't like or use the message pane, and will try and see if
there's a way to make this workaround work without the pane (perhaps you
have some suggestions?) but, if not, I'll just have to get used to the
"3xF8" technique you described...

Thanks a lot for sharing your experience!

Maybe you, or someone reading this, could shed some light on these

.. Unicode (UTF-8)
.. Arabic
.. (etc, etc.)
.. Vietnamese
.. Western
( ) Apply encoding to all messages in the folder (individual
message text encoding settings and auto-detection will be ignored)

If the latter option isn't checked, what's the option for, since the
folder has (supposedly) already-received messages, which don't need
editing? Does it mean that SM would try to use the chosen text encoding
to display messages in the folder, if they don't have any encoding
attached to them?

--
Thanks beforehand for your attention, and I hope to hear from you soon.

s) Alexander Yudenitsch <***@postpro.net>

Alexander Yudenitsch

2016-10-29 03:53:30 UTC