Inline styles (like bold, italic or strikethrough) might be active or inactive. This information is both for user experience and have an impact on editor behavior. It is very important to correctly set style state because it has big impact on UX. There are multiple situations when deducing style state is not trivial and the editor is expected to make educated guess to enable better experience for the user.
Inline style state is represented through UI, most often by changing the look of the button connected with certain feature. Style state has different meaning, depending whether some content in the editor is selected or not.
If content is selected, it is expected that style state reflects state of that content. For example, if bold text is selected, bold style state should be “active” to:
If content is not selected, it is expected that newly typed text will have applied all active styles. For example, if user placed caret somewhere in the content and used bold feature, it state should become “active” and any typed character should have bold applied. After using bold feature again, it’s state should become “inactive”.
If style state cannot be directly reflected by selection content, it should reflect what is expected editor behavior.
If it is expected that selected content should be wholly bold after using bold feature, the style state should be “inactive”. It may be wise to set style to inactive if there is at least one inactive element in selection, because if the user wanted to remove a style, they would be more precise in selecting exactly the part to remove. However this behavior probably needs more argumentation
On the other hand, if user types a new letter with selection set it may be expected that the newly typed character have same styles as the place where it will be put and certainly it should have styles reflecting as styles states. If selection is on f[oba]r and bold state is inactive, what should be style of newly typed letter?
What if given style have multiple possible values? I.e. font color.
As long as there is content that is applicable for given style, content that cannot be styled should not have any impact on style state.
Note: just basing on browsers, this should not have any impact on style state. This makes sense, because style state should be connected with selection contents and not the way it was made.
Each range should be checked in same way as single-range selection and the results should be “added”. For example, if one range has only bold text but the other range has non-bold text, style state should be “inactive”.
Similar to 1. Style state should reflect state of character before or after caret. Sometimes there might be no characters before/after, i.e. when caret is at the beginning or end of paragraph, or before or after an image). By default it is advised to first look left because it is expected that the user want to continue typing in the context before the caret.
There is a nice feature in Firefox, though. When user uses left and right arrows to navigate caret through content, the caret looks at the side it “came from”. So, when user clicks left, the caret looks to right. For example, if caret was in bold text and by arrows it was moved to a place between bold and non-bold character, style state remain bolded. This could called “style retention”. It greatly enhances UX by giving the user more ability to manipulate style state.
When user creates new paragraph it is expected that they want to use same styles that before breaking the paragraph. Style state should be retained. When navigating to empty paragraph, styles states should be recovered to the last known styles states encountered in that paragraph.
After user removes selected content it is expected that they want to continue typing in the context before the caret. Styles state should be taken from the character before.
Taken into consideration question and answers above, this is a proposal of algorithm for deciding styles state when selection is made in editor or caret position is changed.