Repository navigation
Fix NEL in forHtmlUnquotedAttribute and NEL/LS pass-through in the XML 1.1 encoders #136
Description
Problem
A review of main @ 94fd425 (1.4.0) found two encoders that emit a control character in a form real parsers do not preserve, contrary to what the Javadoc promises. forHtmlUnquotedAttribute encodes U+0085 (NEL) as …, which HTML5 parsers decode to U+2026 (HORIZONTAL ELLIPSIS) under the windows-1252 remap in the WHATWG "Numeric character reference end state". The XML 1.1 modes (forXml11, forXml11Content, forXml11Attribute) pass U+0085 and U+2028 (LS) through raw, which a conforming XML 1.1 processor normalises to LF (a space in attribute values) per XML 1.1 §2.11, while the references … and 
 round-trip exactly. Neither is a security issue; both are fidelity defects. #105 records only the wrong forXml11 Javadoc range; this issue covers the behaviour.
Evidence
HTML unquoted attribute
core/src/main/java/org/owasp/encoder/HTMLEncoder.java:325-330:case Unicode.NEL->encode(ch)emits….forHtmlUnquotedAttribute("\u0085")returns…. Every other C1 control in this mode is replaced with-.- jsoup 1.23.1 and validator.nu htmlparser 1.4 both parse
<p title=…>to atitleof U+2026. Both retain a raw U+0085 in an unquoted attribute value; it is not a terminator. Encode.java:398-400documents the row "U+0085 -> �";Encode.java:425-427says C0 and C1 controls are replaced by a hyphen.HTMLEncoderTest.java:60asserts.encode("…", "\u0085").
XML 1.1
core/src/main/java/org/owasp/encoder/XMLEncoder.java:254and:382(ch == Unicode.NEL) exempt NEL from the C1 branch infirstEncodedOffsetandencodeArrays; U+2028 falls through the finalelseat:453-458.- On JDK 17 against the 1.4.0 core jar, all three XML 1.1 methods return input containing U+0085 and U+2028 unchanged. The JDK Xerces parser with
<?xml version="1.1"?>turns raw NEL and raw U+2028 into LF in content and a space in an attribute;x…yandx
yround-trip exactly. - In a sweep of U+007F-U+009F, U+0085 is the only code point XML 1.1 mode leaves raw; the rest become
&#xNN;. XML11EncoderTest.java:85("nel-char"),:221-224(the U+0085 skip intestXML11C1ControlChars) and:237-241(testXML11NEL) assert the pass-through.Encode.java:875andXMLEncoder.java:112say[#x7F-#x9F]is encoded.- XML 1.1 §2.11 requires processors to normalise #x85 and #x2028 to #xA on input; §1.3 exempts whitespace from the character-reference rule, so leaving NEL raw is conformant but lossy. XML 1.0 mode needs no change: a 1.0 parser keeps raw NEL.
Impact
The one C1 control the unquoted-attribute encoder tries to preserve is exactly the one browsers rewrite, so a value read back (for example from a form field) contains an ellipsis. XML 1.1 users silently get NEL and LS turned into line feeds or spaces by every conforming parser. Both fixes change output, so they belong in a minor release with a CHANGELOG entry. The matching Javadoc changes belong here, not in #105, so behaviour about to change is not documented twice.
Acceptance criteria
-
forHtmlUnquotedAttributetreats U+0085 like the other C1 controls (replaced with-), or passes it raw with the rationale documented; it no longer emits…, andfirstEncodedOffsetagrees withencodeArrays. - The "U+0085 -> �" row at
Encode.java:398-400is removed or corrected andHTMLEncoderTest.java:60is updated to the new output. - In
Version.XML_1_1only, U+0085 is emitted as…and U+2028 as
; XML 1.0 output is unchanged. -
maxEncodedLengthfor XML 1.1 accounts for the 8-character
reference, either by raising the bound aboven * CONTROL_CHAR_REF_LENGTH(6) or with a dedicated constant. -
XML11EncoderTest"nel-char" (line 85),testXML11C1ControlChars(lines 221-224) andtestXML11NEL(lines 237-241) expect the references. - The
forXml11Javadoc atEncode.java:875andXMLEncoder.java:112states the set actually encoded, superseding the range correction in Fix Javadoc and TLD descriptions that disagree with encoder behavior #105. - CHANGELOG entry describing both output changes.
Related: #105
Suggested target: 1.5.0
Activity
tirupatilakshmidevi6-hub commentedon Sep 25, 2026
Hi, I would like to work on this issue. I’ll investigate the current HTML/XML encoder behavior for NEL and LINE SEPARATOR, update the relevant encoding logic and length calculations, and add the required tests and documentation changes.
jmanico commentedon Sep 25, 2026
Thank you go for it!