`HTMLParser.unknown_decl` receives corrupted data (`'CDATA['`) when parsing an empty `CDATA` section

[T90REAL]

Bug report

Bug description:

When parsing the input <![CDATA[]]>, the unknown_decl hook is incorrectly called with the corrupted, partial string 'CDATA['.

A correct parser has only two possible-and-correct behaviors:

• If CDATA is supported: Call handle_cdata('').
• If the declaration is "unrecognized," the unknown_decl hook must receive the entire content inside <!...>, which would be '[CDATA[]]'.

The actual result ('CDATA[') matches neither of those. I use the private _set_support_cdata(True) method here since I think it was the only available trigger to activate this specific code path to expose the bug.

from html.parser import HTMLParser

class CdataBugParser(HTMLParser):
    def __init__(self):
        super().__init__()
        self.unknown_decls = []

    def unknown_decl(self, data):
        self.unknown_decls.append(data)

html_input = "<![CDATA[]]>"
parser = CdataBugParser()
parser._set_support_cdata(True)
parser.feed(html_input)
print(parser.unknown_decls)

['CDATA[']

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[serhiy-storchaka]

This looks weird, but this behavior is preserved for backward compatibility.

When the parser recognized a CDATA section <![CDATA[content]]>, it calls the unknown_decl hook, passing the matched string with removed <![ and ]]> (note unbalanced brackets). It was for such long time, that changing this would break user code.

According to specifications, if the CDATA section is recognized, its content should be interpreted as normal data, so the parser should call handle_data(), or maybe a new handle_cdata() which calls handle_data() by default, and pass the string with removed <![CDATA[ and ]]>. But we should think about backward compatibility. This was in my plans.

[serhiy-storchaka]

See also #137877.