HTML Entity Refs and xmerl

According to [erlang-bugs] xmerl and standard HTML entity refs, currently xmerl_scan only recognizes the very limited set of entity references. In brief, if you try to xmerl:scan xml text that includes standard HTML entity refs, such as nbsp, iexcl, pound, frac14, etc. you'll encounter something like:

16> edoc:file("exprecs.erl").
2670- fatal: {unknown_entity_ref,nbsp}
2580- fatal: error_scanning_entity_ref
exprecs.erl, in module header: at line 28: error in XML parser: {fatal,
                         {error_scanning_entity_ref,
                             {file,file_name_unknown},
                             {line,86},
                             {col,18}}}.
** exited: error **

Ulf Wiger said:

... I realize that xmerl can be customized with a rules function which, for example, can handle entity references...

So I take a try by writing a piece of code (html_entity_refs.erl) to parse a HTML entity ref DTD file to ets rules, then: 

xmerl_scan:string(XmlText, [{rules, html_entity_refs:get_xmerl_rules()}]).

Yes, this works. But for a 3MB testing file, the parsing took about 30 seconds.

How about convert these entity refs to utf-8 chars first, then apply xmerl_scan to it?
I wrote another piece of code, and now: 

xmerl_scan:string(html_entity_refs:decode_for_xml(XmlText)).

This time, the decoding+parsing time is about 5 seconds, it's 6 times faster than ets rules solution.

The html_entity_refs.erl can be got from:
http://caoyuan.googlecode.com/svn/trunk/erlang/html_entity_refs.erl

Comments

No comments.