Skip to content

Upgrading charset-normalizer 2 to 3

Breaking2.1.1 → 3.5.2Breaking18Platform5New19Changed33Fixed34Other10

Full comparisonBack to charset-normalizer
BreakingUpgrade verdict

17 breaking changes and 5 platform requirement changes.

In plain terms
  • UTF-7 will no longer appear as "detected" without a recognized SIG/mark (is unreliable/conflicts with ASCII).
  • Coherence detector no longer returns 'Simple English' instead returns 'English'.
  • Coherence detector no longer returns 'Classical Chinese' instead returns 'Chinese'.
  • Redundant UTF7 BOM marker.
Must act Top 5 of 22 that need code or config changes

Python requirement changed in 3.4.1: >=3.6.0 → >=3.7.

What you must change

Breaking17
UTF-7 will no longer appear as "detected" without a recognized SIG/mark (is unreliable/conflicts with ASCII).
3.0.0GitHub ReleaseOriginalView source
Coherence detector no longer returns 'Simple English' instead returns 'English'.
3.0.0GitHub ReleaseOriginalView source
Coherence detector no longer returns 'Classical Chinese' instead returns 'Chinese'.
3.0.0GitHub ReleaseOriginalView source
Redundant UTF7 BOM marker.
3.4.8GitHub ReleaseOriginalView source
setuptools-scm as a build dependency.
setuptools-scm
3.4.4GitHub ReleaseOriginalView source
build-requirements.txt as per using pyproject.toml native build configuration.
build-requirements.txtpyproject.toml
3.4.1GitHub ReleaseOriginalView source
setup.cfg in favor of pyproject.toml metadata configuration.
setup.cfgpyproject.toml
3.4.1GitHub ReleaseOriginalView source
Unused utils.range_scan function.
utils.range_scan
3.4.1GitHub ReleaseOriginalView source
Support for Python 3.6 (PR ).
3.1.0GitHub ReleaseOriginalView source
Method first() and best() from CharsetMatch.
first()best()
3.0.0GitHub ReleaseOriginalView source
Class aliases CharsetDetector, CharsetDoctor, CharsetNormalizerMatch and CharsetNormalizerMatches.
3.0.0GitHub ReleaseOriginalView source
Top-level function normalize.
normalize
3.0.0GitHub ReleaseOriginalView source
Properties chaos_secondary_pass, coherence_non_latin and w_counter from CharsetMatch.
chaos_secondary_passcoherence_non_latinw_counter
3.0.0GitHub ReleaseOriginalView source
Support for the backport unicodedata2.
unicodedata2
3.0.0GitHub ReleaseOriginalView source
bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile).
bin/integration.pybin/serve.py
3.4.1GitHub ReleaseOriginalView source
(internal) Redundant utils.is_ascii function and unused function is_private_use_only.
3.3.0GitHub ReleaseOriginalView source
(internal) charset_normalizer.assets is moved inside charset_normalizer.constant.
3.3.0GitHub ReleaseOriginalView source
Platform5
Optional mypyc compilation upgraded to version 1.15 for Python >= 3.9.
3.4.2GitHub ReleaseOriginalView source
Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8.
3.4.1GitHub ReleaseOriginalView source
Optional mypyc compilation upgraded to version 1.6.1 for Python >= 3.8.
3.3.1GitHub ReleaseOriginalView source
Optional mypyc compilation upgraded to version 1.5.1 for Python >= 3.8.
3.3.0GitHub ReleaseOriginalView source
Speedup provided using mypy/c 0.990 on Python >= 3.7.
3.0.1GitHub ReleaseOriginalView source