Upgrading spacy 1 to 2
Breaking1.10.1 → 2.3.9Breaking31Security1Platform2Deprecated5New134Changed88Fixed486Other76
BreakingUpgrade verdict
31 breaking changes and 2 platform requirement changes and a security fix.
In plain terms
- The from_glove helper for loading GloVe vectors has been removed.
- Manage spaCy warnings with Python's standard warnings filters instead of SPACY_WARNING_IGNORE.
- The old lemmatization tables on Language.Defaults are gone; edit lemmatization tables through nlp.vocab.lookups instead.
- Lemmatization data now lives in a separate package. Pre-trained models still work, but blank models or languages without models need the extra data installed.
Must act Top 5 of 39 that need code or config changes
spacy publishes an official migration guide for version 2. It covers more than the release notes do.
Migrating from spaCy 1.x ↗Requires Python !=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7 from 2.3.0.dev1 (none stated before).
Adds blis <0.8.0,>=0.4.0, catalogue <1.1.0,>=0.0.7, cymem <2.1.0,>=2.0.2, murmurhash <1.1.0,>=0.28.0, numpy >=1.15.0, plac <1.2.0,>=0.9.6, preshed <3.1.0,>=3.0.2, requests <3.0.0,>=2.13.0, setuptools, srsly <1.1.0,>=1.0.2, thinc <7.5.0,>=7.4.1, tqdm <5.0.0,>=4.38.0, wasabi <1.1.0,>=0.4.0.
Installs 17 packages in all (was 0).
What you must change
Breaking31
Removes Vectors.from_glove.
Vectors.from_gloveReplaces custom warnings with native Python
warnings; SPACY_WARNING_IGNORE is no longer used.SPACY_WARNING_IGNORERemoves unused
Language.Defaults attributes lemma_rules, lemma_index, lemma_exc and lemma_lookup; use nlp.vocab.lookups instead.Language.Defaultslemma_ruleslemma_indexlemma_exclemma_lookupnlp.vocab.lookupsLemmatization tables move to the separate spacy-lookups-data package, not installed by default.
spacy-lookups-dataspacy.blank()The spacy download command no longer sets --no-deps by default, so model package dependencies are now installed too.
spacy download--no-depsThe Sentencizer default punctuation has been extended with more characters common in various languages, which may change results.
Sentencizerpunct_charsSerialization methods take a single
exclude argument; disable on Language serialization is renamed to exclude.to_diskfrom_diskto_bytesfrom_bytesexcludedisableLanguageResolves strings to hash values instead of mapping them to integer IDs, so the string-to-int mapping no longer depends on vocabulary state.
Training new models requires installing
spacy-lookups-data, now with lemmatization and normalization tables.spacy-lookups-dataFine-grained POS tags change for many provided models due to updated Universal Dependencies training data.
French, Italian, Portuguese and Spanish tagsets add merged contraction tags such as
ADP_DET.ADP_DETLemmatization tables are now part of Vocab and serialized with it, adding data to serialized objects and files.
VocabLemmatizer is now initialized with a Lookups instance instead of separate dict arguments.
LemmatizerLookupsThe Dutch model is trained on a new NER corpus (custom labelled UD instead of WikiNER), so predictions may differ significantly.
biluo_tags_from_offsets is now stricter and raises an error on overlapping entities instead of silently skipping them.
biluo_tags_from_offsetsdebug-dataPipeline components can now overwrite IOB tags of tokens not yet part of an entity, and ent_iob is never reset to unset.
ent_iobdoc.entsThe PhraseMatcher algorithm was rewritten and is now 10x faster, which may produce slightly different but fully correct results on large lists.
PhraseMatcherThe Serbian language class now uses the language code sr instead of rs and is available via spacy.lang.sr.
Serbianspacy.lang.srThe sources in meta.json changed from a list of strings to a list of dicts, which may require updating code using nlp.meta["sources"].
meta.jsonnlp.meta["sources"]Version 2.1.0 requires downloading new models; custom-trained models must be retrained with the new version.
spacy validateMakes NORM a Token attribute instead of a Lexeme attribute to allow context-specific norms in tokenizer exceptions.
TokenLexemeNORMRequires downloading new models; use
spacy validate to find which models need updating.spacy validateCustom-trained models must be retrained with the new version.
Changes Tokenizer.to_bytes to take an exclude list instead of a vocab flag.
Tokenizer.to_bytesChanges the .pos value for several common English words after corrections to the English tag map.
Token.posChanges spacy train to take a comma-separated --pipeline list of component names instead of separate disable flags like --no-parser.
spacy train--pipeline--no-parserChanges spacy init-model to use a single --jsonl-loc argument instead of separate --freqs-loc and --clusters-loc.
spacy init-model--jsonl-loc--freqs-loc--clusters-locOld v1.x models are not compatible with spaCy v2.0.0; custom trained models must be re-trained.
spacy.load() is now only intended for loading models; import empty language classes directly instead.
spacy.load()load()nlp.from_disk()All built-in pipeline components are now Pipe subclasses; training uses begin_training returning an optimizer passed to update.
Pipebegin_trainingupdateDocGoldParseSerialization API is now consistent: all containers and pipeline components have to_disk, from_disk, to_bytes and from_bytes.
to_disk()from_disk()to_bytes()from_bytes()Security1
Escapes HTML tags in displacy.render.
displacy.renderPlatform2
Explicitly requires the pathlib backport on Python 2 only.
pathlibDrops support for EOL Python 2.6 and 3.3.
Deprecations to plan for
Deprecated5
Improves init-model with support for lexical attributes and word vectors, replacing the now deprecated spacy vocab command.
init-modelspacy vocabDeprecated
Doc.merge and Span.merge still work but run slower; use Doc.retokenize for bulk merges.Doc.mergeSpan.mergeDoc.retokenizeDeprecates the n_threads keyword argument on .pipe methods because v2.x models cannot release the global interpreter lock.
n_threadspipeDeprecates Doc.print_tree in favour of a unified Doc.to_json method matching the JSON training data format.
Doc.print_treeDoc.to_jsonRenames the built-in rule-based sentence boundary detector to 'sentencizer' and deprecates the name 'sbd'.
sentencizersbdnlp.create_pipeAPI changes
Generated by DepDelta from published Python source. Not mixed with prose notes.
- added:
function check_changes(root, db) - added:
function get_hash(path) - added:
function hash_add(base, path, db) - added:
function hash_changed(base, path, db) - added:
function load_hashes(filename) - added:
function preserve_cwd(path, func, *args) - added:
function process(base, filename, db) - added:
function process_pyx(fromfile, tofile, language_level=…) - added:
function run(root) - added:
function save_hashes(hash_db, filename) - added:
variable HASH_FILE - added:
variable args - added:
variable parser
- added:
class Reddit(object) - added:
constructor Reddit(file_path, meta_keys=…) - added:
field Reddit.link_re - added:
field Reddit.post_format_re - added:
field Reddit.pre_format_re - added:
field Reddit.url_re - added:
function main(path) - added:
method Reddit.get_meta(item) - added:
method Reddit.is_valid(comment) - added:
method Reddit.iter_files() - added:
method Reddit.strip_tags(text) - added:
variable BrokenPipeError - added:
variable devnull
- added:
class Corpus(object) - added:
constructor Corpus(directory, nlp) - added:
function iter_dir(loc) - added:
function main(lang, in_dir, out_loc, negative=…, n_workers=…, window=…, size=…, min_count=…, nr_iter=…) - added:
variable logger
- added:
export ud_evaluate - added:
export ud_train
- added:
class TestAlignment(unittest.TestCase) - added:
class UDError(Exception) - added:
function evaluate(gold_ud, system_ud, deprel_weights=…, check_parse=…) - added:
function evaluate_wrapper(args) - added:
function load_conllu(file, check_parse=…) - added:
function load_conllu_file(path) - added:
function load_deprel_weights(weights_file) - added:
function main() - added:
method TestAlignment.test_alignment() - added:
method TestAlignment.test_equal() - added:
method TestAlignment.test_equal_with_multiword() - added:
method TestAlignment.test_exception() - added:
variable DEPREL - added:
variable DEPS - added:
variable FEATS - added:
variable FORM - added:
variable HEAD - added:
variable ID - added:
variable LEMMA - added:
variable MISC - added:
variable UPOS - added:
variable XPOS
- added:
function fetch_all_treebanks(ud_dir, languages, corpus, best_per_language) - added:
function get_freq_tuples(my_list, print_total_threshold) - added:
function load_default_model_sentencizer(lang) - added:
function load_model(modelname, add_sentencizer=…) - added:
function main(out_path, ud_dir, check_parse=…, langs=…, exclude_trained_models=…, exclude_multi=…, hide_freq=…, corpus=…, best_per_language=…) - added:
function run_all_evals(models, treebanks, out_file, check_parse, print_freq_tasks) - added:
function run_single_eval(nlp, loading_time, print_name, text_path, gold_ud, tmp_output_path, out_file, print_header, check_parse, print_freq_tasks) - added:
function split_text(text) - added:
variable ALL_LANGUAGES - added:
variable EVAL_NO_PARSE - added:
variable EVAL_PARSE - added:
variable PRINT_FREQ - added:
variable PRINT_TOTAL - added:
variable space_re
- added:
function evaluate(nlp, text_loc, gold_loc, sys_loc, limit=…) - added:
function get_token_split_end(token) - added:
function get_token_split_start(token) - added:
function guess_fused_orths(word, ud_forms) - added:
function initialize_pipeline(nlp, docs, golds, config, device) - added:
function load_nlp(experiments_dir, corpus) - added:
function main(test_data_dir, experiment_dir, corpus) - added:
function print_results(name, ud_scores) - added:
function read_conllu(file_) - added:
function split_text(text) - added:
function write_conllu(docs, file_) - added:
variable Fused_begin - added:
variable Fused_inside - added:
variable space_re
- added:
class Config(object) - added:
class Dataset(object) - added:
class TreebankPaths(object) - added:
constructor Config(vectors=…, max_doc_length=…, multitask_tag=…, multitask_sent=…, multitask_dep=…, multitask_vectors=…, bilstm_depth=…, nr_epoch=…, min_batch_size=…, max_batch_size=…, batch_by_words=…, dropout=…, conv_depth=…, subword_features=…, vectors_dir=…, pretrained_tok2vec=…) - added:
constructor Dataset(path, section) - added:
constructor TreebankPaths(ud_path, treebank, **cfg) - added:
function evaluate(nlp, text_loc, gold_loc, sys_loc, limit=…) - added:
function get_token_conllu(token, i) - added:
function golds_to_gold_tuples(docs, golds) - added:
function initialize_pipeline(nlp, docs, golds, config, device) - added:
function load_nlp(corpus, config, vectors=…) - added:
function main(ud_dir, parses_dir, corpus, config=…, limit=…, gpu_device=…, vectors_dir=…, use_oracle_segments=…) - added:
function print_progress(itn, losses, ud_scores) - added:
function read_conllu(file_) - added:
function read_data(nlp, conllu_file, text_file, raw_text=…, oracle_segments=…, max_doc_length=…, limit=…) - added:
function split_text(text) - added:
function write_conllu(docs, file_) - added:
static method Config.load(loc, vectors_dir=…) - added:
variable space_re - added:
variable torch
- removed:
export bn - removed:
export de - removed:
export en - removed:
export es - removed:
export fi - removed:
export fr - removed:
export he - removed:
export hu - removed:
export info - removed:
export it - removed:
export ja - removed:
export nb - removed:
export nl - removed:
export pt - removed:
export resolve_model_name - removed:
export ru - removed:
export sv - removed:
export th - removed:
export zh - added:
export Errors - added:
export Warnings - added:
export cli_info - added:
export component - added:
export pipeline - added:
export registry - added:
function blank(name, **kwargs) - added:
function info(model=…, markdown=…, silent=…)
- added:
function analyze_all_pipes(pipeline, warn=…) - added:
function analyze_pipes(pipeline, name, pipe, index, warn=…) - added:
function dot_to_dict(values) - added:
function get_assigns_for_attr(pipeline, attr) - added:
function get_requires_for_attr(pipeline, attr) - added:
function print_summary(nlp, pretty=…, no_print=…) - added:
function validate_attrs(values)
- removed:
class Bengali(Language) - removed:
class Bengali.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Bengali.Defaults.infixes - removed:
field Bengali.Defaults.lemma_rules - removed:
field Bengali.Defaults.lex_attr_getters - removed:
field Bengali.Defaults.prefixes - removed:
field Bengali.Defaults.stop_words - removed:
field Bengali.Defaults.suffixes - removed:
field Bengali.Defaults.tag_map - removed:
field Bengali.Defaults.tokenizer_exceptions - removed:
field Bengali.lang
- removed:
export LEMMA_RULES - removed:
export MORPH_RULES - removed:
export STOP_WORDS - removed:
variable STOP_WORDS - removed:
variable TAG_MAP - removed:
variable TOKENIZER_EXCEPTIONS - removed:
variable TOKENIZER_INFIXES - removed:
variable TOKENIZER_PREFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable LEMMA_RULES
- removed:
variable MORPH_RULES
- removed:
variable TOKENIZER_INFIXES - removed:
variable TOKENIZER_PREFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable STOP_WORDS
- removed:
variable TAG_MAP
- removed:
variable ABBREVIATIONS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
export model - removed:
export train_config - added:
export debug_data - added:
export evaluate - added:
export init_model - added:
export pretrain - added:
export profile - added:
export validate
- removed:
function check_dirs(input_file, output_path) - changed:
function convert(input_file, output_dir, *args) → function convert(input_file, output_dir=…, file_type=…, n_sents=…, seg_sents=…, model=…, morphology=…, converter=…, lang=…) - added:
function autodetect_ner_format(input_data) - added:
variable FILE_TYPES - added:
variable FILE_TYPES_STDOUT
- added:
export conll_ner2json - added:
export iob2json - added:
export ner_jsonl2json
- added:
function conll_ner2json(input_data, n_sents=…, seg_sents=…, model=…, no_print=…, **kwargs) - added:
function n_sents_info(msg, n_sents) - added:
function segment_docs(input_data, n_sents, doc_delimiter) - added:
function segment_sents_and_docs(doc, n_sents, doc_delimiter, model=…, msg=…)
- changed:
function conllu2json(input_path, output_path, n_sents=…, use_morphology=…) → function conllu2json(input_data, n_sents=…, use_morphology=…, lang=…, **_) - changed:
function generate_sentence(sent) → function generate_sentence(sent, has_ner_tags) - changed:
function read_conllx(input_path, use_morphology=…, n=…) → function read_conllx(input_data, use_morphology=…, n=…) - added:
function is_ner(tag) - added:
function simplify_tags(iob)
- added:
function iob2json(input_data, n_sents=…, no_print=…, *args, **kwargs) - added:
function merge_sentences(docs, n_sents) - added:
function read_iob(raw_sents)
- added:
function ner_jsonl2json(input_data, lang=…, n_sents=…, use_morphology=…, **_)
- added:
function debug_data(lang, train_path, dev_path, tag_map_path=…, base_model=…, pipeline=…, ignore_warnings=…, verbose=…, no_format=…) - added:
variable BLANK_MODEL_MIN_THRESHOLD - added:
variable BLANK_MODEL_THRESHOLD - added:
variable DEP_LABEL_THRESHOLD - added:
variable NEW_LABEL_THRESHOLD
- removed:
function check_error_depr(model) - removed:
function check_shortcut(model) - changed:
function download(model=…, direct=…) → function download(model, direct=…, *pip_args) - changed:
function download_model(filename) → function download_model(filename, user_pip_args=…) - added:
function require_package(name)
- added:
function evaluate(model, data_path, gpu_id=…, gold_preproc=…, displacy_path=…, displacy_limit=…, return_scores=…) - added:
function render_parses(docs, output_path, model_name=…, limit=…, deps=…, ents=…)
- removed:
function get_spacy_data() - removed:
function print_info(data, title, markdown) - changed:
function info(model=…, markdown=…) → function info(model=…, markdown=…, silent=…) - added:
function print_markdown(data, title=…)
- added:
function add_vectors(nlp, vectors_loc, truncate_vectors, prune_vectors, name=…) - added:
function create_model(lang, lex_attrs, name=…, base_model=…) - added:
function init_model(lang, output_dir, freqs_loc=…, clusters_loc=…, jsonl_loc=…, vectors_loc=…, truncate_vectors=…, prune_vectors=…, vectors_name=…, model_name=…, omit_extra_lookups=…, base_model=…) - added:
function open_file(loc) - added:
function read_attrs_from_deprecated(freqs_loc, clusters_loc) - added:
function read_clusters(clusters_loc) - added:
function read_freqs(freqs_loc, max_length=…, min_doc_freq=…, min_freq=…) - added:
function read_vectors(vectors_loc, truncate_vectors=…) - added:
variable DEFAULT_OOV_PROB - added:
variable ftfy
- removed:
function get_meta(package_path, package) - removed:
function is_package(name) - removed:
function link_package(package_name, link_name, force=…) - removed:
function symlink(model_path, link_name, force) - changed:
function link(origin, link_name, force=…) → function link(origin, link_name, force=…, model_path=…)
- removed:
function check_dirs(freqs_data, clusters_data, vectors_data) - removed:
function check_unzip(file_path) - removed:
function create_model(model_path, vectors_path, vocab, oov_prob) - removed:
function model(lang, model_dir, freqs_data, clusters_data, vectors_data) - removed:
function populate_vocab(vocab, clusters, probs, oov_prob) - removed:
function read_clusters(clusters_path) - removed:
function read_probs(freqs_path, max_length=…, min_doc_freq=…, min_freq=…)
- removed:
function check_dirs(input_path, output_path, meta_path) - removed:
function create_dirs(package_path, force) - removed:
function get_template(filepath) - removed:
function validate_meta(meta, keys) - changed:
function generate_meta() → function generate_meta(model_path, existing_meta, msg) - changed:
function package(input_dir, output_dir, meta_path, force) → function package(input_dir, output_dir, meta_path=…, create_meta=…, force=…) - added:
variable TEMPLATE_INIT - added:
variable TEMPLATE_MANIFEST - added:
variable TEMPLATE_SETUP
- added:
class ProgressTracker(object) - added:
constructor ProgressTracker(frequency=…) - added:
function create_pretraining_model(nlp, tok2vec, objective=…, nr_char=…) - added:
function get_vectors_loss(ops, docs, prediction, objective=…) - added:
function make_docs(nlp, batch, min_length, max_length) - added:
function make_update(model, docs, optimizer, drop=…, objective=…) - added:
function pretrain(texts_loc, vectors_model, output_dir, width=…, conv_depth=…, cnn_pieces=…, sa_depth=…, cnn_window=…, bilstm_depth=…, use_chars=…, embed_rows=…, loss_func=…, use_vectors=…, dropout=…, n_iter=…, batch_size=…, max_length=…, min_length=…, seed=…, n_save_every=…, init_tok2vec=…, epoch_start=…) - added:
method ProgressTracker.update(epoch, loss, docs)
- added:
function parse_texts(nlp, texts) - added:
function profile(model, inputs=…, n_texts=…)
- removed:
function check_dirs(output_path, train_path, dev_path) - removed:
function evaluate(Language, gold_tuples, output_path) - removed:
function print_progress(itn, nr_weight, nr_active_feat, **scores) - removed:
function print_results(scorer) - removed:
function train_config(config) - removed:
function train_model(Language, train_data, dev_data, output_path, tagger_cfg, parser_cfg, entity_cfg, n_iter) - changed:
function train(language, output_dir, train_data, dev_data, n_iter, tagger, parser, ner, parser_L1) → function train(lang, output_path, train_path, dev_path, raw_text=…, base_model=…, pipeline=…, replace_components=…, vectors=…, width=…, conv_depth=…, cnn_window=…, cnn_pieces=…, bilstm_depth=…, embed_rows=…, n_iter=…, n_early_stopping=…, n_examples=…, use_gpu=…, version=…, meta_path=…, init_tok2vec=…, parser_multitasks=…, entity_multitasks=…, noise_level=…, orth_variant_level=…, eval_beam_widths=…
- added:
function get_model_links(compat) - added:
function get_model_pkgs(compat, all_models) - added:
function get_model_row(compat, name, data, msg, model_type=…) - added:
function is_compat(compat, name, version) - added:
function is_model_path(model_path) - added:
function reformat_version(version) - added:
function validate()
- removed:
variable fix_text - removed:
variable intern - removed:
variable json_dumps - added:
function b_to_str(b_str) - added:
function import_file(name, loc) - added:
function symlink_remove(link) - added:
function unescape_unicode(string) - added:
variable CudaStream - added:
variable class_types - added:
variable copy_array - added:
variable copy_reg - added:
variable cupy - added:
variable is_python_pre_3_5 - added:
variable izip - added:
variable path2str - added:
variable pickle
- removed:
class German(Language) - removed:
class German.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field German.Defaults.lex_attr_getters - removed:
field German.Defaults.stop_words - removed:
field German.Defaults.tag_map - removed:
field German.Defaults.tokenizer_exceptions - removed:
field German.lang
- removed:
export STOP_WORDS - removed:
export TAG_MAP - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TAG_MAP - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
variable TAG_MAP
- removed:
variable ORTH_ONLY - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
class ModelDownload - removed:
function align_tokens(ref, indices) - removed:
function detokenize(token_rules, words) - removed:
function fix_glove_vectors_loading(overrides) - removed:
function match_best_version(target_name, target_version, path) - removed:
function read_lang_data(package) - removed:
function resolve_model_name(name) - removed:
function split_data_name(name) - removed:
static method ModelDownload.de(*args, **kwargs) - removed:
static method ModelDownload.en(*args, **kwargs) - removed:
static method ModelDownload.load(lang)
- added:
export DependencyRenderer - added:
export Doc - added:
export EntityRenderer - added:
export Errors - added:
export Span - added:
export Warnings - added:
export b_to_str - added:
export is_in_jupyter - added:
function app(environ, start_response) - added:
function get_doc_settings(doc) - added:
function parse_deps(orig_doc, options=…) - added:
function parse_ents(doc, options=…) - added:
function render(docs, style=…, page=…, minify=…, jupyter=…, options=…, manual=…) - added:
function serve(docs, style=…, page=…, minify=…, options=…, manual=…, port=…, host=…) - added:
function set_render_wrapper(func) - added:
variable RENDER_WRAPPER
- added:
class DependencyRenderer(object) - added:
class EntityRenderer(object) - added:
constructor DependencyRenderer(options=…) - added:
constructor EntityRenderer(options=…) - added:
field DependencyRenderer.style - added:
field EntityRenderer.style - added:
method DependencyRenderer.get_arc(x_start, y, y_curve, x_end) - added:
method DependencyRenderer.get_arrowhead(direction, x, y, end) - added:
method DependencyRenderer.get_levels(arcs) - added:
method DependencyRenderer.render(parsed, page=…, minify=…) - added:
method DependencyRenderer.render_arrow(label, start, end, direction, i) - added:
method DependencyRenderer.render_svg(render_id, words, arcs) - added:
method DependencyRenderer.render_word(text, tag, lemma, i) - added:
method EntityRenderer.render(parsed, page=…, minify=…) - added:
method EntityRenderer.render_ents(text, spans, title) - added:
variable DEFAULT_DIR - added:
variable DEFAULT_LANG
- added:
variable TPL_DEP_ARCS - added:
variable TPL_DEP_SVG - added:
variable TPL_DEP_WORDS - added:
variable TPL_DEP_WORDS_LEMMA - added:
variable TPL_ENT - added:
variable TPL_ENTS - added:
variable TPL_ENT_RTL - added:
variable TPL_FIGURE - added:
variable TPL_PAGE - added:
variable TPL_TITLE
- removed:
class English(Language) - removed:
class English.Defaults(Language.Defaults) - removed:
constructor English(**overrides) - removed:
export LANG - removed:
export Language - removed:
export Lemmatizer - removed:
export Tokenizer - removed:
export Vocab - removed:
export fix_glove_vectors_loading - removed:
field English.Defaults.lemma_exc - removed:
field English.Defaults.lemma_index - removed:
field English.Defaults.lemma_rules - removed:
field English.Defaults.lex_attr_getters - removed:
field English.Defaults.morph_rules - removed:
field English.Defaults.stop_words - removed:
field English.Defaults.tag_map - removed:
field English.Defaults.tokenizer_exceptions - removed:
field English.lang
- removed:
export LEMMA_EXC - removed:
export LEMMA_INDEX - removed:
export LEMMA_RULES - removed:
export MORPH_RULES - removed:
export STOP_WORDS - removed:
export TAG_MAP - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TAG_MAP - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
export ADJECTIVES - removed:
export ADJECTIVES_IRREG - removed:
export ADJECTIVE_RULES - removed:
export ADVERBS - removed:
export ADVERBS_IRREG - removed:
export NOUNS - removed:
export NOUNS_IRREG - removed:
export NOUN_RULES - removed:
export PUNCT_RULES - removed:
export VERBS - removed:
export VERBS_IRREG - removed:
export VERB_RULES - removed:
variable EXC - removed:
variable INDEX - removed:
variable RULES
- removed:
variable MORPH_RULES
- removed:
variable TAG_MAP
- removed:
variable ABBREVIATIONS - removed:
variable EXC - removed:
variable EXCLUDE_EXC - removed:
variable ORTH_ONLY - removed:
variable OTHER - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable NUM_WORDS - removed:
variable ORDINAL_WORDS - removed:
variable STOP_WORDS
- added:
class AlignmentError(ValueError) - added:
class Errors(object) - added:
class MatchPatternError(ValueError) - added:
class TempErrors(object) - added:
class Warnings(object) - added:
constructor MatchPatternError(key, errors) - added:
field Errors.E001 - added:
field Errors.E002 - added:
field Errors.E003 - added:
field Errors.E004 - added:
field Errors.E005 - added:
field Errors.E006 - added:
field Errors.E007 - added:
field Errors.E008 - added:
field Errors.E009 - added:
field Errors.E010 - added:
field Errors.E011 - added:
field Errors.E012 - added:
field Errors.E013 - added:
field Errors.E014 - added:
field Errors.E015 - added:
field Errors.E016 - added:
field Errors.E017 - added:
field Errors.E018 - added:
field Errors.E019 - added:
field Errors.E020 - added:
field Errors.E021 - added:
field Errors.E022 - added:
field Errors.E023 - added:
field Errors.E024 - added:
field Errors.E025 - added:
field Errors.E026 - added:
field Errors.E027 - added:
field Errors.E028 - added:
field Errors.E029 - added:
field Errors.E030 - added:
field Errors.E031 - added:
field Errors.E032 - added:
field Errors.E033 - added:
field Errors.E034 - added:
field Errors.E035 - added:
field Errors.E036 - added:
field Errors.E037 - added:
field Errors.E038 - added:
field Errors.E039 - added:
field Errors.E040 - added:
field Errors.E041 - added:
field Errors.E042 - added:
field Errors.E043 - added:
field Errors.E044 - added:
field Errors.E045 - added:
field Errors.E046 - added:
field Errors.E047 - added:
field Errors.E048 - added:
field Errors.E049 - added:
field Errors.E050 - added:
field Errors.E051 - added:
field Errors.E052 - added:
field Errors.E053 - added:
field Errors.E054 - added:
field Errors.E055 - added:
field Errors.E056 - added:
field Errors.E057 - added:
field Errors.E058 - added:
field Errors.E059 - added:
field Errors.E060 - added:
field Errors.E061 - added:
field Errors.E062 - added:
field Errors.E063 - added:
field Errors.E064 - added:
field Errors.E065 - added:
field Errors.E066 - added:
field Errors.E067 - added:
field Errors.E068 - added:
field Errors.E069 - added:
field Errors.E070 - added:
field Errors.E071 - added:
field Errors.E072 - added:
field Errors.E073 - added:
field Errors.E074 - added:
field Errors.E075 - added:
field Errors.E076 - added:
field Errors.E077 - added:
field Errors.E078 - added:
field Errors.E079 - added:
field Errors.E080 - added:
field Errors.E081 - added:
field Errors.E082 - added:
field Errors.E083 - added:
field Errors.E084 - added:
field Errors.E085 - added:
field Errors.E086 - added:
field Errors.E087 - added:
field Errors.E088 - added:
field Errors.E089 - added:
field Errors.E090 - added:
field Errors.E091 - added:
field Errors.E092 - added:
field Errors.E093 - added:
field Errors.E094 - added:
field Errors.E095 - added:
field Errors.E096 - added:
field Errors.E097 - added:
field Errors.E098 - added:
field Errors.E099 - added:
field Errors.E100 - added:
field Errors.E101 - added:
field Errors.E102 - added:
field Errors.E103 - added:
field Errors.E104 - added:
field Errors.E105 - added:
field Errors.E106 - added:
field Errors.E107 - added:
field Errors.E108 - added:
field Errors.E109 - added:
field Errors.E110 - added:
field Errors.E111 - added:
field Errors.E112 - added:
field Errors.E113 - added:
field Errors.E114 - added:
field Errors.E115 - added:
field Errors.E116 - added:
field Errors.E117 - added:
field Errors.E118 - added:
field Errors.E119 - added:
field Errors.E120 - added:
field Errors.E121 - added:
field Errors.E122 - added:
field Errors.E123 - added:
field Errors.E124 - added:
field Errors.E125 - added:
field Errors.E126 - added:
field Errors.E127 - added:
field Errors.E128 - added:
field Errors.E129 - added:
field Errors.E130 - added:
field Errors.E131 - added:
field Errors.E132 - added:
field Errors.E133 - added:
field Errors.E134 - added:
field Errors.E135 - added:
field Errors.E136 - added:
field Errors.E137 - added:
field Errors.E138 - added:
field Errors.E139 - added:
field Errors.E140 - added:
field Errors.E141 - added:
field Errors.E142 - added:
field Errors.E143 - added:
field Errors.E144 - added:
field Errors.E145 - added:
field Errors.E146 - added:
field Errors.E147 - added:
field Errors.E148 - added:
field Errors.E149 - added:
field Errors.E150 - added:
field Errors.E151 - added:
field Errors.E152 - added:
field Errors.E153 - added:
field Errors.E154 - added:
field Errors.E155 - added:
field Errors.E156 - added:
field Errors.E157 - added:
field Errors.E158 - added:
field Errors.E159 - added:
field Errors.E160 - added:
field Errors.E161 - added:
field Errors.E162 - added:
field Errors.E163 - added:
field Errors.E164 - added:
field Errors.E165 - added:
field Errors.E166 - added:
field Errors.E167 - added:
field Errors.E168 - added:
field Errors.E169 - added:
field Errors.E170 - added:
field Errors.E171 - added:
field Errors.E172 - added:
field Errors.E173 - added:
field Errors.E174 - added:
field Errors.E175 - added:
field Errors.E176 - added:
field Errors.E177 - added:
field Errors.E178 - added:
field Errors.E179 - added:
field Errors.E180 - added:
field Errors.E181 - added:
field Errors.E182 - added:
field Errors.E183 - added:
field Errors.E184 - added:
field Errors.E185 - added:
field Errors.E186 - added:
field Errors.E187 - added:
field Errors.E188 - added:
field Errors.E189 - added:
field Errors.E190 - added:
field Errors.E191 - added:
field Errors.E192 - added:
field Errors.E193 - added:
field Errors.E194 - added:
field Errors.E195 - added:
field Errors.E196 - added:
field Errors.E197 - added:
field Errors.E198 - added:
field Errors.E199 - added:
field Errors.E200 - added:
field Errors.E201 - added:
field Errors.E202 - added:
field TempErrors.T003 - added:
field TempErrors.T004 - added:
field TempErrors.T007 - added:
field TempErrors.T008 - added:
field Warnings.W001 - added:
field Warnings.W002 - added:
field Warnings.W003 - added:
field Warnings.W004 - added:
field Warnings.W005 - added:
field Warnings.W006 - added:
field Warnings.W007 - added:
field Warnings.W008 - added:
field Warnings.W009 - added:
field Warnings.W010 - added:
field Warnings.W011 - added:
field Warnings.W012 - added:
field Warnings.W013 - added:
field Warnings.W014 - added:
field Warnings.W015 - added:
field Warnings.W016 - added:
field Warnings.W017 - added:
field Warnings.W018 - added:
field Warnings.W019 - added:
field Warnings.W020 - added:
field Warnings.W021 - added:
field Warnings.W022 - added:
field Warnings.W023 - added:
field Warnings.W024 - added:
field Warnings.W025 - added:
field Warnings.W026 - added:
field Warnings.W027 - added:
field Warnings.W028 - added:
field Warnings.W029 - added:
field Warnings.W030 - added:
field Warnings.W031 - added:
field Warnings.W032 - added:
field Warnings.W033 - added:
field Warnings.W034 - added:
field Warnings.W035 - added:
function add_codes(err_cls)
- removed:
class Spanish(Language) - removed:
class Spanish.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Spanish.Defaults.lex_attr_getters - removed:
field Spanish.Defaults.stop_words - removed:
field Spanish.Defaults.tag_map - removed:
field Spanish.Defaults.tokenizer_exceptions - removed:
field Spanish.lang
- removed:
export STOP_WORDS - removed:
export TAG_MAP - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TAG_MAP - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
variable TAG_MAP
- removed:
variable ORTH_ONLY - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
class Finnish(Language) - removed:
class Finnish.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Finnish.Defaults.lex_attr_getters - removed:
field Finnish.Defaults.stop_words - removed:
field Finnish.Defaults.tokenizer_exceptions - removed:
field Finnish.lang
- removed:
export STOP_WORDS - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
variable TOKENIZER_EXCEPTIONS
- removed:
class French(Language) - removed:
class FrenchDefaults(BaseDefaults) - removed:
export BaseDefaults - removed:
export LANG - removed:
export Language - removed:
export TOKENIZER_INFIXES - removed:
export TOKENIZER_SUFFIXES - removed:
field French.Defaults - removed:
field French.lang - removed:
field FrenchDefaults.infixes - removed:
field FrenchDefaults.lex_attr_getters - removed:
field FrenchDefaults.stop_words - removed:
field FrenchDefaults.suffixes - removed:
field FrenchDefaults.token_match - removed:
static method FrenchDefaults.create_tokenizer(nlp=…)
- removed:
export STOP_WORDS - removed:
export TOKEN_MATCH - removed:
export get_tokenizer_exceptions - removed:
variable STOP_WORDS
- removed:
export TOKENIZER_INFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable NUM_WORDS - removed:
variable ORDINAL_WORDS - removed:
variable STOP_WORDS
- removed:
function get_tokenizer_exceptions() - removed:
variable TOKEN_MATCH
- added:
variable GIT_VERSION
- removed:
class Hebrew(Language) - removed:
class Hebrew.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Hebrew.Defaults.lex_attr_getters - removed:
field Hebrew.Defaults.stop_words - removed:
field Hebrew.Defaults.tokenizer_exceptions - removed:
field Hebrew.lang
- removed:
export STOP_WORDS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
class Hungarian(Language) - removed:
class Hungarian.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
export TOKEN_MATCH - removed:
field Hungarian.Defaults.infixes - removed:
field Hungarian.Defaults.lex_attr_getters - removed:
field Hungarian.Defaults.prefixes - removed:
field Hungarian.Defaults.stop_words - removed:
field Hungarian.Defaults.suffixes - removed:
field Hungarian.Defaults.token_match - removed:
field Hungarian.Defaults.tokenizer_exceptions - removed:
field Hungarian.lang
- removed:
export STOP_WORDS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS - removed:
variable TOKENIZER_INFIXES - removed:
variable TOKENIZER_PREFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable TOKENIZER_INFIXES - removed:
variable TOKENIZER_PREFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable STOP_WORDS
- removed:
variable ABBREVIATIONS - removed:
variable NUMERIC_EXP - removed:
variable NUMS - removed:
variable ORD_NUM_OR_DATE - removed:
variable OTHER_EXC - removed:
variable TIME_EXP - removed:
variable TOKEN_MATCH
- removed:
class Italian(Language) - removed:
class Italian.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Italian.Defaults.lex_attr_getters - removed:
field Italian.Defaults.stop_words - removed:
field Italian.Defaults.tokenizer_exceptions - removed:
field Italian.lang
- removed:
export STOP_WORDS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
class Japanese(Language) - removed:
class JapaneseDefaults(BaseDefaults) - removed:
class JapaneseTagger(object) - removed:
class JapaneseTokenizer(object) - removed:
constructor JapaneseTagger(vocab) - removed:
constructor JapaneseTokenizer(cls, nlp=…) - removed:
export BaseDefaults - removed:
export Doc - removed:
export LANG - removed:
export Language - removed:
export Tagger - removed:
export Tokenizer - removed:
field Japanese.Defaults - removed:
field Japanese.lang - removed:
field JapaneseDefaults.tag_map - removed:
function detailed_tokens(tokenizer, text) - removed:
function resolve_pos(token) - removed:
function try_mecab_import() - removed:
method Japanese.make_doc(text) - removed:
static method JapaneseDefaults.create_tagger(tokenizer) - removed:
static method JapaneseDefaults.create_tokenizer(nlp=…) - removed:
variable DETAILS_KEY - removed:
variable ShortUnitWord
- removed:
export STOP_WORDS - removed:
export TAG_MAP - removed:
variable STOP_WORDS - removed:
variable TAG_MAP
- removed:
variable STOP_WORDS
- removed:
variable TAG_MAP
- added:
class Afrikaans(Language) - added:
field Afrikaans.Defaults - added:
field Afrikaans.lang
- added:
variable STOP_WORDS
- added:
class Amharic(Language) - added:
field Amharic.Defaults - added:
field Amharic.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Arabic(Language) - added:
field Arabic.Defaults - added:
field Arabic.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Bulgarian(Language) - added:
field Bulgarian.Defaults - added:
field Bulgarian.lang
- added:
variable STOP_WORDS
- added:
class Bengali(Language) - added:
field Bengali.Defaults - added:
field Bengali.lang
- added:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Catalan(Language) - added:
field Catalan.Defaults - added:
field Catalan.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable ELISION - added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
variable ALPHA - added:
variable ALPHA_LOWER - added:
variable ALPHA_UPPER - added:
variable CONCAT_ICONS - added:
variable CONCAT_QUOTES - added:
variable CURRENCY - added:
variable HYPHENS - added:
variable ICONS - added:
variable LATIN - added:
variable LATIN_BASIC - added:
variable LATIN_LOWER - added:
variable LATIN_LOWER_BASIC - added:
variable LATIN_UPPER - added:
variable LATIN_UPPER_BASIC - added:
variable LIST_CURRENCY - added:
variable LIST_ELLIPSES - added:
variable LIST_HYPHENS - added:
variable LIST_ICONS - added:
variable LIST_PUNCT - added:
variable LIST_QUOTES - added:
variable LIST_UNITS - added:
variable PUNCT - added:
variable UNITS - added:
variable group_chars - added:
variable merge_chars - added:
variable split_chars
- added:
class Czech(Language) - added:
field Czech.Defaults - added:
field Czech.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Danish(Language) - added:
field Danish.Defaults - added:
field Danish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class German(Language) - added:
field German.Defaults - added:
field German.lang
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Greek(Language) - added:
field Greek.Defaults - added:
field Greek.lang
- added:
function get_pos_from_wiktionary()
- added:
class GreekLemmatizer(Lemmatizer) - added:
method GreekLemmatizer.lemmatize(string, index, exceptions, rules)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
function merge_chars(char) - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES - added:
variable UNITS
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class English(Language) - added:
field English.Defaults - added:
field English.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES
- added:
variable STOP_WORDS - added:
variable contractions
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Spanish(Language) - added:
field Spanish.Defaults - added:
field Spanish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function is_verb_token(token) - added:
function noun_bounds(doc, root, np_left_deps, np_right_deps, stop_deps) - added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Estonian(Language) - added:
field Estonian.Defaults - added:
field Estonian.lang
- added:
variable STOP_WORDS
- added:
class Basque(Language) - added:
field Basque.Defaults - added:
field Basque.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Persian(Language) - added:
field Persian.Defaults - added:
field Persian.lang
- added:
variable VERBS_EXC - added:
variable narrative_ends - added:
variable present_ends - added:
variable simple_ends - added:
variable verb_roots - added:
variable with_nots
- added:
function like_num(text) - added:
variable LEX_ATTRS - added:
variable MIM - added:
variable YE_NUN - added:
variable ZWNJ_O_MIM
- added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Finnish(Language) - added:
field Finnish.Defaults - added:
field Finnish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable DASHES - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class French(Language) - added:
field French.Defaults - added:
field French.lang
- added:
class FrenchLemmatizer(Lemmatizer) - added:
method FrenchLemmatizer.adj(string, morphology=…) - added:
method FrenchLemmatizer.lemmatize(string, index, exceptions, rules) - added:
method FrenchLemmatizer.lookup(string, orth=…) - added:
method FrenchLemmatizer.noun(string, morphology=…) - added:
method FrenchLemmatizer.punct(string, morphology=…) - added:
method FrenchLemmatizer.verb(string, morphology=…)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable ELISION - added:
variable HYPHENS - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
function lower_first_letter(text) - added:
function upper_first_letter(text) - added:
variable FR_BASE_EXCEPTIONS - added:
variable TOKENIZER_EXCEPTIONS - added:
variable TOKEN_MATCH - added:
variable orig_elision - added:
variable orig_hyphen
- added:
class Irish(Language) - added:
field Irish.Defaults - added:
field Irish.lang
- added:
function deduplicate(word) - added:
function devoice(word) - added:
function ends_dentals(word) - added:
function ends_with_vowel(word) - added:
function starts_with_vowel(word) - added:
variable broad_vowels - added:
variable consonants - added:
variable slender_vowels - added:
variable vowels
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Gujarati(Language) - added:
field Gujarati.Defaults - added:
field Gujarati.lang
- added:
variable STOP_WORDS
- added:
class Hebrew(Language) - added:
field Hebrew.Defaults - added:
field Hebrew.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Hindi(Language) - added:
field Hindi.Defaults - added:
field Hindi.lang
- added:
function like_num(text) - added:
function norm(string) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Croatian(Language) - added:
field Croatian.Defaults - added:
field Croatian.lang
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Hungarian(Language) - added:
field Hungarian.Defaults - added:
field Hungarian.lang
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS - added:
variable TOKEN_MATCH
- added:
class Armenian(Language) - added:
field Armenian.Defaults - added:
field Armenian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Indonesian(Language) - added:
field Indonesian.Defaults - added:
field Indonesian.lang
- added:
function is_currency(text) - added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable CURRENCY - added:
variable HTML_PREFIX - added:
variable HTML_SUFFIX - added:
variable LIST_CURRENCY - added:
variable MONTHS - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES - added:
variable UNITS
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Icelandic(Language) - added:
field Icelandic.Defaults - added:
field Icelandic.lang
- added:
variable STOP_WORDS
- added:
class Italian(Language) - added:
field Italian.Defaults - added:
field Italian.lang
- added:
variable ELISION - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Japanese(Language) - added:
field Japanese.Defaults - added:
field Japanese.lang - added:
method Japanese.make_doc(text)
- added:
variable STOP_WORDS
- added:
function noun_chunks(obj) - added:
variable SYNTAX_ITERATORS - added:
variable labels
- added:
variable TAG_BIGRAM_MAP
- added:
variable TAG_MAP
- added:
variable TAG_ORTH_MAP
- added:
class Kannada(Language) - added:
field Kannada.Defaults - added:
field Kannada.lang
- added:
variable STOP_WORDS
- added:
class Korean(Language) - added:
field Korean.Defaults - added:
field Korean.lang - added:
method Korean.make_doc(text)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Kyrgyz(Language) - added:
field Kyrgyz.Defaults - added:
field Kyrgyz.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Luxembourgish(Language) - added:
field Luxembourgish.Defaults - added:
field Luxembourgish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable ELISION - added:
variable TOKENIZER_INFIXES - added:
variable abbrev
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
function is_alpha(string) - added:
function is_ascii(text) - added:
function is_bracket(text) - added:
function is_currency(text) - added:
function is_digit(string) - added:
function is_left_punct(text) - added:
function is_lower(string) - added:
function is_punct(text) - added:
function is_quote(text) - added:
function is_right_punct(text) - added:
function is_space(string) - added:
function is_stop(string, stops=…) - added:
function is_title(string) - added:
function is_upper(string) - added:
function like_email(text) - added:
function like_num(text) - added:
function like_url(text) - added:
function lower(string) - added:
function prefix(string) - added:
function suffix(string) - added:
function word_shape(text) - added:
variable LEX_ATTRS
- added:
class Ligurian(Language) - added:
field Ligurian.Defaults - added:
field Ligurian.lang
- added:
variable ELISION - added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Lithuanian(Language) - added:
field Lithuanian.Defaults - added:
field Lithuanian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Latvian(Language) - added:
field Latvian.Defaults - added:
field Latvian.lang
- added:
variable STOP_WORDS
- added:
class Macedonian(Language) - added:
field Macedonian.Defaults - added:
field Macedonian.lang
- added:
class MacedonianLemmatizer(Lemmatizer) - added:
method MacedonianLemmatizer.lemmatize(string, index, exceptions, rules)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES - added:
variable sum_aux
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Malayalam(Language) - added:
field Malayalam.Defaults - added:
field Malayalam.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Marathi(Language) - added:
field Marathi.Defaults - added:
field Marathi.lang
- added:
variable STOP_WORDS
- added:
class Norwegian(Language) - added:
field Norwegian.Defaults - added:
field Norwegian.lang
- added:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Nepali(Language) - added:
field Nepali.Defaults - added:
field Nepali.lang
- added:
function like_num(text) - added:
function norm(string) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Dutch(Language) - added:
field Dutch.Defaults - added:
field Dutch.lang
- added:
class DutchLemmatizer(Lemmatizer) - added:
field DutchLemmatizer.univ_pos_name_variants - added:
method DutchLemmatizer.lemmatize(string, index, exceptions, rules) - added:
method DutchLemmatizer.lookup(string, orth=…)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS - added:
variable abbrevs
- added:
variable BASE_NORMS
- added:
class Polish(Language) - added:
field Polish.Defaults - added:
field Polish.lang
- added:
class PolishLemmatizer(Lemmatizer) - added:
method PolishLemmatizer.lemmatize(string, index, exceptions, rules) - added:
method PolishLemmatizer.lemmatize_adj(string, morphology, lookup_table) - added:
method PolishLemmatizer.lemmatize_noun(string, morphology, lookup_table) - added:
method PolishLemmatizer.lemmatize_verb(string, morphology, lookup_table) - added:
method PolishLemmatizer.lookup(string, orth=…)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Portuguese(Language) - added:
field Portuguese.Defaults - added:
field Portuguese.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
class Romanian(Language) - added:
field Romanian.Defaults - added:
field Romanian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Russian(Language) - added:
field Russian.Defaults - added:
field Russian.lang
- added:
class RussianLemmatizer(Lemmatizer) - added:
constructor RussianLemmatizer(lookups=…) - added:
function oc2ud(oc_tag) - added:
method RussianLemmatizer.lookup(string, orth=…) - added:
static method RussianLemmatizer.normalize_univ_pos(univ_pos) - added:
variable PUNCT_RULES
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Sanskrit(Language) - added:
field Sanskrit.Defaults - added:
field Sanskrit.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Sinhala(Language) - added:
field Sinhala.Defaults - added:
field Sinhala.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Slovak(Language) - added:
field Slovak.Defaults - added:
field Slovak.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Slovenian(Language) - added:
field Slovenian.Defaults - added:
field Slovenian.lang
- added:
variable STOP_WORDS
- added:
class Albanian(Language) - added:
field Albanian.Defaults - added:
field Albanian.lang
- added:
variable STOP_WORDS
- added:
class Serbian(Language) - added:
field Serbian.Defaults - added:
field Serbian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Swedish(Language) - added:
field Swedish.Defaults - added:
field Swedish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
variable ABBREVIATIONS - added:
variable TOKENIZER_EXCEPTIONS
- added:
class Tamil(Language) - added:
field Tamil.Defaults - added:
field Tamil.lang
- added:
function like_num(text) - added:
function suffix_filter(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Telugu(Language) - added:
field Telugu.Defaults - added:
field Telugu.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Thai(Language) - added:
field Thai.Defaults - added:
field Thai.lang - added:
method Thai.make_doc(text)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Tigrinya(Language) - added:
field Tigrinya.Defaults - added:
field Tigrinya.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Tagalog(Language) - added:
field Tagalog.Defaults - added:
field Tagalog.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
variable BASE_EXCEPTIONS - added:
variable TOKEN_MATCH - added:
variable URL_MATCH - added:
variable URL_PATTERN - added:
variable emoticons
- added:
class Turkish(Language) - added:
field Turkish.Defaults - added:
field Turkish.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable MORPH_RULES
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike) - added:
variable SYNTAX_ITERATORS
- added:
variable TOKENIZER_EXCEPTIONS - added:
variable TOKEN_MATCH
- added:
class Tatar(Language) - added:
field Tatar.Defaults - added:
field Tatar.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Ukrainian(Language) - added:
field Ukrainian.Defaults - added:
field Ukrainian.lang
- added:
class UkrainianLemmatizer(Lemmatizer) - added:
constructor UkrainianLemmatizer(lookups=…) - added:
function oc2ud(oc_tag) - added:
method UkrainianLemmatizer.lookup(string, orth=…) - added:
static method UkrainianLemmatizer.normalize_univ_pos(univ_pos) - added:
variable PUNCT_RULES
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class Urdu(Language) - added:
field Urdu.Defaults - added:
field Urdu.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class Vietnamese(Language) - added:
field Vietnamese.Defaults - added:
field Vietnamese.lang - added:
method Vietnamese.make_doc(text)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- added:
class MultiLanguage(Language) - added:
field MultiLanguage.Defaults - added:
field MultiLanguage.lang
- added:
class Yoruba(Language) - added:
field Yoruba.Defaults - added:
field Yoruba.lang
- added:
function like_num(text) - added:
function strip_accents_text(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class Chinese(Language) - added:
field Chinese.Defaults - added:
field Chinese.lang - added:
method Chinese.make_doc(text)
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TAG_MAP
- removed:
field BaseDefaults.entity_features - removed:
field BaseDefaults.lemma_exc - removed:
field BaseDefaults.lemma_index - removed:
field BaseDefaults.lemma_rules - removed:
field BaseDefaults.parser_features - removed:
field BaseDefaults.tagger_features - removed:
method Language.end_training(path=…) - removed:
method Language.save_to_directory(path) - removed:
static method BaseDefaults.add_vectors(nlp=…) - removed:
static method BaseDefaults.create_entity(nlp=…, **cfg) - removed:
static method BaseDefaults.create_matcher(nlp=…) - removed:
static method BaseDefaults.create_parser(nlp=…, **cfg) - removed:
static method BaseDefaults.create_pipeline(nlp=…) - removed:
static method BaseDefaults.create_tagger(nlp=…) - removed:
static method Language.setup_directory(path, **configs) - removed:
static method Language.train(path, gold_tuples, **configs) - changed:
constructor Language(**overrides) → constructor Language(vocab=…, make_doc=…, max_length=…, meta=…, **kwargs) - changed:
method Language.pipe(texts, tag=…, parse=…, entity=…, n_threads=…, batch_size=…) → method Language.pipe(texts, as_tuples=…, n_threads=…, batch_size=…, disable=…, cleanup=…, component_cfg=…, n_process=…) - changed:
static method BaseDefaults.create_lemmatizer(nlp=…) → static method BaseDefaults.create_lemmatizer(nlp=…, lookups=…) - added:
class DisabledPipes(list) - added:
class component(object) - added:
constructor DisabledPipes(nlp, *names) - added:
constructor component(name=…, assigns=…, requires=…, retokenizes=…) - added:
field BaseDefaults.is_base_form - added:
field BaseDefaults.paired_orth_variants - added:
field BaseDefaults.pipe_names - added:
field BaseDefaults.resources - added:
field BaseDefaults.single_orth_variants - added:
field BaseDefaults.syntax_iterators - added:
field BaseDefaults.url_match - added:
field BaseDefaults.writing_system - added:
field Language.factories - added:
method DisabledPipes.restore() - added:
method Language.add_pipe(component, name=…, before=…, after=…, first=…, last=…) - added:
method Language.begin_training(get_gold_tuples=…, sgd=…, component_cfg=…, **cfg) - added:
method Language.create_pipe(name, config=…) - added:
method Language.disable_pipes(*names) - added:
method Language.evaluate(docs_golds, verbose=…, batch_size=…, scorer=…, component_cfg=…) - added:
method Language.from_bytes(bytes_data, exclude=…, disable=…, **kwargs) - added:
method Language.from_disk(path, exclude=…, disable=…) - added:
method Language.get_pipe(name) - added:
method Language.has_pipe(name) - added:
method Language.make_doc(text) - added:
method Language.preprocess_gold(docs_golds) - added:
method Language.rehearse(docs, sgd=…, losses=…, config=…) - added:
method Language.remove_pipe(name) - added:
method Language.rename_pipe(old_name, new_name) - added:
method Language.replace_pipe(name, component) - added:
method Language.resume_training(sgd=…, **cfg) - added:
method Language.to_bytes(exclude=…, disable=…, **kwargs) - added:
method Language.to_disk(path, exclude=…, disable=…) - added:
method Language.update(docs, golds, drop=…, sgd=…, losses=…, component_cfg=…) - added:
method Language.use_params(params, **cfg) - added:
property Language.entity - added:
property Language.linker - added:
property Language.matcher - added:
property Language.meta - added:
property Language.parser - added:
property Language.path - added:
property Language.pipe_factories - added:
property Language.pipe_labels - added:
property Language.pipe_names - added:
property Language.tagger - added:
property Language.tensorizer - added:
static method BaseDefaults.create_lookups(nlp=…) - added:
variable ENABLE_PIPELINE_ANALYSIS
- removed:
variable ABBREVIATIONS
- removed:
variable EMOTICONS
- removed:
variable ENTITY_RULES - removed:
variable FALSE_POSITIVES
- removed:
variable TOKENIZER_INFIXES - removed:
variable TOKENIZER_PREFIXES - removed:
variable TOKENIZER_SUFFIXES
- removed:
variable TAG_MAP
- removed:
variable TOKEN_MATCH
- removed:
function expand_exc(excs, search, replace) - removed:
function strings_to_exc(orths) - removed:
function update_exc(exc, additions) - removed:
variable DET_LEMMA - removed:
variable ENT_ID - removed:
variable PRON_LEMMA - removed:
variable unicode
- removed:
function lemmatize(string, index, exceptions, rules) - removed:
method Lemmatizer.is_base_form(univ_pos, morphology=…) - changed:
constructor Lemmatizer(index, exceptions, rules) → constructor Lemmatizer(lookups, is_base_form=…, *args, **kwargs) - changed:
static method Lemmatizer.load(path, index=…, exc=…, rules=…) → static method Lemmatizer.load(*args, **kwargs) - added:
method Lemmatizer.adp(string, morphology=…) - added:
method Lemmatizer.det(string, morphology=…) - added:
method Lemmatizer.lemmatize(string, index, exceptions, rules) - added:
method Lemmatizer.lookup(string, orth=…) - added:
method Lemmatizer.num(string, morphology=…) - added:
method Lemmatizer.pron(string, morphology=…)
- added:
class Lookups(object) - added:
class Table(OrderedDict) - added:
constructor Lookups() - added:
constructor Table(name=…, data=…) - added:
method Lookups.add_table(name, data=…) - added:
method Lookups.from_bytes(bytes_data, **kwargs) - added:
method Lookups.from_disk(path, filename=…, **kwargs) - added:
method Lookups.get_table(name, default=…) - added:
method Lookups.has_table(name) - added:
method Lookups.remove_table(name) - added:
method Lookups.to_bytes(**kwargs) - added:
method Lookups.to_disk(path, filename=…, **kwargs) - added:
method Table.from_bytes(bytes_data) - added:
method Table.get(key, default=…) - added:
method Table.set(key, value) - added:
method Table.to_bytes() - added:
property Lookups.tables - added:
static method Table.from_dict(data, name=…) - added:
variable UNSET
- added:
export DependencyMatcher - added:
export Matcher - added:
export PhraseMatcher
- added:
export FeedForward - added:
export LayerNormalizedMaxout - added:
export Tok2Vec
- added:
function FeedForward(config) - added:
function LayerNormalizedMaxout(config)
- added:
function CharacterEmbed(config) - added:
function Doc2Feats(config) - added:
function MaxoutWindowEncoder(config) - added:
function MishWindowEncoder(config) - added:
function MultiHashEmbed(config) - added:
function PretrainedVectors(config) - added:
function Tok2Vec(config) - added:
function TorchBiLSTMEncoder(config)
- removed:
function align_chars(raw, ptb) - removed:
function align_section(raw_paragraphs, ptb_files) - removed:
function do_web(src_dir, onto_dir, out_dir) - removed:
function do_wsj(odc_dir, ptb_dir, out_dir) - removed:
function get_alignment(raw_by_para, ptb_by_file) - removed:
function get_sections(odc_dir, ptb_dir, out_dir) - removed:
function group_into_files(sents) - removed:
function group_into_paras(sents) - removed:
function main(odc_dir, onto_dir, out_dir) - removed:
function may_mkdir(parent, *subdirs) - removed:
function read_odc(section_loc) - removed:
function read_ptb_sec(ptb_sec_dir)
- removed:
function parse(sent_text, strip_bad_periods=…) - removed:
function split(text)
- removed:
function parse(string, strip_bad_periods=…) - removed:
function reform_string(tok) - removed:
function split(text) - removed:
function tags_to_entities(tags) - removed:
variable tag_re
- removed:
function sgml_extract(text_data) - removed:
variable datetime_re - removed:
variable docid_re - removed:
variable doctype_re - removed:
variable headline_re - removed:
variable post_re - removed:
variable postdate_re - removed:
variable poster_re - removed:
variable tag_re
- removed:
function parse(sent_text, strip_bad_periods=…) - removed:
function split(text)
- removed:
class Norwegian(Language) - removed:
class Norwegian.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Norwegian.Defaults.lex_attr_getters - removed:
field Norwegian.Defaults.stop_words - removed:
field Norwegian.Defaults.tokenizer_exceptions - removed:
field Norwegian.lang
- removed:
export MORPH_RULES - removed:
export STOP_WORDS - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable MORPH_RULES
- removed:
variable STOP_WORDS
- removed:
variable ORTH_ONLY - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
class Dutch(Language) - removed:
class Dutch.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Dutch.Defaults.lex_attr_getters - removed:
field Dutch.Defaults.stop_words - removed:
field Dutch.Defaults.tokenizer_exceptions - removed:
field Dutch.lang
- removed:
export STOP_WORDS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable NUM_WORDS - removed:
variable ORDINAL_WORDS - removed:
variable STOP_WORDS
- removed:
export DependencyTree - removed:
export PatternParser
- removed:
class PatternParser(object) - removed:
class Reader(object) - removed:
class Token(tuple) - removed:
class TokenStream(object) - removed:
class TokenStreamIterator(object) - removed:
constructor Reader(text) - removed:
constructor TokenStream(generator) - removed:
constructor TokenStreamIterator(stream) - removed:
field PatternParser.EDGE_BLOCK_BEGIN - removed:
field PatternParser.TOKEN_BLOCK_BEGIN - removed:
field PatternParser.TOKEN_BLOCK_END - removed:
field PatternParser.WHITESPACE - removed:
field PatternParser.name_re - removed:
field PatternParser.newline_re - removed:
field PatternParser.whitespace_re - removed:
field Token.lineno - removed:
field Token.type - removed:
field Token.value - removed:
method Reader.consume(count=…) - removed:
method Reader.find(needle, start=…, end=…) - removed:
method Reader.next() - removed:
method Reader.remaining() - removed:
method Token.hash() - removed:
method TokenStream.close() - removed:
method TokenStream.look() - removed:
method TokenStream.push(token) - removed:
static method PatternParser.add_node(node_token, pattern, name_token=…) - removed:
static method PatternParser.check_pattern(pattern) - removed:
static method PatternParser.compile_expression(expr) - removed:
static method PatternParser.parse(query) - removed:
static method PatternParser.parse_edge_attributes(string) - removed:
static method PatternParser.parse_edge_def(stream, pattern) - removed:
static method PatternParser.parse_node_attributes(string) - removed:
static method PatternParser.parse_node_def(stream, pattern) - removed:
static method PatternParser.tokenize(text) - removed:
variable TOKEN_INITIAL
- removed:
class DependencyPattern(Tree) - removed:
class DependencyTree(Tree) - removed:
class PatternMatch(object) - removed:
class Tree(object) - removed:
constructor DependencyTree(doc) - removed:
constructor PatternMatch(mapping, pattern, tree) - removed:
constructor Tree() - removed:
function find_adjacent_nodes(dep_tree, node, target_dep, node_attributes) - removed:
function match_edge(token_dep, target_dep) - removed:
function match_token(token, target_attributes, ignore_special_key=…, lower=…) - removed:
function subtree_in_graph(dep_tree_node, dep_tree, pattern_node, pattern) - removed:
method DependencyTree.match(pattern) - removed:
method DependencyTree.match_nodes(attr_dict, **kwargs) - removed:
method Tree.add_edge(u, v, dep=…) - removed:
method Tree.add_node(node, attr_dict=…) - removed:
method Tree.edges_iter(origin=…, data=…) - removed:
method Tree.is_connected() - removed:
method Tree.nodes_iter() - removed:
method Tree.number_of_edges() - removed:
method Tree.number_of_nodes() - removed:
property DependencyPattern.root_node - removed:
variable logger
- added:
export DependencyParser - added:
export EntityLinker - added:
export EntityRecognizer - added:
export EntityRuler - added:
export Morphologizer - added:
export Pipe - added:
export SentenceSegmenter - added:
export Sentencizer - added:
export SimilarityHook - added:
export Tagger - added:
export Tensorizer - added:
export TextCategorizer - added:
export merge_entities - added:
export merge_noun_chunks - added:
export merge_subtokens
- added:
class EntityRuler(object) - added:
constructor EntityRuler(nlp, phrase_matcher_attr=…, validate=…, **cfg) - added:
method EntityRuler.add_patterns(patterns) - added:
method EntityRuler.from_bytes(patterns_bytes, **kwargs) - added:
method EntityRuler.from_disk(path, **kwargs) - added:
method EntityRuler.to_bytes(**kwargs) - added:
method EntityRuler.to_disk(path, **kwargs) - added:
property EntityRuler.ent_ids - added:
property EntityRuler.labels - added:
property EntityRuler.patterns - added:
static method EntityRuler.from_nlp(nlp, **cfg) - added:
variable DEFAULT_ENT_ID_SEP
- added:
function merge_entities(doc) - added:
function merge_noun_chunks(doc) - added:
function merge_subtokens(doc, label=…)
- added:
class SentenceSegmenter(object) - added:
class SimilarityHook(Pipe) - added:
constructor SentenceSegmenter(vocab, strategy=…) - added:
constructor SimilarityHook(vocab, model=…, **cfg) - added:
method SimilarityHook.begin_training(_=…, pipeline=…, sgd=…, **kwargs) - added:
method SimilarityHook.pipe(docs, **kwargs) - added:
method SimilarityHook.predict(doc1, doc2) - added:
method SimilarityHook.update(doc1_doc2, golds, sgd=…, drop=…) - added:
static method SentenceSegmenter.split_on_punct(doc) - added:
static method SimilarityHook.Model(length)
- removed:
class Portuguese(Language) - removed:
class Portuguese.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Portuguese.Defaults.lex_attr_getters - removed:
field Portuguese.Defaults.stop_words - removed:
field Portuguese.Defaults.tokenizer_exceptions - removed:
field Portuguese.lang
- removed:
export STOP_WORDS - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable NUM_WORDS - removed:
variable ORDINAL_WORDS - removed:
variable STOP_WORDS
- removed:
variable CONTRACTIONS - removed:
variable ORTH_ONLY - removed:
variable TOKENIZER_EXCEPTIONS - removed:
variable adverbs - removed:
variable demonstrative_pronouns - removed:
variable personal_pronoun - removed:
variable undefined_pronouns
- removed:
class Russian(Language) - removed:
class RussianDefaults(Language.Defaults) - removed:
class RussianTokenizer(object) - removed:
constructor RussianTokenizer(spacy_tokenizer, cls, nlp=…) - removed:
export Doc - removed:
export LANG - removed:
export Language - removed:
field Russian.Defaults - removed:
field Russian.lang - removed:
field RussianDefaults.lex_attr_getters - removed:
field RussianDefaults.stop_words - removed:
field RussianDefaults.tokenizer_exceptions - removed:
static method RussianDefaults.create_tokenizer(nlp=…)
- removed:
export STOP_WORDS - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
variable TOKENIZER_EXCEPTIONS
- changed:
constructor Scorer(eval_punct=…) → constructor Scorer(eval_punct=…, pipeline=…) - changed:
method Scorer.score(tokens, gold, verbose=…, punct_labels=…) → method Scorer.score(doc, gold, verbose=…, punct_labels=…) - added:
class ROCAUCScore(object) - added:
constructor ROCAUCScore() - added:
method ROCAUCScore.score_set(cand, gold) - added:
property ROCAUCScore.score - added:
property Scorer.ents_per_type - added:
property Scorer.las_per_type - added:
property Scorer.textcat_score - added:
property Scorer.textcats_per_cat
- removed:
class Swedish(Language) - removed:
class Swedish.Defaults(Language.Defaults) - removed:
export LANG - removed:
export Language - removed:
field Swedish.Defaults.lex_attr_getters - removed:
field Swedish.Defaults.stop_words - removed:
field Swedish.Defaults.tokenizer_exceptions - removed:
field Swedish.lang
- removed:
export STOP_WORDS - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable LEMMA_RULES
- removed:
variable MORPH_RULES
- removed:
variable STOP_WORDS
- removed:
variable ABBREVIATIONS - removed:
variable EXC - removed:
variable ORTH_ONLY - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
class Thai(Language) - removed:
class ThaiDefaults(BaseDefaults) - removed:
export BaseDefaults - removed:
export Doc - removed:
export LANG - removed:
export Language - removed:
export TOKENIZER_EXCEPTIONS - removed:
export Tokenizer - removed:
field Thai.Defaults - removed:
field Thai.lang - removed:
field ThaiDefaults.lex_attr_getters - removed:
field ThaiDefaults.stop_words - removed:
field ThaiDefaults.tag_map - removed:
field ThaiDefaults.tokenizer_exceptions - removed:
method Thai.make_doc(text)
- removed:
export STOP_WORDS - removed:
export TAG_MAP - removed:
export TOKENIZER_EXCEPTIONS - removed:
variable STOP_WORDS - removed:
variable TAG_MAP - removed:
variable TOKENIZER_EXCEPTIONS
- removed:
variable STOP_WORDS
- removed:
variable TAG_MAP
- removed:
variable TOKENIZER_EXCEPTIONS
- added:
export Doc - added:
export DocBin - added:
export Span - added:
export Token
- removed:
function POS_tree(root, light=…, flat=…) - removed:
function format_POS(token, light, flat) - removed:
function merge_ents(doc) - removed:
function parse_tree(doc, light=…, flat=…)
- added:
class Underscore(object) - added:
constructor Underscore(extensions, obj, start=…, end=…) - added:
field Underscore.doc_extensions - added:
field Underscore.mutable_types - added:
field Underscore.span_extensions - added:
field Underscore.token_extensions - added:
function get_ext_args(**kwargs) - added:
function is_writable_attr(ext) - added:
method Underscore.get(name) - added:
method Underscore.has(name) - added:
method Underscore.set(name, value) - added:
static method Underscore.get_state() - added:
static method Underscore.load_state(state)
- removed:
class Trainer(object) - removed:
constructor Trainer(nlp, gold_tuples) - removed:
method Trainer.epochs(nr_epoch, augment_data=…, gold_preproc=…) - removed:
method Trainer.evaluate(dev_sents, gold_preproc=…) - removed:
method Trainer.make_docs(raw_text, paragraph_tuples) - removed:
method Trainer.make_golds(docs, paragraph_tuples) - removed:
method Trainer.update(doc, gold)
- removed:
function check_renamed_kwargs(renamed, kwargs) - removed:
function get_raw_input(description, default=…) - removed:
function parse_package_meta(package_path, package, require=…) - removed:
function print_markdown(data, **kwargs) - removed:
function print_msg(*text, **kwargs) - removed:
function print_table(data, **kwargs) - removed:
function read_json(location) - removed:
function sys_exit(*messages, **kwargs) - removed:
variable LANGUAGES - changed:
function get_lang_class(name) → function get_lang_class(lang) - added:
class DummyTokenizer(object) - added:
class SimpleFrozenDict(dict) - added:
class registry(object) - added:
field registry.architectures - added:
field registry.displacy_colors - added:
field registry.factories - added:
field registry.languages - added:
field registry.lookups - added:
function add_lookups(default_func, *lookups) - added:
function compounding(start, stop, compound) - added:
function decaying(start, stop, decay) - added:
function env_opt(name, default=…) - added:
function escape_html(text) - added:
function expand_exc(excs, search, replace) - added:
function filter_spans(spans) - added:
function fix_random_seed(seed=…) - added:
function from_bytes(bytes_data, setters, exclude) - added:
function from_disk(path, readers, exclude) - added:
function get_async(stream, numpy_array) - added:
function get_component_name(component) - added:
function get_cuda_stream(require=…, non_blocking=…) - added:
function get_json_validator(schema) - added:
function get_model_meta(path) - added:
function get_module_path(module) - added:
function get_package_path(name) - added:
function get_serialization_exclude(serializers, exclude, kwargs) - added:
function get_words_and_spaces(words, text) - added:
function is_in_jupyter() - added:
function is_package(name) - added:
function itershuffle(iterable, bufsize=…) - added:
function lang_class_is_loaded(lang) - added:
function load_language_data(path) - added:
function load_model(name, **overrides) - added:
function load_model_from_init_py(init_file, **overrides) - added:
function load_model_from_link(name, **overrides) - added:
function load_model_from_package(name, **overrides) - added:
function load_model_from_path(model_path, meta=…, **overrides) - added:
function make_layer(arch_config) - added:
function minibatch(items, size=…) - added:
function minibatch_by_words(items, size, tuples=…, count_words=…) - added:
function minify_html(html) - added:
function set_env_log(value) - added:
function stepping(start, stop, steps) - added:
function to_bytes(getters, exclude) - added:
function to_disk(path, writers, exclude) - added:
function update_exc(base_exceptions, *addition_dicts) - added:
function use_gpu(gpu_id) - added:
function validate_json(data, validator) - added:
function validate_schema(schema) - added:
method DummyTokenizer.from_bytes(_bytes_data, **kwargs) - added:
method DummyTokenizer.from_disk(_path, **kwargs) - added:
method DummyTokenizer.pipe(texts, **kwargs) - added:
method DummyTokenizer.to_bytes(**kwargs) - added:
method DummyTokenizer.to_disk(_path, **kwargs) - added:
method SimpleFrozenDict.pop(key, default=…) - added:
method SimpleFrozenDict.update(other) - added:
variable OOV_RANK - added:
variable cupy - added:
variable jsonschema
- removed:
class Chinese(Language) - removed:
export Doc - removed:
export Language - removed:
field Chinese.lang - removed:
method Chinese.make_doc(text)