Upgrading spacy 2 to 3
Breaking2.3.9 → 3.8.18Breaking3Platform1New10Changed7Fixed3Other6
Fetching 68 releases from GitHub…17 / 68
BreakingUpgrade verdict
3 breaking changes and a platform requirement change.
In plain terms
- I've taken some lengths to explain this because migrating off a dependency after breaking changes can be a sensitive topic.
- To support Python 3.13 we therefore create a new module, spacy.pipeline.factories, which contains the factory function implementations. getattr import shims have been added to the previous locations…
- A matching change has been made to the catalogue registry decorators.
- Drop support for Python 3.9 (end-of-life).
Must act 4 items need code or config changes
Python requirement changed in 3.8.18: !=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7 → <3.16,>=3.10.
Adds click <9.0.0,>=8.2.1, confection <2.0.0,>=1.3.4, jinja2, packaging >=20.0, pydantic <3.0.0,>=2.0.0, spacy-legacy <3.1.0,>=3.0.11, spacy-loggers <2.0.0,>=1.0.0, typer <1.0.0,>=0.3.0, weasel <2.0.0,>=1.0.1.
Drops blis, plac.
Installs 43 packages in all (was 17).
What you must change
Breaking3
I've taken some lengths to explain this because migrating off a dependency after breaking changes can be a sensitive topic.
To support Python 3.13 we therefore create a new module, spacy.pipeline.factories, which contains the factory function implementations. getattr import shims have been added to the previous locations…
spacy.pipeline.factories__getattr__A matching change has been made to the catalogue registry decorators.
spacy.registrationsPlatform1
Drop support for Python 3.9 (end-of-life).
API changes
Generated by DepDelta from published Python source. Not mixed with prose notes.
- removed:
function check_changes(root, db) - removed:
function get_hash(path) - removed:
function hash_add(base, path, db) - removed:
function hash_changed(base, path, db) - removed:
function load_hashes(filename) - removed:
function preserve_cwd(path, func, *args) - removed:
function process(base, filename, db) - removed:
function process_pyx(fromfile, tofile, language_level=…) - removed:
function run(root) - removed:
function save_hashes(hash_db, filename) - removed:
variable HASH_FILE - removed:
variable args - removed:
variable parser
- removed:
class Reddit(object) - removed:
constructor Reddit(file_path, meta_keys=…) - removed:
field Reddit.link_re - removed:
field Reddit.post_format_re - removed:
field Reddit.pre_format_re - removed:
field Reddit.url_re - removed:
function main(path) - removed:
method Reddit.get_meta(item) - removed:
method Reddit.is_valid(comment) - removed:
method Reddit.iter_files() - removed:
method Reddit.strip_tags(text) - removed:
variable BrokenPipeError - removed:
variable devnull
- removed:
class Corpus(object) - removed:
constructor Corpus(directory, nlp) - removed:
function iter_dir(loc) - removed:
function main(lang, in_dir, out_loc, negative=…, n_workers=…, window=…, size=…, min_count=…, nr_iter=…) - removed:
variable logger
- removed:
export ud_evaluate - removed:
export ud_train
- removed:
class TestAlignment(unittest.TestCase) - removed:
class UDError(Exception) - removed:
function evaluate(gold_ud, system_ud, deprel_weights=…, check_parse=…) - removed:
function evaluate_wrapper(args) - removed:
function load_conllu(file, check_parse=…) - removed:
function load_conllu_file(path) - removed:
function load_deprel_weights(weights_file) - removed:
function main() - removed:
method TestAlignment.test_alignment() - removed:
method TestAlignment.test_equal() - removed:
method TestAlignment.test_equal_with_multiword() - removed:
method TestAlignment.test_exception() - removed:
variable DEPREL - removed:
variable DEPS - removed:
variable FEATS - removed:
variable FORM - removed:
variable HEAD - removed:
variable ID - removed:
variable LEMMA - removed:
variable MISC - removed:
variable UPOS - removed:
variable XPOS
- removed:
function fetch_all_treebanks(ud_dir, languages, corpus, best_per_language) - removed:
function get_freq_tuples(my_list, print_total_threshold) - removed:
function load_default_model_sentencizer(lang) - removed:
function load_model(modelname, add_sentencizer=…) - removed:
function main(out_path, ud_dir, check_parse=…, langs=…, exclude_trained_models=…, exclude_multi=…, hide_freq=…, corpus=…, best_per_language=…) - removed:
function run_all_evals(models, treebanks, out_file, check_parse, print_freq_tasks) - removed:
function run_single_eval(nlp, loading_time, print_name, text_path, gold_ud, tmp_output_path, out_file, print_header, check_parse, print_freq_tasks) - removed:
function split_text(text) - removed:
variable ALL_LANGUAGES - removed:
variable EVAL_NO_PARSE - removed:
variable EVAL_PARSE - removed:
variable PRINT_FREQ - removed:
variable PRINT_TOTAL - removed:
variable space_re
- removed:
function evaluate(nlp, text_loc, gold_loc, sys_loc, limit=…) - removed:
function get_token_split_end(token) - removed:
function get_token_split_start(token) - removed:
function guess_fused_orths(word, ud_forms) - removed:
function initialize_pipeline(nlp, docs, golds, config, device) - removed:
function load_nlp(experiments_dir, corpus) - removed:
function main(test_data_dir, experiment_dir, corpus) - removed:
function print_results(name, ud_scores) - removed:
function read_conllu(file_) - removed:
function split_text(text) - removed:
function write_conllu(docs, file_) - removed:
variable Fused_begin - removed:
variable Fused_inside - removed:
variable space_re
- removed:
class Config(object) - removed:
class Dataset(object) - removed:
class TreebankPaths(object) - removed:
constructor Config(vectors=…, max_doc_length=…, multitask_tag=…, multitask_sent=…, multitask_dep=…, multitask_vectors=…, bilstm_depth=…, nr_epoch=…, min_batch_size=…, max_batch_size=…, batch_by_words=…, dropout=…, conv_depth=…, subword_features=…, vectors_dir=…, pretrained_tok2vec=…) - removed:
constructor Dataset(path, section) - removed:
constructor TreebankPaths(ud_path, treebank, **cfg) - removed:
function evaluate(nlp, text_loc, gold_loc, sys_loc, limit=…) - removed:
function get_token_conllu(token, i) - removed:
function golds_to_gold_tuples(docs, golds) - removed:
function initialize_pipeline(nlp, docs, golds, config, device) - removed:
function load_nlp(corpus, config, vectors=…) - removed:
function main(ud_dir, parses_dir, corpus, config=…, limit=…, gpu_device=…, vectors_dir=…, use_oracle_segments=…) - removed:
function print_progress(itn, losses, ud_scores) - removed:
function read_conllu(file_) - removed:
function read_data(nlp, conllu_file, text_file, raw_text=…, oracle_segments=…, max_doc_length=…, limit=…) - removed:
function split_text(text) - removed:
function write_conllu(docs, file_) - removed:
static method Config.load(loc, vectors_dir=…) - removed:
variable space_re - removed:
variable torch
- removed:
export Warnings - removed:
export cli_info - removed:
export component - removed:
function info(model=…, markdown=…, silent=…) - changed:
function blank(name, **kwargs) → function blank(name: str, *, vocab: Union[Vocab, bool]=…, config: Union[Dict[str, Any], Config]=…, meta: Dict[str, Any]=…) -> Language - changed:
function load(name, **overrides) → function load(name: Union[str, Path], *, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, config: Union[Dict[str, Any], Config]=…) -> Language - added:
export ConfigSchema - added:
export ConfigSchemaInit - added:
export ConfigSchemaNlp - added:
export ConfigSchemaPretrain - added:
export ConfigSchemaTraining - added:
export Example - added:
export Language - added:
export REGISTRY_POPULATED - added:
export Vocab - added:
export info - added:
export logger - added:
export populate_registry - added:
export setup_default_warnings
- removed:
function analyze_all_pipes(pipeline, warn=…) - removed:
function analyze_pipes(pipeline, name, pipe, index, warn=…) - removed:
function dot_to_dict(values) - removed:
function get_assigns_for_attr(pipeline, attr) - removed:
function get_requires_for_attr(pipeline, attr) - removed:
function print_summary(nlp, pretty=…, no_print=…) - removed:
function validate_attrs(values)
- removed:
export init_model - removed:
export link - removed:
export train - added:
export app - added:
export apply - added:
export assemble_cli - added:
export benchmark_speed_cli - added:
export debug_config - added:
export debug_diff - added:
export debug_model - added:
export download_module - added:
export fill_config - added:
export find_function - added:
export find_threshold - added:
export init_config - added:
export init_pipeline_cli - added:
export project_assets - added:
export project_clone - added:
export project_document - added:
export project_pull - added:
export project_push - added:
export project_run - added:
export project_update_dvc - added:
export setup_cli - added:
export train_cli - added:
function link(*args, **kwargs)
- added:
function apply(data_path: Path, output_file: Path, model: str, json_field: str, batch_size: int, n_process: int) - added:
function apply_cli(model: str=…, data_path: Path=…, output_file: Path=…, code_path: Optional[Path]=…, text_key: str=…, force_overwrite: bool=…, use_gpu: int=…, batch_size: int=…, n_process: int=…) - added:
variable DocOrStrStream - added:
variable code_help - added:
variable force_msg - added:
variable gold_help - added:
variable out_help - added:
variable path_help
- added:
function assemble_cli(ctx: typer.Context, config_path: Path=…, output_path: Path=…, code_path: Optional[Path]=…, verbose: bool=…)
- added:
class Quartiles - added:
class time_context - added:
constructor Quartiles(sample: numpy.ndarray) -> None - added:
field Quartiles.iqr: float - added:
field Quartiles.q1: float - added:
field Quartiles.q2: float - added:
field Quartiles.q3: float - added:
function annotate(nlp: Language, docs: List[Doc], batch_size: Optional[int]) -> numpy.ndarray - added:
function benchmark(nlp: Language, docs: List[Doc], n_batches: int, batch_size: int, shuffle: bool) -> numpy.ndarray - added:
function benchmark_speed_cli(ctx: typer.Context, model: str=…, data_path: Path=…, batch_size: Optional[int]=…, no_shuffle: bool=…, use_gpu: int=…, n_batches: int=…, warmup_epochs: int=…, code_path: Optional[Path]=…) - added:
function bootstrap(x, statistic=…, iterations=…) -> numpy.ndarray - added:
function count_tokens(docs: Iterable[Doc]) -> int - added:
function print_mean_with_ci(sample: numpy.ndarray) - added:
function print_outliers(sample: numpy.ndarray) - added:
function warmup(nlp: Language, docs: List[Doc], warmup_epochs: int, batch_size: Optional[int]) -> numpy.ndarray
- removed:
variable FILE_TYPES - changed:
function autodetect_ner_format(input_data) → function autodetect_ner_format(input_data: str) -> Optional[str] - changed:
function convert(input_file, output_dir=…, file_type=…, n_sents=…, seg_sents=…, model=…, morphology=…, converter=…, lang=…) → function convert(input_path: Path, output_dir: Union[str, Path], *, file_type: str=…, n_sents: int=…, seg_sents: bool=…, model: Optional[str]=…, morphology: bool=…, merge_subtokens: bool=…, converter: str, ner_map: Optional[Path]=…, lang: Optional[str]=…, concatenate: bool=…, silent: bool=…, msg: Optional[Printer]=…) -> None - changed:
variable CONVERTERS → variable CONVERTERS: Mapping[str, Callable[..., Iterable[Doc]]] - added:
class FileTypes(str, Enum) - added:
field FileTypes.json - added:
field FileTypes.spacy - added:
function convert_cli(input_path: str=…, output_dir: Path=…, file_type: FileTypes=…, n_sents: int=…, seg_sents: bool=…, model: Optional[str]=…, morphology: bool=…, merge_subtokens: bool=…, converter: str=…, ner_map: Optional[Path]=…, lang: Optional[str]=…, concatenate: bool=…) - added:
function verify_cli_args(msg: Printer, input_path: Path, output_dir: Union[str, Path], file_type: str, converter: str, ner_map: Optional[Path]) - added:
variable AUTO
- removed:
export conll_ner2json - removed:
export conllu2json - removed:
export iob2json - removed:
export ner_jsonl2json
- removed:
function conll_ner2json(input_data, n_sents=…, seg_sents=…, model=…, no_print=…, **kwargs) - removed:
function n_sents_info(msg, n_sents) - removed:
function segment_docs(input_data, n_sents, doc_delimiter) - removed:
function segment_sents_and_docs(doc, n_sents, doc_delimiter, model=…, msg=…)
- removed:
function conllu2json(input_data, n_sents=…, use_morphology=…, lang=…, **_) - removed:
function create_doc(sentences, id) - removed:
function generate_sentence(sent, has_ner_tags) - removed:
function is_ner(tag) - removed:
function read_conllx(input_data, use_morphology=…, n=…) - removed:
function simplify_tags(iob)
- removed:
function iob2json(input_data, n_sents=…, no_print=…, *args, **kwargs) - removed:
function merge_sentences(docs, n_sents) - removed:
function read_iob(raw_sents)
- removed:
function ner_jsonl2json(input_data, lang=…, n_sents=…, use_morphology=…, **_)
- added:
function debug_config(config_path: Path, *, overrides: Dict[str, Any]=…, show_funcs: bool=…, show_vars: bool=…) - added:
function debug_config_cli(ctx: typer.Context, config_path: Path=…, code_path: Optional[Path]=…, show_funcs: bool=…, show_vars: bool=…) - added:
function get_registered_funcs(config: Config) -> List[Dict[str, Optional[Union[str, int]]]] - added:
function get_variables(config: Config) -> Dict[str, Any]
- changed:
function debug_data(lang, train_path, dev_path, tag_map_path=…, base_model=…, pipeline=…, ignore_warnings=…, verbose=…, no_format=…) → function debug_data(config_path: Path, *, config_overrides: Dict[str, Any]=…, ignore_warnings: bool=…, verbose: bool=…, no_format: bool=…, silent: bool=…) - added:
function debug_data_cli(ctx: typer.Context, config_path: Path=…, code_path: Optional[Path]=…, ignore_warnings: bool=…, verbose: bool=…, no_format: bool=…) - added:
variable BOUNDARY_DISTINCT_THRESHOLD - added:
variable SPAN_DISTINCT_THRESHOLD - added:
variable SPAN_LENGTH_THRESHOLD_PERCENTAGE
- added:
function debug_diff(config_path: Path, compare_to: Optional[Path], gpu: bool, optimize: Optimizations, pretraining: bool, markdown: bool) - added:
function debug_diff_cli(ctx: typer.Context, config_path: Path=…, compare_to: Optional[Path]=…, optimize: Optimizations=…, gpu: bool=…, pretraining: bool=…, markdown: bool=…)
- added:
function debug_model(config, resolved_train_config, nlp, pipe, *, print_settings: Optional[Dict[str, Any]]=…) - added:
function debug_model_cli(ctx: typer.Context, config_path: Path=…, component: str=…, layers: str=…, dimensions: bool=…, parameters: bool=…, gradients: bool=…, attributes: bool=…, P0: bool=…, P1: bool=…, P2: bool=…, P3: bool=…, use_gpu: int=…)
- removed:
function get_json(url, desc) - removed:
function require_package(name) - changed:
function download(model, direct=…, *pip_args) → function download(model: str, direct: bool=…, sdist: bool=…, custom_url: Optional[str]=…, *pip_args) -> None - changed:
function download_model(filename, user_pip_args=…) → function download_model(filename: str, user_pip_args: Optional[Sequence[str]]=…, custom_url: Optional[str]=…) -> None - changed:
function get_compatibility() → function get_compatibility() -> dict - changed:
function get_version(model, comp) → function get_version(model: str, comp: dict) -> str - added:
function download_cli(ctx: typer.Context, model: str=…, direct: bool=…, sdist: bool=…, url: str=…) - added:
function get_latest_version(model: str) -> str - added:
function get_model_filename(model_name: str, version: str, sdist: bool=…) -> str
- changed:
function evaluate(model, data_path, gpu_id=…, gold_preproc=…, displacy_path=…, displacy_limit=…, return_scores=…) → function evaluate(model: str, data_path: Path, output: Optional[Path]=…, use_gpu: int=…, gold_preproc: bool=…, displacy_path: Optional[Path]=…, displacy_limit: int=…, silent: bool=…, spans_key: str=…, per_component: bool=…) -> Dict[str, Any] - changed:
function render_parses(docs, output_path, model_name=…, limit=…, deps=…, ents=…) → function render_parses(docs: List[Doc], output_path: Path, model_name: str=…, limit: int=…, deps: bool=…, ents: bool=…, spans: bool=…) - added:
function evaluate_cli(model: str=…, data_path: Path=…, output: Optional[Path]=…, code_path: Optional[Path]=…, use_gpu: int=…, gold_preproc: bool=…, displacy_path: Optional[Path]=…, displacy_limit: int=…, per_component: bool=…, spans_key: str=…) - added:
function handle_scores_per_type(scores: Dict[str, Any], data: Dict[str, Any]=…, *, spans_key: str=…, silent: bool=…) -> Dict[str, Any] - added:
function print_prf_per_type(msg: Printer, scores: Dict[str, Dict[str, float]], name: str, type: str) -> None - added:
function print_textcats_auc_per_cat(msg: Printer, scores: Dict[str, Dict[str, float]]) -> None
- added:
function find_function(func_name: str, registry_name: str) -> Tuple[str, int] - added:
function find_function_cli(func_name: str=…, registry_name: Optional[str]=…)
- added:
function find_threshold(model: str, data_path: Path, pipe_name: str, threshold_key: str, scores_key: str, *, n_trials: int=…, use_gpu: int=…, gold_preproc: bool=…, silent: bool=…) -> Tuple[float, float, Dict[float, float]] - added:
function find_threshold_cli(model: str=…, data_path: Path=…, pipe_name: str=…, threshold_key: str=…, scores_key: str=…, n_trials: int=…, code_path: Optional[Path]=…, use_gpu: int=…, gold_preproc: bool=…, verbose: bool=…)
- removed:
function list_models() - removed:
function print_markdown(data, title=…) - changed:
function info(model=…, markdown=…, silent=…) → function info(model: Optional[str]=…, *, markdown: bool=…, silent: bool=…, exclude: Optional[List[str]]=…, url: bool=…) -> Union[str, dict] - added:
function get_markdown(data: Dict[str, Any], title: Optional[str]=…, exclude: Optional[List[str]]=…) -> str - added:
function info_cli(model: Optional[str]=…, markdown: bool=…, silent: bool=…, exclude: str=…, url: bool=…) - added:
function info_installed_model_url(model: str) -> Optional[str] - added:
function info_model(model: str, *, silent: bool=…) -> Dict[str, Any] - added:
function info_model_url(model: str) -> Dict[str, Any] - added:
function info_spacy() -> Dict[str, Any]
- added:
class InitValues - added:
class Optimizations(str, Enum) - added:
field InitValues.force_overwrite - added:
field InitValues.gpu - added:
field InitValues.lang - added:
field InitValues.optimize - added:
field InitValues.pipeline - added:
field InitValues.pretraining - added:
field Optimizations.accuracy - added:
field Optimizations.efficiency - added:
function fill_config(output_file: Path, base_path: Path, *, pretraining: bool=…, diff: bool=…, silent: bool=…) -> Tuple[Config, Config] - added:
function has_spacy_transformers() -> bool - added:
function init_config(*, lang: str=…, pipeline: List[str]=…, optimize: str=…, gpu: bool=…, pretraining: bool=…, silent: bool=…) -> Config - added:
function init_config_cli(output_file: Path=…, lang: str=…, pipeline: str=…, optimize: Optimizations=…, gpu: bool=…, pretraining: bool=…, force_overwrite: bool=…) - added:
function init_fill_config_cli(base_path: Path=…, output_file: Path=…, pretraining: bool=…, diff: bool=…, code_path: Optional[Path]=…) - added:
function save_config(config: Config, output_file: Path, is_stdout: bool=…, silent: bool=…) -> None - added:
function validate_config_for_pretrain(config: Config, msg: Printer) -> None - added:
variable RECOMMENDATIONS - added:
variable ROOT - added:
variable TEMPLATE_PATH
- removed:
function add_vectors(nlp, vectors_loc, truncate_vectors, prune_vectors, name=…) - removed:
function create_model(lang, lex_attrs, name=…, base_model=…) - removed:
function init_model(lang, output_dir, freqs_loc=…, clusters_loc=…, jsonl_loc=…, vectors_loc=…, truncate_vectors=…, prune_vectors=…, vectors_name=…, model_name=…, omit_extra_lookups=…, base_model=…) - removed:
function open_file(loc) - removed:
function read_attrs_from_deprecated(freqs_loc, clusters_loc) - removed:
function read_clusters(clusters_loc) - removed:
function read_freqs(freqs_loc, max_length=…, min_doc_freq=…, min_freq=…) - removed:
function read_vectors(vectors_loc, truncate_vectors=…) - removed:
variable DEFAULT_OOV_PROB - removed:
variable ftfy
- added:
function init_labels_cli(ctx: typer.Context, config_path: Path=…, output_path: Path=…, code_path: Optional[Path]=…, verbose: bool=…, use_gpu: int=…) - added:
function init_pipeline_cli(ctx: typer.Context, config_path: Path=…, output_path: Path=…, code_path: Optional[Path]=…, verbose: bool=…, use_gpu: int=…) - added:
function init_vectors_cli(lang: str=…, vectors_loc: Path=…, output_dir: Path=…, prune: int=…, truncate: int=…, mode: str=…, name: Optional[str]=…, verbose: bool=…, jsonl_loc: Optional[Path]=…, attr: str=…) - added:
function update_lexemes(nlp: Language, jsonl_loc: Path) -> None
- removed:
function link(origin, link_name, force=…, model_path=…)
- changed:
function create_file(file_path, contents) → function create_file(file_path: Path, contents: str) -> None - changed:
function generate_meta(model_path, existing_meta, msg) → function generate_meta(existing_meta: Dict[str, Any], msg: Printer) -> Dict[str, Any] - changed:
function package(input_dir, output_dir, meta_path=…, create_meta=…, force=…) → function package(input_dir: Path, output_dir: Path, meta_path: Optional[Path]=…, code_paths: List[Path]=…, name: Optional[str]=…, version: Optional[str]=…, create_meta: bool=…, create_sdist: bool=…, create_wheel: bool=…, require_parent: bool=…, force: bool=…, silent: bool=…) -> None - added:
function generate_readme(meta: Dict[str, Any]) -> str - added:
function get_build_formats(formats: List[str]) -> Tuple[bool, bool] - added:
function get_meta(model_path: Union[str, Path], existing_meta: Dict[str, Any], require_parent: bool=…) -> Dict[str, Any] - added:
function get_third_party_dependencies(config: Config, exclude: List[str]=…) -> List[str] - added:
function has_build() -> bool - added:
function has_wheel() -> bool - added:
function package_cli(input_dir: Path=…, output_dir: Path=…, code_paths: str=…, meta_path: Optional[Path]=…, create_meta: bool=…, name: Optional[str]=…, version: Optional[str]=…, build: str=…, force: bool=…, require_parent: bool=…) - added:
variable FILENAMES_DOCS
- removed:
class ProgressTracker(object) - removed:
constructor ProgressTracker(frequency=…) - removed:
function create_pretraining_model(nlp, tok2vec, objective=…, nr_char=…) - removed:
function get_vectors_loss(ops, docs, prediction, objective=…) - removed:
function make_docs(nlp, batch, min_length, max_length) - removed:
function make_update(model, docs, optimizer, drop=…, objective=…) - removed:
function pretrain(texts_loc, vectors_model, output_dir, width=…, conv_depth=…, cnn_pieces=…, sa_depth=…, cnn_window=…, bilstm_depth=…, use_chars=…, embed_rows=…, loss_func=…, use_vectors=…, dropout=…, n_iter=…, batch_size=…, max_length=…, min_length=…, seed=…, n_save_every=…, init_tok2vec=…, epoch_start=…) - removed:
method ProgressTracker.update(epoch, loss, docs) - added:
function pretrain_cli(ctx: typer.Context, config_path: Path=…, output_dir: Path=…, code_path: Optional[Path]=…, resume_path: Optional[Path]=…, epoch_resume: Optional[int]=…, use_gpu: int=…, skip_last: bool=…) - added:
function verify_cli_args(config_path, output_dir, resume_path, epoch_resume)
- changed:
function parse_texts(nlp, texts) → function parse_texts(nlp: Language, texts: Sequence[str]) -> None - changed:
function profile(model, inputs=…, n_texts=…) → function profile(model: str, inputs: Optional[Path]=…, n_texts: int=…) -> None - added:
function profile_cli(ctx: typer.Context, model: str=…, inputs: Optional[Path]=…, n_texts: int=…)
- changed:
function train(lang, output_path, train_path, dev_path, raw_text=…, base_model=…, pipeline=…, replace_components=…, vectors=…, width=…, conv_depth=…, cnn_window=…, cnn_pieces=…, bilstm_depth=…, embed_rows=…, n_iter=…, n_early_stopping=…, n_examples=…, use_gpu=…, version=…, meta_path=…, init_tok2vec=…, parser_multitasks=…, entity_multitasks=…, noise_level=…, orth_variant_level=…, eval_beam_widths=… → function train(config_path: Union[str, Path], output_path: Optional[Union[str, Path]]=…, *, use_gpu: int=…, overrides: Dict[str, Any]=…) - added:
function train_cli(ctx: typer.Context, config_path: Path=…, output_path: Optional[Path]=…, code_path: Optional[Path]=…, verbose: bool=…, use_gpu: int=…)
- removed:
function get_model_links(compat) - removed:
function get_model_row(compat, name, data, msg, model_type=…) - removed:
function is_compat(compat, name, version) - removed:
function is_model_path(model_path) - changed:
function get_model_pkgs(compat, all_models) → function get_model_pkgs(silent: bool=…) -> Tuple[dict, dict] - changed:
function reformat_version(version) → function reformat_version(version: str) -> str - changed:
function validate() → function validate() -> None - added:
function validate_cli()
- removed:
function b_to_str(b_str) - removed:
function import_file(name, loc) - removed:
function is_config(python2=…, python3=…, windows=…, linux=…, osx=…) - removed:
function symlink_remove(link) - removed:
function symlink_to(orig, dest) - removed:
function unescape_unicode(string) - removed:
variable basestring_ - removed:
variable bytes_ - removed:
variable class_types - removed:
variable input_ - removed:
variable is_python2 - removed:
variable is_python3 - removed:
variable is_python_pre_3_5 - removed:
variable izip - removed:
variable path2str - removed:
variable unicode_
- removed:
export b_to_str - changed:
function get_doc_settings(doc) → function get_doc_settings(doc: Doc) -> Dict[str, Any] - changed:
function parse_deps(orig_doc, options=…) → function parse_deps(orig_doc: Union[Doc, Span], options: Dict[str, Any]=…) -> Dict[str, Any] - changed:
function parse_ents(doc, options=…) → function parse_ents(doc: Doc, options: Dict[str, Any]=…) -> Dict[str, Any] - changed:
function render(docs, style=…, page=…, minify=…, jupyter=…, options=…, manual=…) → function render(docs: Union[Iterable[Union[Doc, Span, dict]], Doc, Span, dict], style: str=…, page: bool=…, minify: bool=…, jupyter: Optional[bool]=…, options: Dict[str, Any]=…, manual: bool=…) -> str - changed:
function serve(docs, style=…, page=…, minify=…, options=…, manual=…, port=…, host=…) → function serve(docs: Union[Iterable[Doc], Doc], style: str=…, page: bool=…, minify: bool=…, options: Dict[str, Any]=…, manual: bool=…, port: int=…, host: str=…, auto_select_port: bool=…) -> None - changed:
function set_render_wrapper(func) → function set_render_wrapper(func: Callable[[str], str]) -> None - added:
export SpanRenderer - added:
export find_available_port - added:
function parse_spans(doc: Doc, options: Dict[str, Any]=…) -> Dict[str, Any]
- changed:
class DependencyRenderer(object) → class DependencyRenderer - changed:
class EntityRenderer(object) → class EntityRenderer - changed:
constructor DependencyRenderer(options=…) → constructor DependencyRenderer(options: Dict[str, Any]=…) -> None - changed:
constructor EntityRenderer(options=…) → constructor EntityRenderer(options: Dict[str, Any]=…) -> None - changed:
method DependencyRenderer.get_arc(x_start, y, y_curve, x_end) → method DependencyRenderer.get_arc(x_start: int, y: int, y_curve: int, x_end: int) -> str - changed:
method DependencyRenderer.get_arrowhead(direction, x, y, end) → method DependencyRenderer.get_arrowhead(direction: str, x: int, y: int, end: int) -> str - changed:
method DependencyRenderer.get_levels(arcs) → method DependencyRenderer.get_levels(arcs: List[Dict[str, Any]]) -> Dict[Tuple[int, int, str], int] - changed:
method DependencyRenderer.render(parsed, page=…, minify=…) → method DependencyRenderer.render(parsed: List[Dict[str, Any]], page: bool=…, minify: bool=…) -> str - changed:
method DependencyRenderer.render_arrow(label, start, end, direction, i) → method DependencyRenderer.render_arrow(label: str, start: int, end: int, direction: str, i: int) -> str - changed:
method DependencyRenderer.render_svg(render_id, words, arcs) → method DependencyRenderer.render_svg(render_id: Union[int, str], words: List[Dict[str, Any]], arcs: List[Dict[str, Any]]) -> str - changed:
method DependencyRenderer.render_word(text, tag, lemma, i) → method DependencyRenderer.render_word(text: str, tag: str, lemma: Optional[str], i: int) -> str - changed:
method EntityRenderer.render(parsed, page=…, minify=…) → method EntityRenderer.render(parsed: List[Dict[str, Any]], page: bool=…, minify: bool=…) -> str - changed:
method EntityRenderer.render_ents(text, spans, title) → method EntityRenderer.render_ents(text: str, spans: List[Dict[str, Any]], title: Optional[str]) -> str - added:
class SpanRenderer - added:
constructor SpanRenderer(options: Dict[str, Any]=…) -> None - added:
field SpanRenderer.style - added:
method SpanRenderer.render(parsed: List[Dict[str, Any]], page: bool=…, minify: bool=…) -> str - added:
method SpanRenderer.render_spans(tokens: List[str], spans: List[Dict[str, Any]], title: Optional[str]) -> str - added:
variable DEFAULT_ENTITY_COLOR - added:
variable DEFAULT_LABEL_COLORS
- added:
variable TPL_KB_LINK - added:
variable TPL_SPAN - added:
variable TPL_SPANS - added:
variable TPL_SPAN_RTL - added:
variable TPL_SPAN_SLICE - added:
variable TPL_SPAN_SLICE_RTL - added:
variable TPL_SPAN_START - added:
variable TPL_SPAN_START_RTL
- removed:
class AlignmentError(ValueError) - removed:
class TempErrors(object) - removed:
field Errors.E009 - removed:
field Errors.E013 - removed:
field Errors.E014 - removed:
field Errors.E015 - removed:
field Errors.E020 - removed:
field Errors.E021 - removed:
field Errors.E023 - removed:
field Errors.E032 - removed:
field Errors.E034 - removed:
field Errors.E038 - removed:
field Errors.E049 - removed:
field Errors.E051 - removed:
field Errors.E061 - removed:
field Errors.E066 - removed:
field Errors.E069 - removed:
field Errors.E070 - removed:
field Errors.E072 - removed:
field Errors.E075 - removed:
field Errors.E076 - removed:
field Errors.E077 - removed:
field Errors.E078 - removed:
field Errors.E086 - removed:
field Errors.E092 - removed:
field Errors.E104 - removed:
field Errors.E105 - removed:
field Errors.E108 - removed:
field Errors.E113 - removed:
field Errors.E114 - removed:
field Errors.E116 - removed:
field Errors.E124 - removed:
field Errors.E127 - removed:
field Errors.E128 - removed:
field Errors.E129 - removed:
field Errors.E131 - removed:
field Errors.E135 - removed:
field Errors.E136 - removed:
field Errors.E137 - removed:
field Errors.E138 - removed:
field Errors.E142 - removed:
field Errors.E144 - removed:
field Errors.E151 - removed:
field Errors.E156 - removed:
field Errors.E162 - removed:
field Errors.E167 - removed:
field Errors.E168 - removed:
field Errors.E172 - removed:
field Errors.E173 - removed:
field Errors.E174 - removed:
field Errors.E186 - removed:
field Errors.E188 - removed:
field Errors.E201 - removed:
field TempErrors.T003 - removed:
field TempErrors.T004 - removed:
field TempErrors.T007 - removed:
field TempErrors.T008 - removed:
field Warnings.W001 - removed:
field Warnings.W002 - removed:
field Warnings.W003 - removed:
field Warnings.W004 - removed:
field Warnings.W009 - removed:
field Warnings.W010 - removed:
field Warnings.W013 - removed:
field Warnings.W014 - removed:
field Warnings.W015 - removed:
field Warnings.W016 - removed:
field Warnings.W019 - removed:
field Warnings.W020 - removed:
field Warnings.W022 - removed:
field Warnings.W023 - removed:
field Warnings.W025 - removed:
field Warnings.W029 - removed:
field Warnings.W031 - removed:
field Warnings.W032 - removed:
field Warnings.W034 - removed:
function add_codes(err_cls) - changed:
class Errors(object) → class Errors - changed:
class Warnings(object) → class Warnings - added:
class ErrorsWithCodes(type) - added:
field Errors.E1000 - added:
field Errors.E1001 - added:
field Errors.E1002 - added:
field Errors.E1003 - added:
field Errors.E1004 - added:
field Errors.E1005 - added:
field Errors.E1007 - added:
field Errors.E1008 - added:
field Errors.E1010 - added:
field Errors.E1011 - added:
field Errors.E1012 - added:
field Errors.E1013 - added:
field Errors.E1014 - added:
field Errors.E1015 - added:
field Errors.E1016 - added:
field Errors.E1017 - added:
field Errors.E1018 - added:
field Errors.E1019 - added:
field Errors.E1020 - added:
field Errors.E1021 - added:
field Errors.E1022 - added:
field Errors.E1023 - added:
field Errors.E1024 - added:
field Errors.E1025 - added:
field Errors.E1026 - added:
field Errors.E1027 - added:
field Errors.E1028 - added:
field Errors.E1029 - added:
field Errors.E1030 - added:
field Errors.E1031 - added:
field Errors.E1032 - added:
field Errors.E1033 - added:
field Errors.E1034 - added:
field Errors.E1035 - added:
field Errors.E1036 - added:
field Errors.E1037 - added:
field Errors.E1038 - added:
field Errors.E1039 - added:
field Errors.E1040 - added:
field Errors.E1041 - added:
field Errors.E1042 - added:
field Errors.E1043 - added:
field Errors.E1044 - added:
field Errors.E1045 - added:
field Errors.E1046 - added:
field Errors.E1047 - added:
field Errors.E1048 - added:
field Errors.E1049 - added:
field Errors.E1050 - added:
field Errors.E1051 - added:
field Errors.E1052 - added:
field Errors.E1053 - added:
field Errors.E1054 - added:
field Errors.E1055 - added:
field Errors.E1056 - added:
field Errors.E1057 - added:
field Errors.E203 - added:
field Errors.E849 - added:
field Errors.E850 - added:
field Errors.E851 - added:
field Errors.E853 - added:
field Errors.E854 - added:
field Errors.E855 - added:
field Errors.E856 - added:
field Errors.E857 - added:
field Errors.E858 - added:
field Errors.E859 - added:
field Errors.E860 - added:
field Errors.E861 - added:
field Errors.E862 - added:
field Errors.E863 - added:
field Errors.E864 - added:
field Errors.E865 - added:
field Errors.E866 - added:
field Errors.E867 - added:
field Errors.E868 - added:
field Errors.E869 - added:
field Errors.E870 - added:
field Errors.E871 - added:
field Errors.E872 - added:
field Errors.E873 - added:
field Errors.E874 - added:
field Errors.E875 - added:
field Errors.E879 - added:
field Errors.E884 - added:
field Errors.E885 - added:
field Errors.E886 - added:
field Errors.E887 - added:
field Errors.E888 - added:
field Errors.E889 - added:
field Errors.E890 - added:
field Errors.E891 - added:
field Errors.E892 - added:
field Errors.E893 - added:
field Errors.E894 - added:
field Errors.E895 - added:
field Errors.E896 - added:
field Errors.E897 - added:
field Errors.E898 - added:
field Errors.E899 - added:
field Errors.E900 - added:
field Errors.E901 - added:
field Errors.E902 - added:
field Errors.E903 - added:
field Errors.E904 - added:
field Errors.E905 - added:
field Errors.E906 - added:
field Errors.E908 - added:
field Errors.E909 - added:
field Errors.E910 - added:
field Errors.E911 - added:
field Errors.E912 - added:
field Errors.E913 - added:
field Errors.E914 - added:
field Errors.E915 - added:
field Errors.E916 - added:
field Errors.E917 - added:
field Errors.E918 - added:
field Errors.E919 - added:
field Errors.E920 - added:
field Errors.E921 - added:
field Errors.E922 - added:
field Errors.E923 - added:
field Errors.E924 - added:
field Errors.E925 - added:
field Errors.E926 - added:
field Errors.E927 - added:
field Errors.E928 - added:
field Errors.E929 - added:
field Errors.E930 - added:
field Errors.E931 - added:
field Errors.E940 - added:
field Errors.E941 - added:
field Errors.E942 - added:
field Errors.E943 - added:
field Errors.E944 - added:
field Errors.E945 - added:
field Errors.E947 - added:
field Errors.E948 - added:
field Errors.E949 - added:
field Errors.E952 - added:
field Errors.E953 - added:
field Errors.E954 - added:
field Errors.E955 - added:
field Errors.E956 - added:
field Errors.E957 - added:
field Errors.E958 - added:
field Errors.E959 - added:
field Errors.E960 - added:
field Errors.E961 - added:
field Errors.E962 - added:
field Errors.E963 - added:
field Errors.E964 - added:
field Errors.E965 - added:
field Errors.E966 - added:
field Errors.E967 - added:
field Errors.E968 - added:
field Errors.E969 - added:
field Errors.E970 - added:
field Errors.E971 - added:
field Errors.E972 - added:
field Errors.E973 - added:
field Errors.E974 - added:
field Errors.E976 - added:
field Errors.E977 - added:
field Errors.E978 - added:
field Errors.E980 - added:
field Errors.E981 - added:
field Errors.E982 - added:
field Errors.E983 - added:
field Errors.E984 - added:
field Errors.E985 - added:
field Errors.E986 - added:
field Errors.E989 - added:
field Errors.E991 - added:
field Errors.E992 - added:
field Errors.E993 - added:
field Errors.E997 - added:
field Errors.E999 - added:
field Warnings.W036 - added:
field Warnings.W086 - added:
field Warnings.W087 - added:
field Warnings.W088 - added:
field Warnings.W089 - added:
field Warnings.W090 - added:
field Warnings.W091 - added:
field Warnings.W092 - added:
field Warnings.W093 - added:
field Warnings.W094 - added:
field Warnings.W095 - added:
field Warnings.W096 - added:
field Warnings.W100 - added:
field Warnings.W101 - added:
field Warnings.W102 - added:
field Warnings.W103 - added:
field Warnings.W104 - added:
field Warnings.W105 - added:
field Warnings.W107 - added:
field Warnings.W108 - added:
field Warnings.W109 - added:
field Warnings.W110 - added:
field Warnings.W111 - added:
field Warnings.W112 - added:
field Warnings.W113 - added:
field Warnings.W114 - added:
field Warnings.W115 - added:
field Warnings.W116 - added:
field Warnings.W117 - added:
field Warnings.W118 - added:
field Warnings.W119 - added:
field Warnings.W120 - added:
field Warnings.W121 - added:
field Warnings.W122 - added:
field Warnings.W123 - added:
field Warnings.W124 - added:
field Warnings.W125 - added:
field Warnings.W126 - added:
field Warnings.W127 - added:
function filter_warning(action: Literal['default', 'error', 'ignore', 'always', 'module', 'once'], error_msg: str) - added:
function setup_default_warnings() - added:
variable OLD_MODEL_SHORTCUTS
- added:
export Candidate - added:
export InMemoryLookupKB - added:
export KnowledgeBase - added:
export get_candidates - added:
export get_candidates_batch
- added:
class Azerbaijani(Language) - added:
field Azerbaijani.Defaults - added:
field Azerbaijani.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
variable MORPH_RULES
- added:
class Tibetan(Language) - added:
field Tibetan.Defaults - added:
field Tibetan.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
class CatalanLemmatizer(Lemmatizer) - added:
method CatalanLemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
static method CatalanLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES - added:
variable UNITS
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- removed:
variable TAG_MAP
- added:
variable COMBINING_DIACRITICS
- removed:
variable TAG_MAP
- removed:
variable MORPH_RULES
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- added:
class LowerSorbian(Language) - added:
field LowerSorbian.Defaults - added:
field LowerSorbian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- removed:
method GreekLemmatizer.lemmatize(string, index, exceptions, rules) - added:
method GreekLemmatizer.rule_lemmatize(token: Token) -> List[str]
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- added:
class EnglishLemmatizer(Lemmatizer) - added:
method EnglishLemmatizer.is_base_form(token: Token) -> bool
- removed:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- added:
class SpanishLemmatizer(Lemmatizer) - added:
method SpanishLemmatizer.lemmatize_adj(word: str, features: List[str], rule: str, index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_adv(word: str, features: List[str], rule: str, index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_det(word: str, features: List[str], rule: str, index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_noun(word: str, features: List[str], rule: str, index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_num(word: str, features: List[str], rule: str, index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_pron(word: str, features: List[str], rule: Optional[str], index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_verb(word: str, features: List[str], rule: Optional[str], index: List[str]) -> List[str] - added:
method SpanishLemmatizer.lemmatize_verb_pron(word: str, features: List[str], rule: Optional[str], index: List[str]) -> List[str] - added:
method SpanishLemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
method SpanishLemmatizer.select_rule(pos: str, features: List[str]) -> Optional[str] - added:
static method SpanishLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- removed:
function is_verb_token(token) - removed:
function noun_bounds(doc, root, np_left_deps, np_right_deps, stop_deps) - changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- added:
variable conj_contraction_bases - added:
variable conj_contraction_negations
- added:
class Faroese(Language) - added:
field Faroese.Defaults - added:
field Faroese.lang
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
method FrenchLemmatizer.adj(string, morphology=…) - removed:
method FrenchLemmatizer.lemmatize(string, index, exceptions, rules) - removed:
method FrenchLemmatizer.lookup(string, orth=…) - removed:
method FrenchLemmatizer.noun(string, morphology=…) - removed:
method FrenchLemmatizer.punct(string, morphology=…) - removed:
method FrenchLemmatizer.verb(string, morphology=…) - added:
method FrenchLemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
static method FrenchLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- removed:
function deduplicate(word) - removed:
function devoice(word) - removed:
function ends_dentals(word) - removed:
function ends_with_vowel(word) - removed:
function starts_with_vowel(word) - removed:
variable broad_vowels - removed:
variable consonants - removed:
variable slender_vowels - removed:
variable vowels
- added:
class IrishLemmatizer(Lemmatizer) - added:
function demutate(word: str, is_hpref: bool=…) -> str - added:
function unponc(word: str) -> str - added:
method IrishLemmatizer.pos_lookup_lemmatize(token: Token) -> List[str] - added:
static method IrishLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- removed:
variable TAG_MAP
- added:
class Scottish(Language) - added:
field Scottish.Defaults - added:
field Scottish.lang
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class AncientGreek(Language) - added:
field AncientGreek.Defaults - added:
field AncientGreek.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
variable TAG_MAP
- added:
class UpperSorbian(Language) - added:
field UpperSorbian.Defaults - added:
field UpperSorbian.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- added:
variable TOKENIZER_EXCEPTIONS
- added:
class HaitianCreole(Language) - added:
field HaitianCreole.Defaults - added:
field HaitianCreole.lang
- added:
class HaitianCreoleLemmatizer(Lemmatizer) - added:
method HaitianCreoleLemmatizer.is_base_form(token: Token) -> bool - added:
method HaitianCreoleLemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
static method HaitianCreoleLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- added:
function like_num(text) - added:
function norm_custom(text) - added:
variable LEX_ATTRS - added:
variable NORM_MAP
- added:
variable ELISION - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable STOP_WORDS - added:
variable contractions
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- added:
variable TAG_MAP
- added:
function make_variants(base, first_norm, second_orth, second_norm) - added:
variable TOKENIZER_EXCEPTIONS
- removed:
variable TAG_MAP
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- added:
class ItalianLemmatizer(Lemmatizer) - added:
method ItalianLemmatizer.lemmatize_adj(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method ItalianLemmatizer.lemmatize_adp(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method ItalianLemmatizer.lemmatize_det(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method ItalianLemmatizer.lemmatize_noun(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method ItalianLemmatizer.lemmatize_pron(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method ItalianLemmatizer.pos_lookup_lemmatize(token: Token) -> List[str] - added:
static method ItalianLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- removed:
variable TAG_MAP
- removed:
method Japanese.make_doc(text)
- changed:
function noun_chunks(obj) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- added:
class Kurmanji(Language) - added:
field Kurmanji.Defaults - added:
field Kurmanji.lang
- added:
function is_digit(text) - added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable STOP_WORDS
- removed:
method Korean.make_doc(text)
- added:
variable TOKENIZER_INFIXES
- added:
class Latin(Language) - added:
field Latin.Defaults - added:
field Latin.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS - added:
variable roman_numerals_compile
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
variable TAG_MAP
- changed:
function is_alpha(string) → function is_alpha(string: str) -> bool - changed:
function is_ascii(text) → function is_ascii(text: str) -> bool - changed:
function is_bracket(text) → function is_bracket(text: str) -> bool - changed:
function is_currency(text) → function is_currency(text: str) -> bool - changed:
function is_digit(string) → function is_digit(string: str) -> bool - changed:
function is_left_punct(text) → function is_left_punct(text: str) -> bool - changed:
function is_lower(string) → function is_lower(string: str) -> bool - changed:
function is_punct(text) → function is_punct(text: str) -> bool - changed:
function is_quote(text) → function is_quote(text: str) -> bool - changed:
function is_right_punct(text) → function is_right_punct(text: str) -> bool - changed:
function is_space(string) → function is_space(string: str) -> bool - changed:
function is_stop(string, stops=…) → function is_stop(string: str, stops: Set[str]=…) -> bool - changed:
function is_title(string) → function is_title(string: str) -> bool - changed:
function is_upper(string) → function is_upper(string: str) -> bool - changed:
function like_email(text) → function like_email(text: str) -> bool - changed:
function like_num(text) → function like_num(text: str) -> bool - changed:
function like_url(text) → function like_url(text: str) -> bool - changed:
function lower(string) → function lower(string: str) -> str - changed:
function prefix(string) → function prefix(string: str) -> str - changed:
function suffix(string) → function suffix(string: str) -> str - changed:
function word_shape(text) → function word_shape(text: str) -> str - added:
function get_lang(text: str, lang: str=…) -> str
- added:
class Luganda(Language) - added:
field Luganda.Defaults - added:
field Luganda.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- removed:
variable MORPH_RULES
- removed:
variable TAG_MAP
- added:
variable mod_base_exceptions
- removed:
method MacedonianLemmatizer.lemmatize(string, index, exceptions, rules) - added:
method MacedonianLemmatizer.rule_lemmatize(token: Token) -> List[str]
- removed:
variable MORPH_RULES - removed:
variable sum_aux
- added:
class Malay(Language) - added:
field Malay.Defaults - added:
field Malay.lang
- added:
function is_currency(text) - added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable CURRENCY - added:
variable HTML_PREFIX - added:
variable HTML_SUFFIX - added:
variable LIST_CURRENCY - added:
variable MONTHS - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES - added:
variable UNITS
- added:
variable STOP_WORDS
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
variable MORPH_RULES
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- removed:
field DutchLemmatizer.univ_pos_name_variants - removed:
method DutchLemmatizer.lemmatize(string, index, exceptions, rules) - removed:
method DutchLemmatizer.lookup(string, orth=…) - added:
method DutchLemmatizer.lookup_lemmatize(token: Token) -> List[str] - added:
method DutchLemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
static method DutchLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- removed:
variable TAG_MAP
- added:
class NorwegianNynorsk(Language) - added:
field NorwegianNynorsk.Defaults - added:
field NorwegianNynorsk.lang
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable TOKENIZER_EXCEPTIONS
- removed:
method PolishLemmatizer.lemmatize(string, index, exceptions, rules) - removed:
method PolishLemmatizer.lookup(string, orth=…) - changed:
method PolishLemmatizer.lemmatize_adj(string, morphology, lookup_table) → method PolishLemmatizer.lemmatize_adj(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - changed:
method PolishLemmatizer.lemmatize_noun(string, morphology, lookup_table) → method PolishLemmatizer.lemmatize_noun(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - changed:
method PolishLemmatizer.lemmatize_verb(string, morphology, lookup_table) → method PolishLemmatizer.lemmatize_verb(string: str, morphology: dict, lookup_table: Dict[str, str]) -> List[str] - added:
method PolishLemmatizer.pos_lookup_lemmatize(token: Token) -> List[str] - added:
static method PolishLemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- removed:
variable TAG_MAP
- added:
function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]] - added:
variable SYNTAX_ITERATORS
- removed:
variable TAG_MAP
- added:
variable COMBINING_DIACRITICS_TOKENIZER_INFIXES - added:
variable COMBINING_DIACRITICS_TOKENIZER_SUFFIXES
- removed:
variable TAG_MAP
- removed:
method RussianLemmatizer.lookup(string, orth=…) - removed:
static method RussianLemmatizer.normalize_univ_pos(univ_pos) - changed:
constructor RussianLemmatizer(lookups=…) → constructor RussianLemmatizer(vocab: Vocab, model: Optional[Model], name: str=…, *, mode: str=…, overwrite: bool=…, scorer: Optional[Callable]=…) -> None - changed:
function oc2ud(oc_tag) → function oc2ud(oc_tag: str) -> Tuple[str, Dict[str, str]] - added:
method RussianLemmatizer.pymorphy2_lemmatize(token: Token) -> List[str] - added:
method RussianLemmatizer.pymorphy2_lookup_lemmatize(token: Token) -> List[str] - added:
method RussianLemmatizer.pymorphy3_lemmatize(token: Token) -> List[str] - added:
method RussianLemmatizer.pymorphy3_lookup_lemmatize(token: Token) -> List[str]
- removed:
variable TAG_MAP
- removed:
variable TAG_MAP
- added:
function is_currency(text) - added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable CONCAT_QUOTES - added:
variable HYPHENS_PERMITTED - added:
variable INCLUDE_SPECIAL - added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_PREFIXES - added:
variable TOKENIZER_SUFFIXES
- added:
variable TOKENIZER_EXCEPTIONS - added:
variable abbrv
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- removed:
variable MORPH_RULES
- added:
variable TOKENIZER_INFIXES - added:
variable TOKENIZER_SUFFIXES
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
variable TAG_MAP
- removed:
variable TAG_MAP
- removed:
method Thai.make_doc(text)
- removed:
variable TAG_MAP
- added:
class Setswana(Language) - added:
field Setswana.Defaults - added:
field Setswana.lang
- added:
function like_num(text) - added:
variable LEX_ATTRS
- added:
variable TOKENIZER_INFIXES
- added:
variable STOP_WORDS
- removed:
variable TOKEN_MATCH
- removed:
variable MORPH_RULES
- changed:
function noun_chunks(doclike) → function noun_chunks(doclike: Union[Doc, Span]) -> Iterator[Tuple[int, int, int]]
- removed:
function oc2ud(oc_tag) - removed:
method UkrainianLemmatizer.lookup(string, orth=…) - removed:
static method UkrainianLemmatizer.normalize_univ_pos(univ_pos) - removed:
variable PUNCT_RULES - changed:
class UkrainianLemmatizer(Lemmatizer) → class UkrainianLemmatizer(RussianLemmatizer) - changed:
constructor UkrainianLemmatizer(lookups=…) → constructor UkrainianLemmatizer(vocab: Vocab, model: Optional[Model], name: str=…, *, mode: str=…, overwrite: bool=…, scorer: Optional[Callable]=…) -> None
- removed:
variable TAG_MAP
- removed:
variable TAG_MAP
- removed:
method Vietnamese.make_doc(text)
- removed:
variable TAG_MAP
- removed:
field MultiLanguage.Defaults
- removed:
method Chinese.make_doc(text)
- removed:
variable TAG_MAP
- removed:
class component(object) - removed:
constructor component(name=…, assigns=…, requires=…, retokenizes=…) - removed:
field BaseDefaults.is_base_form - removed:
field BaseDefaults.morph_rules - removed:
field BaseDefaults.paired_orth_variants - removed:
field BaseDefaults.pipe_names - removed:
field BaseDefaults.resources - removed:
field BaseDefaults.single_orth_variants - removed:
field BaseDefaults.tag_map - removed:
method Language.preprocess_gold(docs_golds) - removed:
property Language.entity - removed:
property Language.linker - removed:
property Language.matcher - removed:
property Language.meta - removed:
property Language.parser - removed:
property Language.pipe_factories - removed:
property Language.pipe_labels - removed:
property Language.pipe_names - removed:
property Language.tagger - removed:
property Language.tensorizer - removed:
static method BaseDefaults.create_lemmatizer(nlp=…, lookups=…) - removed:
static method BaseDefaults.create_lookups(nlp=…) - removed:
static method BaseDefaults.create_tokenizer(nlp=…) - removed:
static method BaseDefaults.create_vocab(nlp=…) - removed:
variable ENABLE_PIPELINE_ANALYSIS - changed:
class BaseDefaults(object) → class BaseDefaults - changed:
class Language(object) → class Language - changed:
constructor DisabledPipes(nlp, *names) → constructor DisabledPipes(nlp: Language, names: List[str]) -> None - changed:
constructor Language(vocab=…, make_doc=…, max_length=…, meta=…, **kwargs) → constructor Language(vocab: Union[Vocab, bool]=…, *, max_length: int=…, meta: Dict[str, Any]=…, create_tokenizer: Optional[Callable[[Language], Callable[[str], Doc]]]=…, create_vectors: Optional[Callable[[Vocab], BaseVectors]]=…, batch_size: int=…, **kwargs) -> None - changed:
field BaseDefaults.infixes → field BaseDefaults.infixes: Optional[Sequence[Union[str, Pattern]]] - changed:
field BaseDefaults.lex_attr_getters → field BaseDefaults.lex_attr_getters: Dict[int, Callable[[str], Any]] - changed:
field BaseDefaults.prefixes → field BaseDefaults.prefixes: Optional[Sequence[Union[str, Pattern]]] - changed:
field BaseDefaults.stop_words → field BaseDefaults.stop_words: Set[str] - changed:
field BaseDefaults.suffixes → field BaseDefaults.suffixes: Optional[Sequence[Union[str, Pattern]]] - changed:
field BaseDefaults.syntax_iterators → field BaseDefaults.syntax_iterators: Dict[str, Callable] - changed:
field BaseDefaults.token_match → field BaseDefaults.token_match: Optional[Callable] - changed:
field BaseDefaults.tokenizer_exceptions → field BaseDefaults.tokenizer_exceptions: Dict[str, List[dict]] - changed:
field BaseDefaults.url_match → field BaseDefaults.url_match: Optional[Callable] - changed:
field Language.lang → field Language.lang: Optional[str] - changed:
method DisabledPipes.restore() → method DisabledPipes.restore() -> None - changed:
method Language.add_pipe(component, name=…, before=…, after=…, first=…, last=…) → method Language.add_pipe(factory_name: str, name: Optional[str]=…, *, before: Optional[Union[str, int]]=…, after: Optional[Union[str, int]]=…, first: Optional[bool]=…, last: Optional[bool]=…, source: Optional[Language]=…, config: Dict[str, Any]=…, raw_config: Optional[Config]=…, validate: bool=…) -> PipeCallable - changed:
method Language.begin_training(get_gold_tuples=…, sgd=…, component_cfg=…, **cfg) → method Language.begin_training(get_examples: Optional[Callable[[], Iterable[Example]]]=…, *, sgd: Optional[Optimizer]=…) -> Optimizer - changed:
method Language.create_pipe(name, config=…) → method Language.create_pipe(factory_name: str, name: Optional[str]=…, *, config: Dict[str, Any]=…, raw_config: Optional[Config]=…, validate: bool=…) -> PipeCallable - changed:
method Language.disable_pipes(*names) → method Language.disable_pipes(*names) -> DisabledPipes - changed:
method Language.evaluate(docs_golds, verbose=…, batch_size=…, scorer=…, component_cfg=…) → method Language.evaluate(examples: Iterable[Example], *, batch_size: Optional[int]=…, scorer: Optional[Scorer]=…, component_cfg: Optional[Dict[str, Dict[str, Any]]]=…, scorer_cfg: Optional[Dict[str, Any]]=…, per_component: bool=…) -> Dict[str, Any] - changed:
method Language.from_bytes(bytes_data, exclude=…, disable=…, **kwargs) → method Language.from_bytes(bytes_data: bytes, *, exclude: Iterable[str]=…) -> Language - changed:
method Language.from_disk(path, exclude=…, disable=…) → method Language.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…, overrides: Dict[str, Any]=…) -> Language - changed:
method Language.get_pipe(name) → method Language.get_pipe(name: str) -> PipeCallable - changed:
method Language.has_pipe(name) → method Language.has_pipe(name: str) -> bool - changed:
method Language.make_doc(text) → method Language.make_doc(text: str) -> Doc - changed:
method Language.pipe(texts, as_tuples=…, n_threads=…, batch_size=…, disable=…, cleanup=…, component_cfg=…, n_process=…) → method Language.pipe(texts: Iterable[Union[str, Doc]], *, as_tuples: Literal[False]=…, batch_size: Optional[int]=…, disable: Iterable[str]=…, component_cfg: Optional[Dict[str, Dict[str, Any]]]=…, n_process: int=…) -> Iterator[Doc] | (texts: Iterable[Tuple[Union[str, Doc], _AnyContext]], *, as_tuples: Literal[True]=…, batch_size: Optional[int]=…, disable: Iterable[str]=…, component_cfg: Optional[D… - changed:
method Language.rehearse(docs, sgd=…, losses=…, config=…) → method Language.rehearse(examples: Iterable[Example], *, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…, component_cfg: Optional[Dict[str, Dict[str, Any]]]=…, exclude: Iterable[str]=…) -> Dict[str, float] - changed:
method Language.remove_pipe(name) → method Language.remove_pipe(name: str) -> Tuple[str, PipeCallable] - changed:
method Language.rename_pipe(old_name, new_name) → method Language.rename_pipe(old_name: str, new_name: str) -> None - changed:
method Language.replace_pipe(name, component) → method Language.replace_pipe(name: str, factory_name: str, *, config: Dict[str, Any]=…, validate: bool=…) -> PipeCallable - changed:
method Language.resume_training(sgd=…, **cfg) → method Language.resume_training(*, sgd: Optional[Optimizer]=…) -> Optimizer - changed:
method Language.to_bytes(exclude=…, disable=…, **kwargs) → method Language.to_bytes(*, exclude: Iterable[str]=…) -> bytes - changed:
method Language.to_disk(path, exclude=…, disable=…) → method Language.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - changed:
method Language.update(docs, golds, drop=…, sgd=…, losses=…, component_cfg=…) → method Language.update(examples: Iterable[Example], _: Optional[Any]=…, *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…, component_cfg: Optional[Dict[str, Dict[str, Any]]]=…, exclude: Iterable[str]=…, annotates: Iterable[str]=…) - changed:
method Language.use_params(params, **cfg) → method Language.use_params(params: Optional[dict]) - added:
class FactoryMeta - added:
field BaseDefaults.config: Config - added:
field FactoryMeta.assigns: Iterable[str] - added:
field FactoryMeta.default_config: Optional[Dict[str, Any]] - added:
field FactoryMeta.default_score_weights: Optional[Dict[str, Optional[float]]] - added:
field FactoryMeta.factory: str - added:
field FactoryMeta.requires: Iterable[str] - added:
field FactoryMeta.retokenizes: bool - added:
field FactoryMeta.scores: Iterable[str] - added:
field Language.default_config - added:
function create_tokenizer() -> Callable[[Language], Tokenizer] - added:
function load_lookups_data(lang, tables) - added:
method Language.analyze_pipes(*, keys: List[str]=…, pretty: bool=…) -> Optional[Dict[str, Any]] - added:
method Language.create_optimizer() - added:
method Language.create_pipe_from_source(source_name: str, source: Language, *, name: str) -> Tuple[PipeCallable, str] - added:
method Language.disable_pipe(name: str) -> None - added:
method Language.enable_pipe(name: str) -> None - added:
method Language.get_pipe_config(name: str) -> Config - added:
method Language.get_pipe_meta(name: str) -> FactoryMeta - added:
method Language.initialize(get_examples: Optional[Callable[[], Iterable[Example]]]=…, *, sgd: Optional[Optimizer]=…) -> Optimizer - added:
method Language.memory_zone(mem: Optional[Pool]=…) -> Iterator[Pool] - added:
method Language.replace_listeners(tok2vec_name: str, pipe_name: str, listeners: Iterable[str]) -> None - added:
method Language.select_pipes(*, disable: Optional[Union[str, Iterable[str]]]=…, enable: Optional[Union[str, Iterable[str]]]=…) -> DisabledPipes - added:
method Language.set_error_handler(error_handler: Callable[[str, PipeCallable, List[Doc], Exception], NoReturn]) - added:
property Language.component_names -> List[str] - added:
property Language.components -> List[Tuple[str, PipeCallable]] - added:
property Language.config -> Config - added:
property Language.disabled -> List[str] - added:
property Language.factory_names -> List[str] - added:
property Language.meta -> Dict[str, Any] - added:
property Language.pipe_factories -> Dict[str, str] - added:
property Language.pipe_labels -> Dict[str, List[str]] - added:
property Language.pipe_names -> List[str] - added:
property Language.pipeline -> List[Tuple[str, PipeCallable]] - added:
static method Language.component(name: str, *, assigns: Iterable[str]=…, requires: Iterable[str]=…, retokenizes: bool=…, func: Optional[PipeCallable]=…) -> Callable[..., Any] - added:
static method Language.factory(name: str, *, default_config: Dict[str, Any]=…, assigns: Iterable[str]=…, requires: Iterable[str]=…, retokenizes: bool=…, default_score_weights: Dict[str, Optional[float]]=…, func: Optional[Callable]=…) -> Callable - added:
static method Language.from_config(config: Union[Dict[str, Any], Config]=…, *, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, meta: Dict[str, Any]=…, auto_fill: bool=…, validate: bool=…) -> Language - added:
static method Language.get_factory_meta(name: str) -> FactoryMeta - added:
static method Language.get_factory_name(name: str) -> str - added:
static method Language.has_factory(name: str) -> bool - added:
static method Language.set_factory_meta(name: str, value: FactoryMeta) -> None - added:
variable DEFAULT_CONFIG - added:
variable DEFAULT_CONFIG_PATH - added:
variable DEFAULT_CONFIG_PRETRAIN_PATH - added:
variable PipeCallable
- removed:
class Lemmatizer(object) - removed:
constructor Lemmatizer(lookups, is_base_form=…, *args, **kwargs) - removed:
method Lemmatizer.adj(string, morphology=…) - removed:
method Lemmatizer.adp(string, morphology=…) - removed:
method Lemmatizer.det(string, morphology=…) - removed:
method Lemmatizer.lemmatize(string, index, exceptions, rules) - removed:
method Lemmatizer.lookup(string, orth=…) - removed:
method Lemmatizer.noun(string, morphology=…) - removed:
method Lemmatizer.num(string, morphology=…) - removed:
method Lemmatizer.pron(string, morphology=…) - removed:
method Lemmatizer.punct(string, morphology=…) - removed:
method Lemmatizer.verb(string, morphology=…) - removed:
static method Lemmatizer.load(*args, **kwargs)
- added:
class Lexeme - added:
constructor Lexeme(vocab: Vocab, orth: int) -> None - added:
field Lexeme.cluster: int - added:
field Lexeme.flags: int - added:
field Lexeme.is_alpha: bool - added:
field Lexeme.is_ascii: bool - added:
field Lexeme.is_bracket: bool - added:
field Lexeme.is_currency: bool - added:
field Lexeme.is_digit: bool - added:
field Lexeme.is_left_punct: bool - added:
field Lexeme.is_lower: bool - added:
field Lexeme.is_punct: bool - added:
field Lexeme.is_quote: bool - added:
field Lexeme.is_right_punct: bool - added:
field Lexeme.is_space: bool - added:
field Lexeme.is_stop: bool - added:
field Lexeme.is_title: bool - added:
field Lexeme.is_upper: bool - added:
field Lexeme.lang: int - added:
field Lexeme.lang_: str - added:
field Lexeme.like_email: bool - added:
field Lexeme.like_num: bool - added:
field Lexeme.like_url: bool - added:
field Lexeme.lower: int - added:
field Lexeme.lower_: str - added:
field Lexeme.norm: int - added:
field Lexeme.norm_: str - added:
field Lexeme.orth: int - added:
field Lexeme.prefix: int - added:
field Lexeme.prefix_: str - added:
field Lexeme.prob: float - added:
field Lexeme.rank: int - added:
field Lexeme.sentiment: float - added:
field Lexeme.shape: int - added:
field Lexeme.shape_: str - added:
field Lexeme.suffix: int - added:
field Lexeme.suffix_: str - added:
field Lexeme.vector: Floats1d - added:
method Lexeme.check_flag(flag_id: int) -> bool - added:
method Lexeme.set_attrs(**attrs: Any) -> None - added:
method Lexeme.set_flag(flag_id: int, value: bool) -> None - added:
method Lexeme.similarity(other: Union[Doc, Span, Token, Lexeme]) -> float - added:
property Lexeme.has_vector -> bool - added:
property Lexeme.is_oov -> bool - added:
property Lexeme.orth_ -> str - added:
property Lexeme.text -> str - added:
property Lexeme.vector_norm -> float
- removed:
property Lookups.tables - changed:
class Lookups(object) → class Lookups - changed:
constructor Lookups() → constructor Lookups() -> None - changed:
constructor Table(name=…, data=…) → constructor Table(name: Optional[str]=…, data: Optional[dict]=…) -> None - changed:
method Lookups.add_table(name, data=…) → method Lookups.add_table(name: str, data: dict=…) -> Table - changed:
method Lookups.from_bytes(bytes_data, **kwargs) → method Lookups.from_bytes(bytes_data: bytes, **kwargs) -> Lookups - changed:
method Lookups.from_disk(path, filename=…, **kwargs) → method Lookups.from_disk(path: Union[str, Path], filename: str=…, **kwargs) -> Lookups - changed:
method Lookups.get_table(name, default=…) → method Lookups.get_table(name: str, default: Any=…) -> Table - changed:
method Lookups.has_table(name) → method Lookups.has_table(name: str) -> bool - changed:
method Lookups.remove_table(name) → method Lookups.remove_table(name: str) -> Table - changed:
method Lookups.to_bytes(**kwargs) → method Lookups.to_bytes(**kwargs) -> bytes - changed:
method Lookups.to_disk(path, filename=…, **kwargs) → method Lookups.to_disk(path: Union[str, Path], filename: str=…, **kwargs) -> None - changed:
method Table.from_bytes(bytes_data) → method Table.from_bytes(bytes_data: bytes) -> Table - changed:
method Table.get(key, default=…) → method Table.get(key: Union[str, int], default: Optional[Any]=…) -> Any - changed:
method Table.set(key, value) → method Table.set(key: Union[str, int], value: Any) -> None - changed:
method Table.to_bytes() → method Table.to_bytes() -> bytes - changed:
static method Table.from_dict(data, name=…) → static method Table.from_dict(data: dict, name: Optional[str]=…) -> Table - added:
function load_lookups(lang: str, tables: List[str], strict: bool=…) -> Lookups - added:
method Lookups.set_table(name: str, table: Table) -> None - added:
property Lookups.tables -> List[str]
- added:
export levenshtein
- added:
class DependencyMatcher - added:
constructor DependencyMatcher(vocab: Vocab, *, validate: bool=…) -> None - added:
field DependencyMatcher.vocab: Vocab - added:
function unpickle_matcher(vocab: Vocab, patterns: Dict[str, Any], callbacks: Dict[str, Callable[..., Any]]) -> DependencyMatcher - added:
method DependencyMatcher.add(key: Union[str, int], patterns: List[List[Dict[str, Any]]], *, on_match: Optional[Callable[[DependencyMatcher, Doc, int, List[Tuple[int, List[int]]]], Any]]=…) -> None - added:
method DependencyMatcher.get(key: Union[str, int], default: Optional[Any]=…) -> Tuple[Optional[Callable[[DependencyMatcher, Doc, int, List[Tuple[int, List[int]]]], Any]], List[List[Dict[str, Any]]]] - added:
method DependencyMatcher.has_key(key: Union[str, int]) -> bool - added:
method DependencyMatcher.remove(key: Union[str, int]) -> None
- added:
class Matcher - added:
constructor Matcher(vocab: Vocab, validate: bool=…, fuzzy_compare: Callable[[str, str, int], bool]=…) -> None - added:
method Matcher.add(key: Union[str, int], patterns: List[List[Dict[str, Any]]], *, on_match: Optional[Callable[[Matcher, Doc, int, List[Tuple[Any, ...]]], Any]]=…, greedy: Optional[str]=…) -> None - added:
method Matcher.get(key: Union[str, int], default: Optional[Any]=…) -> Tuple[Optional[Callable[[Any], Any]], List[List[Dict[Any, Any]]]] - added:
method Matcher.has_key(key: Union[str, int]) -> bool - added:
method Matcher.pipe(docs: Iterable[Tuple[Doc, Any]], batch_size: int=…, return_matches: bool=…, as_tuples: bool=…) -> Union[Iterator[Tuple[Tuple[Doc, Any], Any]], Iterator[Tuple[Doc, Any]], Iterator[Doc]] - added:
method Matcher.remove(key: str) -> None
- added:
class PhraseMatcher - added:
constructor PhraseMatcher(vocab: Vocab, attr: Optional[Union[int, str]]=…, validate: bool=…) -> None - added:
method PhraseMatcher.add(key: str, docs: List[Doc], *, on_match: Optional[Callable[[Matcher, Doc, int, List[Tuple[Any, ...]]], Any]]=…) -> None - added:
method PhraseMatcher.remove(key: str) -> None
- removed:
export FeedForward - removed:
export LayerNormalizedMaxout - removed:
export Tok2Vec - added:
export create_models_with_nvtx_range
- added:
function create_models_and_pipes_with_nvtx_range(forward_color: int=…, backprop_color: int=…, additional_pipe_functions: Optional[Dict[str, List[str]]]=…) -> Callable[[Language], Language] - added:
function create_models_with_nvtx_range(forward_color: int=…, backprop_color: int=…) -> Callable[[Language], Language] - added:
function models_with_nvtx_range(nlp, forward_color: int, backprop_color: int) - added:
function nvtx_range_wrapper_for_pipe_method(self, func, *args, **kwargs) - added:
function pipes_with_nvtx_range(nlp, additional_pipe_functions: Optional[Dict[str, List[str]]]) - added:
variable DEFAULT_NVTX_ANNOTATABLE_PIPE_METHODS
- removed:
function FeedForward(config) - removed:
function LayerNormalizedMaxout(config)
- added:
function extract_ngrams(ngram_size: int, attr: int=…) -> Model - added:
function forward(model: Model, docs, is_train: bool)
- added:
function extract_spans() -> Model[Tuple[Ragged, Ragged], Ragged] - added:
function forward(model: Model, source_spans: Tuple[Ragged, Ragged], is_train: bool) -> Tuple[Ragged, Callable] - added:
function init(model, X=…, Y=…)
- added:
function FeatureExtractor(columns: Union[List[str], List[int], List[Union[int, str]]]) -> Model[List[Doc], List[Ints2d]] - added:
function forward(model: Model[List[Doc], List[Ints2d]], docs, is_train: bool) -> Tuple[List[Ints2d], Callable]
- added:
function build_nel_encoder(tok2vec: Model, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_span_maker(n_sents: int=…) -> Model - added:
function create_candidates() -> Callable[[KnowledgeBase, Span], Iterable[Candidate]] - added:
function create_candidates_batch() -> Callable[[KnowledgeBase, Iterable[Span]], Iterable[Iterable[Candidate]]] - added:
function empty_kb(entity_vector_length: int) -> Callable[[Vocab], KnowledgeBase] - added:
function empty_kb_for_config() -> Callable[[Vocab, int], KnowledgeBase] - added:
function load_kb(kb_path: Path) -> Callable[[Vocab], KnowledgeBase] - added:
function span_maker_forward(model, docs: List[Doc], is_train) -> Tuple[Ragged, Callable]
- added:
function build_cloze_characters_multi_task_model(vocab: Vocab, tok2vec: Model, maxout_pieces: int, hidden_size: int, nr_char: int) -> Model - added:
function build_cloze_multi_task_model(vocab: Vocab, tok2vec: Model, maxout_pieces: int, hidden_size: int) -> Model - added:
function build_masked_language_model(vocab: Vocab, wrapped_model: Model, mask_prob: float=…) -> Model - added:
function build_multi_task_model(tok2vec: Model, maxout_pieces: int, token_vector_width: int, nO: Optional[int]=…) -> Model - added:
function create_pretrain_characters(maxout_pieces: int, hidden_size: int, n_characters: int) -> Callable[[Vocab, Model], Model] - added:
function create_pretrain_vectors(maxout_pieces: int, hidden_size: int, loss: str) -> Callable[[Vocab, Model], Model] - added:
function get_characters_loss(ops, docs, prediction, nr_char) - added:
function get_vectors_loss(ops, docs, prediction, distance)
- added:
function build_tb_parser_model(tok2vec: Model[List[Doc], List[Floats2d]], state_type: Literal['parser', 'ner'], extra_state_tokens: bool, hidden_width: int, maxout_pieces: int, use_upper: bool, nO: Optional[int]=…) -> Model - added:
function resize_output(model, new_nO)
- added:
function build_finder_model(tok2vec: Model[InT, List[Floats2d]], scorer: Model[OutT, OutT]) -> Model[InT, OutT] - added:
function flattener() -> Model[List[Floats2d], Floats2d] - added:
variable InT - added:
variable OutT
- added:
function build_linear_logistic(nO=…, nI=…) -> Model[Floats2d, Floats2d] - added:
function build_mean_max_reducer(hidden_size: int) -> Model[Ragged, Floats2d] - added:
function build_spancat_model(tok2vec: Model[List[Doc], List[Floats2d]], reducer: Model[Ragged, Floats2d], scorer: Model[Floats2d, Floats2d]) -> Model[Tuple[List[Doc], Ragged], Floats2d]
- added:
function build_tagger_model(tok2vec: Model[List[Doc], List[Floats2d]], nO: Optional[int]=…, normalize=…) -> Model[List[Doc], List[Floats2d]]
- added:
function build_bow_text_classifier(exclusive_classes: bool, ngram_size: int, no_output_layer: bool, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_bow_text_classifier_v3(exclusive_classes: bool, ngram_size: int, no_output_layer: bool, length: int=…, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_reduce_text_classifier(tok2vec: Model, exclusive_classes: bool, use_reduce_first: bool, use_reduce_last: bool, use_reduce_max: bool, use_reduce_mean: bool, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_simple_cnn_text_classifier(tok2vec: Model, exclusive_classes: bool, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_text_classifier_lowdata(width: int, dropout: Optional[float], nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_text_classifier_v2(tok2vec: Model[List[Doc], List[Floats2d]], linear_model: Model[List[Doc], Floats2d], nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function build_textcat_parametric_attention_v1(tok2vec: Model[List[Doc], List[Floats2d]], exclusive_classes: bool, nO: Optional[int]=…) -> Model[List[Doc], Floats2d] - added:
function init_ensemble_textcat(model, X, Y) -> Model - added:
function resize_and_set_ref(model, new_nO, resizable_layer) - added:
variable NEG_VALUE
- added:
function BiLSTMEncoder(width: int, depth: int, dropout: float) -> Model[List[Floats2d], List[Floats2d]] - added:
function CharacterEmbed(width: int, rows: int, nM: int, nC: int, include_static_vectors: bool, feature: Union[int, str]=…) -> Model[List[Doc], List[Floats2d]] - added:
function MaxoutWindowEncoder(width: int, window_size: int, maxout_pieces: int, depth: int) -> Model[List[Floats2d], List[Floats2d]] - added:
function MishWindowEncoder(width: int, window_size: int, depth: int) -> Model[List[Floats2d], List[Floats2d]] - added:
function MultiHashEmbed(width: int, attrs: Union[List[str], List[int], List[Union[str, int]]], rows: List[int], include_static_vectors: bool) -> Model[List[Doc], List[Floats2d]] - added:
function build_Tok2Vec_model(embed: Model[List[Doc], List[Floats2d]], encode: Model[List[Floats2d], List[Floats2d]]) -> Model[List[Doc], List[Floats2d]] - added:
function build_hash_embed_cnn_tok2vec(*, width: int, depth: int, embed_size: int, window_size: int, maxout_pieces: int, subword_features: bool, pretrained_vectors: Optional[bool]) -> Model[List[Doc], List[Floats2d]] - added:
function get_tok2vec_width(model: Model) - added:
function tok2vec_listener_v1(width: int, upstream: str=…)
- added:
function StaticVectors(nO: Optional[int]=…, nM: Optional[int]=…, *, dropout: Optional[float]=…, init_W: Callable=…, key_attr: str=…) -> Model[List[Doc], Ragged] - added:
function forward(model: Model[List[Doc], Ragged], docs: List[Doc], is_train: bool) -> Tuple[Ragged, Callable] - added:
function init(init_W: Callable, model: Model[List[Doc], Ragged], X: Optional[List[Doc]]=…, Y: Optional[Ragged]=…) -> Model[List[Doc], Ragged]
- added:
function TransitionModel(tok2vec, lower, upper, resize_output, dropout=…, unseen_classes=…) - added:
function forward(model, X, is_train) - added:
function init(model, X=…, Y=…)
- removed:
function CharacterEmbed(config) - removed:
function Doc2Feats(config) - removed:
function MaxoutWindowEncoder(config) - removed:
function MishWindowEncoder(config) - removed:
function MultiHashEmbed(config) - removed:
function PretrainedVectors(config) - removed:
function Tok2Vec(config) - removed:
function TorchBiLSTMEncoder(config)
- added:
function analyze_pipes(nlp: Language, *, keys: List[str]=…) -> Dict[str, Dict[str, Union[List[str], Dict]]] - added:
function get_attr_info(nlp: Language, attr: str) -> Dict[str, List[str]] - added:
function print_pipe_analysis(analysis: Dict[str, Dict[str, Union[List[str], Dict]]], *, keys: List[str]=…) -> None - added:
function validate_attrs(values: Iterable[str]) -> Iterable[str] - added:
variable DEFAULT_KEYS
- removed:
export SentenceSegmenter - removed:
export SimilarityHook - removed:
export Tensorizer - added:
export AttributeRuler - added:
export EditTreeLemmatizer - added:
export Lemmatizer - added:
export MultiLabel_TextCategorizer - added:
export SentenceRecognizer - added:
export SpanCategorizer - added:
export SpanFinder - added:
export SpanRuler - added:
export Tok2Vec - added:
export TrainablePipe
- added:
class AttributeRuler(Pipe) - added:
constructor AttributeRuler(vocab: Vocab, name: str=…, *, validate: bool=…, scorer: Optional[Callable]=…) -> None - added:
function attribute_ruler_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
function make_attribute_ruler_scorer() - added:
method AttributeRuler.add(patterns: Iterable[MatcherPatternType], attrs: Dict, index: int=…) -> None - added:
method AttributeRuler.add_patterns(patterns: Iterable[AttributeRulerPatternType]) -> None - added:
method AttributeRuler.clear() -> None - added:
method AttributeRuler.from_bytes(bytes_data: bytes, exclude: Iterable[str]=…) -> AttributeRuler - added:
method AttributeRuler.from_disk(path: Union[Path, str], exclude: Iterable[str]=…) -> AttributeRuler - added:
method AttributeRuler.initialize(get_examples: Optional[Callable[[], Iterable[Example]]], *, nlp: Optional[Language]=…, patterns: Optional[Iterable[AttributeRulerPatternType]]=…, tag_map: Optional[TagMapType]=…, morph_rules: Optional[MorphRulesType]=…) -> None - added:
method AttributeRuler.load_from_morph_rules(morph_rules: Dict[str, Dict[str, Dict[Union[int, str], Union[int, str]]]]) -> None - added:
method AttributeRuler.load_from_tag_map(tag_map: Dict[str, Dict[Union[int, str], Union[int, str]]]) -> None - added:
method AttributeRuler.match(doc: Doc) - added:
method AttributeRuler.set_annotations(doc, matches) - added:
method AttributeRuler.to_bytes(exclude: Iterable[str]=…) -> bytes - added:
method AttributeRuler.to_disk(path: Union[Path, str], exclude: Iterable[str]=…) -> None - added:
property AttributeRuler.patterns -> List[AttributeRulerPatternType] - added:
variable AttributeRulerPatternType - added:
variable MatcherPatternType - added:
variable MorphRulesType - added:
variable TagMapType
- added:
class EditTreeLemmatizer(TrainablePipe) - added:
constructor EditTreeLemmatizer(vocab: Vocab, model: Model, name: str=…, *, backoff: Optional[str]=…, min_tree_freq: int=…, overwrite: bool=…, top_k: int=…, scorer: Optional[Callable]=…) - added:
method EditTreeLemmatizer.from_bytes(bytes_data, *, exclude=…) - added:
method EditTreeLemmatizer.from_disk(path, exclude=…) - added:
method EditTreeLemmatizer.get_loss(examples: Iterable[Example], scores: List[Floats2d]) -> Tuple[float, List[Floats2d]] - added:
method EditTreeLemmatizer.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, labels: Optional[Dict]=…) - added:
method EditTreeLemmatizer.predict(docs: Iterable[Doc]) -> List[Ints2d] - added:
method EditTreeLemmatizer.set_annotations(docs: Iterable[Doc], batch_tree_ids) - added:
method EditTreeLemmatizer.to_bytes(*, exclude=…) - added:
method EditTreeLemmatizer.to_disk(path, exclude=…) - added:
property EditTreeLemmatizer.hide_labels -> bool - added:
property EditTreeLemmatizer.label_data -> Dict - added:
property EditTreeLemmatizer.labels -> Tuple[int, ...] - added:
variable DEFAULT_EDIT_TREE_LEMMATIZER_MODEL - added:
variable TOP_K_GUARDRAIL - added:
variable default_model_config
- added:
class EntityLinker(TrainablePipe) - added:
constructor EntityLinker(vocab: Vocab, model: Model, name: str=…, *, labels_discard: Iterable[str], n_sents: int, incl_prior: bool, incl_context: bool, entity_vector_length: int, get_candidates: Callable[[KnowledgeBase, Span], Iterable[Candidate]], get_candidates_batch: Callable[[KnowledgeBase, Iterable[Span]], Iterable[Iterable[Candidate]]], generate_empty_kb: Callable[[Vocab, int], KnowledgeBas… - added:
field EntityLinker.NIL - added:
function entity_linker_score(examples, **kwargs) - added:
function make_entity_linker_scorer() - added:
method EntityLinker.add_label(label) - added:
method EntityLinker.batch_has_learnable_example(examples) - added:
method EntityLinker.from_bytes(bytes_data, *, exclude=…) - added:
method EntityLinker.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> EntityLinker - added:
method EntityLinker.get_loss(examples: Iterable[Example], sentence_encodings: Floats2d) - added:
method EntityLinker.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, kb_loader: Optional[Callable[[Vocab], KnowledgeBase]]=…) - added:
method EntityLinker.predict(docs: Iterable[Doc]) -> List[str] - added:
method EntityLinker.rehearse(examples, *, sgd=…, losses=…, **config) - added:
method EntityLinker.set_annotations(docs: Iterable[Doc], kb_ids: List[str]) -> None - added:
method EntityLinker.set_kb(kb_loader: Callable[[Vocab], KnowledgeBase]) - added:
method EntityLinker.to_bytes(*, exclude=…) - added:
method EntityLinker.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - added:
method EntityLinker.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
method EntityLinker.validate_kb() -> None - added:
variable BACKWARD_OVERWRITE - added:
variable DEFAULT_NEL_MODEL - added:
variable default_model_config
- removed:
property EntityRuler.ent_ids - removed:
property EntityRuler.labels - removed:
property EntityRuler.patterns - removed:
static method EntityRuler.from_nlp(nlp, **cfg) - changed:
class EntityRuler(object) → class EntityRuler(Pipe) - changed:
constructor EntityRuler(nlp, phrase_matcher_attr=…, validate=…, **cfg) → constructor EntityRuler(nlp: Language, name: str=…, *, phrase_matcher_attr: Optional[Union[int, str]]=…, matcher_fuzzy_compare: Callable=…, validate: bool=…, overwrite_ents: bool=…, ent_id_sep: str=…, patterns: Optional[List[PatternType]]=…, scorer: Optional[Callable]=…) -> None - changed:
method EntityRuler.add_patterns(patterns) → method EntityRuler.add_patterns(patterns: List[PatternType]) -> None - changed:
method EntityRuler.from_bytes(patterns_bytes, **kwargs) → method EntityRuler.from_bytes(patterns_bytes: bytes, *, exclude: Iterable[str]=…) -> EntityRuler - changed:
method EntityRuler.from_disk(path, **kwargs) → method EntityRuler.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> EntityRuler - changed:
method EntityRuler.to_bytes(**kwargs) → method EntityRuler.to_bytes(*, exclude: Iterable[str]=…) -> bytes - changed:
method EntityRuler.to_disk(path, **kwargs) → method EntityRuler.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - added:
function entity_ruler_score(examples, **kwargs) - added:
function make_entity_ruler_scorer() - added:
method EntityRuler.clear() -> None - added:
method EntityRuler.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, patterns: Optional[Sequence[PatternType]]=…) - added:
method EntityRuler.match(doc: Doc) - added:
method EntityRuler.remove(ent_id: str) -> None - added:
method EntityRuler.set_annotations(doc, matches) - added:
property EntityRuler.ent_ids -> Tuple[Optional[str], ...] - added:
property EntityRuler.labels -> Tuple[str, ...] - added:
property EntityRuler.patterns -> List[PatternType] - added:
variable PatternType
- added:
function make_attribute_ruler(nlp: Language, name: str, validate: bool, scorer: Optional[Callable]) - added:
function make_beam_ner(nlp: Language, name: str, model: Model, moves: Optional[TransitionSystem], update_with_oracle_cut_size: int, beam_width: int, beam_density: float, beam_update_prob: float, incorrect_spans_key: Optional[str], scorer: Optional[Callable]) - added:
function make_beam_parser(nlp: Language, name: str, model: Model, moves: Optional[TransitionSystem], update_with_oracle_cut_size: int, learn_tokens: bool, min_action_freq: int, beam_width: int, beam_density: float, beam_update_prob: float, scorer: Optional[Callable]) - added:
function make_doc_cleaner(nlp: Language, name: str, *, attrs: Dict[str, Any], silent: bool) - added:
function make_edit_tree_lemmatizer(nlp: Language, name: str, model: Model, backoff: Optional[str], min_tree_freq: int, overwrite: bool, top_k: int, scorer: Optional[Callable]) - added:
function make_entity_linker(nlp: Language, name: str, model: Model, *, labels_discard: Iterable[str], n_sents: int, incl_prior: bool, incl_context: bool, entity_vector_length: int, get_candidates: Callable[[KnowledgeBase, Span], Iterable[Candidate]], get_candidates_batch: Callable[[KnowledgeBase, Iterable[Span]], Iterable[Iterable[Candidate]]], generate_empty_kb: Callable[[Vocab, int], KnowledgeB… - added:
function make_entity_ruler(nlp: Language, name: str, phrase_matcher_attr: Optional[Union[int, str]], matcher_fuzzy_compare: Callable, validate: bool, overwrite_ents: bool, ent_id_sep: str, scorer: Optional[Callable]) - added:
function make_future_entity_ruler(nlp: Language, name: str, phrase_matcher_attr: Optional[Union[int, str]], matcher_fuzzy_compare: Callable, validate: bool, overwrite_ents: bool, scorer: Optional[Callable], ent_id_sep: str) - added:
function make_lemmatizer(nlp: Language, model: Optional[Model], name: str, mode: str, overwrite: bool, scorer: Optional[Callable]) - added:
function make_morphologizer(nlp: Language, model: Model, name: str, overwrite: bool, extend: bool, label_smoothing: float, scorer: Optional[Callable]) - added:
function make_multilabel_textcat(nlp: Language, name: str, model: Model[List[Doc], List[Floats2d]], threshold: float, scorer: Optional[Callable]) -> MultiLabel_TextCategorizer - added:
function make_ner(nlp: Language, name: str, model: Model, moves: Optional[TransitionSystem], update_with_oracle_cut_size: int, incorrect_spans_key: Optional[str], scorer: Optional[Callable]) - added:
function make_nn_labeller(nlp: Language, name: str, model: Model, labels: Optional[dict], target: str) - added:
function make_parser(nlp: Language, name: str, model: Model, moves: Optional[TransitionSystem], update_with_oracle_cut_size: int, learn_tokens: bool, min_action_freq: int, scorer: Optional[Callable]) - added:
function make_sentencizer(nlp: Language, name: str, punct_chars: Optional[List[str]], overwrite: bool, scorer: Optional[Callable]) - added:
function make_senter(nlp: Language, name: str, model: Model, overwrite: bool, scorer: Optional[Callable]) - added:
function make_span_finder(nlp: Language, name: str, model: Model[Iterable[Doc], Floats2d], spans_key: str, threshold: float, max_length: Optional[int], min_length: Optional[int], scorer: Optional[Callable]) -> SpanFinder - added:
function make_span_ruler(nlp: Language, name: str, spans_key: Optional[str], spans_filter: Optional[Callable[[Iterable[Span], Iterable[Span]], Iterable[Span]]], annotate_ents: bool, ents_filter: Callable[[Iterable[Span], Iterable[Span]], Iterable[Span]], phrase_matcher_attr: Optional[Union[int, str]], matcher_fuzzy_compare: Callable, validate: bool, overwrite: bool, scorer: Optional[Callable]) - added:
function make_spancat(nlp: Language, name: str, suggester: Suggester, model: Model[Tuple[List[Doc], Ragged], Floats2d], spans_key: str, scorer: Optional[Callable], threshold: float, max_positive: Optional[int]) -> SpanCategorizer - added:
function make_spancat_singlelabel(nlp: Language, name: str, suggester: Suggester, model: Model[Tuple[List[Doc], Ragged], Floats2d], spans_key: str, negative_weight: float, allow_overlap: bool, scorer: Optional[Callable]) -> SpanCategorizer - added:
function make_tagger(nlp: Language, name: str, model: Model, overwrite: bool, scorer: Optional[Callable], neg_prefix: str, label_smoothing: float) - added:
function make_textcat(nlp: Language, name: str, model: Model[List[Doc], List[Floats2d]], threshold: float, scorer: Optional[Callable]) -> TextCategorizer - added:
function make_tok2vec(nlp: Language, name: str, model: Model) -> Tok2Vec - added:
function make_token_splitter(nlp: Language, name: str, *, min_length: int=…, split_length: int=…) - added:
function register_factories() -> None - added:
variable FACTORIES_REGISTERED
- changed:
function merge_entities(doc) → function merge_entities(doc: Doc) - changed:
function merge_noun_chunks(doc) → function merge_noun_chunks(doc: Doc) -> Doc - changed:
function merge_subtokens(doc, label=…) → function merge_subtokens(doc: Doc, label: str=…) -> Doc - added:
class DocCleaner - added:
class TokenSplitter - added:
constructor DocCleaner(attrs: Dict[str, Any], *, silent: bool=…) - added:
constructor TokenSplitter(min_length: int=…, split_length: int=…) - added:
method DocCleaner.from_bytes(data, **kwargs) - added:
method DocCleaner.from_disk(path, **kwargs) - added:
method DocCleaner.to_bytes(**kwargs) - added:
method DocCleaner.to_disk(path, **kwargs) - added:
method TokenSplitter.from_bytes(data, **kwargs) - added:
method TokenSplitter.from_disk(path, **kwargs) - added:
method TokenSplitter.to_bytes(**kwargs) - added:
method TokenSplitter.to_disk(path, **kwargs)
- removed:
class SentenceSegmenter(object) - removed:
class SimilarityHook(Pipe) - removed:
constructor SentenceSegmenter(vocab, strategy=…) - removed:
constructor SimilarityHook(vocab, model=…, **cfg) - removed:
method SimilarityHook.begin_training(_=…, pipeline=…, sgd=…, **kwargs) - removed:
method SimilarityHook.pipe(docs, **kwargs) - removed:
method SimilarityHook.predict(doc1, doc2) - removed:
method SimilarityHook.update(doc1_doc2, golds, sgd=…, drop=…) - removed:
static method SentenceSegmenter.split_on_punct(doc) - removed:
static method SimilarityHook.Model(length)
- added:
export EntityLinker_v1
- added:
class EntityLinker_v1(TrainablePipe) - added:
constructor EntityLinker_v1(vocab: Vocab, model: Model, name: str=…, *, labels_discard: Iterable[str], n_sents: int, incl_prior: bool, incl_context: bool, entity_vector_length: int, get_candidates: Callable[[KnowledgeBase, Span], Iterable[Candidate]], overwrite: bool=…, scorer: Optional[Callable]=…) -> None - added:
field EntityLinker_v1.NIL - added:
function entity_linker_score(examples, **kwargs) - added:
method EntityLinker_v1.add_label(label) - added:
method EntityLinker_v1.from_bytes(bytes_data, *, exclude=…) - added:
method EntityLinker_v1.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> EntityLinker_v1 - added:
method EntityLinker_v1.get_loss(examples: Iterable[Example], sentence_encodings: Floats2d) - added:
method EntityLinker_v1.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, kb_loader: Optional[Callable[[Vocab], KnowledgeBase]]=…) - added:
method EntityLinker_v1.predict(docs: Iterable[Doc]) -> List[str] - added:
method EntityLinker_v1.rehearse(examples, *, sgd=…, losses=…, **config) - added:
method EntityLinker_v1.set_annotations(docs: Iterable[Doc], kb_ids: List[str]) -> None - added:
method EntityLinker_v1.set_kb(kb_loader: Callable[[Vocab], KnowledgeBase]) - added:
method EntityLinker_v1.to_bytes(*, exclude=…) - added:
method EntityLinker_v1.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - added:
method EntityLinker_v1.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
method EntityLinker_v1.validate_kb() -> None - added:
variable BACKWARD_OVERWRITE
- added:
class Lemmatizer(Pipe) - added:
constructor Lemmatizer(vocab: Vocab, model: Optional[Model], name: str=…, *, mode: str=…, overwrite: bool=…, scorer: Optional[Callable]=…) -> None - added:
function lemmatizer_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
function make_lemmatizer_scorer() - added:
method Lemmatizer.from_bytes(bytes_data: bytes, *, exclude: Iterable[str]=…) -> Lemmatizer - added:
method Lemmatizer.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> Lemmatizer - added:
method Lemmatizer.initialize(get_examples: Optional[Callable[[], Iterable[Example]]]=…, *, nlp: Optional[Language]=…, lookups: Optional[Lookups]=…) - added:
method Lemmatizer.is_base_form(token: Token) -> bool - added:
method Lemmatizer.lookup_lemmatize(token: Token) -> List[str] - added:
method Lemmatizer.rule_lemmatize(token: Token) -> List[str] - added:
method Lemmatizer.to_bytes(*, exclude: Iterable[str]=…) -> bytes - added:
method Lemmatizer.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) - added:
property Lemmatizer.mode - added:
static method Lemmatizer.get_lookups_config(mode: str) -> Tuple[List[str], List[str]]
- added:
class Pipe - added:
function deserialize_config(path: Path) -> Any - added:
method Pipe.get_error_handler() -> Callable[[str, Pipe, List[Doc], Exception], NoReturn] - added:
method Pipe.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Language=…) -> None - added:
method Pipe.pipe(stream: Iterable[Doc], *, batch_size: int=…) -> Iterator[Doc] - added:
method Pipe.score(examples: Iterable[Example], **kwargs: Any) -> Dict[str, Union[float, Dict[str, float]]] - added:
method Pipe.set_error_handler(error_handler: Callable[[str, Pipe, List[Doc], Exception], NoReturn]) -> None - added:
property Pipe.hide_labels -> bool - added:
property Pipe.is_trainable -> bool - added:
property Pipe.label_data -> Any - added:
property Pipe.labels -> Tuple[str, ...]
- added:
class SpanFinder(TrainablePipe) - added:
constructor SpanFinder(nlp: Language, model: Model[Iterable[Doc], Floats2d], name: str=…, *, spans_key: str=…, threshold: float=…, max_length: Optional[int]=…, min_length: Optional[int]=…, scorer: Optional[Callable]=…) -> None - added:
function make_span_finder_scorer() - added:
function span_finder_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
method SpanFinder.get_loss(examples, scores) -> Tuple[float, Floats2d] - added:
method SpanFinder.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…) -> None - added:
method SpanFinder.predict(docs: Iterable[Doc]) - added:
method SpanFinder.set_annotations(docs: Iterable[Doc], scores: Floats2d) -> None - added:
method SpanFinder.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
variable DEFAULT_SPAN_FINDER_MODEL - added:
variable span_finder_default_config
- added:
class SpanRuler(Pipe) - added:
constructor SpanRuler(nlp: Language, name: str=…, *, spans_key: Optional[str]=…, spans_filter: Optional[Callable[[Iterable[Span], Iterable[Span]], Iterable[Span]]]=…, annotate_ents: bool=…, ents_filter: Callable[[Iterable[Span], Iterable[Span]], Iterable[Span]]=…, phrase_matcher_attr: Optional[Union[int, str]]=…, matcher_fuzzy_compare: Callable=…, validate: bool=…, overwrite: bool=…, scorer: Opti… - added:
function make_overlapping_labeled_spans_scorer(spans_key: str=…) - added:
function make_preserve_existing_ents_filter() - added:
function make_prioritize_new_ents_filter() - added:
function overlapping_labeled_spans_score(examples: Iterable[Example], *, spans_key=…, **kwargs) -> Dict[str, Any] - added:
function prioritize_existing_ents_filter(entities: Iterable[Span], spans: Iterable[Span]) -> List[Span] - added:
function prioritize_new_ents_filter(entities: Iterable[Span], spans: Iterable[Span]) -> List[Span] - added:
method SpanRuler.add_patterns(patterns: List[PatternType]) -> None - added:
method SpanRuler.clear() -> None - added:
method SpanRuler.from_bytes(bytes_data: bytes, *, exclude: Iterable[str]=…) -> SpanRuler - added:
method SpanRuler.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> SpanRuler - added:
method SpanRuler.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, patterns: Optional[Sequence[PatternType]]=…) - added:
method SpanRuler.match(doc: Doc) - added:
method SpanRuler.remove(label: str) -> None - added:
method SpanRuler.remove_by_id(pattern_id: str) -> None - added:
method SpanRuler.set_annotations(doc, matches) - added:
method SpanRuler.to_bytes(*, exclude: Iterable[str]=…) -> bytes - added:
method SpanRuler.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - added:
property SpanRuler.ids -> Tuple[str, ...] - added:
property SpanRuler.key -> Optional[str] - added:
property SpanRuler.labels -> Tuple[str, ...] - added:
property SpanRuler.patterns -> List[PatternType] - added:
variable DEFAULT_SPANS_KEY - added:
variable PatternType
- added:
class SpanCategorizer(TrainablePipe) - added:
class Suggester(Protocol) - added:
constructor SpanCategorizer(vocab: Vocab, model: Model[Tuple[List[Doc], Ragged], Floats2d], suggester: Suggester, name: str=…, *, add_negative_label: bool=…, spans_key: str=…, negative_weight: Optional[float]=…, allow_overlap: Optional[bool]=…, max_positive: Optional[int]=…, threshold: Optional[float]=…, scorer: Optional[Callable]=…) -> None - added:
function build_ngram_range_suggester(min_size: int, max_size: int) -> Suggester - added:
function build_ngram_suggester(sizes: List[int]) -> Suggester - added:
function build_preset_spans_suggester(spans_key: str) -> Suggester - added:
function make_spancat_scorer() - added:
function ngram_suggester(docs: Iterable[Doc], sizes: List[int], *, ops: Optional[Ops]=…) -> Ragged - added:
function preset_spans_suggester(docs: Iterable[Doc], spans_key: str, *, ops: Optional[Ops]=…) -> Ragged - added:
function spancat_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
method SpanCategorizer.add_label(label: str) -> int - added:
method SpanCategorizer.get_loss(examples: Iterable[Example], spans_scores: Tuple[Ragged, Floats2d]) -> Tuple[float, float] - added:
method SpanCategorizer.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, labels: Optional[List[str]]=…) -> None - added:
method SpanCategorizer.predict(docs: Iterable[Doc]) - added:
method SpanCategorizer.set_annotations(docs: Iterable[Doc], indices_scores) -> None - added:
method SpanCategorizer.set_candidates(docs: Iterable[Doc], *, candidates_key: str=…) -> None - added:
method SpanCategorizer.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
property SpanCategorizer.key -> str - added:
property SpanCategorizer.label_data -> List[str] - added:
property SpanCategorizer.labels -> Tuple[str] - added:
variable DEFAULT_SPANCAT_MODEL - added:
variable DEFAULT_SPANCAT_SINGLELABEL_MODEL - added:
variable DEFAULT_SPANS_KEY - added:
variable spancat_default_config - added:
variable spancat_singlelabel_default_config
- added:
class TextCategorizer(TrainablePipe) - added:
constructor TextCategorizer(vocab: Vocab, model: Model, name: str=…, *, threshold: float, scorer: Optional[Callable]=…) -> None - added:
function make_textcat_scorer() - added:
function textcat_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
method TextCategorizer.add_label(label: str) -> int - added:
method TextCategorizer.get_loss(examples: Iterable[Example], scores) -> Tuple[float, float] - added:
method TextCategorizer.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, labels: Optional[Iterable[str]]=…, positive_label: Optional[str]=…) -> None - added:
method TextCategorizer.predict(docs: Iterable[Doc]) - added:
method TextCategorizer.rehearse(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
method TextCategorizer.set_annotations(docs: Iterable[Doc], scores) -> None - added:
method TextCategorizer.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float] - added:
property TextCategorizer.label_data -> List[str] - added:
property TextCategorizer.labels -> Tuple[str] - added:
property TextCategorizer.support_missing_values - added:
variable DEFAULT_SINGLE_TEXTCAT_MODEL - added:
variable single_label_bow_config - added:
variable single_label_cnn_config - added:
variable single_label_default_config
- added:
class MultiLabel_TextCategorizer(TextCategorizer) - added:
constructor MultiLabel_TextCategorizer(vocab: Vocab, model: Model, name: str=…, *, threshold: float, scorer: Optional[Callable]=…) -> None - added:
function make_textcat_multilabel_scorer() - added:
function textcat_multilabel_score(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
method MultiLabel_TextCategorizer.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…, labels: Optional[Iterable[str]]=…) - added:
property MultiLabel_TextCategorizer.support_missing_values - added:
variable DEFAULT_MULTI_TEXTCAT_MODEL - added:
variable multi_label_bow_config - added:
variable multi_label_cnn_config - added:
variable multi_label_default_config
- added:
class Tok2Vec(TrainablePipe) - added:
class Tok2VecListener(Model) - added:
constructor Tok2Vec(vocab: Vocab, model: Model, name: str=…) -> None - added:
constructor Tok2VecListener(upstream_name: str, width: int) -> None - added:
field Tok2VecListener.name - added:
function forward(model: Tok2VecListener, inputs, is_train: bool) - added:
method Tok2Vec.add_label(label) - added:
method Tok2Vec.add_listener(listener: Tok2VecListener, component_name: str) -> None - added:
method Tok2Vec.find_listeners(component) -> None - added:
method Tok2Vec.get_loss(examples, scores) -> None - added:
method Tok2Vec.initialize(get_examples: Callable[[], Iterable[Example]], *, nlp: Optional[Language]=…) - added:
method Tok2Vec.predict(docs: Iterable[Doc]) - added:
method Tok2Vec.remove_listener(listener: Tok2VecListener, component_name: str) -> bool - added:
method Tok2Vec.set_annotations(docs: Sequence[Doc], tokvecses) -> None - added:
method Tok2Vec.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) - added:
method Tok2VecListener.receive(batch_id: int, outputs, backprop) -> None - added:
method Tok2VecListener.verify_inputs(inputs) -> bool - added:
property Tok2Vec.listeners -> List[Tok2VecListener] - added:
property Tok2Vec.listening_components -> List[str] - added:
static method Tok2VecListener.get_batch_id(inputs: Iterable[Doc]) -> int - added:
variable DEFAULT_TOK2VEC_MODEL - added:
variable default_model_config
- added:
function populate_registry() -> None - added:
variable REGISTRY_POPULATED
- added:
class ConfigSchema(BaseModel) - added:
class ConfigSchemaInit(BaseModel) - added:
class ConfigSchemaNlp(BaseModel) - added:
class ConfigSchemaPretrain(BaseModel) - added:
class ConfigSchemaPretrainEmpty(BaseModel) - added:
class ConfigSchemaTraining(BaseModel) - added:
class DocJSONSchema(BaseModel) - added:
class ModelMetaSchema(BaseModel) - added:
class RecommendationSchema(BaseModel) - added:
class RecommendationTrf(BaseModel) - added:
class RecommendationTrfItem(BaseModel) - added:
class TokenPattern(BaseModel) - added:
class TokenPatternNumber(BaseModel) - added:
class TokenPatternOperatorSimple(str, Enum) - added:
class TokenPatternSchema(BaseModel) - added:
class TokenPatternString(BaseModel) - added:
field ConfigSchema.components: Dict[str, Dict[str, Any]] - added:
field ConfigSchema.corpora: Dict[str, Reader] - added:
field ConfigSchema.initialize: ConfigSchemaInit - added:
field ConfigSchema.model_config - added:
field ConfigSchema.nlp: ConfigSchemaNlp - added:
field ConfigSchema.pretraining: Union[ConfigSchemaPretrain, ConfigSchemaPretrainEmpty] - added:
field ConfigSchema.training: ConfigSchemaTraining - added:
field ConfigSchemaInit.after_init: Optional[Callable[[Language], Language]] - added:
field ConfigSchemaInit.before_init: Optional[Callable[[Language], Language]] - added:
field ConfigSchemaInit.components: Dict[StrictStr, Dict[StrictStr, Any]] - added:
field ConfigSchemaInit.init_tok2vec: Optional[StrictStr] - added:
field ConfigSchemaInit.lookups: Optional[Lookups] - added:
field ConfigSchemaInit.model_config - added:
field ConfigSchemaInit.tokenizer: Dict[StrictStr, Any] - added:
field ConfigSchemaInit.vectors: Optional[StrictStr] - added:
field ConfigSchemaInit.vocab_data: Optional[StrictStr] - added:
field ConfigSchemaNlp.after_creation: Optional[Callable[[Language], Language]] - added:
field ConfigSchemaNlp.after_pipeline_creation: Optional[Callable[[Language], Language]] - added:
field ConfigSchemaNlp.batch_size: Optional[int] - added:
field ConfigSchemaNlp.before_creation: Optional[Callable[[Type[Language]], Type[Language]]] - added:
field ConfigSchemaNlp.disabled: List[StrictStr] - added:
field ConfigSchemaNlp.lang: StrictStr - added:
field ConfigSchemaNlp.model_config - added:
field ConfigSchemaNlp.pipeline: List[StrictStr] - added:
field ConfigSchemaNlp.tokenizer: Callable - added:
field ConfigSchemaNlp.vectors: Callable - added:
field ConfigSchemaPretrain.batcher: Batcher - added:
field ConfigSchemaPretrain.component: str - added:
field ConfigSchemaPretrain.corpus: StrictStr - added:
field ConfigSchemaPretrain.dropout: StrictFloat - added:
field ConfigSchemaPretrain.layer: str - added:
field ConfigSchemaPretrain.max_epochs: StrictInt - added:
field ConfigSchemaPretrain.model_config - added:
field ConfigSchemaPretrain.n_save_epoch: Optional[StrictInt] - added:
field ConfigSchemaPretrain.n_save_every: Optional[StrictInt] - added:
field ConfigSchemaPretrain.objective: Callable[[Vocab, Model], Model] - added:
field ConfigSchemaPretrain.optimizer: Optimizer - added:
field ConfigSchemaPretrainEmpty.model_config - added:
field ConfigSchemaTraining.accumulate_gradient: StrictInt - added:
field ConfigSchemaTraining.annotating_components: List[str] - added:
field ConfigSchemaTraining.batcher: Batcher - added:
field ConfigSchemaTraining.before_to_disk: Optional[Callable[[Language], Language]] - added:
field ConfigSchemaTraining.before_update: Optional[Callable[[Language, Dict[str, Any]], None]] - added:
field ConfigSchemaTraining.dev_corpus: StrictStr - added:
field ConfigSchemaTraining.dropout: StrictFloat - added:
field ConfigSchemaTraining.eval_frequency: StrictInt - added:
field ConfigSchemaTraining.frozen_components: List[str] - added:
field ConfigSchemaTraining.gpu_allocator: Optional[StrictStr] - added:
field ConfigSchemaTraining.logger: Logger - added:
field ConfigSchemaTraining.max_epochs: StrictInt - added:
field ConfigSchemaTraining.max_steps: StrictInt - added:
field ConfigSchemaTraining.model_config - added:
field ConfigSchemaTraining.optimizer: Optimizer - added:
field ConfigSchemaTraining.patience: StrictInt - added:
field ConfigSchemaTraining.score_weights: Dict[StrictStr, Optional[Union[StrictFloat, StrictInt]]] - added:
field ConfigSchemaTraining.seed: Optional[StrictInt] - added:
field ConfigSchemaTraining.train_corpus: StrictStr - added:
field DocJSONSchema.cats: Optional[Dict[StrictStr, StrictFloat]] - added:
field DocJSONSchema.ents: Optional[List[Dict[StrictStr, Union[StrictInt, StrictStr]]]] - added:
field DocJSONSchema.sents: Optional[List[Dict[StrictStr, StrictInt]]] - added:
field DocJSONSchema.spans: Optional[Dict[StrictStr, List[Dict[StrictStr, Union[StrictStr, StrictInt]]]]] - added:
field DocJSONSchema.text: StrictStr - added:
field DocJSONSchema.tokens: List[Dict[StrictStr, Union[StrictStr, StrictInt]]] - added:
field DocJSONSchema.underscore_doc: Optional[Dict[StrictStr, Any]] - added:
field DocJSONSchema.underscore_span: Optional[Dict[StrictStr, List[Dict[StrictStr, Any]]]] - added:
field DocJSONSchema.underscore_token: Optional[Dict[StrictStr, List[Dict[StrictStr, Any]]]] - added:
field ModelMetaSchema.author: StrictStr - added:
field ModelMetaSchema.description: StrictStr - added:
field ModelMetaSchema.email: StrictStr - added:
field ModelMetaSchema.labels: Dict[str, List[str]] - added:
field ModelMetaSchema.lang: StrictStr - added:
field ModelMetaSchema.license: StrictStr - added:
field ModelMetaSchema.name: StrictStr - added:
field ModelMetaSchema.parent_package: StrictStr - added:
field ModelMetaSchema.performance: Dict[str, Any] - added:
field ModelMetaSchema.pipeline: List[StrictStr] - added:
field ModelMetaSchema.requirements: List[StrictStr] - added:
field ModelMetaSchema.sources: Optional[Union[List[StrictStr], List[Dict[str, str]]]] - added:
field ModelMetaSchema.spacy_git_version: StrictStr - added:
field ModelMetaSchema.spacy_version: StrictStr - added:
field ModelMetaSchema.url: StrictStr - added:
field ModelMetaSchema.vectors: Dict[str, Any] - added:
field ModelMetaSchema.version: StrictStr - added:
field RecommendationSchema.has_letters: bool - added:
field RecommendationSchema.transformer: Optional[RecommendationTrf] - added:
field RecommendationSchema.word_vectors: Optional[str] - added:
field RecommendationTrf.accuracy: RecommendationTrfItem - added:
field RecommendationTrf.efficiency: RecommendationTrfItem - added:
field RecommendationTrfItem.name: str - added:
field RecommendationTrfItem.size_factor: int - added:
field TokenPattern.dep: Optional[StringValue] - added:
field TokenPattern.ent_id: Optional[StringValue] - added:
field TokenPattern.ent_iob: Optional[IobValue] - added:
field TokenPattern.ent_kb_id: Optional[StringValue] - added:
field TokenPattern.ent_type: Optional[StringValue] - added:
field TokenPattern.is_alpha: Optional[StrictBool] - added:
field TokenPattern.is_ascii: Optional[StrictBool] - added:
field TokenPattern.is_bracket: Optional[StrictBool] - added:
field TokenPattern.is_currency: Optional[StrictBool] - added:
field TokenPattern.is_digit: Optional[StrictBool] - added:
field TokenPattern.is_left_punct: Optional[StrictBool] - added:
field TokenPattern.is_lower: Optional[StrictBool] - added:
field TokenPattern.is_punct: Optional[StrictBool] - added:
field TokenPattern.is_quote: Optional[StrictBool] - added:
field TokenPattern.is_right_punct: Optional[StrictBool] - added:
field TokenPattern.is_sent_start: Optional[StrictBool] - added:
field TokenPattern.is_space: Optional[StrictBool] - added:
field TokenPattern.is_stop: Optional[StrictBool] - added:
field TokenPattern.is_title: Optional[StrictBool] - added:
field TokenPattern.is_upper: Optional[StrictBool] - added:
field TokenPattern.lemma: Optional[StringValue] - added:
field TokenPattern.length: Optional[NumberValue] - added:
field TokenPattern.like_email: Optional[StrictBool] - added:
field TokenPattern.like_num: Optional[StrictBool] - added:
field TokenPattern.like_url: Optional[StrictBool] - added:
field TokenPattern.lower: Optional[StringValue] - added:
field TokenPattern.model_config - added:
field TokenPattern.morph: Optional[StringValue] - added:
field TokenPattern.norm: Optional[StringValue] - added:
field TokenPattern.op: Optional[TokenPatternOperator] - added:
field TokenPattern.orth: Optional[StringValue] - added:
field TokenPattern.pos: Optional[StringValue] - added:
field TokenPattern.sent_start: Optional[StrictBool] - added:
field TokenPattern.shape: Optional[StringValue] - added:
field TokenPattern.spacy: Optional[StrictBool] - added:
field TokenPattern.tag: Optional[StringValue] - added:
field TokenPattern.text: Optional[StringValue] - added:
field TokenPattern.underscore: Optional[Dict[StrictStr, UnderscoreValue]] - added:
field TokenPatternNumber.EQ: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.GEQ: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.GT: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.IN: Optional[List[StrictInt]] - added:
field TokenPatternNumber.INTERSECTS: Optional[List[StrictInt]] - added:
field TokenPatternNumber.IS_SUBSET: Optional[List[StrictInt]] - added:
field TokenPatternNumber.IS_SUPERSET: Optional[List[StrictInt]] - added:
field TokenPatternNumber.LEQ: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.LT: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.NEQ: Optional[Union[StrictInt, StrictFloat]] - added:
field TokenPatternNumber.NOT_IN: Optional[List[StrictInt]] - added:
field TokenPatternNumber.REGEX: Optional[StrictStr] - added:
field TokenPatternNumber.model_config - added:
field TokenPatternOperatorSimple.exclamation - added:
field TokenPatternOperatorSimple.plus - added:
field TokenPatternOperatorSimple.question - added:
field TokenPatternOperatorSimple.star - added:
field TokenPatternSchema.model_config - added:
field TokenPatternSchema.pattern: List[TokenPattern] - added:
field TokenPatternString.FUZZY1: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY2: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY3: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY4: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY5: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY6: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY7: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY8: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY9: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.FUZZY: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.IN: Optional[List[StrictStr]] - added:
field TokenPatternString.INTERSECTS: Optional[List[StrictStr]] - added:
field TokenPatternString.IS_SUBSET: Optional[List[StrictStr]] - added:
field TokenPatternString.IS_SUPERSET: Optional[List[StrictStr]] - added:
field TokenPatternString.NOT_IN: Optional[List[StrictStr]] - added:
field TokenPatternString.REGEX: Optional[Union[StrictStr, TokenPatternString]] - added:
field TokenPatternString.model_config - added:
function get_arg_model(func: Callable, *, exclude: Iterable[str]=…, name: str=…, strict: bool=…) -> type[BaseModel] - added:
function validate(schema: Type[BaseModel], obj: Dict[str, Any]) -> List[str] - added:
function validate_init_settings(func: Callable, settings: Dict[str, Any], *, section: Optional[str]=…, name: str=…, exclude: Iterable[str]=…) -> Dict[str, Any] - added:
function validate_token_pattern(obj: list) -> List[str] - added:
static method TokenPattern.raise_for_none(v) - added:
static method TokenPatternNumber.raise_for_none(v) - added:
static method TokenPatternString.raise_for_none(v) - added:
variable ArgSchemaConfig - added:
variable ArgSchemaConfigExtra - added:
variable Batcher - added:
variable CONFIG_SCHEMAS - added:
variable IobValue - added:
variable ItemT - added:
variable Logger - added:
variable NumberValue - added:
variable Reader - added:
variable StringValue - added:
variable TokenPatternOperator - added:
variable TokenPatternOperatorMinMax - added:
variable UnderscoreValue
- removed:
property PRFScore.fscore - removed:
property PRFScore.precision - removed:
property PRFScore.recall - removed:
property Scorer.ents_f - removed:
property Scorer.ents_p - removed:
property Scorer.ents_per_type - removed:
property Scorer.ents_r - removed:
property Scorer.las - removed:
property Scorer.las_per_type - removed:
property Scorer.scores - removed:
property Scorer.tags_acc - removed:
property Scorer.textcat_score - removed:
property Scorer.textcats_per_cat - removed:
property Scorer.token_acc - removed:
property Scorer.uas - changed:
class PRFScore(object) → class PRFScore - changed:
class ROCAUCScore(object) → class ROCAUCScore - changed:
class Scorer(object) → class Scorer - changed:
constructor PRFScore() → constructor PRFScore(*, tp: int=…, fp: int=…, fn: int=…) -> None - changed:
constructor ROCAUCScore() → constructor ROCAUCScore() -> None - changed:
constructor Scorer(eval_punct=…, pipeline=…) → constructor Scorer(nlp: Optional[Language]=…, default_lang: str=…, default_pipeline: Iterable[str]=…, **cfg) -> None - changed:
method PRFScore.score_set(cand, gold) → method PRFScore.score_set(cand: set, gold: set) -> None - changed:
method ROCAUCScore.score_set(cand, gold) → method ROCAUCScore.score_set(cand, gold) -> None - changed:
method Scorer.score(doc, gold, verbose=…, punct_labels=…) → method Scorer.score(examples: Iterable[Example], *, per_component: bool=…) -> Dict[str, Any] - added:
function get_ner_prf(examples: Iterable[Example], **kwargs) -> Dict[str, Any] - added:
function trapezoid(y, x=…, dx=…, axis=…) - added:
method PRFScore.to_dict() -> Dict[str, float] - added:
method ROCAUCScore.is_binary() - added:
property PRFScore.fscore -> float - added:
property PRFScore.precision -> float - added:
property PRFScore.recall -> float - added:
static method Scorer.score_cats(examples: Iterable[Example], attr: str, *, getter: Callable[[Doc, str], Any]=…, labels: Iterable[str]=…, multi_label: bool=…, positive_label: Optional[str]=…, threshold: Optional[float]=…, **cfg) -> Dict[str, Any] - added:
static method Scorer.score_deps(examples: Iterable[Example], attr: str, *, getter: Callable[[Token, str], Any]=…, head_attr: str=…, head_getter: Callable[[Token, str], Token]=…, ignore_labels: Iterable[str]=…, missing_values: Set[Any]=…, **cfg) -> Dict[str, Any] - added:
static method Scorer.score_links(examples: Iterable[Example], *, negative_labels: Iterable[str], **cfg) -> Dict[str, Any] - added:
static method Scorer.score_spans(examples: Iterable[Example], attr: str, *, getter: Callable[[Doc, str], Iterable[Span]]=…, has_annotation: Optional[Callable[[Doc], bool]]=…, labeled: bool=…, allow_overlap: bool=…, **cfg) -> Dict[str, Any] - added:
static method Scorer.score_token_attr(examples: Iterable[Example], attr: str, *, getter: Callable[[Token, str], Any]=…, missing_values: Set[Any]=…, **cfg) -> Dict[str, Any] - added:
static method Scorer.score_token_attr_per_feat(examples: Iterable[Example], attr: str, *, getter: Callable[[Token, str], Any]=…, missing_values: Set[Any]=…, **cfg) -> Dict[str, Any] - added:
static method Scorer.score_tokenization(examples: Iterable[Example], **cfg) -> Dict[str, Any] - added:
variable DEFAULT_PIPELINE - added:
variable MISSING_VALUES
- added:
class StringStore - added:
constructor StringStore(strings: Optional[Iterable[str]]=…, freeze: bool=…) -> None - added:
function get_string_id(key: Union[str, int]) -> int - added:
method StringStore.add(string: str) -> int - added:
method StringStore.as_int(key: Union[bytes, str, int]) -> int - added:
method StringStore.as_string(key: Union[bytes, str, int]) -> str - added:
method StringStore.from_bytes(bytes_data: bytes, **kwargs: Any) -> StringStore - added:
method StringStore.from_disk(path: Union[str, Path]) -> StringStore - added:
method StringStore.to_bytes(**kwargs: Any) -> bytes - added:
method StringStore.to_disk(path: Union[str, Path]) -> None
- added:
export MorphAnalysis - added:
export SpanGroup
- added:
class MorphAnalysis - added:
constructor MorphAnalysis(vocab: Vocab, features: Union[Dict[str, str], str]=…) -> None - added:
method MorphAnalysis.get(field: Any, default: Optional[List[str]]) -> List[str] - added:
method MorphAnalysis.to_dict() -> Dict[str, str] - added:
method MorphAnalysis.to_json() -> str - added:
static method MorphAnalysis.from_id(vocab: Vocab, key: Any) -> MorphAnalysis
- added:
class Span - added:
class SpanMethod(Protocol) - added:
constructor Span(doc: Doc, start: int, end: int, label: Union[str, int]=…, vector: Optional[Floats1d]=…, vector_norm: Optional[float]=…, kb_id: Union[str, int]=…, span_id: Union[str, int]=…) -> None - added:
field Span.end: int - added:
field Span.end_char: int - added:
field Span.ent_id: int - added:
field Span.ent_id_: str - added:
field Span.id: int - added:
field Span.id_: str - added:
field Span.kb_id: int - added:
field Span.kb_id_: str - added:
field Span.label: int - added:
field Span.label_: str - added:
field Span.start: int - added:
field Span.start_char: int - added:
method Span.as_doc(*, copy_user_data: bool=…) -> Doc - added:
method Span.char_span(start_idx: int, end_idx: int, label: Union[int, str]=…, kb_id: Union[int, str]=…, vector: Optional[Floats1d]=…, id: Union[int, str]=…, alignment_mode: str=…, span_id: Union[int, str]=…) -> Span - added:
method Span.get_lca_matrix() -> Ints2d - added:
method Span.similarity(other: Union[Doc, Span, Token, Lexeme]) -> float - added:
property Span.conjuncts -> Tuple[Token] - added:
property Span.doc -> Doc - added:
property Span.ents -> Tuple[Span] - added:
property Span.has_vector -> bool - added:
property Span.lefts -> Iterator[Token] - added:
property Span.lemma_ -> str - added:
property Span.n_lefts -> int - added:
property Span.n_rights -> int - added:
property Span.noun_chunks -> Iterator[Span] - added:
property Span.orth_ -> str - added:
property Span.rights -> Iterator[Token] - added:
property Span.root -> Token - added:
property Span.sent -> Span - added:
property Span.sentiment -> float - added:
property Span.subtree -> Iterator[Token] - added:
property Span.tensor -> FloatsXd - added:
property Span.text -> str - added:
property Span.text_with_ws -> str - added:
property Span.vector -> Floats1d - added:
property Span.vector_norm -> float - added:
property Span.vocab -> Vocab - added:
static method Span.get_extension(name: str) -> Tuple[Optional[Any], Optional[SpanMethod], Optional[Callable[[Span], Any]], Optional[Callable[[Span, Any], None]]] - added:
static method Span.has_extension(name: str) -> bool - added:
static method Span.remove_extension(name: str) -> Tuple[Optional[Any], Optional[SpanMethod], Optional[Callable[[Span], Any]], Optional[Callable[[Span, Any], None]]] - added:
static method Span.set_extension(name: str, default: Optional[Any]=…, getter: Optional[Callable[[Span], Any]]=…, setter: Optional[Callable[[Span, Any], None]]=…, method: Optional[SpanMethod]=…, force: bool=…) -> None
- added:
class SpanGroup - added:
constructor SpanGroup(doc: Doc, *, name: str=…, attrs: Dict[str, Any]=…, spans: Iterable[Span]=…) -> None - added:
field SpanGroup.attrs: Dict[str, Any] - added:
field SpanGroup.name: str - added:
method SpanGroup.append(span: Span) -> None - added:
method SpanGroup.copy(doc: Optional[Doc]=…) -> SpanGroup - added:
method SpanGroup.extend(spans: Iterable[Span]) -> None - added:
method SpanGroup.from_bytes(bytes_data: bytes) -> SpanGroup - added:
method SpanGroup.to_bytes() -> bytes - added:
property SpanGroup.doc -> Doc - added:
property SpanGroup.has_overlap -> bool
- added:
class Token - added:
class TokenMethod(Protocol) - added:
constructor Token(vocab: Vocab, doc: Doc, offset: int) -> None - added:
field Token.dep: int - added:
field Token.dep_: str - added:
field Token.doc: Doc - added:
field Token.ent_id: int - added:
field Token.ent_id_: str - added:
field Token.ent_kb_id: int - added:
field Token.ent_kb_id_: str - added:
field Token.ent_type: int - added:
field Token.ent_type_: str - added:
field Token.head: Token - added:
field Token.i: int - added:
field Token.is_sent_end: Optional[bool] - added:
field Token.is_sent_start: Optional[bool] - added:
field Token.lemma: int - added:
field Token.lemma_: str - added:
field Token.morph: MorphAnalysis - added:
field Token.norm_: str - added:
field Token.pos: int - added:
field Token.pos_: str - added:
field Token.sent_start: bool - added:
field Token.tag: int - added:
field Token.tag_: str - added:
field Token.vocab: Vocab - added:
method Token.has_dep() -> bool - added:
method Token.has_head() -> bool - added:
method Token.has_morph() -> bool - added:
method Token.is_ancestor(descendant: Token) -> bool - added:
method Token.nbor(i: int=…) -> Token - added:
method Token.similarity(other: Union[Doc, Span, Token, Lexeme]) -> float - added:
property Token.ancestors -> Iterator[Token] - added:
property Token.children -> Iterator[Token] - added:
property Token.cluster -> int - added:
property Token.conjuncts -> Tuple[Token] - added:
property Token.ent_iob -> int - added:
property Token.ent_iob_ -> str - added:
property Token.has_vector -> bool - added:
property Token.idx -> int - added:
property Token.is_alpha -> bool - added:
property Token.is_ascii -> bool - added:
property Token.is_bracket -> bool - added:
property Token.is_currency -> bool - added:
property Token.is_digit -> bool - added:
property Token.is_left_punct -> bool - added:
property Token.is_lower -> bool - added:
property Token.is_oov -> bool - added:
property Token.is_punct -> bool - added:
property Token.is_quote -> bool - added:
property Token.is_right_punct -> bool - added:
property Token.is_space -> bool - added:
property Token.is_stop -> bool - added:
property Token.is_title -> bool - added:
property Token.is_upper -> bool - added:
property Token.lang -> int - added:
property Token.lang_ -> str - added:
property Token.left_edge -> Token - added:
property Token.lefts -> Iterator[Token] - added:
property Token.lex -> Lexeme - added:
property Token.lex_id -> int - added:
property Token.like_email -> bool - added:
property Token.like_num -> bool - added:
property Token.like_url -> bool - added:
property Token.lower -> int - added:
property Token.lower_ -> str - added:
property Token.n_lefts -> int - added:
property Token.n_rights -> int - added:
property Token.norm -> int - added:
property Token.orth -> int - added:
property Token.orth_ -> str - added:
property Token.prefix -> int - added:
property Token.prefix_ -> str - added:
property Token.prob -> float - added:
property Token.rank -> int - added:
property Token.right_edge -> Token - added:
property Token.rights -> Iterator[Token] - added:
property Token.sent -> Span - added:
property Token.sentiment -> float - added:
property Token.shape -> int - added:
property Token.shape_ -> str - added:
property Token.subtree -> Iterator[Token] - added:
property Token.suffix -> int - added:
property Token.suffix_ -> str - added:
property Token.tensor -> Optional[FloatsXd] - added:
property Token.text -> str - added:
property Token.text_with_ws -> str - added:
property Token.vector -> Floats1d - added:
property Token.vector_norm -> float - added:
property Token.whitespace_ -> str - added:
static method Token.get_extension(name: str) -> Tuple[Optional[Any], Optional[TokenMethod], Optional[Callable[[Token], Any]], Optional[Callable[[Token, Any], None]]] - added:
static method Token.has_extension(name: str) -> bool - added:
static method Token.iob_strings() -> Tuple[str] - added:
static method Token.remove_extension(name: str) -> Tuple[Optional[Any], Optional[TokenMethod], Optional[Callable[[Token], Any]], Optional[Callable[[Token, Any], None]]] - added:
static method Token.set_extension(name: str, default: Optional[Any]=…, getter: Optional[Callable[[Token], Any]]=…, setter: Optional[Callable[[Token, Any], None]]=…, method: Optional[TokenMethod]=…, force: bool=…) -> None
- changed:
class Underscore(object) → class Underscore - changed:
constructor Underscore(extensions, obj, start=…, end=…) → constructor Underscore(extensions: Dict[str, Any], obj: Union[Doc, Span, Token], start: Optional[int]=…, end: Optional[int]=…) - changed:
field Underscore.doc_extensions → field Underscore.doc_extensions: Dict[Any, Any] - changed:
field Underscore.span_extensions → field Underscore.span_extensions: Dict[Any, Any] - changed:
field Underscore.token_extensions → field Underscore.token_extensions: Dict[Any, Any] - changed:
function get_ext_args(**kwargs) → function get_ext_args(**kwargs: Any) - changed:
method Underscore.get(name) → method Underscore.get(name: str) -> Any - changed:
method Underscore.has(name) → method Underscore.has(name: str) -> bool - changed:
method Underscore.set(name, value) → method Underscore.set(name: str, value: Any) - changed:
static method Underscore.get_state() → static method Underscore.get_state() -> Tuple[Dict[Any, Any], Dict[Any, Any], Dict[Any, Any]] - changed:
static method Underscore.load_state(state) → static method Underscore.load_state(state: Tuple[Dict[Any, Any], Dict[Any, Any], Dict[Any, Any]]) -> None
- added:
export Alignment - added:
export Corpus - added:
export Example - added:
export JsonlCorpus - added:
export PlainTextCorpus - added:
export biluo_tags_to_offsets - added:
export biluo_tags_to_spans - added:
export biluo_to_iob - added:
export create_copy_from_base_model - added:
export docs_to_json - added:
export dont_augment - added:
export iob_to_biluo - added:
export minibatch_by_padded_size - added:
export minibatch_by_words - added:
export offsets_to_biluo_tags - added:
export orth_variants_augmenter - added:
export read_json_file - added:
export remove_bilu_prefix - added:
export split_bilu_label - added:
export tags_to_entities - added:
export validate_examples - added:
export validate_get_examples
- added:
class Alignment - added:
field Alignment.x2y: AlignmentArray - added:
field Alignment.y2x: AlignmentArray - added:
static method Alignment.from_indices(x2y: List[List[int]], y2x: List[List[int]]) -> Alignment - added:
static method Alignment.from_strings(A: List[str], B: List[str]) -> Alignment
- added:
function combined_augmenter(nlp: Language, example: Example, *, lower_level: float=…, orth_level: float=…, orth_variants: Optional[Dict[str, List[Dict]]]=…, whitespace_level: float=…, whitespace_per_token: float=…, whitespace_variants: Optional[List[str]]=…) -> Iterator[Example] - added:
function construct_modified_raw_text(token_dict) - added:
function create_combined_augmenter(lower_level: float, orth_level: float, orth_variants: Optional[Dict[str, List[Dict]]], whitespace_level: float, whitespace_per_token: float, whitespace_variants: Optional[List[str]]) -> Callable[[Language, Example], Iterator[Example]] - added:
function create_lower_casing_augmenter(level: float) -> Callable[[Language, Example], Iterator[Example]] - added:
function create_orth_variants_augmenter(level: float, lower: float, orth_variants: Dict[str, List[Dict]]) -> Callable[[Language, Example], Iterator[Example]] - added:
function dont_augment(nlp: Language, example: Example) -> Iterator[Example] - added:
function lower_casing_augmenter(nlp: Language, example: Example, *, level: float) -> Iterator[Example] - added:
function make_lowercase_variant(nlp: Language, example: Example) - added:
function make_orth_variants(nlp: Language, raw: str, token_dict: Dict[str, List[str]], orth_variants: Dict[str, List[Dict[str, List[str]]]], *, lower: bool=…) -> Tuple[str, Dict[str, List[str]]] - added:
function make_whitespace_variant(nlp: Language, example: Example, whitespace: str, position: int) -> Example - added:
function orth_variants_augmenter(nlp: Language, example: Example, orth_variants: Dict[str, List[Dict]], *, level: float=…, lower: float=…) -> Iterator[Example]
- added:
function configure_minibatch(size: Sizing, get_length: Optional[Callable[[ItemT], int]]=…) -> BatcherT - added:
function configure_minibatch_by_padded_size(*, size: Sizing, buffer: int, discard_oversize: bool, get_length: Optional[Callable[[ItemT], int]]=…) -> BatcherT - added:
function configure_minibatch_by_words(*, size: Sizing, tolerance: float, discard_oversize: bool, get_length: Optional[Callable[[ItemT], int]]=…) -> BatcherT - added:
function minibatch_by_padded_size(seqs: Iterable[ItemT], size: Sizing, buffer: int=…, discard_oversize: bool=…, get_length: Callable=…) -> Iterable[List[ItemT]] - added:
function minibatch_by_words(seqs: Iterable[ItemT], size: Sizing, tolerance=…, discard_oversize=…, get_length=…) -> Iterable[List[ItemT]] - added:
variable BatcherT - added:
variable ItemT - added:
variable Sizing
- added:
function create_copy_from_base_model(tokenizer: Optional[str]=…, vocab: Optional[str]=…) -> Callable[[Language], Language]
- added:
export conll_ner_to_docs - added:
export conllu_to_docs - added:
export iob_to_docs - added:
export json_to_docs
- added:
function conll_ner_to_docs(input_data, n_sents=…, seg_sents=…, model=…, no_print=…, **kwargs) - added:
function n_sents_info(msg, n_sents) - added:
function segment_docs(input_data, n_sents, doc_delimiter) - added:
function segment_sents_and_docs(doc, n_sents, doc_delimiter, model=…, msg=…)
- added:
function conllu_sentence_to_doc(vocab, lines, ner_tag_pattern, merge_subtokens=…, append_morphology=…, ner_map=…, set_ents=…) - added:
function conllu_to_docs(input_data, n_sents=…, append_morphology=…, ner_map=…, merge_subtokens=…, no_print=…, **_) - added:
function get_entities(lines, tag_pattern, ner_map=…) - added:
function has_ner(input_data, ner_tag_pattern) - added:
function merge_conllu_subtokens(lines, doc) - added:
function read_conllx(input_data, append_morphology=…, merge_subtokens=…, ner_tag_pattern=…, ner_map=…)
- added:
function iob_to_docs(input_data, n_sents=…, no_print=…, *args, **kwargs) - added:
function read_iob(raw_sents, vocab, n_sents)
- added:
function json_to_docs(input_data, model=…, **kwargs)
- added:
class Corpus - added:
class JsonlCorpus - added:
class PlainTextCorpus - added:
constructor Corpus(path: Union[str, Path], *, limit: int=…, gold_preproc: bool=…, max_length: int=…, augmenter: Optional[Callable]=…, shuffle: bool=…) -> None - added:
constructor JsonlCorpus(path: Optional[Union[str, Path]], *, limit: int=…, min_length: int=…, max_length: int=…) -> None - added:
constructor PlainTextCorpus(path: Optional[Union[str, Path]], *, min_length: int=…, max_length: int=…) -> None - added:
field JsonlCorpus.file_type - added:
field PlainTextCorpus.file_type - added:
function create_docbin_reader(path: Optional[Path], gold_preproc: bool, max_length: int=…, limit: int=…, augmenter: Optional[Callable]=…) -> Callable[[Language], Iterable[Example]] - added:
function create_jsonl_reader(path: Optional[Union[str, Path]], min_length: int=…, max_length: int=…, limit: int=…) -> Callable[[Language], Iterable[Example]] - added:
function create_plain_text_reader(path: Optional[Path], min_length: int=…, max_length: int=…) -> Callable[[Language], Iterable[Example]] - added:
function read_labels(path: Union[str, Path], *, require: bool=…) - added:
function walk_corpus(path: Union[str, Path], file_type) -> List[Path] - added:
method Corpus.make_examples(nlp: Language, reference_docs: Iterable[Doc]) -> Iterator[Example] - added:
method Corpus.make_examples_gold_preproc(nlp: Language, reference_docs: Iterable[Doc]) -> Iterator[Example] - added:
method Corpus.read_docbin(vocab: Vocab, locs: Iterable[Union[str, Path]]) -> Iterator[Doc] - added:
variable FILE_TYPE
- added:
function convert_vectors(nlp: Language, vectors_loc: Optional[Path], *, truncate: int, prune: int, name: Optional[str]=…, mode: str=…, attr: str=…) -> None - added:
function ensure_shape(vectors_loc) - added:
function init_nlp(config: Config, *, use_gpu: int=…) -> Language - added:
function init_tok2vec(nlp: Language, pretrain_config: Dict[str, Any], init_config: Dict[str, Any]) -> bool - added:
function init_vocab(nlp: Language, *, data: Optional[Path]=…, lookups: Optional[Lookups]=…, vectors: Optional[str]=…) -> None - added:
function load_vectors_into_model(nlp: Language, name: Union[str, Path], *, add_strings: bool=…) -> None - added:
function open_file(loc: Union[str, Path]) -> IO - added:
function read_vectors(vectors_loc: Path, truncate_vectors: int, *, mode: str=…)
- added:
function biluo_tags_to_offsets(doc: Doc, tags: Iterable[str]) -> List[Tuple[int, int, Union[str, int]]] - added:
function biluo_tags_to_spans(doc: Doc, tags: Iterable[str]) -> List[Span] - added:
function biluo_to_iob(tags: Iterable[str]) -> List[str] - added:
function doc_to_biluo_tags(doc: Doc, missing: str=…) - added:
function iob_to_biluo(tags: Iterable[str]) -> List[str] - added:
function offsets_to_biluo_tags(doc: Doc, entities: Iterable[Tuple[int, int, Union[str, int]]], missing: str=…) -> List[str] - added:
function remove_bilu_prefix(label: str) -> str - added:
function split_bilu_label(label: str) -> Tuple[str, str] - added:
function tags_to_entities(tags: Iterable[str]) -> List[Tuple[str, int, int]] - added:
variable biluo_tags_from_offsets - added:
variable offsets_from_biluo_tags - added:
variable spans_from_biluo_tags
- added:
function console_logger(progress_bar: bool=…, console_output: bool=…, output_file: Optional[Union[str, Path]]=…) - added:
function console_logger_v3(progress_bar: Optional[str]=…, console_output: bool=…, output_file: Optional[Union[str, Path]]=…) - added:
function setup_table(*, cols: List[str], widths: List[int], max_width: int=…) -> Tuple[List[str], List[int], List[str]]
- added:
function clean_output_dir(path: Optional[Path]) -> None - added:
function create_before_to_disk_callback(callback: Optional[Callable[[Language], Language]]) -> Callable[[Language], Language] - added:
function create_evaluation_callback(nlp: Language, dev_corpus: Callable, weights: Dict[str, float]) -> Callable[[], Tuple[float, Dict[str, float]]] - added:
function create_train_batches(nlp: Language, corpus: Callable[[Language], Iterable[Example]], batcher: Callable[[Iterable[Example]], Iterable[Example]], max_epochs: int) - added:
function subdivide_batch(batch, accumulate_gradient) - added:
function train(nlp: Language, output_path: Optional[Path]=…, *, use_gpu: int=…, stdout: IO=…, stderr: IO=…) -> Tuple[Language, Optional[Path]] - added:
function train_while_improving(nlp: Language, optimizer: Optimizer, train_data, evaluate, *, dropout: float, eval_frequency: int, accumulate_gradient: int, patience: int, max_steps: int, exclude: List[str], annotating_components: List[str], before_update: Optional[Callable[[Language, Dict[str, Any]], None]]) - added:
function update_meta(training: Union[Dict[str, Any], Config], nlp: Language, info: Dict[str, Any]) -> None - added:
variable DIR_MODEL_BEST - added:
variable DIR_MODEL_LAST
- added:
class ProgressTracker - added:
constructor ProgressTracker(frequency=…) - added:
function create_pretraining_model(nlp, pretrain_config) - added:
function ensure_docs(examples_or_docs: Iterable[Union[Doc, Example]]) -> List[Doc] - added:
function get_tok2vec_ref(nlp, pretrain_config) - added:
function make_update(model: Model, docs: Iterable[Doc], optimizer: Optimizer, objective_func: Callable) -> float - added:
function pretrain(config: Config, output_dir: Path, resume_path: Optional[Path]=…, epoch_resume: Optional[int]=…, use_gpu: int=…, silent: bool=…, skip_last: bool=…) - added:
method ProgressTracker.update(epoch, loss, docs)
- added:
class InitializableComponent(Protocol) - added:
class ListenedToComponent(Protocol) - added:
class TrainableComponent(Protocol) - added:
field ListenedToComponent.listener_map: Dict[str, Sequence[Model]] - added:
field ListenedToComponent.listeners: Sequence[Model] - added:
field ListenedToComponent.listening_components: List[str] - added:
field ListenedToComponent.model: Any - added:
field TrainableComponent.is_trainable: bool - added:
field TrainableComponent.model: Any - added:
method InitializableComponent.initialize(get_examples: Callable[[], Iterable[Example]], nlp: Language, **kwargs: Any) - added:
method ListenedToComponent.add_listener(listener: Model, component_name: str) -> None - added:
method ListenedToComponent.find_listeners(component) -> None - added:
method ListenedToComponent.remove_listener(listener: Model, component_name: str) -> bool - added:
method TrainableComponent.finish_update(sgd: Optimizer) -> None - added:
method TrainableComponent.update(examples: Iterable[Example], *, drop: float=…, sgd: Optional[Optimizer]=…, losses: Optional[Dict[str, float]]=…) -> Dict[str, float]
- removed:
function compounding(start, stop, compound) - removed:
function decaying(start, stop, decay) - removed:
function env_opt(name, default=…) - removed:
function fix_random_seed(seed=…) - removed:
function get_component_name(component) - removed:
function get_data_path(require_exists=…) - removed:
function get_json_validator(schema) - removed:
function get_serialization_exclude(serializers, exclude, kwargs) - removed:
function itershuffle(iterable, bufsize=…) - removed:
function load_model_from_link(name, **overrides) - removed:
function make_layer(arch_config) - removed:
function minibatch_by_words(items, size, tuples=…, count_words=…) - removed:
function set_data_path(path) - removed:
function set_env_log(value) - removed:
function stepping(start, stop, steps) - removed:
function use_gpu(gpu_id) - removed:
function validate_json(data, validator) - removed:
function validate_schema(schema) - removed:
variable jsonschema - changed:
class DummyTokenizer(object) → class DummyTokenizer - changed:
class registry(object) → class registry(thinc.registry) - changed:
function add_lookups(default_func, *lookups) → function add_lookups(default_func: Callable[[str], Any], *lookups) -> Callable[[str], Any] - changed:
function compile_infix_regex(entries) → function compile_infix_regex(entries: Iterable[Union[str, Pattern]]) -> Pattern - changed:
function compile_prefix_regex(entries) → function compile_prefix_regex(entries: Iterable[Union[str, Pattern]]) -> Pattern - changed:
function compile_suffix_regex(entries) → function compile_suffix_regex(entries: Iterable[Union[str, Pattern]]) -> Pattern - changed:
function ensure_path(path) → function ensure_path(path: Any) -> Any - changed:
function escape_html(text) → function escape_html(text: str) -> str - changed:
function expand_exc(excs, search, replace) → function expand_exc(excs: Dict[str, List[dict]], search: str, replace: str) -> Dict[str, List[dict]] - changed:
function filter_spans(spans) → function filter_spans(spans: Iterable[Span]) -> List[Span] - changed:
function from_bytes(bytes_data, setters, exclude) → function from_bytes(bytes_data: bytes, setters: Dict[str, Callable[[bytes], Any]], exclude: Iterable[str]) -> None - changed:
function from_disk(path, readers, exclude) → function from_disk(path: Union[str, Path], readers: Dict[str, Callable[[Path], None]], exclude: Iterable[str]) -> Path - changed:
function get_cuda_stream(require=…, non_blocking=…) → function get_cuda_stream(require: bool=…, non_blocking: bool=…) -> Optional[CudaStream] - changed:
function get_lang_class(lang) → function get_lang_class(lang: str) -> Type[Language] - changed:
function get_model_meta(path) → function get_model_meta(path: Union[str, Path]) -> Dict[str, Any] - changed:
function get_module_path(module) → function get_module_path(module: ModuleType) -> Path - changed:
function get_package_path(name) → function get_package_path(name: str) -> Path - changed:
function get_words_and_spaces(words, text) → function get_words_and_spaces(words: Iterable[str], text: str) -> Tuple[List[str], List[bool]] - changed:
function is_in_jupyter() → function is_in_jupyter() -> bool - changed:
function is_package(name) → function is_package(name: str) -> bool - changed:
function lang_class_is_loaded(lang) → function lang_class_is_loaded(lang: str) -> bool - changed:
function load_language_data(path) → function load_language_data(path: Union[str, Path]) -> Union[dict, list] - changed:
function load_model(name, **overrides) → function load_model(name: Union[str, Path], *, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, config: Union[Dict[str, Any], Config]=…) -> Language - changed:
function load_model_from_init_py(init_file, **overrides) → function load_model_from_init_py(init_file: Union[Path, str], *, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, config: Union[Dict[str, Any], Config]=…) -> Language - changed:
function load_model_from_package(name, **overrides) → function load_model_from_package(name: str, *, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, config: Union[Dict[str, Any], Config]=…) -> Language - changed:
function load_model_from_path(model_path, meta=…, **overrides) → function load_model_from_path(model_path: Path, *, meta: Optional[Dict[str, Any]]=…, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, config: Union[Dict[str, Any], Config]=…) -> Language - changed:
function minibatch(items, size=…) → function minibatch(items, size) - changed:
function minify_html(html) → function minify_html(html: str) -> str - changed:
function normalize_slice(length, start, stop, step=…) → function normalize_slice(length: int, start: int, stop: int, step: Optional[int]=…) -> Tuple[int, int] - changed:
function read_regex(path) → function read_regex(path: Union[str, Path]) -> Pattern - changed:
function set_lang_class(name, cls) → function set_lang_class(name: str, cls: Type[Language]) -> None - changed:
function to_bytes(getters, exclude) → function to_bytes(getters: Dict[str, Callable[[], bytes]], exclude: Iterable[str]) -> bytes - changed:
function to_disk(path, writers, exclude) → function to_disk(path: Union[str, Path], writers: Dict[str, Callable[[Path], None]], exclude: Iterable[str]) -> Path - changed:
function update_exc(base_exceptions, *addition_dicts) → function update_exc(base_exceptions: Dict[str, List[dict]], *addition_dicts) -> Dict[str, List[dict]] - changed:
method DummyTokenizer.from_bytes(_bytes_data, **kwargs) → method DummyTokenizer.from_bytes(data: bytes, **kwargs) -> DummyTokenizer - changed:
method DummyTokenizer.from_disk(_path, **kwargs) → method DummyTokenizer.from_disk(path: Union[str, Path], **kwargs) -> DummyTokenizer - changed:
method DummyTokenizer.to_disk(_path, **kwargs) → method DummyTokenizer.to_disk(path: Union[str, Path], **kwargs) -> None - added:
class ENV_VARS - added:
class SimpleFrozenList(list) - added:
constructor SimpleFrozenDict(*args, error: str=…, **kwargs) -> None - added:
constructor SimpleFrozenList(*args, error: str=…) -> None - added:
field ENV_VARS.CONFIG_OVERRIDES - added:
field registry.augmenters - added:
field registry.batchers - added:
field registry.callbacks - added:
field registry.cli - added:
field registry.lemmatizers - added:
field registry.loggers - added:
field registry.misc - added:
field registry.models - added:
field registry.readers - added:
field registry.scorers - added:
field registry.tokenizers - added:
field registry.vectors - added:
function all_equal(iterable) - added:
function check_bool_env_var(env_var: str) -> bool - added:
function check_lexeme_norms(vocab, component_name) - added:
function combine_score_weights(weights: List[Dict[str, Optional[float]]], overrides: Dict[str, Optional[float]]=…) -> Dict[str, Optional[float]] - added:
function copy_config(config: Union[Dict[str, Any], Config]) -> Config - added:
function create_default_optimizer() -> Optimizer - added:
function dict_to_dot(obj: Dict[str, dict], *, for_overrides: bool=…) -> Dict[str, Any] - added:
function dot_to_dict(values: Dict[str, Any]) -> Dict[str, dict] - added:
function dot_to_object(config: Config, section: str) - added:
function filter_chain_spans(*spans: Iterable[Span]) -> List[Span] - added:
function find_available_port(start: int, host: str, auto_select: bool=…) -> int - added:
function find_matching_language(lang: str) -> Optional[str] - added:
function from_dict(msg: Dict[str, Any], setters: Dict[str, Callable[[Any], Any]], exclude: Iterable[str]) -> Dict[str, Any] - added:
function get_arg_names(func: Callable) -> List[str] - added:
function get_base_version(version: str) -> str - added:
function get_installed_models() -> List[str] - added:
function get_minor_version(version: str) -> Optional[str] - added:
function get_minor_version_range(version: str) -> str - added:
function get_model_lower_version(constraint: str) -> Optional[str] - added:
function get_object_name(obj: Any) -> str - added:
function get_package_version(name: str) -> Optional[str] - added:
function get_sourced_components(config: Union[Dict[str, Any], Config]) -> Dict[str, Dict[str, Any]] - added:
function ignore_error(proc_name, proc, docs, e) - added:
function import_file(name: str, loc: Union[str, Path]) -> ModuleType - added:
function is_compatible_version(version: str, constraint: str, prereleases: bool=…) -> Optional[bool] - added:
function is_cython_func(func: Callable) -> bool - added:
function is_in_interactive() -> bool - added:
function is_minor_version_match(version_a: str, version_b: str) -> bool - added:
function is_prerelease_version(version: str) -> bool - added:
function is_same_func(func1: Callable, func2: Callable) -> bool - added:
function is_unconstrained_version(constraint: str, prereleases: bool=…) -> Optional[bool] - added:
function load_config(path: Union[str, Path], overrides: Dict[str, Any]=…, interpolate: bool=…) -> Config - added:
function load_config_from_str(text: str, overrides: Dict[str, Any]=…, interpolate: bool=…) - added:
function load_meta(path: Union[str, Path]) -> Dict[str, Any] - added:
function load_model_from_config(config: Union[Dict[str, Any], Config], *, meta: Dict[str, Any]=…, vocab: Union[Vocab, bool]=…, disable: Union[str, Iterable[str]]=…, enable: Union[str, Iterable[str]]=…, exclude: Union[str, Iterable[str]]=…, auto_fill: bool=…, validate: bool=…) -> Language - added:
function make_first_longest_spans_filter() - added:
function make_tempdir() -> Generator[Path, None, None] - added:
function packages_distributions() -> Dict[str, List[str]] - added:
function raise_error(proc_name, proc, docs, e) - added:
function replace_model_node(model: Model, target: Model, replacement: Model) -> None - added:
function resolve_dot_names(config: Config, dot_names: List[Optional[str]]) -> Tuple[Any, ...] - added:
function run_command(command: Union[str, List[str]], *, stdin: Optional[Any]=…, capture: bool=…) -> subprocess.CompletedProcess - added:
function set_dot_to_object(config: Config, section: str, value: Any) -> None - added:
function split_command(command: str) -> List[str] - added:
function split_requirement(requirement: str) -> Tuple[str, str] - added:
function to_dict(getters: Dict[str, Callable[[], Any]], exclude: Iterable[str]) -> Dict[str, Any] - added:
function to_ternary_int(val) -> int - added:
function walk_dict(node: Dict[str, Any], parent: List[str]=…, *, for_overrides: bool=…) -> Iterator[Tuple[List[str], Any]] - added:
function warn_if_jupyter_cupy() - added:
function working_dir(path: Union[str, Path]) -> Iterator[Path] - added:
method SimpleFrozenList.append(*args, **kwargs) - added:
method SimpleFrozenList.clear(*args, **kwargs) - added:
method SimpleFrozenList.extend(*args, **kwargs) - added:
method SimpleFrozenList.insert(*args, **kwargs) - added:
method SimpleFrozenList.pop(*args, **kwargs) - added:
method SimpleFrozenList.remove(*args, **kwargs) - added:
method SimpleFrozenList.reverse(*args, **kwargs) - added:
method SimpleFrozenList.sort(*args, **kwargs) - added:
static method registry.ensure_populated() -> None - added:
static method registry.find(registry_name: str, func_name: str) -> Dict[str, Optional[Union[str, int]]] - added:
static method registry.get(registry_name: str, func_name: str) -> Callable - added:
static method registry.get_registry_names() -> List[str] - added:
static method registry.has(registry_name: str, func_name: str) -> bool - added:
variable CONFIG_SECTION_ORDER - added:
variable DEFAULT_OOV_PROB - added:
variable LANG_ALIASES - added:
variable LEXEME_NORM_LANGS - added:
variable logger - added:
variable logger_stream_handler
- added:
class Vocab - added:
constructor Vocab(lex_attr_getters: Optional[Dict[str, Callable[[str], Any]]]=…, strings: Optional[Union[List[str], StringStore]]=…, lookups: Optional[Lookups]=…, oov_prob: float=…, vectors_name: Optional[str]=…, writing_system: Dict[str, Any]=…, get_noun_chunks: Optional[Callable[[Union[Doc, Span]], Iterator[Span]]]=…) -> None - added:
field Vocab.cfg: Dict[str, Any] - added:
field Vocab.get_noun_chunks: Optional[Callable[[Union[Doc, Span]], Iterator[Span]]] - added:
field Vocab.lookups: Lookups - added:
field Vocab.morphology: Morphology - added:
field Vocab.strings: StringStore - added:
field Vocab.vectors: Vectors - added:
field Vocab.writing_system: Dict[str, Any] - added:
function create_vocab(lang: Optional[str], defaults: Any, vectors_name: Optional[str]=…) -> Vocab - added:
function pickle_vocab(vocab: Vocab) -> Any - added:
function unpickle_vocab(sstore: StringStore, vectors: Any, morphology: Any, _unused_object: Any, lex_attr_getters: Any, lookups: Any, get_noun_chunks: Any) -> Vocab - added:
method Vocab.add_flag(flag_getter: Callable[[str], bool], flag_id: int=…) -> int - added:
method Vocab.deduplicate_vectors() -> None - added:
method Vocab.from_bytes(bytes_data: bytes, *, exclude: Iterable[str]=…) -> Vocab - added:
method Vocab.from_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> Vocab - added:
method Vocab.get_vector(orth: Union[int, str], minn: Optional[int]=…, maxn: Optional[int]=…) -> FloatsXd - added:
method Vocab.has_vector(orth: Union[int, str]) -> bool - added:
method Vocab.memory_zone(mem: Optional[Pool]=…) -> Iterator[Pool] - added:
method Vocab.prune_vectors(nr_row: int, batch_size: int=…) -> Dict[str, float] - added:
method Vocab.reset_vectors(*, width: Optional[int]=…, shape: Optional[int]=…) -> None - added:
method Vocab.set_vector(orth: Union[int, str], vector: Floats1d) -> None - added:
method Vocab.to_bytes(*, exclude: Iterable[str]=…) -> bytes - added:
method Vocab.to_disk(path: Union[str, Path], *, exclude: Iterable[str]=…) -> None - added:
property Vocab.lang -> str - added:
property Vocab.vectors_length -> int