Upgrading dask 2021 to 2022
Unknown2021.12.0 → 2022.12.1
UnknownUpgrade verdict
None of the 27 releases in this range publish notes, so breaking changes can't be ruled out.
In plain terms
- Some releases in this range publish no notes, so DepDelta can't rule out breaking changes.
- Their commits are listed with this comparison.
Must act Unknown for releases without notes
Releases without notes may still need changes. Check their commits before upgrading.
Python requirement changed in 2022.2.1: >=3.7 → >=3.8.
Adds click >=7.0.
Installs 8 packages in all (was 7).
API changes
Generated by DepDelta from published Python source. Not mixed with prose notes.
- added:
export abs - added:
export expand_dims - added:
export left_shift - added:
export ndim - added:
export positive - added:
export right_shift
- changed:
function percentile(a, q, interpolation=…) → function percentile(a, q, method=…) - added:
class ArrayBackendEntrypoint(DaskBackendEntrypoint) - added:
class NumpyBackendEntrypoint(ArrayBackendEntrypoint) - added:
property ArrayBackendEntrypoint.RandomState - added:
property NumpyBackendEntrypoint.RandomState - added:
static method ArrayBackendEntrypoint.arange(start, /, stop=…, step=…, *, dtype=…, meta=…, **kwargs) - added:
static method ArrayBackendEntrypoint.empty(shape, *, dtype=…, meta=…, **kwargs) - added:
static method ArrayBackendEntrypoint.full(shape, fill_value, *, dtype=…, meta=…, **kwargs) - added:
static method ArrayBackendEntrypoint.ones(shape, *, dtype=…, meta=…, **kwargs) - added:
static method ArrayBackendEntrypoint.zeros(shape, *, dtype=…, meta=…, **kwargs) - added:
variable array_creation_dispatch
- removed:
variable take_along_axis
- removed:
function getem(arr, chunks, getitem=…, shape=…, out_name=…, lock=…, asarray=…, dtype=…) - removed:
property Array.chunksize - removed:
property Array.itemsize - removed:
property Array.nbytes - removed:
property Array.ndim - removed:
property Array.shape - removed:
property Array.size - changed:
constructor BlockView(array: Array) -> BlockView → constructor BlockView(array: Array) - changed:
function asanyarray(a, dtype=…, order=…, *, like=…) → function asanyarray(a, dtype=…, order=…, *, like=…, inline_array=…) - changed:
function insert_to_ooc(keys: list, chunks: tuple[tuple[int, ...], ...], out, name: str, *, lock: Lock | bool=…, region: slice | None=…, return_stored: bool=…, load_stored: bool=…) -> dict → function insert_to_ooc(keys: list, chunks: tuple[tuple[int, ...], ...], out: ArrayLike, name: str, *, lock: Lock | bool=…, region: tuple[slice, ...] | slice | None=…, return_stored: bool=…, load_stored: bool=…) -> dict - changed:
function load_chunk(out, index, lock) → function load_chunk(out: A, index: slice, lock: Any) -> A - changed:
function load_store_chunk(x, out, index, lock, return_stored, load_stored) → function load_store_chunk(x: Any, out: Any, index: slice, lock: Any, return_stored: bool, load_stored: bool) - changed:
function map_blocks(func, *args, name=…, token=…, dtype=…, chunks=…, drop_axis=…, new_axis=…, meta=…, **kwargs) → function map_blocks(func, *args, name=…, token=…, dtype=…, chunks=…, drop_axis=…, new_axis=…, enforce_ndim=…, meta=…, **kwargs) - changed:
function retrieve_from_ooc(keys: Collection[Hashable], dsk_pre: Mapping, dsk_post: Mapping) -> dict → function retrieve_from_ooc(keys: Collection[Hashable], dsk_pre: Mapping, dsk_post: Mapping) -> dict[tuple, Any] - changed:
function store(sources: Array | Collection[Array], targets, lock: bool | Lock=…, regions: tuple[slice, ...] | Collection[tuple[slice, ...]] | None=…, compute: bool=…, return_stored: bool=…, **kwargs) → function store(sources: Array | Collection[Array], targets: ArrayLike | Delayed | Collection[ArrayLike | Delayed], lock: bool | Lock=…, regions: tuple[slice, ...] | Collection[tuple[slice, ...]] | None=…, compute: bool=…, return_stored: bool=…, **kwargs) - changed:
function store_chunk(x, out, index, lock, return_stored) → function store_chunk(x: ArrayLike, out: ArrayLike, index: slice, lock: Any, return_stored: bool) - changed:
function to_zarr(arr, url, component=…, storage_options=…, overwrite=…, compute=…, return_stored=…, **kwargs) → function to_zarr(arr, url, component=…, storage_options=…, overwrite=…, region=…, compute=…, return_stored=…, **kwargs) - changed:
method Array.argmax(axis=…, split_every=…, out=…) → method Array.argmax(axis=…, *, keepdims=…, split_every=…, out=…) - changed:
method Array.argmin(axis=…, split_every=…, out=…) → method Array.argmin(axis=…, *, keepdims=…, split_every=…, out=…) - changed:
method Array.reshape(*shape, merge_chunks=…) → method Array.reshape(*shape, merge_chunks=…, limit=…) - added:
function apply_and_enforce(*args, **kwargs) - added:
function graph_from_arraylike(arr, chunks, shape, name, getitem=…, lock=…, asarray=…, dtype=…, inline_array=…) -> HighLevelGraph - added:
property Array.chunksize -> tuple[T_IntOrNaN, ...] - added:
property Array.itemsize -> int - added:
property Array.nbytes -> T_IntOrNaN - added:
property Array.ndim -> int - added:
property Array.shape -> tuple[T_IntOrNaN, ...] - added:
property Array.size -> T_IntOrNaN - added:
variable A - added:
variable DEFAULT_GET - added:
variable T_IntOrNaN
- changed:
function diag(v) → function diag(v, k=…)
- added:
class CupyBackendEntrypoint(ArrayBackendEntrypoint) - added:
constructor CupyBackendEntrypoint() - added:
property CupyBackendEntrypoint.RandomState - added:
static method CupyBackendEntrypoint.arange(*args, like=…, **kwargs) - added:
static method CupyBackendEntrypoint.empty(*args, **kwargs) - added:
static method CupyBackendEntrypoint.full(*args, **kwargs) - added:
static method CupyBackendEntrypoint.ones(*args, **kwargs) - added:
static method CupyBackendEntrypoint.zeros(*args, **kwargs)
- added:
variable nannumel_lookup - added:
variable numel_lookup
- changed:
function solve(a, b, sym_pos=…) → function solve(a, b, sym_pos=…, assume_a=…)
- changed:
function average(a, axis=…, weights=…, returned=…) → function average(a, axis=…, weights=…, returned=…, keepdims=…) - added:
function count(a, axis=…, keepdims=…, split_every=…) - added:
function empty_like(a, **kwargs) - added:
function ones_like(a, **kwargs) - added:
function zeros_like(a, **kwargs)
- added:
function percentile(a, q, method=…) - added:
variable ArrayLike - added:
variable DTypeLike
- changed:
function map_overlap(func, *args, depth=…, boundary=…, trim=…, align_arrays=…, **kwargs) → function map_overlap(func, *args, depth=…, boundary=…, trim=…, align_arrays=…, allow_rechunk=…, **kwargs) - changed:
function overlap(x, depth, boundary) → function overlap(x, depth, boundary, *, allow_rechunk=…)
- changed:
function merge_percentiles(finalq, qs, vals, interpolation=…, Ns=…, raise_on_nan=…) → function merge_percentiles(finalq, qs, vals, method=…, Ns=…, raise_on_nan=…) - changed:
function percentile(a, q, interpolation=…, method=…) → function percentile(a, q, method=…, internal_method=…, **kwargs)
- removed:
function doc_wraps(func)
- removed:
function make_arg_reduction(func, argfunc, is_nan_func=…) - removed:
variable argmax - removed:
variable argmin - removed:
variable nanargmax - removed:
variable nanargmin - changed:
function arg_agg(func, argfunc, data, axis=…, **kwargs) → function arg_agg(argfunc, data, axis=…, keepdims=…, **kwargs) - changed:
function arg_combine(func, argfunc, data, axis=…, **kwargs) → function arg_combine(argfunc, data, axis=…, **kwargs) - changed:
function arg_reduction(x, chunk, combine, agg, axis=…, split_every=…, out=…) → function arg_reduction(x, chunk, combine, agg, axis=…, keepdims=…, split_every=…, out=…) - changed:
function nanarg_agg(func, argfunc, data, axis=…, **kwargs) → function nanarg_agg(argfunc, data, axis=…, keepdims=…, **kwargs) - changed:
function reduction(x, chunk, aggregate, axis=…, keepdims=…, dtype=…, split_every=…, combine=…, name=…, out=…, concatenate=…, output_size=…, meta=…) → function reduction(x, chunk, aggregate, axis=…, keepdims=…, dtype=…, split_every=…, combine=…, name=…, out=…, concatenate=…, output_size=…, meta=…, weights=…) - added:
function argmax(a, axis=…, keepdims=…, split_every=…, out=…) - added:
function argmin(a, axis=…, keepdims=…, split_every=…, out=…) - added:
function chunk_max(x, axis=…, keepdims=…) - added:
function chunk_min(x, axis=…, keepdims=…) - added:
function nanargmax(a, axis=…, keepdims=…, split_every=…, out=…) - added:
function nanargmin(a, axis=…, keepdims=…, split_every=…, out=…)
- changed:
function reshape(x, shape, merge_chunks=…) → function reshape(x, shape, merge_chunks=…, limit=…)
- changed:
function aligned_coarsen_chunks(chunks: List[int], multiple: int) -> Tuple[int] → function aligned_coarsen_chunks(chunks: list[int], multiple: int) -> tuple[int, ...] - changed:
function average(a, axis=…, weights=…, returned=…) → function average(a, axis=…, weights=…, returned=…, keepdims=…) - added:
function expand_dims(a, axis) - added:
function ndim(a)
- removed:
function cached_cumsum(seq, initial_zero=…)
- changed:
function wrap_elemwise(numpy_ufunc, array_wrap=…, source=…) → function wrap_elemwise(numpy_ufunc, source=…) - added:
variable abs - added:
variable left_shift - added:
variable positive - added:
variable right_shift
- changed:
function assert_eq(a, b, check_shape=…, check_graph=…, check_meta=…, check_chunks=…, check_type=…, **kwargs) → function assert_eq(a, b, check_shape=…, check_graph=…, check_meta=…, check_chunks=…, check_ndim=…, check_type=…, check_dtype=…, equal_nan=…, scheduler=…, **kwargs) - changed:
function assert_eq_shape(a, b, check_nan=…) → function assert_eq_shape(a, b, check_ndim=…, check_nan=…)
- added:
class CreationDispatch(Generic[BackendEntrypointType]) - added:
class DaskBackendEntrypoint - added:
constructor CreationDispatch(module_name: str, default: str, entrypoint_class: type[BackendEntrypointType], name: str | None=…) - added:
function detect_entrypoints(entry_point_name) - added:
method CreationDispatch.dispatch(backend: str) - added:
method CreationDispatch.register_backend(name: str, backend: BackendEntrypointType) -> BackendEntrypointType - added:
method CreationDispatch.register_inplace(backend: str, name: str | None=…) -> Callable[[Callable[BackendFuncParams, BackendFuncReturn]], Callable[BackendFuncParams, BackendFuncReturn]] - added:
property CreationDispatch.backend -> str - added:
variable BackendEntrypointType - added:
variable BackendFuncParams - added:
variable BackendFuncReturn
- changed:
constructor Bag(dsk, name, npartitions) → constructor Bag(dsk: Mapping, name: str, npartitions: int) - changed:
constructor Item(dsk, key) → constructor Item(dsk, key, layer=…) - changed:
method Bag.to_dataframe(meta=…, columns=…) → method Bag.to_dataframe(meta=…, columns=…, optimize_graph=…) - added:
variable DEFAULT_GET
- changed:
function choices(population, k=…) → function choices(population, k=…, split_every=…) - changed:
function sample(population, k) → function sample(population, k, split_every=…)
- changed:
function assert_eq(a, b) → function assert_eq(a, b, scheduler=…)
- changed:
function get_collection_names(collection) -> Set[str] → function get_collection_names(collection) -> set[str] - changed:
function is_dask_collection(x) → function is_dask_collection(x) -> bool - changed:
function visualize(*args, filename=…, traverse=…, optimize_graph=…, maxval=…, **kwargs) → function visualize(*args, filename=…, traverse=…, optimize_graph=…, maxval=…, engine: Literal['cytoscape', 'ipycytoscape', 'graphviz'] | None=…, **kwargs)
- changed:
constructor Blockwise(output: str, output_indices: Iterable[str], dsk: Mapping[str, tuple], indices: Iterable[Tuple[Union[str, BlockwiseDep], Optional[Iterable[str]]]], numblocks: Mapping[str, Sequence[int]], concatenate: bool=…, new_axes: Mapping[str, int]=…, output_blocks: Set[Tuple[int, ...]]=…, annotations: Mapping[str, Any]=…, io_deps: Optional[Mapping[str, BlockwiseDep]]=…) → constructor Blockwise(output: str, output_indices: Iterable[str], dsk: Mapping[str, tuple], indices: Iterable[tuple[str | BlockwiseDep, Iterable[str] | None]], numblocks: Mapping[str, Sequence[int]], concatenate: bool | None=…, new_axes: Mapping[str, int] | None=…, output_blocks: set[tuple[int, ...]] | None=…, annotations: Mapping[str, Any] | None=…, io_deps: Mapping[str, BlockwiseDep] | None=…) - changed:
constructor BlockwiseDepDict(mapping: dict, numblocks: Optional[Tuple[int, ...]]=…, produces_tasks: bool=…) → constructor BlockwiseDepDict(mapping: dict, numblocks: tuple[int, ...] | None=…, produces_tasks: bool=…, produces_keys: bool=…) - changed:
field Blockwise.concatenate: Optional[bool] → field Blockwise.concatenate: bool | None - changed:
field Blockwise.indices: Tuple[Tuple[str, Optional[Tuple[str, ...]]], ...] → field Blockwise.indices: tuple[tuple[str, tuple[str, ...] | None], ...] - changed:
field Blockwise.output_blocks: Optional[Set[Tuple[int, ...]]] → field Blockwise.output_blocks: set[tuple[int, ...]] | None - changed:
field Blockwise.output_indices: Tuple[str, ...] → field Blockwise.output_indices: tuple[str, ...] - changed:
field BlockwiseDep.numblocks: Tuple[int, ...] → field BlockwiseDep.numblocks: tuple[int, ...] - changed:
method Blockwise.clone(keys: set, seed: Hashable, bind_to: Hashable=…) -> Tuple[Layer, bool] → method Blockwise.clone(keys: set, seed: Hashable, bind_to: Hashable=…) -> tuple[Layer, bool] - changed:
method Blockwise.cull(keys: set, all_hlg_keys: Iterable) -> Tuple[Layer, Mapping[Hashable, set]] → method Blockwise.cull(keys: set, all_hlg_keys: Iterable) -> tuple[Layer, Mapping[Hashable, set]] - changed:
method BlockwiseDep.get(idx: Tuple[int, ...], default) -> Any → method BlockwiseDep.get(idx: tuple[int, ...], default) -> Any - added:
class BlockIndex(BlockwiseDep) - added:
constructor BlockIndex(numblocks: tuple[int, ...]) - added:
field BlockIndex.produces_tasks: bool - added:
field Blockwise.io_deps: Mapping[str, BlockwiseDep] - added:
property BlockwiseDep.produces_keys -> bool - added:
property BlockwiseDepDict.produces_keys -> bool
- removed:
function open_file(*arg, **kwargs) - removed:
function open_files(*arg, **kwargs)
- changed:
field Callback.active → field Callback.active: ClassVar[set[tuple[Callable | None, ...]]] - changed:
method Callback.register() → method Callback.register() -> None - changed:
method Callback.unregister() → method Callback.unregister() -> None
- added:
function cli() - added:
function docs() - added:
function info() - added:
function run_cli() - added:
function versions() - added:
variable CONTEXT_SETTINGS
- removed:
function prod(iterable, *, start=…) - added:
function entry_points(group=…)
- changed:
constructor set(arg=…, config=…, lock=…, **kwargs) → constructor set(arg: Mapping | None=…, config: dict=…, lock: threading.Lock=…, **kwargs) - changed:
function canonical_name(k, config) → function canonical_name(k: str, config: dict) -> str - changed:
function check_deprecations(key: str, deprecations: dict=…) → function check_deprecations(key: str, deprecations: dict=…) -> str - changed:
function collect(paths=…, env=…) → function collect(paths: list[str]=…, env: Mapping[str, str] | None=…) -> dict - changed:
function collect_env(env=…) → function collect_env(env: Mapping[str, str] | None=…) -> dict - changed:
function collect_yaml(paths=…) → function collect_yaml(paths: Sequence[str]=…) -> list[dict] - changed:
function deserialize(data) → function deserialize(data: str) -> Any - changed:
function ensure_file(source, destination=…, comment=…) → function ensure_file(source: str, destination: str | None=…, comment: bool=…) -> None - changed:
function expand_environment_variables(config) → function expand_environment_variables(config: Any) -> Any - changed:
function get(key, default=…, config=…, override_with=…) → function get(key: str, default: Any=…, config: dict=…, override_with: Any | None=…) -> Any - changed:
function merge(*dicts) → function merge(*dicts: Mapping) -> dict - changed:
function refresh(config=…, defaults=…, **kwargs) → function refresh(config: dict=…, defaults: list[Mapping]=…, **kwargs) -> None - changed:
function rename(aliases, config=…) → function rename(aliases: Mapping, config: dict=…) -> None - changed:
function serialize(data) → function serialize(data: Any) -> str - changed:
function update(old, new, priority=…) → function update(old: dict, new: Mapping, priority: Literal['old', 'new']=…) -> dict - changed:
function update_defaults(new, config=…, defaults=…) → function update_defaults(new: Mapping, config: dict=…, defaults: list[Mapping]=…) -> None - changed:
variable config → variable config: dict - changed:
variable defaults → variable defaults: list[Mapping] - added:
field set.config: dict
- removed:
export from_bcolz - added:
export dask - added:
export from_dict - added:
export from_map - added:
export read_sql - added:
export read_sql_query
- added:
class CachedAccessor - added:
constructor CachedAccessor(name, accessor) - added:
function register_dataframe_accessor(name) - added:
function register_index_accessor(name) - added:
function register_series_accessor(name) - added:
method StringAccessor.rsplit(pat=…, n=…, expand=…)
- removed:
function make_meta_index(x, index=…) - removed:
function make_meta_pandas(x, index=…) - changed:
variable meta_object_types → variable meta_object_types: tuple[type, ...] - added:
class DataFrameBackendEntrypoint(DaskBackendEntrypoint) - added:
class PandasBackendEntrypoint(DataFrameBackendEntrypoint) - added:
function get_grouper_pandas(obj) - added:
function get_pyarrow_schema_pandas(obj) - added:
static method DataFrameBackendEntrypoint.from_dict(data: dict, *, npartitions: int, **kwargs) - added:
static method DataFrameBackendEntrypoint.read_csv(urlpath: str | list, **kwargs) - added:
static method DataFrameBackendEntrypoint.read_hdf(pattern: str | list, key: str, **kwargs) - added:
static method DataFrameBackendEntrypoint.read_json(url_path: str | list, **kwargs) - added:
static method DataFrameBackendEntrypoint.read_orc(path: str | list, **kwargs) - added:
static method DataFrameBackendEntrypoint.read_parquet(path: str | list, **kwargs) - added:
variable dataframe_creation_dispatch
- removed:
method Series.cat() - removed:
method Series.dt() - removed:
method Series.str() - changed:
function cov_corr_agg(data, cols, min_periods=…, corr=…, scalar=…) → function cov_corr_agg(data, cols, min_periods=…, corr=…, scalar=…, like_df=…) - changed:
function partitionwise_graph(func, name, *args, **kwargs) → function partitionwise_graph(func, layer_name, *args, **kwargs) - changed:
method DataFrame.set_index(other, drop=…, sorted=…, npartitions=…, divisions=…, inplace=…, **kwargs) → method DataFrame.set_index(other: str | Series, drop: bool=…, sorted: bool=…, npartitions: int | Literal['auto'] | None=…, divisions: Sequence | None=…, inplace: bool=…, **kwargs) - changed:
method DataFrame.sort_values(by, npartitions=…, ascending=…, na_position=…, sort_function=…, sort_function_kwargs=…, **kwargs) → method DataFrame.sort_values(by: str, npartitions: int | Literal['auto'] | None=…, ascending: bool=…, na_position: Literal['first'] | Literal['last']=…, sort_function: Callable[[pd.DataFrame], pd.DataFrame] | None=…, sort_function_kwargs: Mapping[str, Any] | None=…, **kwargs) -> DataFrame - changed:
method Series.nunique(split_every=…) → method Series.nunique(split_every=…, dropna=…) - added:
field Series.cat - added:
field Series.dt - added:
field Series.str - added:
method DataFrame.nunique(split_every=…, dropna=…, axis=…) - added:
method Index.memory_usage(deep=…) - added:
method Series.median(method=…) - added:
method Series.median_approximate(method=…) - added:
method Series.view(dtype) - added:
static method DataFrame.from_dict(data, *, npartitions, orient=…, dtype=…, columns=…) - added:
variable DEFAULT_GET - added:
variable GROUP_KEYS_DEFAULT
- added:
variable grouper_dispatch - added:
variable meta_lib_from_array - added:
variable pyarrow_schema_dispatch
- changed:
method DataFrameGroupBy.agg(arg, split_every=…, split_out=…) → method DataFrameGroupBy.agg(arg=…, split_every=…, split_out=…, shuffle=…, **kwargs) - changed:
method DataFrameGroupBy.aggregate(arg, split_every=…, split_out=…) → method DataFrameGroupBy.aggregate(arg=…, split_every=…, split_out=…, shuffle=…, **kwargs) - changed:
method SeriesGroupBy.agg(arg, split_every=…, split_out=…) → method SeriesGroupBy.agg(arg=…, split_every=…, split_out=…, shuffle=…, **kwargs) - changed:
method SeriesGroupBy.aggregate(arg, split_every=…, split_out=…) → method SeriesGroupBy.aggregate(arg=…, split_every=…, split_out=…, shuffle=…, **kwargs) - changed:
method SeriesGroupBy.head(n=…, split_every=…, split_out=…) → method SeriesGroupBy.head(n=…, split_every=…, split_out=…, shuffle=…) - changed:
method SeriesGroupBy.tail(n=…, split_every=…, split_out=…) → method SeriesGroupBy.tail(n=…, split_every=…, split_out=…, shuffle=…) - changed:
method SeriesGroupBy.unique(split_every=…, split_out=…) → method SeriesGroupBy.unique(split_every=…, split_out=…, shuffle=…) - changed:
method SeriesGroupBy.value_counts(split_every=…, split_out=…) → method SeriesGroupBy.value_counts(split_every=…, split_out=…, shuffle=…) - added:
variable SORT_SPLIT_OUT_WARNING
- removed:
export dataframe_from_ctable - removed:
export from_bcolz - added:
export from_dict - added:
export from_map - added:
export read_sql - added:
export read_sql_query
- changed:
class CSVFunctionWrapper → class CSVFunctionWrapper(DataFrameIOFunction) - added:
property CSVFunctionWrapper.columns
- changed:
class HDFFunctionWrapper → class HDFFunctionWrapper(DataFrameIOFunction) - added:
property HDFFunctionWrapper.columns - added:
variable MP_GET
- removed:
function dataframe_from_ctable(x, slc, columns=…, categories=…, lock=…) - removed:
function from_bcolz(x, chunksize=…, categorize=…, index=…, lock=…, **kwargs) - changed:
function from_delayed(dfs, meta=…, divisions=…, prefix=…, verify_meta=…) → function from_delayed(dfs: Delayed | distributed.Future | Iterable[Delayed | distributed.Future], meta=…, divisions: tuple | Literal['sorted'] | None=…, prefix: str=…, verify_meta: bool=…) -> DataFrame | Series - changed:
function from_pandas(data, npartitions=…, chunksize=…, sort=…, name=…) → function from_pandas(data: pd.DataFrame, npartitions: int | None=…, chunksize: int | None=…, sort: bool=…, name: str | None=…) -> DataFrame | (data: pd.Series, npartitions: int | None=…, chunksize: int | None=…, sort: bool=…, name: str | None=…) -> Series - added:
function from_dict(data, npartitions, orient=…, dtype=…, columns=…, constructor=…) - added:
function from_map(func, *iterables, args=…, meta=…, divisions=…, label=…, token=…, enforce_metadata=…, **kwargs)
- changed:
function read_json(url_path, orient=…, lines=…, storage_options=…, blocksize=…, sample=…, encoding=…, errors=…, compression=…, meta=…, engine=…, **kwargs) → function read_json(url_path, orient=…, lines=…, storage_options=…, blocksize=…, sample=…, encoding=…, errors=…, compression=…, meta=…, engine=…, include_path_column=…, path_converter=…, **kwargs) - changed:
function read_json_chunk(chunk, encoding, errors, engine, kwargs, meta=…) → function read_json_chunk(chunk, encoding, errors, engine, column_name, path, path_dtype, kwargs, meta=…) - changed:
function read_json_file(f, orient, lines, engine, kwargs) → function read_json_file(f, orient, lines, engine, column_name, path, path_dtype, kwargs) - added:
function add_path_column(df, column_name, path, dtype)
- changed:
class ORCFunctionWrapper → class ORCFunctionWrapper(DataFrameIOFunction) - added:
property ORCFunctionWrapper.columns
- removed:
class ArrowLegacyEngine(ArrowDatasetEngine) - removed:
static method ArrowLegacyEngine.multi_support() - removed:
variable ArrowEngine - changed:
static method ArrowDatasetEngine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, split_row_groups=…, chunksize=…, aggregate_files=…, ignore_metadata_file=…, metadata_task_size=…, **kwargs) → static method ArrowDatasetEngine.read_metadata(fs, paths, categories=…, index=…, use_nullable_dtypes=…, gather_statistics=…, filters=…, split_row_groups=…, chunksize=…, aggregate_files=…, ignore_metadata_file=…, metadata_task_size=…, parquet_file_extension=…, **kwargs) - changed:
static method ArrowDatasetEngine.read_partition(fs, pieces, columns, index, categories=…, partitions=…, filters=…, schema=…, **kwargs) → static method ArrowDatasetEngine.read_partition(fs, pieces, columns, index, use_nullable_dtypes=…, categories=…, partitions=…, filters=…, schema=…, **kwargs) - changed:
static method ArrowDatasetEngine.write_metadata(parts, fmd, fs, path, append=…, **kwargs) → static method ArrowDatasetEngine.write_metadata(parts, meta, fs, path, append=…, **kwargs) - added:
static method ArrowDatasetEngine.extract_filesystem(urlpath, filesystem, dataset_options, open_file_options, storage_options) - added:
variable PYARROW_NULLABLE_DTYPE_MAPPING - added:
variable partitioning_supported - added:
variable pre_buffer_supported
- changed:
function read_parquet(path, columns=…, filters=…, categories=…, index=…, storage_options=…, engine=…, gather_statistics=…, ignore_metadata_file=…, metadata_task_size=…, split_row_groups=…, chunksize=…, aggregate_files=…, **kwargs) → function read_parquet(path, columns=…, filters=…, categories=…, index=…, storage_options=…, engine=…, use_nullable_dtypes: bool=…, calculate_divisions=…, ignore_metadata_file=…, metadata_task_size=…, split_row_groups=…, chunksize=…, aggregate_files=…, parquet_file_extension=…, filesystem=…, **kwargs)
- changed:
static method FastParquetEngine.initialize_write(df, fs, path, append=…, partition_on=…, ignore_divisions=…, division_info=…, schema=…, object_encoding=…, index_cols=…, **kwargs) → static method FastParquetEngine.initialize_write(df, fs, path, append=…, partition_on=…, ignore_divisions=…, division_info=…, schema=…, object_encoding=…, index_cols=…, custom_metadata=…, **kwargs) - changed:
static method FastParquetEngine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, split_row_groups=…, chunksize=…, aggregate_files=…, ignore_metadata_file=…, metadata_task_size=…, **kwargs) → static method FastParquetEngine.read_metadata(fs, paths, categories=…, index=…, use_nullable_dtypes=…, gather_statistics=…, filters=…, split_row_groups=…, chunksize=…, aggregate_files=…, ignore_metadata_file=…, metadata_task_size=…, parquet_file_extension=…, **kwargs) - changed:
static method FastParquetEngine.read_partition(fs, pieces, columns, index, categories=…, root_cats=…, root_file_scheme=…, base_path=…, **kwargs) → static method FastParquetEngine.read_partition(fs, pieces, columns, index, use_nullable_dtypes=…, categories=…, root_cats=…, root_file_scheme=…, base_path=…, **kwargs) - changed:
static method FastParquetEngine.write_metadata(parts, fmd, fs, path, append=…, **kwargs) → static method FastParquetEngine.write_metadata(parts, meta, fs, path, append=…, **kwargs) - added:
static method FastParquetEngine.pf_to_pandas(pf, fs=…, columns=…, categories=…, index=…, open_file_options=…, **kwargs)
- changed:
static method Engine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, **kwargs) → static method Engine.read_metadata(fs, paths, categories=…, index=…, use_nullable_dtypes=…, gather_statistics=…, filters=…, **kwargs) - changed:
static method Engine.read_partition(fs, piece, columns, index, **kwargs) → static method Engine.read_partition(fs, piece, columns, index, use_nullable_dtypes=…, **kwargs) - added:
static method Engine.extract_filesystem(urlpath, filesystem, dataset_options, open_file_options, storage_options)
- changed:
function read_sql_table(table, uri, index_col, divisions=…, npartitions=…, limits=…, columns=…, bytes_per_chunk=…, head_rows=…, schema=…, meta=…, engine_kwargs=…, **kwargs) → function read_sql_table(table_name, con, index_col, divisions=…, npartitions=…, limits=…, columns=…, bytes_per_chunk=…, head_rows=…, schema=…, meta=…, engine_kwargs=…, **kwargs) - changed:
function to_sql(df, name: str, uri: str, schema=…, if_exists: str=…, index: bool=…, index_label=…, chunksize=…, dtype=…, method=…, compute=…, parallel=…) → function to_sql(df, name: str, uri: str, schema=…, if_exists: str=…, index: bool=…, index_label=…, chunksize=…, dtype=…, method=…, compute=…, parallel=…, engine_kwargs=…) - added:
function read_sql(sql, con, index_col, **kwargs) - added:
function read_sql_query(sql, con, index_col, divisions=…, npartitions=…, limits=…, bytes_per_chunk=…, head_rows=…, meta=…, engine_kwargs=…, **kwargs)
- added:
class DataFrameIOFunction(Protocol) - added:
method DataFrameIOFunction.project_columns(columns) - added:
property DataFrameIOFunction.columns - added:
variable fsspec_parquet
- removed:
function monotonic_decreasing_aggregate(concatenated) - removed:
function monotonic_decreasing_chunk(x) - removed:
function monotonic_increasing_aggregate(concatenated) - removed:
function monotonic_increasing_chunk(x) - added:
function pivot_agg_first(df) - added:
function pivot_agg_last(df) - added:
function pivot_first(df, index, columns, values) - added:
function pivot_last(df, index, columns, values) - added:
variable monotonic_decreasing_aggregate - added:
variable monotonic_decreasing_chunk - added:
variable monotonic_decreasing_combine - added:
variable monotonic_increasing_aggregate - added:
variable monotonic_increasing_chunk - added:
variable monotonic_increasing_combine
- changed:
function merge_chunk(lhs, *args, empty_index_dtype=…, categorical_columns=…, **kwargs) → function merge_chunk(lhs, *args, result_meta, **kwargs)
- changed:
function map_overlap(func, df, before, after, *args, **kwargs) → function map_overlap(func, df, before, after, *args, meta=…, enforce_metadata=…, transform_divisions=…, align_dataframes=…, **kwargs) - changed:
function overlap_chunk(func, prev_part, current_part, next_part, before, after, args, kwargs) → function overlap_chunk(func, before, after, *args, **kwargs) - added:
variable CombinedOutput
- changed:
function compute_and_set_divisions(df, **kwargs) → function compute_and_set_divisions(df: DataFrame, **kwargs) -> DataFrame - changed:
function fix_overlap(ddf, overlap) → function fix_overlap(ddf, mins, maxes, lens) - changed:
function remove_nans(divisions) → function remove_nans(divisions: Sequence) -> List - changed:
function set_index(df, index, npartitions=…, shuffle=…, compute=…, drop=…, upsample=…, divisions=…, partition_size=…, **kwargs) → function set_index(df: DataFrame, index: Union[str, Series], npartitions: Optional[Union[int, Literal['auto']]]=…, shuffle: Optional[str]=…, compute: bool=…, drop: bool=…, upsample: float=…, divisions: Optional[Sequence]=…, partition_size: float=…, **kwargs) -> DataFrame - changed:
function set_partition(df, index, divisions, max_branch=…, drop=…, shuffle=…, compute=…) → function set_partition(df: DataFrame, index: Union[str, Series], divisions: Sequence, max_branch: int=…, drop: bool=…, shuffle: Optional[str]=…, compute: Optional[bool]=…) -> DataFrame - changed:
function set_sorted_index(df, index, drop=…, divisions=…, **kwargs) → function set_sorted_index(df: DataFrame, index: Union[str, Series], drop: bool=…, divisions: Optional[Sequence]=…, **kwargs) -> DataFrame - changed:
function sort_values(df, by, npartitions=…, ascending=…, na_position=…, upsample=…, partition_size=…, sort_function=…, sort_function_kwargs=…, **kwargs) → function sort_values(df: DataFrame, by: Union[str, List[str]], npartitions: Optional[Union[int, Literal['auto']]]=…, ascending: Union[bool, List[bool]]=…, na_position: Union[Literal['first'], Literal['last']]=…, upsample: float=…, partition_size: float=…, sort_function: Optional[Callable[[pd.DataFrame], pd.DataFrame]]=…, sort_function_kwargs: Optional[Mapping[str, Any]]=…, **kwargs) -> DataFrame - added:
function compute_divisions(df: DataFrame, col: Optional[Any]=…, **kwargs) -> Tuple
- removed:
function getnanos(rule)
- removed:
function is_dataframe_like(df) - removed:
function is_index_like(s) - removed:
function is_series_like(s) - changed:
function assert_divisions(ddf) → function assert_divisions(ddf, scheduler=…) - changed:
function assert_eq(a, b, check_names=…, check_dtype=…, check_divisions=…, check_index=…, **kwargs) → function assert_eq(a, b, check_names=…, check_dtype=…, check_divisions=…, check_index=…, sort_results=…, scheduler=…, **kwargs) - changed:
function insert_meta_param_description(*args, **kwargs) → function insert_meta_param_description(func: T) -> T | (pad: int) -> Callable[[T], T] - changed:
variable meta_object_types → variable meta_object_types: tuple[type, ...] - added:
class AttributeNotImplementedError(NotImplementedError, AttributeError) - added:
function meta_frame_constructor(like) - added:
function meta_series_constructor(like) - added:
variable T
- changed:
constructor Delayed(key, dsk, length=…) → constructor Delayed(key, dsk, length=…, layer=…)
- added:
function BOKEH_VERSION()
- removed:
variable graphviz - added:
function cytoscape_graph(dsk, filename: str | None=…, format: str | None=…, *, rankdir: str=…, node_sep: float=…, edge_sep: float=…, spacing_factor: float=…, node_style: dict[str, str] | None=…, edge_style: dict[str, str] | None=…, **kwargs)
- changed:
function bind(children: T, parents, *, omit=…, seed: Hashable=…, assume_layers: bool=…, split_every: SplitEvery=…) -> T → function bind(children: T, parents, *, omit=…, seed: Hashable | None=…, assume_layers: bool=…, split_every: float | Literal[False] | None=…) -> T - changed:
function checkpoint(*collections, split_every: SplitEvery=…) -> Delayed → function checkpoint(*collections, split_every: float | Literal[False] | None=…) -> Delayed - changed:
function wait_on(*collections, split_every: SplitEvery=…) → function wait_on(*collections, split_every: float | Literal[False] | None=…)
- removed:
method HighLevelGraph.keyset() -> AbstractSet - removed:
property HighLevelGraph.dicts - changed:
class Layer(collections.abc.Mapping) → class Layer(Mapping) - changed:
constructor HighLevelGraph(layers: Mapping[str, Mapping], dependencies: Mapping[str, AbstractSet], key_dependencies: Optional[Dict[Hashable, AbstractSet]]=…) → constructor HighLevelGraph(layers: Mapping[str, Mapping], dependencies: Mapping[str, Set], key_dependencies: dict[Hashable, Set] | None=…) - changed:
constructor Layer(annotations: Mapping[str, Any]=…, collection_annotations: Mapping[str, Any]=…) → constructor Layer(annotations: Mapping[str, Any] | None=…, collection_annotations: Mapping[str, Any] | None=…) - changed:
field HighLevelGraph.dependencies: Mapping[str, AbstractSet] → field HighLevelGraph.dependencies: Mapping[str, Set] - changed:
field HighLevelGraph.key_dependencies: Dict[Hashable, AbstractSet] → field HighLevelGraph.key_dependencies: dict[Hashable, Set] - changed:
field Layer.annotations: Optional[Mapping[str, Any]] → field Layer.annotations: Mapping[str, Any] | None - changed:
field Layer.collection_annotations: Optional[Mapping[str, Any]] → field Layer.collection_annotations: Mapping[str, Any] | None - changed:
method HighLevelGraph.get_all_dependencies() -> Dict[Hashable, AbstractSet] → method HighLevelGraph.get_all_dependencies() -> dict[Hashable, Set] - changed:
method HighLevelGraph.keys() -> AbstractSet → method HighLevelGraph.keys() -> KeysView - changed:
method Layer.cull(keys: set, all_hlg_keys: Iterable) -> Tuple[Layer, Mapping[Hashable, set]] → method Layer.cull(keys: set, all_hlg_keys: Iterable) -> tuple[Layer, Mapping[Hashable, set]] - changed:
method Layer.get_output_keys() -> AbstractSet → method Layer.get_output_keys() -> Set
- removed:
class BlockwiseCreateArray(Blockwise) - removed:
class CreateArrayDeps(BlockwiseDep) - removed:
class DataFrameLayer(Layer) - removed:
constructor BlockwiseCreateArray(name, func, shape, chunks) - removed:
constructor CreateArrayDeps(chunks: tuple) - removed:
method DataFrameLayer.project_columns(output_columns) - changed:
class BroadcastJoinLayer(DataFrameLayer) → class BroadcastJoinLayer(Layer) - changed:
class DataFrameIOLayer(Blockwise, DataFrameLayer) → class DataFrameIOLayer(Blockwise) - changed:
class SimpleShuffleLayer(DataFrameLayer) → class SimpleShuffleLayer(Layer) - changed:
constructor DataFrameIOLayer(name, columns, inputs, io_func, label=…, produces_tasks=…, annotations=…) → constructor DataFrameIOLayer(name, columns, inputs, io_func, label=…, produces_tasks=…, creation_info=…, annotations=…) - added:
class ArrayBlockwiseDep(BlockwiseDep) - added:
class ArrayChunkShapeDep(ArrayBlockwiseDep) - added:
class ArraySliceDep(ArrayBlockwiseDep) - added:
class DataFrameTreeReduction(Layer) - added:
constructor ArrayBlockwiseDep(chunks: tuple[tuple[int, ...], ...]) - added:
constructor ArraySliceDep(chunks: tuple[tuple[int, ...], ...]) - added:
constructor DataFrameTreeReduction(name: str, name_input: str, npartitions_input: int, concat_func: Callable, tree_node_func: Callable, finalize_func: Callable | None=…, split_every: int=…, split_out: int | None=…, output_partitions: list[int] | None=…, tree_node_name: str | None=…, annotations: dict[str, Any] | None=…) - added:
field ArrayBlockwiseDep.chunks: tuple[tuple[int, ...], ...] - added:
field ArrayBlockwiseDep.numblocks: tuple[int, ...] - added:
field ArrayBlockwiseDep.produces_tasks: bool - added:
field ArraySliceDep.starts: tuple[tuple[int, ...], ...] - added:
field DataFrameTreeReduction.concat_func: Callable - added:
field DataFrameTreeReduction.finalize_func: Callable | None - added:
field DataFrameTreeReduction.height: int - added:
field DataFrameTreeReduction.name: str - added:
field DataFrameTreeReduction.name_input: str - added:
field DataFrameTreeReduction.npartitions_input: int - added:
field DataFrameTreeReduction.output_partitions: list[int] - added:
field DataFrameTreeReduction.split_every: int - added:
field DataFrameTreeReduction.split_out: int - added:
field DataFrameTreeReduction.tree_node_func: Callable - added:
field DataFrameTreeReduction.tree_node_name: str - added:
field DataFrameTreeReduction.widths: list[int] - added:
method DataFrameTreeReduction.cull(keys, all_keys) - added:
method DataFrameTreeReduction.get_output_keys() - added:
method DataFrameTreeReduction.is_materialized() - added:
property DataFrameIOLayer.columns
- changed:
function get_sync(dsk, keys, **kwargs) → function get_sync(dsk: Mapping, keys: Sequence[Hashable] | Hashable, **kwargs)
- changed:
function get(dsk, keys, num_workers=…, func_loads=…, func_dumps=…, optimize_graph=…, pool=…, chunksize=…, **kwargs) → function get(dsk: Mapping, keys: Sequence[Hashable] | Hashable, num_workers=…, func_loads=…, func_dumps=…, optimize_graph=…, pool=…, initializer=…, chunksize=…, **kwargs) - changed:
function initialize_worker_process() → function initialize_worker_process(user_initializer=…) - changed:
function remote_exception(exc, tb) → function remote_exception(exc: Exception, tb) -> Exception - changed:
function reraise(exc, tb) → function reraise(exc, tb=…) - changed:
variable exceptions → variable exceptions: dict[type[Exception], type[Exception]] - added:
function default_initializer()
- removed:
function getsizeof(x) - removed:
variable getsizeof - added:
variable logger
- changed:
function get(dsk, result, cache=…, num_workers=…, pool=…, **kwargs) → function get(dsk: Mapping, keys: Sequence[Hashable] | Hashable, cache=…, num_workers=…, pool=…, **kwargs) - changed:
variable default_pool → variable default_pool: Executor | None - changed:
variable pools → variable pools: defaultdict[threading.Thread, dict[int, Executor]]
- added:
class DaskCollection(Protocol) - added:
class HLGDaskCollection(DaskCollection, Protocol) - added:
class PostPersistCallable(Protocol[CollType_co]) - added:
class SchedulerGetCallable(Protocol) - added:
method DaskCollection.compute(**kwargs: Any) -> Any - added:
method DaskCollection.persist(**kwargs: Any) -> CollType - added:
method DaskCollection.visualize(filename: str=…, format: str | None=…, optimize_graph: bool=…, **kwargs: Any) -> DisplayObject | None - added:
variable CollType - added:
variable CollType_co - added:
variable PostComputeCallable
- removed:
field methodcaller.func - removed:
function factors(n) - removed:
function ignoring(*exceptions) - removed:
function noop_context() - changed:
class cached_property(_cached_property) → class cached_property(functools.cached_property) - changed:
constructor SerializableLock(token=…) → constructor SerializableLock(token: Hashable | None=…) - changed:
function derived_from(original_klass, version=…, ua_args=…, skipblocks=…) → function derived_from(original_klass, version=…, ua_args=…, skipblocks=…, inconsistencies=…) - changed:
function ensure_bytes(s) → function ensure_bytes(s) -> bytes - changed:
function ensure_not_exists(filename) → function ensure_not_exists(filename) -> None - changed:
function ensure_unicode(s) → function ensure_unicode(s) -> str - changed:
function format_time(n) → function format_time(n: float) -> str - changed:
function funcname(func) → function funcname(func) -> str - changed:
function get_named_args(func) → function get_named_args(func) -> list[str] - changed:
function ignore_warning(doc, cls, name, extra=…, skipblocks=…) → function ignore_warning(doc, cls, name, extra=…, skipblocks=…, inconsistencies=…) - changed:
function is_arraylike(x) → function is_arraylike(x) -> bool - changed:
function is_cupy_type(x) → function is_cupy_type(x) -> bool - changed:
function is_dataframe_like(df) → function is_dataframe_like(df) -> bool - changed:
function is_index_like(s) → function is_index_like(s) -> bool - changed:
function is_integer(i) → function is_integer(i) -> bool - changed:
function is_series_like(s) → function is_series_like(s) -> bool - changed:
function natural_sort_key(s) → function natural_sort_key(s: str) -> list[str | int] - changed:
function parse_bytes(s) → function parse_bytes(s: float | str) -> int - changed:
function parse_timedelta(s, default=…) → function parse_timedelta(s: None, default: str | Literal[False]=…) -> None | (s: str | float | timedelta, default: str | Literal[False]=…) -> float - changed:
function pseudorandom(n, p, random_state=…) → function pseudorandom(n: int, p, random_state=…) - changed:
function random_state_data(n, random_state=…) → function random_state_data(n: int, random_state=…) -> list - changed:
function typename(typ, short=…) → function typename(typ: Any, short: bool=…) -> str - added:
field SerializableLock.lock: Lock - added:
field SerializableLock.token: Hashable - added:
field methodcaller.method: str - added:
function cached_cumsum(seq, initial_zero=…) - added:
function ensure_set(s: Set[T], *, copy: bool=…) -> set[T] - added:
function is_namedtuple_instance(obj: Any) -> bool - added:
function maybe_pluralize(count, noun, plural_form=…) - added:
function show_versions() -> None - added:
property methodcaller.func -> str - added:
variable T
- added:
function slowadd(a, b, delay=…)
- changed:
function get_environment(*args, **kwargs) → function get_environment() - changed:
function get_template(*args, **kwargs) → function get_template(name: str)