Upgrading dask 1 to 2
Unknown1.2.2 → 2.30.0
UnknownUpgrade verdict
None of the 39 releases in this range publish notes, so breaking changes can't be ruled out.
In plain terms
- Some releases in this range publish no notes, so DepDelta can't rule out breaking changes.
- Their commits are listed with this comparison.
Must act Unknown for releases without notes
Releases without notes may still need changes. Check their commits before upgrading.
Python requirement changed in 2.7.0: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.* → >=3.6.
Adds pyyaml.
Installs 1 package in all (was 0).
API changes
Generated by DepDelta from published Python source. Not mixed with prose notes.
- removed:
export set_options
- added:
export divmod - added:
export median - added:
export moveaxis - added:
export nanmedian - added:
export register_chunk_type - added:
export rollaxis - added:
export shape - added:
export unify_chunks - added:
export union1d
- added:
function register_cupyx()
- changed:
function blockwise(func, out_ind, *args, **kwargs) → function blockwise(func, out_ind, *args, name=…, token=…, dtype=…, adjust_chunks=…, new_axes=…, align_arrays=…, concatenate=…, meta=…, **kwargs)
- changed:
function coarsen(reduction, x, axes, trim_excess=…) → function coarsen(reduction, x, axes, trim_excess=…, **kwargs)
- added:
function is_valid_array_chunk(array) - added:
function is_valid_chunk_type(type) - added:
function register_chunk_type(type)
- changed:
function from_array(x, chunks=…, name=…, lock=…, asarray=…, fancy=…, getitem=…) → function from_array(x, chunks=…, name=…, lock=…, asarray=…, fancy=…, getitem=…, meta=…) - changed:
function from_delayed(value, shape, dtype, name=…) → function from_delayed(value, shape, dtype=…, meta=…, name=…) - changed:
function map_blocks(func, *args, **kwargs) → function map_blocks(func, *args, name=…, token=…, dtype=…, chunks=…, drop_axis=…, new_axis=…, meta=…, **kwargs) - changed:
function stack(seq, axis=…) → function stack(seq, axis=…, allow_unknown_chunksizes=…) - changed:
method Array.rechunk(chunks, threshold=…, block_size_limit=…) → method Array.rechunk(chunks=…, threshold=…, block_size_limit=…, balance=…) - changed:
method Array.to_dask_dataframe(columns=…, index=…) → method Array.to_dask_dataframe(columns=…, index=…, meta=…) - changed:
method Array.view(dtype, order=…) → method Array.view(dtype=…, order=…) - added:
function check_if_handled_given_other(f) - added:
function implements(*numpy_functions) - added:
method Array.compute_chunk_sizes() - added:
method Array.to_svg(size=…) - added:
property Array.dtype - added:
property Array.partitions - added:
variable unknown_chunk_message
- changed:
function empty_like(a, dtype=…, chunks=…) → function empty_like(a, dtype=…, order=…, chunks=…, name=…, shape=…) - changed:
function eye(N, chunks, M=…, k=…, dtype=…) → function eye(N, chunks=…, M=…, k=…, dtype=…) - changed:
function full_like(a, fill_value, dtype=…, chunks=…) → function full_like(a, fill_value, order=…, dtype=…, chunks=…, name=…, shape=…) - changed:
function ones_like(a, dtype=…, chunks=…) → function ones_like(a, dtype=…, order=…, chunks=…, name=…, shape=…) - changed:
function pad(array, pad_width, mode, **kwargs) → function pad(array, pad_width, mode=…, **kwargs) - changed:
function pad_edge(array, pad_width, mode, *args) → function pad_edge(array, pad_width, mode, **kwargs) - changed:
function pad_reuse(array, pad_width, mode, *args) → function pad_reuse(array, pad_width, mode, **kwargs) - changed:
function pad_stats(array, pad_width, mode, *args) → function pad_stats(array, pad_width, mode, stat_length) - changed:
function zeros_like(a, dtype=…, chunks=…) → function zeros_like(a, dtype=…, order=…, chunks=…, name=…, shape=…)
- changed:
function compression_matrix(data, q, n_power_iter=…, seed=…) → function compression_matrix(data, q, n_power_iter=…, seed=…, compute=…) - changed:
function svd(a) → function svd(a, coerce_signs=…) - changed:
function svd_compressed(a, k, n_power_iter=…, seed=…) → function svd_compressed(a, k, n_power_iter=…, seed=…, compute=…, coerce_signs=…)
- added:
function moveaxis(a, source, destination) - added:
function rollaxis(a, axis, start=…)
- changed:
function map_overlap(x, func, depth, boundary=…, trim=…, **kwargs) → function map_overlap(func, *args, depth=…, boundary=…, trim=…, align_arrays=…, **kwargs)
- changed:
method RandomState.beta(a, b, size=…, chunks=…) → method RandomState.beta(a, b, size=…, chunks=…, **kwargs) - changed:
method RandomState.binomial(n, p, size=…, chunks=…) → method RandomState.binomial(n, p, size=…, chunks=…, **kwargs) - changed:
method RandomState.chisquare(df, size=…, chunks=…) → method RandomState.chisquare(df, size=…, chunks=…, **kwargs) - changed:
method RandomState.exponential(scale=…, size=…, chunks=…) → method RandomState.exponential(scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.f(dfnum, dfden, size=…, chunks=…) → method RandomState.f(dfnum, dfden, size=…, chunks=…, **kwargs) - changed:
method RandomState.gamma(shape, scale=…, size=…, chunks=…) → method RandomState.gamma(shape, scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.geometric(p, size=…, chunks=…) → method RandomState.geometric(p, size=…, chunks=…, **kwargs) - changed:
method RandomState.gumbel(loc=…, scale=…, size=…, chunks=…) → method RandomState.gumbel(loc=…, scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.hypergeometric(ngood, nbad, nsample, size=…, chunks=…) → method RandomState.hypergeometric(ngood, nbad, nsample, size=…, chunks=…, **kwargs) - changed:
method RandomState.laplace(loc=…, scale=…, size=…, chunks=…) → method RandomState.laplace(loc=…, scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.logistic(loc=…, scale=…, size=…, chunks=…) → method RandomState.logistic(loc=…, scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.lognormal(mean=…, sigma=…, size=…, chunks=…) → method RandomState.lognormal(mean=…, sigma=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.logseries(p, size=…, chunks=…) → method RandomState.logseries(p, size=…, chunks=…, **kwargs) - changed:
method RandomState.multinomial(n, pvals, size=…, chunks=…) → method RandomState.multinomial(n, pvals, size=…, chunks=…, **kwargs) - changed:
method RandomState.negative_binomial(n, p, size=…, chunks=…) → method RandomState.negative_binomial(n, p, size=…, chunks=…, **kwargs) - changed:
method RandomState.noncentral_chisquare(df, nonc, size=…, chunks=…) → method RandomState.noncentral_chisquare(df, nonc, size=…, chunks=…, **kwargs) - changed:
method RandomState.noncentral_f(dfnum, dfden, nonc, size=…, chunks=…) → method RandomState.noncentral_f(dfnum, dfden, nonc, size=…, chunks=…, **kwargs) - changed:
method RandomState.normal(loc=…, scale=…, size=…, chunks=…) → method RandomState.normal(loc=…, scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.pareto(a, size=…, chunks=…) → method RandomState.pareto(a, size=…, chunks=…, **kwargs) - changed:
method RandomState.poisson(lam=…, size=…, chunks=…) → method RandomState.poisson(lam=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.power(a, size=…, chunks=…) → method RandomState.power(a, size=…, chunks=…, **kwargs) - changed:
method RandomState.randint(low, high=…, size=…, chunks=…, dtype=…) → method RandomState.randint(low, high=…, size=…, chunks=…, dtype=…, **kwargs) - changed:
method RandomState.random_integers(low, high=…, size=…, chunks=…) → method RandomState.random_integers(low, high=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.random_sample(size=…, chunks=…) → method RandomState.random_sample(size=…, chunks=…, **kwargs) - changed:
method RandomState.rayleigh(scale=…, size=…, chunks=…) → method RandomState.rayleigh(scale=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.standard_cauchy(size=…, chunks=…) → method RandomState.standard_cauchy(size=…, chunks=…, **kwargs) - changed:
method RandomState.standard_exponential(size=…, chunks=…) → method RandomState.standard_exponential(size=…, chunks=…, **kwargs) - changed:
method RandomState.standard_gamma(shape, size=…, chunks=…) → method RandomState.standard_gamma(shape, size=…, chunks=…, **kwargs) - changed:
method RandomState.standard_normal(size=…, chunks=…) → method RandomState.standard_normal(size=…, chunks=…, **kwargs) - changed:
method RandomState.standard_t(df, size=…, chunks=…) → method RandomState.standard_t(df, size=…, chunks=…, **kwargs) - changed:
method RandomState.tomaxint(size=…, chunks=…) → method RandomState.tomaxint(size=…, chunks=…, **kwargs) - changed:
method RandomState.triangular(left, mode, right, size=…, chunks=…) → method RandomState.triangular(left, mode, right, size=…, chunks=…, **kwargs) - changed:
method RandomState.uniform(low=…, high=…, size=…, chunks=…) → method RandomState.uniform(low=…, high=…, size=…, chunks=…, **kwargs) - changed:
method RandomState.vonmises(mu, kappa, size=…, chunks=…) → method RandomState.vonmises(mu, kappa, size=…, chunks=…, **kwargs) - changed:
method RandomState.wald(mean, scale, size=…, chunks=…) → method RandomState.wald(mean, scale, size=…, chunks=…, **kwargs) - changed:
method RandomState.weibull(a, size=…, chunks=…) → method RandomState.weibull(a, size=…, chunks=…, **kwargs) - changed:
method RandomState.zipf(a, size=…, chunks=…) → method RandomState.zipf(a, size=…, chunks=…, **kwargs) - added:
method RandomState.permutation(x) - added:
variable permutation
- changed:
function rechunk(x, chunks, threshold=…, block_size_limit=…) → function rechunk(x, chunks=…, threshold=…, block_size_limit=…, balance=…)
- changed:
function mean_agg(pairs, dtype=…, axis=…, **kwargs) → function mean_agg(pairs, dtype=…, axis=…, computing_meta=…, **kwargs) - changed:
function mean_chunk(x, sum=…, numel=…, dtype=…, **kwargs) → function mean_chunk(x, sum=…, numel=…, dtype=…, computing_meta=…, **kwargs) - changed:
function mean_combine(pairs, sum=…, numel=…, dtype=…, axis=…, **kwargs) → function mean_combine(pairs, sum=…, numel=…, dtype=…, axis=…, computing_meta=…, **kwargs) - changed:
function moment_agg(pairs, order=…, ddof=…, dtype=…, sum=…, axis=…, **kwargs) → function moment_agg(pairs, order=…, ddof=…, dtype=…, sum=…, axis=…, computing_meta=…, **kwargs) - changed:
function moment_chunk(A, order=…, sum=…, numel=…, dtype=…, **kwargs) → function moment_chunk(A, order=…, sum=…, numel=…, dtype=…, computing_meta=…, **kwargs) - changed:
function moment_combine(pairs, order=…, ddof=…, dtype=…, sum=…, axis=…, **kwargs) → function moment_combine(pairs, order=…, ddof=…, dtype=…, sum=…, axis=…, computing_meta=…, **kwargs) - changed:
function partial_reduce(func, x, split_every, keepdims=…, dtype=…, name=…) → function partial_reduce(func, x, split_every, keepdims=…, dtype=…, name=…, reduced_meta=…) - changed:
function reduction(x, chunk, aggregate, axis=…, keepdims=…, dtype=…, split_every=…, combine=…, name=…, out=…, concatenate=…, output_size=…) → function reduction(x, chunk, aggregate, axis=…, keepdims=…, dtype=…, split_every=…, combine=…, name=…, out=…, concatenate=…, output_size=…, meta=…) - added:
function median(a, axis=…, keepdims=…, out=…) - added:
function nanmedian(a, axis=…, keepdims=…, out=…) - added:
function safe_sqrt(a)
- changed:
function apply_along_axis(func1d, axis, arr, *args, **kwargs) → function apply_along_axis(func1d, axis, arr, *args, dtype=…, shape=…, **kwargs) - changed:
function coarsen(reduction, x, axes, trim_excess=…) → function coarsen(reduction, x, axes, trim_excess=…, **kwargs) - changed:
function unravel_index(indices, dims, order=…) → function unravel_index(indices, shape, order=…) - added:
function aligned_coarsen_chunks(chunks: List[int], multiple: int) -> List[int] - added:
function iscomplexobj(x) - added:
function shape(array) - added:
function union1d(ar1, ar2)
- changed:
function slice_array(out_name, in_name, blockdims, index) → function slice_array(out_name, in_name, blockdims, index, itemsize) - changed:
function slice_with_newaxes(out_name, in_name, blockdims, index) → function slice_with_newaxes(out_name, in_name, blockdims, index, itemsize) - changed:
function slice_wrap_lists(out_name, in_name, blockdims, index) → function slice_wrap_lists(out_name, in_name, blockdims, index, itemsize) - changed:
function take(outname, inname, chunks, index, axis=…) → function take(outname, inname, chunks, index, itemsize, axis=…) - added:
function cached_cumsum(seq, initial_zero=…) - added:
function make_block_sorted_slices(index, chunks) - added:
function shuffle_slice(x, index)
- added:
function draw_sizes(shape, size=…) - added:
function grid_points(chunks, sizes) - added:
function ratio_response(x) - added:
function svg(chunks, size=…, **kwargs) - added:
function svg_1d(chunks, sizes=…, **kwargs) - added:
function svg_2d(chunks, offset=…, skew=…, size=…, sizes=…) - added:
function svg_3d(chunks, size=…, sizes=…, offset=…) - added:
function svg_grid(x, y, offset=…, skew=…, size=…) - added:
function svg_lines(x1, y1, x2, y2, max_n=…) - added:
function svg_nd(chunks, size=…) - added:
variable text_style
- removed:
function copy_docstring(target, source=…) - changed:
function wrap_elemwise(numpy_ufunc, array_wrap=…) → function wrap_elemwise(numpy_ufunc, array_wrap=…, source=…) - added:
function divmod(x, y)
- changed:
function assert_eq(a, b, check_shape=…, check_graph=…, **kwargs) → function assert_eq(a, b, check_shape=…, check_graph=…, check_meta=…, **kwargs) - added:
function compute_meta(func, _dtype, *args, **kwargs) - added:
function empty_like_safe(a, shape, **kwargs) - added:
function full_like_safe(a, fill_value, shape, **kwargs) - added:
function meta_from_array(x, ndim=…, dtype=…) - added:
function ones_like_safe(a, shape, **kwargs) - added:
function svd_flip(u, v, u_based_decision=…) - added:
function zeros_like_safe(a, shape, **kwargs)
- removed:
variable full - added:
function broadcast_trick(func) - added:
function full(shape, fill_value, *args, **kwargs) - added:
function full_like(a, fill_value, *args, **kwargs) - added:
function wrap_func_like(func, *args, **kwargs) - added:
function wrap_func_like_safe(func, func_like, *args, **kwargs) - added:
variable empty_like - added:
variable w_like
- added:
function barrier(*args) - added:
function foldby_combine2(combine, acc, x) - added:
function getitem(x, key) - added:
function groupby_tasks_group_hash(x, hash, grouper) - added:
function var_aggregate(x, ddof) - added:
function var_chunk(seq)
- changed:
function map_chunk(f, args, bag_kwargs, kwargs) → function map_chunk(f, iters, iter_kwarg_keys=…, kwargs=…) - changed:
method Bag.repartition(npartitions) → method Bag.repartition(npartitions=…, partition_size=…) - added:
function repartition_npartitions(bag, npartitions) - added:
function repartition_size(bag, size) - added:
function total_mem_usage(partition)
- added:
function choices(population, k=…) - added:
function sample(population, k)
- changed:
function file_to_blocks(lazy_file) → function file_to_blocks(include_path, lazy_file) - changed:
function read_text(urlpath, blocksize=…, compression=…, encoding=…, errors=…, linedelimiter=…, collection=…, storage_options=…, files_per_partition=…) → function read_text(urlpath, blocksize=…, compression=…, encoding=…, errors=…, linedelimiter=…, collection=…, storage_options=…, files_per_partition=…, include_path=…) - added:
function attach_path(block, path)
- changed:
function blockwise(func, output, output_indices, *arrind_pairs, **kwargs) → function blockwise(func, output, output_indices, *arrind_pairs, numblocks=…, concatenate=…, new_axes=…, dependencies=…, **kwargs) - added:
function fuse_roots(graph: HighLevelGraph, keys: list) - added:
function lol_product(head, values)
- removed:
export ignoring - removed:
export local - added:
export open_file - added:
variable fsspec
- removed:
function get_xz_blocks(fp) - removed:
function noop_file(file, **kwargs) - removed:
function xz_decompress(data, check) - removed:
variable compress - removed:
variable decompress - removed:
variable files - removed:
variable seekable_files
- removed:
class OpenFile(object) - removed:
constructor OpenFile(fs, path, mode=…, compression=…, encoding=…, errors=…) - removed:
function expand_paths_if_needed(paths, mode, num, fs, name_function) - removed:
function get_compression(urlpath, compression) - removed:
function get_fs(protocol, storage_options=…) - removed:
function get_fs_token_paths(urlpath, mode=…, num=…, name_function=…, storage_options=…) - removed:
function get_hdfs_driver(driver=…) - removed:
function get_mapper(fs, path) - removed:
function get_pyarrow_filesystem(fs) - removed:
function infer_options(urlpath) - removed:
function logical_size(fs, path, compression=…) - removed:
function open_files(urlpath, mode=…, compression=…, encoding=…, errors=…, name_function=…, num=…, **kwargs) - removed:
method OpenFile.close()
- removed:
function generic_glob(fs, path_impl, pathname)
- removed:
class HDFS3HadoopFileSystem(object) - removed:
constructor HDFS3HadoopFileSystem(**kwargs) - removed:
field HDFS3HadoopFileSystem.sep - removed:
method HDFS3HadoopFileSystem.glob(path) - removed:
method HDFS3HadoopFileSystem.mkdirs(path) - removed:
method HDFS3HadoopFileSystem.open(path, mode=…, **kwargs) - removed:
method HDFS3HadoopFileSystem.size(path) - removed:
method HDFS3HadoopFileSystem.ukey(path) - removed:
static method HDFS3HadoopFileSystem.from_hdfs3(fs)
- removed:
class HTTPFile(object) - removed:
class HTTPFileSystem(object) - removed:
constructor HTTPFile(url, session=…, block_size=…, **kwargs) - removed:
constructor HTTPFileSystem(simple_links=…, block_size=…, same_schema=…, **storage_options) - removed:
field HTTPFileSystem.sep - removed:
function file_size(url, session, **kwargs) - removed:
method HTTPFile.close() - removed:
method HTTPFile.flush() - removed:
method HTTPFile.read(length=…) - removed:
method HTTPFile.readable() - removed:
method HTTPFile.seek(where, whence=…) - removed:
method HTTPFile.seekable() - removed:
method HTTPFile.tell() - removed:
method HTTPFile.writable() - removed:
method HTTPFile.write() - removed:
method HTTPFileSystem.glob(path) - removed:
method HTTPFileSystem.isdir(path) - removed:
method HTTPFileSystem.ls(url, detail=…) - removed:
method HTTPFileSystem.mkdirs(url) - removed:
method HTTPFileSystem.open(url, mode=…, block_size=…, **kwargs) - removed:
method HTTPFileSystem.size(url) - removed:
method HTTPFileSystem.ukey(url) - removed:
variable DEFAULT_BLOCK_SIZE - removed:
variable ex - removed:
variable ex2
- removed:
class LocalFileSystem(object) - removed:
constructor LocalFileSystem(**storage_options) - removed:
field LocalFileSystem.sep - removed:
method LocalFileSystem.glob(path) - removed:
method LocalFileSystem.mkdirs(path) - removed:
method LocalFileSystem.open(path, mode=…, **kwargs) - removed:
method LocalFileSystem.size(path) - removed:
method LocalFileSystem.ukey(path)
- removed:
class HDFS3Wrapper(pa.filesystem.DaskFileSystem) - removed:
class PyArrowHadoopFileSystem(object) - removed:
constructor PyArrowHadoopFileSystem(**kwargs) - removed:
field PyArrowHadoopFileSystem.sep - removed:
function update_hdfs_options(options) - removed:
method HDFS3Wrapper.isdir(path) - removed:
method HDFS3Wrapper.isfile(path) - removed:
method PyArrowHadoopFileSystem.glob(path) - removed:
method PyArrowHadoopFileSystem.mkdirs(path) - removed:
method PyArrowHadoopFileSystem.open(path, mode=…, **kwargs) - removed:
method PyArrowHadoopFileSystem.size(path) - removed:
method PyArrowHadoopFileSystem.ukey(path) - removed:
static method PyArrowHadoopFileSystem.from_pyarrow(fs)
- removed:
class DaskS3FileSystem(S3FileSystem, object) - removed:
constructor DaskS3FileSystem(key=…, username=…, secret=…, password=…, path=…, host=…, s3=…, **kwargs) - removed:
field DaskS3FileSystem.sep - removed:
method DaskS3FileSystem.mkdirs(path) - removed:
method DaskS3FileSystem.size(path) - removed:
method DaskS3FileSystem.ukey(path)
- removed:
class SeekableFile(object) - removed:
constructor SeekableFile(file) - removed:
function build_name_function(max_int) - removed:
function infer_compression(filename) - removed:
function infer_storage_options(urlpath, inherit_storage_options=…) - removed:
function read_block(f, offset, length, delimiter=…) - removed:
function seek_delimiter(file, delimiter, blocksize) - removed:
function update_storage_options(options, inherited=…) - removed:
method SeekableFile.read1(*args, **kwargs) - removed:
method SeekableFile.readable() - removed:
method SeekableFile.seekable() - removed:
method SeekableFile.writable() - removed:
variable SeekableFile - removed:
variable compressions - added:
function zip_compress(data) - added:
variable compress
- removed:
class BZ2File(BufferedIOBase) - removed:
class GzipFile(BufferedIOBase) - removed:
class LZMAFile - removed:
constructor BZ2File(*args, **kwargs) - removed:
constructor GzipFile(*args, **kwargs) - removed:
constructor LZMAFile(*args, **kwargs) - removed:
function apply(func, args, kwargs=…) - removed:
function bind_method(cls, name, func) - removed:
function get_named_args(func) - removed:
function getargspec(func) - removed:
function gzip_compress(b) - removed:
function gzip_decompress(b) - removed:
function makedirs(name, mode=…, exist_ok=…) - removed:
function reraise(exc, tb=…) - removed:
method BZ2File.close() - removed:
method BZ2File.flush() - removed:
method BZ2File.isatty() - removed:
method BZ2File.read(*args, **kwargs) - removed:
method BZ2File.read1(*args, **kwargs) - removed:
method BZ2File.readable() - removed:
method BZ2File.readline(*args, **kwargs) - removed:
method BZ2File.readlines(*args, **kwargs) - removed:
method BZ2File.seek(*args, **kwargs) - removed:
method BZ2File.seekable() - removed:
method BZ2File.tell() - removed:
method BZ2File.truncate(*args, **kwargs) - removed:
method BZ2File.writable() - removed:
method BZ2File.write(*args, **kwargs) - removed:
method BZ2File.writelines(*args, **kwargs) - removed:
method GzipFile.close() - removed:
method GzipFile.flush(*args, **kwargs) - removed:
method GzipFile.isatty() - removed:
method GzipFile.read(*args, **kwargs) - removed:
method GzipFile.read1(*args, **kwargs) - removed:
method GzipFile.readable() - removed:
method GzipFile.readline(*args, **kwargs) - removed:
method GzipFile.readlines(*args, **kwargs) - removed:
method GzipFile.seek(*args, **kwargs) - removed:
method GzipFile.seekable() - removed:
method GzipFile.tell() - removed:
method GzipFile.truncate(*args, **kwargs) - removed:
method GzipFile.writable() - removed:
method GzipFile.write(*args, **kwargs) - removed:
method GzipFile.writelines(*args, **kwargs) - removed:
property BZ2File.closed - removed:
property GzipFile.closed - removed:
variable BZ2File - removed:
variable FileExistsError - removed:
variable FileNotFoundError - removed:
variable LZMA_AVAILABLE - removed:
variable PY2 - removed:
variable PY3 - removed:
variable apply - removed:
variable long - removed:
variable operator_div - removed:
variable range - removed:
variable reduce - removed:
variable reraise - removed:
variable string_types - removed:
variable zip - added:
variable PY_VERSION
- removed:
variable fn - removed:
variable yaml - added:
function check_deprecations(key: str, deprecations: dict=…) - added:
variable deprecations
- removed:
function set_options(*args, **kwargs)
- added:
function keys_in_tasks(keys, tasks, as_list=…) - added:
variable no_default
- added:
export merge_asof - added:
export to_numeric - added:
export to_sql
- added:
function categorical_dtype(meta, categories=…, ordered=…) - added:
function categorical_dtype_pandas(categories=…, ordered=…) - added:
variable categorical_dtype_dispatch
- changed:
function apply_concat_apply(args, chunk=…, aggregate=…, combine=…, meta=…, token=…, chunk_kwargs=…, aggregate_kwargs=…, combine_kwargs=…, split_every=…, split_out=…, split_out_setup=…, split_out_setup_kwargs=…, **kwargs) → function apply_concat_apply(args, chunk=…, aggregate=…, combine=…, meta=…, token=…, chunk_kwargs=…, aggregate_kwargs=…, combine_kwargs=…, split_every=…, split_out=…, split_out_setup=…, split_out_setup_kwargs=…, sort=…, ignore_index=…, **kwargs) - changed:
function cov_corr_combine(data, corr=…) → function cov_corr_combine(data_in, corr=…) - changed:
function hash_shard(df, nparts, split_out_setup=…, split_out_setup_kwargs=…) → function hash_shard(df, nparts, split_out_setup=…, split_out_setup_kwargs=…, ignore_index=…) - changed:
function map_partitions(func, *args, **kwargs) → function map_partitions(func, *args, meta=…, enforce_metadata=…, transform_divisions=…, **kwargs) - changed:
function pd_split(df, p, random_state=…) → function pd_split(df, p, random_state=…, shuffle=…) - changed:
function to_datetime(arg, **kwargs) → function to_datetime(arg, meta=…, **kwargs) - changed:
method DataFrame.drop(labels, axis=…, errors=…) → method DataFrame.drop(labels=…, axis=…, columns=…, errors=…) - changed:
method DataFrame.to_records(index=…) → method DataFrame.to_records(index=…, lengths=…) - changed:
method Series.value_counts(split_every=…, split_out=…) → method Series.value_counts(sort=…, ascending=…, dropna=…, split_every=…, split_out=…) - added:
function mapseries(base_chunk, concat_map) - added:
function mapseries_combine(index, concat_result) - added:
function prefix_reduction(f, ddf, identity, **kwargs) - added:
function repartition_size(df, size) - added:
function series_map(base_series, map_series) - added:
function suffix_reduction(f, ddf, identity, **kwargs) - added:
function total_mem_usage(df, index=…, deep=…) - added:
method DataFrame.explode(column) - added:
method DataFrame.items() - added:
method DataFrame.melt(id_vars=…, value_vars=…, var_name=…, value_name=…, col_level=…) - added:
method DataFrame.mode(dropna=…, split_every=…) - added:
method DataFrame.pop(item) - added:
method Index.map(arg, na_action=…, meta=…, is_monotonic=…) - added:
method Series.explode() - added:
method Series.mode(dropna=…, split_every=…) - added:
property DataFrame.empty
- added:
export register_dataframe_accessor - added:
export register_index_accessor - added:
export register_series_accessor - added:
variable make_scalar
- added:
method SeriesGroupBy.unique(split_every=…, split_out=…) - added:
method SeriesGroupBy.value_counts(split_every=…, split_out=…)
- added:
export to_sql
- changed:
function read_pandas(reader, urlpath, blocksize=…, collection=…, lineterminator=…, compression=…, sample=…, enforce=…, assume_missing=…, storage_options=…, include_path_column=…, **kwargs) → function read_pandas(reader, urlpath, blocksize=…, lineterminator=…, compression=…, sample=…, enforce=…, assume_missing=…, storage_options=…, include_path_column=…, **kwargs) - changed:
function text_blocks_to_pandas(reader, block_lists, header, head, kwargs, collection=…, enforce=…, specified_dtypes=…, path=…) → function text_blocks_to_pandas(reader, block_lists, header, head, kwargs, enforce=…, specified_dtypes=…, path=…) - changed:
function to_csv(df, filename, name_function=…, compression=…, compute=…, scheduler=…, storage_options=…, header_first_partition_only=…, **kwargs) → function to_csv(df, filename, single_file=…, encoding=…, mode=…, name_function=…, compression=…, compute=…, scheduler=…, storage_options=…, header_first_partition_only=…, compute_kwargs=…, **kwargs) - added:
class CSVSubgraph(Mapping) - added:
constructor CSVSubgraph(name, reader, blocks, is_first, head, header, kwargs, dtypes, columns, enforce, path) - added:
function block_mask(block_lists)
- changed:
function from_array(x, chunksize=…, columns=…) → function from_array(x, chunksize=…, columns=…, meta=…) - changed:
function from_dask_array(x, columns=…, index=…) → function from_dask_array(x, columns=…, index=…, meta=…) - changed:
function from_delayed(dfs, meta=…, divisions=…, prefix=…) → function from_delayed(dfs, meta=…, divisions=…, prefix=…, verify_meta=…)
- changed:
function read_json(url_path, orient=…, lines=…, storage_options=…, blocksize=…, sample=…, encoding=…, errors=…, compression=…, meta=…, **kwargs) → function read_json(url_path, orient=…, lines=…, storage_options=…, blocksize=…, sample=…, encoding=…, errors=…, compression=…, meta=…, engine=…, **kwargs) - changed:
function read_json_chunk(chunk, encoding, errors, kwargs, meta=…) → function read_json_chunk(chunk, encoding, errors, engine, kwargs, meta=…) - changed:
function read_json_file(f, orient, lines, kwargs) → function read_json_file(f, orient, lines, engine, kwargs) - changed:
function to_json(df, url_path, orient=…, lines=…, storage_options=…, compute=…, encoding=…, errors=…, compression=…, **kwargs) → function to_json(df, url_path, orient=…, lines=…, storage_options=…, compute=…, encoding=…, errors=…, compression=…, compute_kwargs=…, **kwargs)
- removed:
function read_parquet(path, columns=…, filters=…, categories=…, index=…, storage_options=…, engine=…, infer_divisions=…) - removed:
function to_parquet(df, path, engine=…, compression=…, write_index=…, append=…, ignore_divisions=…, partition_on=…, storage_options=…, compute=…, **kwargs) - added:
export read_parquet - added:
export read_parquet_part - added:
export to_parquet
- added:
class ArrowEngine(Engine) - added:
static method ArrowEngine.initialize_write(df, fs, path, append=…, partition_on=…, ignore_divisions=…, division_info=…, schema=…, index_cols=…, **kwargs) - added:
static method ArrowEngine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, split_row_groups=…, **kwargs) - added:
static method ArrowEngine.read_partition(fs, piece, columns, index, categories=…, partitions=…, filters=…, schema=…, **kwargs) - added:
static method ArrowEngine.write_metadata(parts, fmd, fs, path, append=…, **kwargs) - added:
static method ArrowEngine.write_partition(df, path, fs, filename, partition_on, return_metadata, fmd=…, compression=…, index_cols=…, schema=…, **kwargs) - added:
variable preserve_ind_supported - added:
variable schema_field_supported
- added:
function read_parquet(path, columns=…, filters=…, categories=…, index=…, storage_options=…, engine=…, gather_statistics=…, split_row_groups=…, chunksize=…, **kwargs) - added:
function to_parquet(df, path, engine=…, compression=…, write_index=…, append=…, ignore_divisions=…, partition_on=…, storage_options=…, write_metadata_file=…, compute=…, compute_kwargs=…, schema=…, **kwargs)
- added:
class FastParquetEngine(Engine) - added:
static method FastParquetEngine.initialize_write(df, fs, path, append=…, partition_on=…, ignore_divisions=…, division_info=…, schema=…, **kwargs) - added:
static method FastParquetEngine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, **kwargs) - added:
static method FastParquetEngine.read_partition(fs, piece, columns, index, categories=…, pf=…, **kwargs) - added:
static method FastParquetEngine.write_metadata(parts, fmd, fs, path, append=…, **kwargs) - added:
static method FastParquetEngine.write_partition(df, path, fs, filename, partition_on, return_metadata, fmd=…, compression=…, **kwargs) - added:
variable paths_to_cats
- added:
class Engine - added:
static method Engine.initialize_write(df, fs, path, append=…, partition_on=…, ignore_divisions=…, division_info=…, **kwargs) - added:
static method Engine.read_metadata(fs, paths, categories=…, index=…, gather_statistics=…, filters=…, **kwargs) - added:
static method Engine.read_partition(fs, piece, columns, index, **kwargs) - added:
static method Engine.write_metadata(parts, meta, fs, path, append=…, **kwargs) - added:
static method Engine.write_partition(df, path, fs, filename, partition_on, return_metadata, **kwargs)
- added:
function to_sql(df, name: str, uri: str, schema=…, if_exists: str=…, index: bool=…, index_label=…, chunksize=…, dtype=…, method=…, compute=…, parallel=…)
- removed:
function var_aggregate(x2, x, n, ddof) - removed:
variable concat_kwargs - changed:
function concat(dfs, axis=…, join=…, uniform=…, filter_warning=…) → function concat(dfs, axis=…, join=…, uniform=…, filter_warning=…, ignore_index=…, **kwargs) - changed:
function concat_pandas(dfs, axis=…, join=…, uniform=…, filter_warning=…) → function concat_pandas(dfs, axis=…, join=…, uniform=…, filter_warning=…, ignore_index=…, **kwargs) - changed:
function value_counts_aggregate(x) → function value_counts_aggregate(x, sort=…, ascending=…, **groupby_kwargs) - changed:
function value_counts_combine(x) → function value_counts_combine(x, sort=…, ascending=…, **groupby_kwargs) - added:
function describe_nonnumeric_aggregate(stats, name) - added:
function describe_numeric_aggregate(stats, name=…, is_timedelta_col=…) - added:
function tolist(obj) - added:
function tolist_pandas(obj) - added:
function var_mixed_concat(numeric_var, timedelta_var, columns) - added:
function wrap_var_reduction(array_var, index) - added:
variable group_split - added:
variable hash_df - added:
variable tolist_dispatch
- changed:
function concat(dfs, axis=…, join=…, interleave_partitions=…) → function concat(dfs, axis=…, join=…, interleave_partitions=…, ignore_unknown_divisions=…, **kwargs) - changed:
function concat_indexed_dataframes(dfs, axis=…, join=…) → function concat_indexed_dataframes(dfs, axis=…, join=…, **kwargs) - changed:
function concat_unindexed_dataframes(dfs) → function concat_unindexed_dataframes(dfs, **kwargs) - changed:
function stack_partitions(dfs, divisions, join=…) → function stack_partitions(dfs, divisions, join=…, **kwargs) - added:
function compute_heads(ddf, by=…) - added:
function compute_tails(ddf, by=…) - added:
function concat_and_unsort(frames, columns) - added:
function get_unsorted_columns(frames) - added:
function merge_asof(left, right, on=…, left_on=…, right_on=…, left_index=…, right_index=…, by=…, left_by=…, right_by=…, suffixes=…, tolerance=…, allow_exact_matches=…, direction=…) - added:
function merge_asof_indexed(left, right, **kwargs) - added:
function merge_asof_padded(left, right, prev=…, next=…, **kwargs) - added:
function most_recent_head(left, right) - added:
function most_recent_head_summary(left, right, by=…) - added:
function most_recent_tail(left, right) - added:
function most_recent_tail_summary(left, right, by=…) - added:
function pair_partitions(L, R) - added:
function warn_dtype_mismatch(left, right, left_on, right_on) - added:
variable allowed_left - added:
variable allowed_right
- added:
function to_numeric(arg, errors=…, meta=…)
- removed:
variable fastparquet - added:
function optimize_read_parquet_getitem(dsk, keys)
- changed:
function get_dummies(data, prefix=…, prefix_sep=…, dummy_na=…, columns=…, sparse=…, drop_first=…, dtype=…) → function get_dummies(data, prefix=…, prefix_sep=…, dummy_na=…, columns=…, sparse=…, drop_first=…, dtype=…, **kwargs)
- changed:
constructor Rolling(obj, window=…, min_periods=…, freq=…, center=…, win_type=…, axis=…) → constructor Rolling(obj, window=…, min_periods=…, center=…, win_type=…, axis=…) - changed:
method Rolling.apply(func, args=…, kwargs=…, **kwds) → method Rolling.apply(func, raw=…, engine=…, engine_kwargs=…, args=…, kwargs=…) - added:
method Rolling.cov()
- removed:
function compute_divisions(df, **kwargs) - changed:
function rearrange_by_column(df, col, npartitions=…, max_branch=…, shuffle=…, compute=…) → function rearrange_by_column(df, col, npartitions=…, max_branch=…, shuffle=…, compute=…, ignore_index=…) - changed:
function rearrange_by_column_tasks(df, column, max_branch=…, npartitions=…) → function rearrange_by_column_tasks(df, column, max_branch=…, npartitions=…, ignore_index=…) - changed:
function shuffle(df, index, shuffle=…, npartitions=…, max_branch=…, compute=…) → function shuffle(df, index, shuffle=…, npartitions=…, max_branch=…, ignore_index=…, compute=…) - changed:
function shuffle_group(df, col, stage, k, npartitions) → function shuffle_group(df, cols, stage, k, npartitions, ignore_index, nfinal) - changed:
function shuffle_group_2(df, col) → function shuffle_group_2(df, cols, ignore_index, nparts) - added:
function cleanup_partd_files(p, keys) - added:
function compute_and_set_divisions(df, **kwargs) - added:
function drop_overlap(df, index) - added:
function ensure_cleanup_on_exception(p) - added:
function fix_overlap(ddf, overlap) - added:
function get_overlap(df, index) - added:
variable logger
- added:
method Resampler.nunique() - added:
method Resampler.quantile() - added:
method Resampler.size()
- removed:
variable HAS_INT_NA - removed:
variable PANDAS_GT_0240 - removed:
variable PANDAS_VERSION - removed:
variable is_interval_dtype - changed:
function strip_unknown_categories(x) → function strip_unknown_categories(x, just_drop_unknown=…) - added:
function check_matching_columns(meta, actual) - added:
function drop_by_shallow_copy(df, columns, errors=…) - added:
function group_split_pandas(df, c, k, ignore_index=…) - added:
function hash_object_pandas(obj, index=…, encoding=…, hash_key=…, categorize=…) - added:
variable group_split_dispatch - added:
variable hash_object_dispatch
- changed:
function box_label(key) → function box_label(key, verbose=…) - changed:
function to_graphviz(dsk, data_attributes=…, function_attributes=…, rankdir=…, graph_attr=…, node_attr=…, edge_attr=…, **kwargs) → function to_graphviz(dsk, data_attributes=…, function_attributes=…, rankdir=…, graph_attr=…, node_attr=…, edge_attr=…, collapse_outputs=…, verbose=…, **kwargs)
- added:
function compute_layer_dependencies(layers) - added:
method HighLevelGraph.keys() - added:
method HighLevelGraph.validate() - added:
method HighLevelGraph.values() - added:
property HighLevelGraph.dependents
- added:
function reraise(exc, tb=…)
- removed:
function fuse_getitem(dsk, func, place) - removed:
function fuse_selections(dsk, head1, head2, merge) - removed:
function key_split(s) - removed:
variable hex_pattern - changed:
function default_fused_keys_renamer(keys) → function default_fused_keys_renamer(keys, max_fused_key_length=…) - added:
class Default(Enum) - added:
field Default.token
- removed:
class ReverseStrComparable(object) - removed:
constructor ReverseStrComparable(obj) - removed:
function ndependents(dependencies, dependents, total_dependencies) - added:
function graph_metrics(dependencies, dependents, total_dependencies)
- removed:
class ShareDict(Mapping) - removed:
constructor ShareDict(dicts=…, dependencies=…) - removed:
function merge(*dicts, **kwargs) - removed:
method ShareDict.items() - removed:
method ShareDict.update(arg) - removed:
method ShareDict.update_with_key(arg, key=…, dependencies=…)
- added:
function register_numba() - added:
function register_pyarrow() - added:
function register_rmm() - added:
function sizeof_array(o) - added:
function sizeof_bytes(o) - added:
function sizeof_memoryview(o) - added:
function sizeof_python_dict(d)
- added:
function cpu_count() - added:
variable CPU_COUNT
- changed:
function derived_from(original_klass, version=…, ua_args=…) → function derived_from(original_klass, version=…, ua_args=…, skipblocks=…) - changed:
function ignore_warning(doc, cls, name, extra=…) → function ignore_warning(doc, cls, name, extra=…, skipblocks=…) - added:
function apply(func, args, kwargs=…) - added:
function format_bytes(n) - added:
function format_time(n) - added:
function get_named_args(func) - added:
function getargspec(func) - added:
function is_dataframe_like(df) - added:
function is_index_like(s) - added:
function is_series_like(s) - added:
function iter_chunks(sizes, max_size) - added:
function key_split(s) - added:
function parse_timedelta(s, default=…) - added:
variable hex_pattern - added:
variable tds2 - added:
variable timedelta_sizes