Upgrading pyarrow 15 to 16
Breaking15.0.2 → 16.1.0Breaking4Platform3New76Changed64Fixed159Other117
BreakingUpgrade verdict
4 breaking changes and 3 platform requirement changes.
In plain terms
- Removes the const qualifier from Buffer::mutable_span_as.
- The old pyarrow.filesystem module is gone; use the newer filesystem APIs instead.
- Removes arrow-glib/gobject-type.h from GLib bindings.
- Renames Function::is_impure() to is_pure() in C++.
Must act Top 5 of 7 that need code or config changes
What you must change
Breaking4
Removes the const qualifier from Buffer::mutable_span_as.
Buffer::mutable_span_asRemoves deprecated pyarrow.filesystem legacy implementations.
pyarrow.filesystemRemoves arrow-glib/gobject-type.h from GLib bindings.
arrow-glib/gobject-type.hRenames Function::is_impure() to is_pure() in C++.
Function::is_impure()Function::is_pure()Platform3
Requires GLib 2.58 or later for timezone support in Ruby CI.
Adds ORC support to Windows wheels.
Requires Go 1.21 or later.
API changes
Generated by DepDelta from published Python source. Not mixed with prose notes.
- removed:
export HdfsFile - removed:
export hdfs - added:
export BinaryViewArray - added:
export BinaryViewScalar - added:
export LargeListViewArray - added:
export LargeListViewScalar - added:
export LargeListViewType - added:
export ListViewArray - added:
export ListViewScalar - added:
export ListViewType - added:
export StringViewArray - added:
export StringViewScalar - added:
export binary_view - added:
export large_list_view - added:
export list_view - added:
export string_view
- removed:
class DaskFileSystem(FileSystem) - removed:
class FileSystem - removed:
class LocalFileSystem(FileSystem) - removed:
class S3FSWrapper(DaskFileSystem) - removed:
constructor DaskFileSystem(fs) - removed:
constructor LocalFileSystem() - removed:
function resolve_filesystem_and_path(where, filesystem=…) - removed:
method DaskFileSystem.delete(path, recursive=…) - removed:
method DaskFileSystem.exists(path) - removed:
method DaskFileSystem.isdir(path) - removed:
method DaskFileSystem.isfile(path) - removed:
method DaskFileSystem.ls(path, detail=…) - removed:
method DaskFileSystem.mkdir(path, create_parents=…) - removed:
method DaskFileSystem.open(path, mode=…) - removed:
method DaskFileSystem.walk(path) - removed:
method FileSystem.cat(path) - removed:
method FileSystem.delete(path, recursive=…) - removed:
method FileSystem.disk_usage(path) - removed:
method FileSystem.exists(path) - removed:
method FileSystem.isdir(path) - removed:
method FileSystem.isfile(path) - removed:
method FileSystem.ls(path) - removed:
method FileSystem.mkdir(path, create_parents=…) - removed:
method FileSystem.mv(path, new_path) - removed:
method FileSystem.open(path, mode=…) - removed:
method FileSystem.read_parquet(path, columns=…, metadata=…, schema=…, use_threads=…, use_pandas_metadata=…) - removed:
method FileSystem.rename(path, new_path) - removed:
method FileSystem.rm(path, recursive=…) - removed:
method FileSystem.stat(path) - removed:
method LocalFileSystem.exists(path) - removed:
method LocalFileSystem.isdir(path) - removed:
method LocalFileSystem.isfile(path) - removed:
method LocalFileSystem.ls(path) - removed:
method LocalFileSystem.mkdir(path, create_parents=…) - removed:
method LocalFileSystem.open(path, mode=…) - removed:
method LocalFileSystem.walk(path) - removed:
method S3FSWrapper.isdir(path) - removed:
method S3FSWrapper.isfile(path) - removed:
method S3FSWrapper.walk(path, refresh=…) - removed:
property FileSystem.pathsep - removed:
property LocalFileSystem.pathsep - removed:
static method LocalFileSystem.get_instance()
- removed:
class HadoopFileSystem(_hdfsio.HadoopFileSystem, FileSystem) - removed:
constructor HadoopFileSystem(host=…, port=…, user=…, kerb_ticket=…, driver=…, extra_conf=…) - removed:
function connect(host=…, port=…, user=…, kerb_ticket=…, extra_conf=…) - removed:
method HadoopFileSystem.delete(path, recursive=…) - removed:
method HadoopFileSystem.exists(path) - removed:
method HadoopFileSystem.isdir(path) - removed:
method HadoopFileSystem.isfile(path) - removed:
method HadoopFileSystem.ls(path, detail=…) - removed:
method HadoopFileSystem.mkdir(path, **kwargs) - removed:
method HadoopFileSystem.rename(path, new_path) - removed:
method HadoopFileSystem.walk(top_path)
- added:
function is_binary_view(t) - added:
function is_large_list_view(t) - added:
function is_list_view(t) - added:
function is_string_view(t)
- added:
function download_tzdata_on_windows()