Skip to content

Upgrading pyarrow 9 to 10

Breaking9.0.0 → 10.0.1Breaking11Platform4Deprecated8New129Changed67Fixed149Other141

Full comparisonBack to pyarrow
BreakingUpgrade verdict

11 breaking changes and 4 platform requirement changes.

In plain terms
  • Removes the IR Consumer in C++.
  • Removes the as.factor to dictionary_encode mapping in R.
  • Go users lose the ValueDescr types.
  • Code relying on pre-C++17 compatibility shims must be updated.
Must act Top 5 of 23 that need code or config changes

What you must change

Breaking11
Removes the IR Consumer in C++.
10.0.0CHANGELOG.mdSummarizedView source
Removes the as.factor to dictionary_encode mapping in R.
as.factordictionary_encode
10.0.0CHANGELOG.mdSummarizedView source
Removes the ValueDescr types from the Go implementation.
ValueDescr
10.0.0CHANGELOG.mdSummarizedView source
Removes pre-C++17 compatibility measures in C++.
10.0.0CHANGELOG.mdSummarizedView source
Removes remaining deprecated Python APIs from versions 1.0.0 and earlier.
10.0.0CHANGELOG.mdSummarizedView source
Removes the C++ string_view backport.
string_view
10.0.0CHANGELOG.mdSummarizedView source
Removes the C++ std::optional backport.
std::optional
10.0.0CHANGELOG.mdSummarizedView source
Removes the C++ Variant class.
Variant
10.0.0CHANGELOG.mdSummarizedView source
Changes the behavior of pull in R to compute instead of collect.
pull
10.0.0CHANGELOG.mdSummarizedView source
Moves MakeArrayOfNull to the array package in Go.
MakeArrayOfNull
10.0.0CHANGELOG.mdSummarizedView source
Removes internal headers from the Substrait API.
10.0.0CHANGELOG.mdSummarizedView source
Platform4
Adds support for Python 3.11.
10.0.1CHANGELOG.mdSummarizedView source
Updates the minimum required vcpkg version for Python packaging.
vcpkg
10.0.1CHANGELOG.mdSummarizedView source
Adds Windows build support for Java in CI.
10.0.0CHANGELOG.mdSummarizedView source
Uses jacoco-maven-plugin 0.8.8 for Java 18 support.
jacoco-maven-plugin
10.0.0CHANGELOG.mdSummarizedView source

Deprecations to plan for

Deprecated8
Adds a deprecation cycle for the pull() change in R.
pull()
10.0.1CHANGELOG.mdSummarizedView source
Deprecates the redundant iterable of ScanTask in Java.
ScanTask
10.0.0CHANGELOG.mdSummarizedView source
Deprecates Plasma.
Plasma
10.0.0CHANGELOG.mdSummarizedView source
Deprecates Plasma in C++.
Plasma
10.0.0CHANGELOG.mdSummarizedView source
Deprecates the Plasma C GLib bindings.
Plasma
10.0.0CHANGELOG.mdSummarizedView source
Deprecates the Plasma Python bindings.
Plasma
10.0.0CHANGELOG.mdSummarizedView source
Deprecates the Plasma Ruby bindings.
Plasma
10.0.0CHANGELOG.mdSummarizedView source
Deprecates the Java Plasma JNI bindings.
Plasma
10.0.0CHANGELOG.mdSummarizedView source