Trying to save a DataFrame using Arrow.jl gives: ArgumentError: type does not have a definite number of fields. Tuples of tuples of ints

Solution 1:

Probably you need to update your packages, because your problem is not reproducible under the current versions of these packages.

PS It is very difficult to find any good reason on earth to save such a structure in a data frame. Transform your data in such a way that each column has an optimal structure for data manipulation (like, Int, Float64,...)

Solution 2:

The problem is fixed by explicitly typing the array before constructing the DataFrame. Here is a fixed working example:

using Arrow, DataFrames

x = ((1,), (1,), (), ());
y = ((1, 2), (), (), ());
T = Union{
    Tuple{Tuple{Int64}, Tuple{Int64}, Tuple{}, Tuple{}},
    Tuple{Tuple{Int64, Int64}, Tuple{}, Tuple{}, Tuple{}}
};
C = T[x, y];
df = DataFrame(col = C);
Arrow.write("test.arrow", df)