Python 3.7 Error: Unsupported Pickle Protocol 5

I'm trying to restore a pickled config file from RLLib (json didn't work as shown in this post), and getting the following error:

config = pickle.load(open(f"{path}/params.pkl", "rb"))

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-28-c964561b863c> in <module>
----> 1 config = pickle.load(open(f"{path}/params.pkl", "rb"))

ValueError: unsupported pickle protocol: 5

Python Version = 3.7.0

How can I open this file in 3.7?

Solution 1:

For pandas users who saved a dataframe to a pickle file with protocol 5 in python 3.8 and need to load it into python 3.6 which only supports protocol 4 (I'm looking at you google colab):

!pip3 install pickle5
import pickle5 as pickle
with open(path_to_protocol5, "rb") as fh:
  data = pickle.load(fh)

Could also save into a protocol-4 pickle from python 3.6

data.to_pickle(path_to_protocol4)

Update: If facing this when loading a model from stable-baselines3:

!pip install --upgrade --quiet cloudpickle pickle5
from stable_baselines3 import PPO
# restart kernel if in jupyter notebook

# Might not need this dict in all cases
custom_objects = {
    "lr_schedule": lambda x: .003,
    "clip_range": lambda x: .02
}
model = PPO.load("path/to/model.zip", custom_objects=custom_objects)

Tested on 2021-05-31 with env:

cloudpickle: 1.6.0
pickle5: 0.0.11 
stable-baselines3: 1.0

Reference: https://brainsteam.co.uk/2021/01/14/pickle-5-madness-with-mlflow/

Solution 2:

Use pickle5 or load it into python 3.8+ and then serialize it to a lower version of it using the protocol parameter.