How to replace a value in pandas, with NaN?

You can replace this just for that column using replace:

df['workclass'].replace('?', np.NaN)

or for the whole df:

df.replace('?', np.NaN)

UPDATE

OK I figured out your problem, by default if you don't pass a separator character then read_csv will use commas ',' as the separator.

Your data and in particular one example where you have a problematic line:

54, ?, 180211, Some-college, 10, Married-civ-spouse, ?, Husband, Asian-Pac-Islander, Male, 0, 0, 60, South, >50K

has in fact a comma and a space as the separator so when you passed the na_value=['?'] this didn't match because all your values have a space character in front of them all which you can't observe.

if you change your line to this:

rawfile = pd.read_csv(filename, header=None, names=DataLabels, sep=',\s', na_values=["?"])

then you should find that it all works:

27      54               NaN  180211  Some-college             10

Use numpy.nan

Numpy - Replace a number with NaN

import numpy as np
df.applymap(lambda x: np.nan if x == '?' else x)

df=df.replace({'?':np.NaN})

Using Dictionary to replace any value by NaN

okay I got it by :

 #========trying to replace ?
    newraw= rawfile.replace('[?]', np.nan, regex=True)
    print newraw[25:40]

There are many ways folks, this is best, if you figure that your CSV file has any object for NAN like "missing", just use

    rawfile = pd.read_csv("Property_train.csv", na_values=["missing"])

How to replace a value in pandas, with NaN?

Related

Recent Posts