Getting Warning: " 'newdata' had 1 row but variables found have 32 rows" on predict.lm
I found peculiarity while using predict and lm function in R. I got different results for data frame and vector for same data.
DataFrame code:
data(mtcars)
fitCar<-lm(mtcars$mpg~mtcars$wt)
predict(fitCar,
data.frame(x=mean(mtcars$wt)),
interval="confidence")
Output:
fit lwr upr
1 23.282611 21.988668 24.57655
2 21.919770 20.752751 23.08679
3 24.885952 23.383008 26.38890
4 20.102650 19.003004 21.20230
5 18.900144 17.771469 20.02882
6 18.793255 17.659216 19.92729
7 18.205363 17.034274 19.37645
8 20.236262 19.136179 21.33635
9 20.450041 19.347720 21.55236
10 18.900144 17.771469 20.02882
11 18.900144 17.771469 20.02882
12 15.533127 14.064349 17.00190
13 17.350247 16.104455 18.59604
14 17.083024 15.809403 18.35664
15 9.226650 6.658271 11.79503
16 8.296712 5.547468 11.04596
17 8.718926 6.052112 11.38574
18 25.527289 23.927797 27.12678
19 28.653805 26.519252 30.78836
20 27.478021 25.554415 29.40163
21 24.111004 22.715653 25.50635
22 18.472586 17.319886 19.62529
23 18.926866 17.799465 20.05427
24 16.762355 15.452833 18.07188
25 16.735633 15.423002 18.04826
26 26.943574 25.112491 28.77466
27 25.847957 24.198041 27.49787
28 29.198941 26.963760 31.43412
29 20.343151 19.242185 21.44412
30 22.480940 21.268498 23.69338
31 18.205363 17.034274 19.37645
32 22.427495 21.219818 23.63517
Warning message:
'newdata' had 1 row but variables found have 32 rows
When I separate out both data into vector, i got different answer
Code for vector
predict(fit,data.frame(x=mean(x)), interval="confidence")
Output:
fit lwr upr
1 20.09 18.99 21.19
What is the reason for this difference?
This is a problem of using different names between your data and your newdata and not a problem between using vectors or dataframes.
When you fit a model with the lm function and then use predict to make predictions, predict tries to find the same names on your newdata. In your first case name x conflicts with mtcars$wt and hence you get the warning.
See here an illustration of what I say:
This is what you did and didn't get an error:
a <- mtcars$mpg
x <- mtcars$wt
#here you use x as a name
fitCar <- lm(a ~ x)
#here you use x again as a name in newdata.
predict(fitCar, data.frame(x = mean(x)), interval = "confidence")
fit lwr upr
1 20.09062 18.99098 21.19027
See that in this case you fit your model using the name x and also predict using the name x in your newdata. This way you get no warnings and it is what you expect.
Let's see what happens when I change the name to something else when I fit the model:
a <- mtcars$mpg
#name it b this time
b <- mtcars$wt
fitCar <- lm(a ~ b)
#here I am using name x as previously
predict(fitCar, data.frame(x = mean(x)), interval = "confidence")
fit lwr upr
1 23.282611 21.988668 24.57655
2 21.919770 20.752751 23.08679
3 24.885952 23.383008 26.38890
4 20.102650 19.003004 21.20230
5 18.900144 17.771469 20.02882
Warning message:
'newdata' had 1 row but variables found have 32 rows
The only thing I did now was to change the name x when fitting the model to b and then predict using the name x in the newdata. As you can see I got the same error as in your question.
Hope this is clear now!
In the formula for lm function do not refer to the variables using the datasetname$variablename pattern. Instead use variablename + variablename ...This will not throw the warning: 'newdata' had nrow(test) row but variables found have nrow(train) rows.
A way around this without making names is to use the following:
fitCar<-lm(mpg ~ wt, mtcars) #here you use x as a name
predict(fitCar,data.frame(wt=mean(mtcars$wt)), interval="confidence")
I got the same problem when I was using variable name in conjuction with data name with the use of $ sign.
So instead of:
fitCar<-lm(mtcars$mpg~mtcars$wt)
predict(fitCar,
data.frame(x=mean(mtcars$wt)),
interval="confidence")
Use this:
fitCar<-lm(mpg~wt , data = mtcars)
predict(fitCar,
wt = mean(mtcars$wt), interval = "confidence")