Prevent unlist to drop NULL values
I have a vector of lists and I use unlist
on them. Some of the elements in the vectors are NULL
and unlist
seems to be dropping them.
How can I prevent this?
Here's a simple (non) working example showing this unwanted feature of unlist
a = c(list("p1"=2, "p2"=5),
list("p1"=3, "p2"=4),
list("p1"=NULL, "p2"=NULL),
list("p1"=4, "p2"=5))
unlist(a)
p1 p2 p1 p2 p1 p2
2 5 3 4 4 5
In this case (one level depth list) this should works too:
a[sapply(a, is.null)] <- NA
unlist(a)
# p1 p2 p1 p2 p1 p2 p1 p2
# 2 5 3 4 NA NA 4 5
The issue here is that you can't have NULL
in the middle of a vector. For example:
> c(1,NULL,3)
[1] 1 3
You can have NA in the middle though. You could could convert it to character and then back to numeric, which automatically converts the NULL values to NA (with a warning):
> b <- as.numeric(as.character(a))
Warning message:
NAs introduced by coercion
then put the names back in, because they've been dropped by the previous operation:
> names(b) <- names(a)
> b
p1 p2 p1 p2 p1 p2 p1 p2
2 5 3 4 NA NA 4 5 `
If you are dealing with a long complex JSON with several levels you should give this a try:
I extracted game log data from nba.com/stats web site. The problem is, some players have a NULL value for 3 point free throws (mostly centers) and jsonlite::fromJSON seems to handle NULL values very well:
#### Player game logs URL: one record per player per game played ####
gameLogsURL <- paste("http://stats.nba.com/stats/leaguegamelog?Counter=1000&Direction=DESC&LeagueID=00&PlayerOrTeam=P&Season=2016-17&SeasonType=Regular+Season&Sorter=PTS")
#### Import game logs data from JSON ####
# use jsonlite::fromJSON to handle NULL values
gameLogsData <- jsonlite::fromJSON(gameLogsURL, simplifyDataFrame = TRUE)
# Save into a data frame and add column names
gameLogs <- data.frame(gameLogsData$resultSets$rowSet)
colnames(gameLogs) <- gameLogsData$resultSets$headers[[1]]