Split up a dataframe by number of rows
Make your own grouping variable.
d <- split(my_data_frame,rep(1:400,each=1000))
You should also consider the ddply
function from the plyr
package, or the group_by()
function from dplyr
.
edited for brevity, after Hadley's comments.
If you don't know how many rows are in the data frame, or if the data frame might be an unequal length of your desired chunk size, you can do
chunk <- 1000
n <- nrow(my_data_frame)
r <- rep(1:ceiling(n/chunk),each=chunk)[1:n]
d <- split(my_data_frame,r)
You could also use
r <- ggplot2::cut_width(1:n,chunk,boundary=0)
For future readers, methods based on the dplyr
and data.table
packages will probably be (much) faster for doing group-wise operations on data frames, e.g. something like
(my_data_frame
%>% mutate(index=rep(1:ngrps,each=full_number)[seq(.data)])
%>% group_by(index)
%>% [mutate, summarise, do()] ...
)
There are also many answers here
I had a similar question and used this:
library(tidyverse)
n = 100 #number of groups
split <- df %>% group_by(row_number() %/% n) %>% group_map(~ .x)
from left to right:
- you assign your result to
split
- you start with
df
as your input dataframe - then you group your data by dividing the
row_number
byn
(number of groups) using modular division. - then you just pass that group through the
group_map
function which returns a list.
So in the end your split
is a list with in each element a group of your dataset.
On the other hand, you could also immediately write your data by replacing the group_map
call by e.g. group_walk(~ write_csv(.x, paste0("file_", .y, ".csv")))
.
You can find more info on these powerful tools on: Cheat sheet of dplyr explaining group_by and also below for: group_map, group_walk follow up functions