ggplot scale_fill_manual within groups

One way would be to add a column of colors in the dataframe and use scale_fill_identity.

library(dplyr)
library(ggplot2)

colors <- c("black","blue","black","green","black","orange",
            "black","pink","black", "yellow")

df %>%
  distinct(type, ripeness) %>%
  mutate(color = colors) %>%
  inner_join(df, by = c('type', 'ripeness')) %>%
  ggplot(aes(x=type, y=count, fill=color)) +
  geom_bar(stat="summary", width = .7, position = position_dodge(width = .75)) +
  theme_light() +
  scale_fill_identity()

PS - I couldn't make "stripes-2" and "stripes-3" color work so changed it to other random color.