Creating a mask for natural earth R giving unexpected results

st_difference() is generic function and I guess it faild to choice appropriate method at your latter case. (I didn't read source code, this is an estimate.)
Maybe it choice not sfc but sf method. (because Polygon is class sf)
you can get desired output by making polygon sfc class or using sf:::st_difference.sfc directly. Below is an example;

x_coord = c(-20,-20,27,27)
y_coord = c(-1,28,28,-1)


polygon <-  cbind(x_coord, y_coord) %>%
  st_linestring() %>%
  st_cast("POLYGON") %>%
  st_sfc(crs = 4326, check_ring_dir = TRUE)# %>%  # keep sfc class
#  st_sf() %>%
#  st_wrap_dateline()

land = ne_countries(scale = "medium", 
                    returnclass = "sf") %>% 
  filter(admin %in% c("Niger", "Burkina Faso", "Nigeria", "Mali", "Chad", "Mauritania",
                      "Gambia", "Senegal", "Guinea", "Cameroon"))

land = st_union(land)
polygon_land_diff = st_difference(polygon, land)

# Plot the data 
plot(st_geometry(land))
plot(st_geometry(polygon), add=TRUE)
plot(st_geometry(polygon_land_diff),add=TRUE, col="green")