When we `fill` the plot with `gender`, it implicitly groups the data according to `gender`. Using `interaction` to tell `ggplot` to group data by both variables.

group-interaction

grand_slam_ages %>%
    mutate(decade = 10 * (year %/% 10)) %>%
    ggplot(aes(decade, age, fill = gender, group = interaction(decade, gender))) +
    geom_boxplot() +
    scale_x_continuous(breaks = seq(1950, 2021, 10)) +
    expand_limits(x = 2020)