How do you interpret 'up to'?

Solution 1:

Usually "up to" means "up to and including", if not further clarified. So yes, you fill in the last box and take the cap off the yellow marker.

Solution 2:

In the case of the markers, I would be more inclined to take the caps off the black, blue, and red, leaving the yellow cap on, simply because it would have been more natural for you to say "take the caps off" or "take all the caps off" and I would be inclined to interpret the "up to the yellow marker" in a way that makes sense for you to have felt it necessary to specify it that way. (Assuming I'm not allowed to ask for clarification, like "What, you mean all of them?")

In the case of the boxes, "up to the last box" can be more readily taken as an intensifier: "Fill them, (all the way) up to the last box." So I would fill all the boxes, even though the "up to" phrase is rather redundant.