Can "broken" mean never working to begin with?

Yes, it can be used that way, since it is simply an adjective that describes an ongoing state, not a statement of what it was in the past, too.

I think the confusion arises from the fact that "broken" is also the past (passive?) participle of "break". If it were used in that sense, then you could make the case that something that doesn't exist yet cannot be "broken".

Being incomplete is not the same as being broken, but something can be incomplete and broken at the same time.

If a car being assembled falls off the assembly line and its axle snaps, it is both incomplete and broken. It won't work because not all of it is there, and it won't work because it is damaged.