Should I refer to "Section 2.3" or "Subsection 2.3"?

I have to agree with you.

Collins Dictionary defines "subsection":

a section of a section; subdivision

It is certainly grammatically correct to use either word, and I think it is semantically correct both ways as well. However, as you say, writing "Subsection 2.3" it introducing redundancy, as it is (as you say) blatantly obvious that section 2.3 is a subsection of section 2.

I would say, however, that "Section 2.3" does not imply a subsection, it seems rather explicit that it is a subsection, to me.

A quick search on Google for "Section 2.3" and "Subsection 2.3": Section has 1.2 million hits. Subsection has 46,000 hits. So it definitely makes much more sense to drop the "sub-".

If we have the document:

1. Birds
   1.1. Parrots

   This is a bunch of information about parrots.

   1.2. Hawks

   This is a bunch of information about hawks.

Both "Parrots" and "Hawks" are sections in their own right. They are simply sections within sections.

This is similar to the folder metaphor in computing. A folder contains many subfolders, but each subfolder is still a complete, and real, folder in its own right.

Princeton University's WordNet defines subsection:

(n) subsection, subdivision (a section of a section; a part of a part; i.e., a part of something already divided)

Furthermore, Merriam-Webster defines section:

a distinct part or portion of something written (as a chapter, law, or newspaper)

So if one takes a section, then takes another logical portion of that, that is another section–also a subsection–that happens to be inside the original section.

E.g. a chapter is a section of a book. A paragraph is a section of a chapter. A sentence is a section of a paragraph. We can logically divide anything into however we like.

Cambridge Advanced Learner's Dictionary defines subsection:

one of the smaller parts into which the main parts of a document or organization are divided
  Further details can be found in section 7 subsection 4 of the report.

From the example, we can see that they have referred to section 7.4 as section 7, subsection 4.

My understanding of all this, is that we can call top-level sections, just "section". But we can call non-top-level sections either "section" OR "subsection".

In law, you clarify what you are talking about when referring to a subsection by saying "Subsection 23 (1)" or in spoken English you would say "subsection twenty-three, one". You can also say "Section twenty-three, Subsection one".

The purpose of saying "subsection" twenty-three (1) is to set in the mind of the listener or reader, that you are going to refer to a subsection, not the section the subsection you are referring to is subordinate to.

Statute Example:

23 All dog leashes shall be solid blue in color unless;

 (1) The dog is a seeing-eye-dog, then the leash used on this type of dog shall be solid yellow in color.

Using the statute example above, a person who is reading this law could say that according to subsection twenty-three, one, a person with a seeing-eye-dog has to use a solid yellow leash.

They could also say that according to section twenty-three, subsection one, a person with a seeing-eye-dog has to use a solid yellow leash.

When referring to a section and a subsection (or a paragraph and sub-paragraph, etc) the abbreviation that corresponds to the highest 'level' of the section (or paragraph, etc) should be used -- for example, using the Section Designation Guidelines from DBSK_FAN and imagining that it was taken from 'Chapter 9 - Witty Examples', you would say 'chapter 9, section 4' - not 'sub-chapter 9, 4'; you would say 'section 4' not 'sub-chapter 4'; you would say 'section 4(a)' - not 'subsection 4(a)'; and you would say 'section 4(a)(i)' not 'subsection 4(a)(i)' nor 'sub-subsection 4(a)(i)' nor 'double-sub-section 4(a)(i)'.

The only times you say 'subsection' at all would be, eg, 'section 4, subsection (a)' (and that's only when actually saying it out loud - like in court or whatever - as opposed to typing or writing it); or when you are writing about section 4 in an essay and then, having made it very clear you are talking solely about chapter 9, section 4, you go on to discuss the correlation or contradiction or whatever between subsections (a) and (b).

Those are the rules you're supposed to abide by at Australian law faculties, anyway. Google the AGLC (Australian Guide to Legal Citation), it clarifies most things. Hope it helps!