How to refer to a specific floor of a building

Are there any differences between "floor" and "Floor" when talking about areas of a building? For example, "The children book section is on the 4th floor" or "The children book section is on the 4th Floor"

And I found a sentence like "Computers are available on the 5th and 6th floors"? Why is "floor" plural?

Solution 1:

Capitalisation implies that the name has been elevated to have meaning in its own right, not just as a literal description. For example, if the mezzanine between the 1st and what was the 2nd floor was converted to be the 2nd floor, what had been the 4th floor would become the 5th floor but might be referred to as "the 4th Floor". Similarly, say a company owned two bookstores, and in the original one a specific range of titles resided on the 4th floor... the employees might refer to the area in the 2nd bookstore that happened to house the same books as "the 4th Floor" even though it was in the basement in that building, perhaps only occupying a part of the floor....

So, if it's literally the 4th floor, keep to lowercase.

"Computers are available on the 5th and 6th floors"? Why is "floor" plural?

Because there are two of them... simple as that. It's much like "you'll find your rooms through the 2nd and 3rd doors", or "the third and sixth dresses you modeled were my favourites"....