Is it possible to complete Fallout 4 without the use of power armour?

Yes, it's perfectly possible to finish the game without using Power Armor, with the one exception of it being a requirement (a quest condition) near the start of the game during the initial quest meeting the Minutemen in the Museum of Freedom, but as you're already 40 hours in and mention this quest I guess you've found and passed this one already. After this you can get out of it and abandon it and nobody will care all that much.

You will miss out on some random passer-by dialogue ("ooh, power armor") but this is flavor rather than substance.

Yes, you can play Fallout 4 without using Power Armor, although you will find some tasks become much easier by using it.

Fallout 4 is designed to be a fairly open game, where you can make choices as to how you play. Features like settlements and faction quests are completely optional (with one exception), and how you spend your time in The Commonwealth is completely up to you.

The only time that using Power Armor is absolutely required is during the first Minuteman quest in Concord, called "When Freedom Calls". It sounds like you're already past that, but if on a future playthrough you want to circumvent the armor altogether, AND you're on the PC version, you can use the console to skip that step by entering:

setstage 001a001c 65

This will tell the game that you've already gotten into the armor and picked up the minigun (even if you haven't), and gives you the next task in the quest (clear Concord of hostiles).

Now, that said, the game expects you use Power Armor occasionally, and certain enemies such as Deathclaws & Behemoths are designed with that in mind. At lower levels, such enemies are more difficult to defeat without using the armor.

There is also a step during the main questline that requires you to

travel into the Glowing Sea, a heavily irradiated area.

You can use a hazmat suit instead, or can stock up on anti-Rad chems, but will be much more vulnerable to the high-powered enemies found there. Also, regardless of which faction you join with, you must later

enter the reactor room at the Mass Fusion building.

Again, you can use a hazmat suit or chems to protect yourself, but will immediately face enemies afterwards and may find yourself vulnerable during the encounter.

Other than those two things, Power Armor is really just a convenience item. Putting points into the right perks & stats can eliminate most of the disadvantages you'd get by NOT using the armor, and no other quests or encounters require it.