Is there a point of no return in Fallout 4?
Is there a quest that marks the endgame, and if so, is it communicated to the player that the endgame is about to take place?
Furthermore, will it lock out certain secondary quests or make factions hostile to the player?
Can one continue playing after the main story line?
Solution 1:
The point of no return is definitely a significant amount further than what kalina stated. The point where you're forced to start making factions hostile towards you is a quest called "Mass Fusion", where your actions during that quest will affect you. There are obviously points in the storyline where you can prematurely make a faction an enemy, but "Mass Fusion" is the point in the storyline where you're forced to make a faction an enemy in order to progress in the storyline. It is a very definitive point in the storyline where you can see the endgame approaching.
Solution 2:
The Molecular Level quest both is and isn't a point of no return, fundamentally it requires you to make a choice - what you do need to consider is that speaking to a faction will provide you the infiltration quest for the Institute but also make the equivalent quest for the other factions unavailable. It is possible to reconnect with the factions you didn't choose after completing Institutionalized though. The actual point of no return varies by faction alignment.
The game is seemingly inconsistent in warning you when you're about to make permanent changes to your faction reputations.
In the case of the Institute quest line, speaking to Allie to start the Mass Fusion quest will pop up a nice warning stating you're about to become permanently hostile to the Brotherhood of Steel. There isn't a subsequent warning when you do the same to the Railroad, you'll just get lots of failed quest spam if you've got quests outstanding but in fairness, the mission is specifically to kill the Railroad so...
For the Railroad, I believe the switch occurs silently at the start of the quest Precipice of War. I definitely didn't receive a message and the conversation to start Precipice of War seemed to flow continually from the conversation that ended Operation Ticonderoga - the Brotherhood of Steel just stormed in and then I got lots of quest failure spam.
For the Brotherhood of Steel, the mission that makes you hostile with the Railroad is Tactical Thinking.
Yes, once you've finished the "story" you can continue playing, but there will be less faction missions available because of who you destroyed in order to finish the story.