How can I generate a rendered preview of selected markdown text or files?

I tried a few things (even a websocket node app I wrote) to do this, however I found an app, Marked, that does exactly what you asked. It constantly reads a text file and shows you HTML/MD on the fly as you type. It's pretty nifty, commercial, but nifty.

The Marked Bonus Pack comes with a service for previewing text selections.

To view a HTML file (I'll leave you to work out exactly how you want to format it and parse the markdown etc) in a chromeless window similar to quicklook you can use the automator actions Get Urls From list > Safari Popup.

This will open the URL (which can be a local file://~/etc) in a box as follows (the following screenshot shows facebook - as you can also set window size and user agent in the safari popup options - I choice iPhone size and Mobile Safari agent to show how you can create a quick and dirty Facebook app using automator in 10 seconds). Clicking OK or cancel closes the app as it's the last Automator step, but you could easily extend it so a canel stops the workflow, but OK continues the actions, which may me to copy the file to an upload folder or whatever.