Why remote desktop (RDP) isn't fast enough for watching videos on a remote PC?

On my iPad, I tried to remote control a computer on my local network and open a video on it, but it wasn't fast enough (the video refresh rate was terrible). I've purposely tried to connect to the local area IP 192.168.0.xxx, my router touts 54Mb/s of transfer rate. What do I need to do to make my view of the video on the remote PC more fluid?

Added (after reading keen's answer): For example, if I want to stream from twitch.tv, that's hard because twitch.tv's ipad app doesn't work the majority of the time, so I can't really go directly to the "source". Other times when i'm browsing a site on the computer, and it streams a video. I want to sit on a sofa and watch it. It's inconvenient for me to type in the address on my ipad's browser again to go directly to the source. And this is a small website that hasn't created an app yet.

Solution 1:

54Mbps (which is 6.75MBps) isn't great speed in front of local disk speed. Despite the number is bigger than Blu-ray's 30-40Mbps bitrate, don't forget its theoretical number and RDP transfers RAW frames which isn't compressed as in video formats. So, video of your question shouldn't be high quality.

Now, do these things before creating RDC: Click on arrow next to "Options" to reveal "Advanced Options". Go to "Display" tab and change all display settings to maximum (like Colors to 32-bit). Go to "Experience" tab and change "Connection Speed" to LAN. Also, set maximum settings of all experience options.
Use maximum experience settings in your iPad client (as you haven't provided details about it, I can't further help).

Update after Question Update:
You can't compare video streaming with RDP streaming. Video formats are compressed formats. It means when you stream videos, compressed data (no redundancy) flows from server to client. But, the scene of RDP is different. With RDP, a part of video file (which is compressed) doesn't flow from server to client. RDP draws frames after seeing the screen. More details on screen (like at the time of FullHD video play) means big sizes of raw frames. As raw frames aren't compressed like videos, its causing you problem on slow network.

However, you aren't out of luck. You can use a nice RDP compression and acceleration software.

Solution 2:

Remote Desktop works by transferring graphical information from the remote computer to your local computer. In the most naive straightforward implementation of this functionality, everything would be rendered on the remote computer. And then the snapshots of already rendered data would be sent over to RD client computer. However, RD implementation is much less naive.

For graphics that consists of "line art" and text drawn through Windows API, Remote Desktop actually performs a remote procedure call: it sends the API call parameters to the client machine and carries out the actual call on the client machine. I.e. the graphics is actually rendered on your local client machine. This is an extremely compact and efficient way to transfer graphics, which is why all "line art" and text drawn through Windows API work very fast over Remote Desktop connection. Basic Windows GUI is one example of such graphical information. In essence, this graphical information is transferred over the network in extremely efficient vector form. This is what makes typical Windows GUI elements to work so well over Remote Desktop connection.

Now, any graphical information that cannot be described as a sequence of vector Windows API calls has to be transferred in bitmap form. That applies to raster images, for example. You probably noticed that ordinary bitmap images are drawn in Remote Desktop client much slower than typical GUI elements. The same applies to video. Video is actually played on remote machine and then the rendered result is transferred to your local client machine as a rapid sequence of bitmaps. This generates a huge amount of network traffic, which easily exceeds the bandwidth of a typical connection. This is why videos are virtually unplayable through Remote Desktop.