Difference between a stand-alone executable file, and an installed executable?

I have noticed on Windows, at least, that you can download a direct, statically-linked executable file and launch it directly, or write your own program and executable it (even dynamically) without having to install it.

That brings me to my main point ... what's the purpose of the installation process? I mean besides maybe the Windows Registry. However, for practicality and usage purposes, it's possible to have a single, independent, stand-alone program that can be run, stored on non-volatile storage, and accessed via the filesystem of whatever device it's on, and executed on the OS. So what's the big deal with all this "install this" business if many a great programs of virtually any magnitude can work perfectly without going through an installation configuration? It puzzles me a bit, and aside from a database or other metadata/access configuration systems, what is the real difference here if the latter(an installed executable)performs and works in the same way as a stand-alone?

Is there a difference here I'm unaware of with a non-installed program versus an installed one?

PS: This doesn't just have to apply to Windows OSes, but any that implement a similar function.

Solution 1:

There are several reasons why programs come as installers rather than standalone executables:

File Size Concerns

Programs with many, large dependencies can bundle Web-based installers that download the dependencies and place them in a common location, so that they can be shared by multiple programs. For example, DirectX is a very large library. If every single program on your system that depends on DirectX just bundled the entire DirectX runtime with it, it would consume a good bit of space. This may not seem to matter in the age of 4 TB hard disks, but consider that SSDs are quite a lot smaller in capacity, and they're coming into common use on ultrabooks, some with as little as 64 GB of storage. And of course there are many other shared libraries aside from DirectX.
Programs that are both very large and continuously updated are best distributed as a collection of many small files, along with a launcher or updater program that checks the Internet for an update, and if any update exists, only download the required changes. If all large programs were shipped as a single monolithic executable, it is very likely that the patch process would require re-downloading the entire executable, as patching the running executable file on disk is near impossible due to file locks. Also, because the updater needs to know where its files are, it often stores that directory path in a well-known location in the registry.

User Convenience Concerns

Installers for very large programs, such as Visual Studio and Microsoft Office, allow the user to de-select the installation of certain features, if the user knows they will never need them. This has 3 potential benefits: it reduces disk space consumption; it can reduce download time and bandwidth consumption if the installer is a web downloader; and it can reduce "clutter" and "bloat" on the user's machine, fewer start menu / desktop shortcuts, fewer startup programs, etc.
Installers for complicated programs often come with configuration options that the user can set up using a user-friendly graphical interface as part of the installer. See for example the MySQL or SQL Server installers, which can take you through the entire process of getting your database server up and running before you even click "Finish" on the installer.
Installers can prompt the user for required information, such as license keys, which only need to be entered once. This can simplify the design of the program itself, and reduce the number of things it has to do and check for when it starts up. This also results in the user having confidence that, once the program is successfully installed, it should "just work" -- there are no more "gotchas" within the program that may hold them up from using it.

Compatibility Concerns

Some programs conflict with other programs. This is a simple and unfortunate fact of software engineering. Before installing a program that has known conflicts with other programs, it is often helpful to first check the system to see whether an incompatible program is installed. The user can then be alerted if so. For example, there is a very dangerous incompatibility potential in older versions of VMware and VirtualBox, that resulted in a Blue Screen of Death, because one program would try to use a special virtualization processor instruction after it was already reserved for user by the other product. If you were to simply provide the end product to the user without an installer, you would have to check for the presence of incompatible products at every start of your program, which could slow down the startup of the program.
Programs may have dependencies on other system components that can only be installed at a system-wide level, not at a per-user level. In order to install these special system components, administrative privileges are usually required, and an installer usually has to be run.

Elevated privileges and special services

Some programs depend on changes to the operating system for their functionality, and these changes can't easily be implemented without some kind of installer to take care of them with administrative privileges. For instance, programs that install drivers or kernel modules, such as Wireshark, can't simply be run, because you absolutely have to ship the kernel-mode components in separate files. In the absolute best case, you would still have to have the user manually unzip an archive, and then run some kind of installer for the device driver. Services are another example of something that requires administrative privileges to install. Installer software is particularly good at obtaining admin rights in an elegant way, without requiring that the main program itself request admin rights every time it runs (this would be an unnecessary security exposure in many cases).

Having given all these reasons for why installers are useful, here are a few observations from the other side:

Many programs, even those that are only available for download as an installer that requires admin privileges, can be forcibly "unpacked" from their installers and run directly without installing them. Other programs, especially open source ones, are repackaged into self-contained executables by PortableApps. It is noteworthy that some programs, when unpacked from their installer, will have reduced functionality, exhibit errors, or other problems.
On operating systems other than Windows, it is almost always possible to simply download (or compile) programs and run them as a regular user, without obtaining root. There are a few exceptions with respect to packages that are a core part of the operating system, but for most user applications, you can run it in your home directory without installing it system-wide using the package manager. Windows is a bit of a special case in that most desktop programs on Windows have an installer, and can usually not be installed any other way.
Even on non-Windows platforms, programs that need the ability to load a kernel module come with some sort of installer, which compiles the kernel module and installs it in the right directory. You can also expect to see an installer in the event that the program is a daemon that will be started using a system service script, e.g. in /etc/init.d. This type of "shrinkwrapped binary" is a less common distribution method on GNU/Linux, but most Linux distributions still provide most software in the form of installable packages, each of which requires root access (admin access) to install.

Conclusions

You asked why we need installers. The short answer is that we don't -- not strictly speaking, anyway. There are vanishingly few examples of applications that cannot, in principle, be bundled into a single self-contained executable with no resources, no installer, etc. Even something as complicated as VMware Workstation could automatically obtain admin privileges, write the hypervisor kernel module out to a file on disk, and install it dynamically on program startup, and ship all of its resources (images, sounds, etc.) bundled inside the data section of the executable.

Using an installer or not is a choice that software producers have to make. There are advantages and disadvantages to using an installer. Many vendors choose to distribute their software both as an installer, and as a standalone binary, or at least as a ZIP file that can simply be unpacked and run. For software that does not absolutely require an installer, this is a very pragmatic way to go, and makes everyone happy. Usually, software that does not ship in any other form than with an installer is software that requires administrative privileges to install some component of itself, since the installer is the most elegant way to obtain the needed privileges.

Personally, I find installers very annoying in my day to day work, because sometimes I want to run a program when I don't have administrative rights on the computer I'm using. I have quite a bit of experience manually unpacking installers to extract the program files within, and then getting those files to run correctly. However, on my personal PC at home where I always have administrative access, I find installers beneficial and convenient, because most installers give me useful options, like whether to create a desktop shortcut, that I would have had to do manually instead without it.

Solution 2:

The installation process considers these items:

whether you have required library (api) installed, such as .net framework version, or direct X.
Install some extra system DLL files into system folder. If system folder already contains these files (same version), then ignore this step. This way you don't need several copies of the same DLL files.
Install shortcuts into your Start menu or Desktop to help you start the application quickly.
As you said, Registry modification. This step can actually be done when the application is first executed in this computer, so it is not extra important.
very important: whether to bundle a software (such as Google desktop bar or Yahoo! bar into your IE). This is the way a lot of independent software creating revenue to the developer. Don't forget that, if you are using free software.

Because we are considering an "executable" application, we don't need to discuss those steps that can install a IE add-on or related things.

Solution 3:

Stand-alone program has no external dependencies.

It doesn't have to be .exe file only, it can have accompanying libraries and data files: Unpack the archive to a folder and run the executable. If you just unpack an archive, no shortcut is added to Start menu, hence you'll have to navigate to the folder where the unpacked application is located and start it from there, or manually create a shortcut for it in Start menu. Many computer users find it difficult.

Easier to Use

An installer guides users through the installation process. You download the installer, .exe or .msi (the former is preferable for non-advanced users), and run it. It picks up the installation folder, usually in Program Files, copies the files, creates shortcut in Start menu. You're done: in the majority of cases you simply click Next several time.

Then go to Start menu and run the application. Some installers provide an option to start the application when installation is complete.

If the application opens files or documents of certain type, the installer registers it with the shell. So that you can click the file to open it.

License Agreement

Many applications, both commercial and free ones, require you to accept the license agreement before you can use their application. Installation doesn't proceed until you acknowledge you agree to the license terms. Even if you didn't read the license, you have agreed to it.

Dependencies

Sometimes it's not enough to simply copy the executable files. Applications often use shared components or special runtime libraries. For example, .Net framework runtime has to installed to run the application written for .Net; even Visual C++ runtime, if it's not statically linked, has to be installed. The installer takes care of ensuring all the dependencies are satisfied.

If an application consists of several .exe and/or .dll files, dynamic linking to Visual C++ runtime reduces disk space. If .exe and .dll are statically linked, then the runtime is duplicated in each and every file.

License terms of a library may not allow statical linking.

Security

If a vulnerability is found in the runtime, it can updated separately from the application. Updates to .Net and Visual C++ runtime are installed automatically via Windows Update.

If executables and libraries are statically linked, then application vendor has to recompile the application and release the updated version. So using shared runtime reduces cost of application maintenance for developers and vendors.

Installing to Program Files also provides more secure environment: the files there can't be modified or deleted without administrator privileges.

Registry

Many Windows applications rely on entries in the registry. If application uses COM, all the objects have to be registered otherwise the application will fail to create the needed object and will not start.