How is virtual memory actually increasing the memory space?

It is not increasing physical memory at all. Its purpose is something else entirely. What it can do is make available other backing stores that allow programs to use more memory than is physically available.

Virtual memory is used to separate and isolate processes from each other and also allow memory access to be diverted to alternative locations.

Virtual memory allows the system to give every process its own memory space isolated from other processes. With programs effectively operating in their own space it gives them full access to the entire address space rather than having to work around other programs which might also need to use the "same" addresses. This has the side effect of increasing reliability and security as processes cannot easily interfere with each other.

The virtual memory space of an application is built up as needed. An application looks (to itself) to be in a single contiguous block of memory but could actually be completely scattered across physical memory.

Virtual memory also allows memory accesses to be trapped and diverted which allows us to use features like a swap file. What this means is that we can push parts of memory that haven't recently been used out to disk and set up a pointer that says "this block of memory is in file x at location y" and then we can free up the physical memory area for use by another application. When an application needs that memory it can be read back from disk, placed at some location of physical RAM (potentially different from where it was before) and mapped back into the same location of virtual memory as it was previously.

In the same way as the page file is used virtual memory can also allow the operating system to do what is effectively "lazy" loading of shared libraries for a program. When the main program tells the operating system that it wants to use a particular library then the operating system can save time by checking the requirements for the library, allocating the space in the virtual memory area for the application, but rather than load the entire library in it can defer loading pages of the library in from disk until they are actually needed. In this way the only parts of the library that get loaded into RAM are the parts that are actually used by the program, parts that are never used never get loaded and so don't waste RAM.

Using these techniques we improve the stability of the system and allow more processes to run in a confined space without them unduly affecting each other. It does not "increase memory", but instead allows us to more effectively use what we have.

The swap file is enabled by virtual memory systems, but in the past was confused as being the virtual memory.


Layman's explanation

The system will have to map each virtual address to physical address when that memory is used, but not all memory is used at the same time. For example, suppose you have 20 tabs in your browser, each taking 1GB of memory. In the OS without virtual memory support, you would need 20GB of RAM for this to work. The trick is, you don't browse all 20 tabs at the same time, so the OS with virtual memory will enable you to use your browser like that with just a couple of GB of RAM, swapping inactive tabs to disk.

More complex aspects

Virtual memory isn't used exclusively for swapping. It's main purpose is actually to avoid RAM fragmentation, which is a big problem on systems without virtual memory management: you may well have 1GB of RAM free, but if it comes in chunks of 10MB, an application requesting 100MB will not be able to work.

Over time, virtual memory found even more uses, notably random file access: many applications like databases will become painfully slow if they are forced to read files sequentially, and work much faster if the OS lets them pretend the whole file is located in (virtual) memory and optimize disk IO and caching based on access patterns.


Virtual memory doesn't increase memory, in the sense of actually adding more main-memory hardware. But it CAN increase the range of usable addresses. So one could have a running program consisting of a code segment and a data (stack & heap) segment, and both of these could occupy a range of virtual addresses larger than the range of physical addresses provided by the physically-real storage space of the machine. The trick is that only a small fraction of those virtual addresses are backed by physical main-memory at any moment [but everything is ultimately backed by disk storage]. This works because of the phenomenon of locality of reference: At any moment, only the instructions in one or more small contiguous sections of the program segment are being executed, and only data in one or more small contiguous sections of the data segment are being operated upon [of course the behavior is actually more complex, but it does follow this pattern for a large fraction of the time]


I understand that virtual memory fools the program by displaying more memory than is actually available.

The original motivation for virtual memory was a form of memory management to provide an address space larger than physical memory.
Software could utilize the full address space of the CPU (e.g. 2^32 address space) while the actual installed physical memory was only a fraction of that number.
Large programs could be portable among computers that used virtual memory without imposing huge (installed) memory requirements.
This use of virtual memory was back in the day of mainframe computers and ferrite core memory (which was low-density physically and expensive).

But ultimately it has to map the logical address to the actually physical address. Now how is it increasing the memory?

Virtual memory has evolved from just a technique to provide more address space for the program.
Virtual memory is a key component in providing security to each process in modern operating systems, so that a process cannot interfere with another process, nor be compromised by another process.
But multiprocessing (do not confuse with multiprocessors) with virtual memory still does provide more apparent memory for the system than physical memory.

Each created process is provided with its own virtual address space, i.e. its own virtual memory.
The amount of physical memory that is actually used (and mapped to the virtual memory) to each process is dynamic. Typically only the virtual memory that contains the code (aka text) and data pages/segments to perform the execution of the process is mapped to physical memory (aka resident in memory).

Nonessential code (because it's not currently executed) and data (because it's not being referenced/processed) does not have to be memory resident all the time. The code and/or data pages/segments can be "swapped out" to the backing store (e.g. swap space or page file on a HDD or SSD), and later "swapped (back) in" as needed (aka "on demand").

Virtual memory facilitates the efficient usage of the finite physical memory among numerous processes, each with its own protected virtual address space. The sum of these virtual memories would typically be larger than the installed physical memory.
The "increased memory" is now from the system perspective, and not just the program perspective.