Why can't Linux developers create a universal packaging format?
Solution 1:
It seems appropriate to quote Joel Spolsky on this one:
(By the way, for those of you who follow the arcane but politically-charged world of blog syndication feed formats, you can see the same thing happening over there. RSS became fragmented with several different versions, inaccurate specs and lots of political fighting, and the attempt to clean everything up by creating yet another format called Atom has resulted in several different versions of RSS plus one version of Atom, inaccurate specs and lots of political fighting. When you try to unify two opposing forces by creating a third alternative, you just end up with three opposing forces. You haven't unified anything and you haven't really fixed anything.)
(emphasis added)
You have (at least) two packaging systems for Linux. That's actually a good thing. A single system will simply create a third system.
Solution 2:
There are many reasons for this, and a bit of history is in order to put things into perspective.
Remember that when we talk about "Linux" what we are generally referring to is one of many different Linux distributions. "Linux" is actually just an operating system kernel.
The original goal of Linux was to create a Unix-based system that would run on PCs (initially the 386). The first step was to create the kernel itself. While Linus Torvalds was working on the kernel Richard Stallman was working on his own Free Unix system, under the GNU (GNU's Not Unix) project. To cut a long story short, the two somewhat converged because GNU had the associated utilities (C compiler / library / build tools, shell, text editors etc.) but no core to run it on, and Linux had the core but no utilities to run on top of it to make it useful for the masses.
This convergence came to be known somewhat officially as GNU/Linux. You will see that a lot of distros still refer to themselves as GNU/Linux distributions.
Because of the Free and open nature of GNU/Linux anyone could pick it up and create a bundled system to their specific tastes. The result was that many different streams of varying configuration methods were used to create these systems, which had the side-effect of creating almost as many different package management systems to fit in with each one.
Each different complete system had its own strong followers who stuck with them over the years, resulting in what we have today: a handful of widely used, deeply-rooted and stable package management systems like RPM, APT / dpkg and Gentoo's Portage.
There are projects, such as Autopackage, which are attempting to solve the problem, but the continuous evolution of the various supported package management system means there are many moving targets to follow.
What some software vendors end up doing is bundling the specific binaries and copies of dependencies they require into one large package which will work on specific systems.