Applications of rings without identity
The most common example of rings without identity occurs in functional analysis, when one considers rings of functions. A typical example is to consider the ring of all functions of compact support on a non-compact space. Obviously, as these rings of functions are very important in $C^*$-algebras and in studying the properties of the space, knowledge about rings without identity is very important for studying these spaces.
Arbitrary direct sums of rings with unity are not rings with unity, which can also be fairly annoying.
It is true that one can always embed a ring (as an ideal, even) into a ring with identity. The most common such embedding is the Dorroh embedding, in which we start with a ring $R$, and consider the ring with underlying set $\mathbb{Z}\times R$ and operations given by $(n,a)+(m,b) = (n+m,a+b)$ and $(n,a)(m,b) = (nm, nb+ma+ab)$. It is not hard to verify that $r\mapsto (0,r)$ embeds $R$ into the Dorroh extension as an ideal. You can preserve the characteristic of $R$ if necessary: if $R$ is of characteristic $n$, then replace $\mathbb{Z}$ with $\mathbb{Z}/n\mathbb{Z}$ in the construction. The extension has other nice properties (ideals of $R$ remain ideals of the extension, for example).
(Luckily, I am currently going over a thesis about embedding rings as ideals into rings with identity, so I can give you some other classical results.)
However, the Dorroh extension does not preserve all ring properties that may be of interest in $R$. For example, a ring is entire if it has no nonzero zero divisors; a ring is prime if whenever $A$ and $B$ are ideals and $AB=0$, then either $A=0$ or $B=0$ (that is, "prime" is the ideal version of "entire"; an entire ring is necessarily prime). For example, if you perform the Dorroh extension on $\mathbb{Z}$ itself (perhaps not realizing it already had a $1$) then $(1,-1)(0,r)=(0,0)$ even though $\mathbb{Z}$ is entire. There are nontrivial examples of this situation as well. Another property not necessarily preserved by the Dorroh extension is being semiprime.
There are other standard embeddings of rings into rings with identity, such as the Szendrei extension (a quotient of the Dorroh extension). But even so there are ring-theoretic properties that may be very hard to maintain in these kinds of embeddings. Among the more difficult ones are simplicity (if $R$ is simple, can we embed $R$ into a simple ring with identity? Yes; Anne Vakarietis, a student of a colleague, just finished putting together the pieces for this in her dissertation). It's known that every commutative $n$-root ring (rings in which every element has an $n$th root) can be embedded in a commutative $n$-root ring with identity, but it is not known if this is possible for noncommutative rings. Likewise, it is not known if every semiprimary ring can be embedded in a semiprimary ring with identity.
And worse, there are some properties that we know cannot be respected by such embeddings. For example, Fuchs and Rangaswamy proved that not every $\pi$-regular ring can be embedded as an ideal in a $\pi$-regular ring with identity (a ring is $\pi$-regular if every element is $n$-regular for some natural number $n$; an element $x$ is $n$-regular if there exists some $y$ such that $x^nyx^n=x^n$; this is a generalization of von Neumann regularity).
So, in summary: yes, rings without identity arise very naturally, and as such they show up when investigating other mathematical objects. And while it is true that one can always embed a ring without identity as an ideal into a ring with identity, this may not be a good thing from the point of view of studying some ring-theoretic properties of these rings.