With character or sign

Solution 1:

& comes from a Latin scribal abbreviation for et (⁊ which was used as an alternative to & in Old English and is still used that way in Modern Irish comes from a different form of the same thing).

c̄ was a Latin scribal abbreviation for cum and p̱ for per, both of which would be used some places where in English we would use with. They had other meanings too, depending on position within a text.

I do not know of their ever seeing extensive use in English. Update: See the comment below about doctors using it in writing prescriptions.

In modern English there is the slash-abbreviation w/ for with. Around the middle of the last century it was common to write two-letter abbreviations with a slash between those two letters, and w/o was so-used for without and by extension removing the o gives w/ for with.

While it is now rare to use a slash in an abbreviation in this way (unless the expanded from has a slash, like i/o for input/output), both w/ and w/o continue to enjoy some use.