"I was by my friend"

In Yiddish Idioms in American English, Lillian Mermin Feinsilver says that one Yiddish idiom is this use of with instead of by. She presents as examples the phrases "it's all right by me" and "this is fine by Mr. Fiedler."

In a footnote, she discusses a related idiom:

The substitution of by for at, as in "I was by his house," noted by the Hermans in their manual (p. 416), appears in Wentworth and Flexner, op. cit., p. 83, with a quotation from Nelson Algren: "I'll buy you a drink by Antek." But this would seem to be still rather parochial.

The works referenced in the footnote are Dictionary of American Slang (1960) by Harold Wentworth and Stuart B. Flexner; and Manual of Foreign Dialects for Radio, Stage and Screen (1958) by Lewis Herman and Marguerite Shalett Herman.

In Supplement II to The American Language (1948), H. L. Mencken (in words identical to Feinsilver's) also lists this a feature of American Yiddish:

the substitution of by for at, as in "I was by his house."

It seems that by is a replacement for with or at rather than nearby.

Though I've never heard this expression, I don't think it's grammatically incorrect.

"By" is frequently used to mean "near" or "next to" or even "with".

Example:

Where's the ketchup? It's over there, by the mustard.

It's also not dissimilar to the phrase "by myself" - if "by myself" means with/near myself, "by my friend" should mean with/near my friend.