Is the following information being said in the implied quote, to the reader, or is it ambiguous

I am referring specifically to the phrase "which we weren't." I cannot tell whether or not he is saying it to the reader or if it is part of his reply. Is it ambiguous?

First example:

"I replied that he should give that careful thought because it might create a narrative that we were investigating him personally, which we weren’t, and because it was very difficult to prove a negative."

I include this second example because it appears to me that it is an explanation to the reader in this circumstance, but again I am not sure. And if the conclusion is different in both examples, I would like to know why.

Second example in which the same language is used:

"I said the White House Counsel should contact the leadership of DOJ to make the request, which was the traditional channel."

Full Context

This is part of the text of James Comey's testimony to a Congressional committee, released on June 7, 2017, for delivery in person June 8, 2017.

Neither of these remarks is an "aside" to the reader. The entire testimony is an account to the Committee hearing the testimony.

"I replied that he should give that careful thought because it might create a narrative that we were investigating him personally, which we weren’t, and because it was very difficult to prove a negative."

In the above quotation, Comey is recounting what he told the President. "which we weren't" is something he is saying that he told the President. "We" refers to the U.S. FBI, of which Comey was the director at the time of his conversations with the President.

In the second quotation Comey is saying that he told the President that the President's request should be made through what is the traditional channel, i.e., the White House Counsel contacts the leadership of the Department of Justice. Again, he's recounting his conversation with the President, not giving additional information to his current audience (the Committee, in open hearing, and thus to anyone who cares to read or listen).