"Really" modification problems

Construct a dialogue:

Me: You know French, don't you?
James: No, I don't know French.
Me: But you had four years of French in high school!
James: Yeah, but I don't REALLY know French. I just SORT OF know French.

I think that in this case, not modifies do, and really—just like sort of, where the not distraction is removed—modifies know.

However:

Me: You know French, don't you?
James: Not really.
Me: Oh, come on! I heard you say hors de concours not ten minutes ago.
James: No, really, I don't know French. Just a few phrases I picked up from novels.

In this case, not really modifies (eliptically) the assertion in my preceding utterance, and really tout seul modifies the following clause, including the not there.

In your example, the meaning is perfectly clear from the context: ‘In spite of my modest ability in some areas of the language, my knowledge of French is far from comprehensive.’ In other words, really modifies the verb and nothing else. Any other interpretation is perverse.

I don't think the sentence is appropriate for describing your knowledge of French. "Really" is one of those squishy adverbs that gets inserted into sentences because people like to insert phatic ("of, relating to, or being speech used for social or emotive purposes rather than for communicating information": m-w.com) remarks because they want to emphasize something that doesn't need emphasizing.

You're not pretending to know French, but you are actually managing to read French newspapers with the help of a dictionary. I know how to read a little bit of French and am learning more every day because I read a French newspaper with a dictionary. That's how my first father-in-law learned to read English after he arrived from Europe during WWII: The New York Times and a dictionary.

In some PhD programs, that's the only language requirement: a reading knowledge of works in your field. I read anthropology books in French without any trouble, but old literary ( Les Misérables as opposed to popular, e.g., Le Petit Prince) novels were beyond my comprehension. I could read the newspaper with the help of a dictionary, and I could read French subtitles in non-French films. I could speak sufficient French to call myself fluent (the words flowed without my having to search for them, and my sentences usually sounded natural, so I wasn't dismissed as a barbarian -- or thrown out of taxicabs, like some of my friends who tried to speak French in France), but my vocabulary was too small to claim more than basic fluency in everyday French and a few specialized topics. I did know French, but I wouldn't say that "I really knew French", because to me that implies being as proficient as a high school graduate (I think the average level of English proficiency in the USA is 9th grade). I wasn't that good at French.

A professional linguist can tell you whether "really" modifies "know" or "not really" is a phrasal adverb that modifies the whole sentence. It's not an interesting question to this amateur linguist (I think "really" modifies "know"), simply because what it does has no effect on how people understand the sentence.

That you need a dictionary to read the newspaper says all that need be said about your French reading skills and your French reading vocabulary. Really!