What is the correct usage of the word "Cartesian"?

If you take a look at the Wikipedia disambiguation page for Cartesian (see http://en.wikipedia.org/wiki/Cartesian ), you will see that every expression in which the term appears relates to the French philosopher and mathematician René Descartes (the adjective 'Cartesian' being derived from the Latinized form of his name, Cartesius).

In my view, your teacher is not only ignorant, she is arrogantly ignorant: the link with Descartes ought to be part of every teacher's general knowledge, irrespective of their specialist subject area(s).

To answer your last question, 'Cartesian' does always begin with a capital letter.

'Cartesian' is an adjective derived from the proper noun 'Descartes'; the possessive form is only used with nouns. (And yes, you do capitalize it; the majority of adjectives derived from proper nouns are capitalized).

As other posters have stated, your teacher is demonstrably wrong. It's difficult to imagine someone making such a basic error, but I would guess that she is unaware that "Cartesian" is the adjective form of "Descartes." She probably assumes that you are attempting to refer to a person named "Cartesian," and, since you are evidently in the process of learning English as a second language, further assumes that you have made a very basic grammatical error in the process.

Inform her that "Cartesian" is an adjective, not a proper noun.