Usage of "what"

Solution 1:

• 1.) This film is better than what we saw yesterday.

You are right, the choice "what" should be acceptable. The choice "which" seems to be ungrammatical.

LONG VERSION: Your example is in the form of a comparison construction that involves a fused relative noun phrase.

Your example is somewhat similar to:

• 2.) This film is better than [yesterday's film].

Notice how in version #2 that the subject "This film" is being compared to the noun phrase "yesterday's film". This is similar to what is occurring in your original #1 version: two films are being compared to each other (or today's film is being compared to the set of films shown yesterday). That is, things are being compared to things.

In your original example #1, your corresponding noun phrase for "yesterday's film" (of #2) is the expression "what we saw yesterday". Your #1 version has the same interpretation as the non-fused relative versions:

• 1.) This film is better than [what we saw yesterday].

• 3.) This film is better than [that which we saw yesterday].

• 4.) This film is better than [the film which we saw yesterday].

• 5.) This film is better than [the one which we saw yesterday].

The non-fused relative versions #3, #4, and #5 expose the relative clause ("which we saw yesterday") and the noun or whatever that it modifies (either #3 "that" or #4 "film" or #5 "one").

The word "what" is common as the head for fused relative noun phrases.

But as for the word "which":

• On the other hand, who, whom and which occur in the fused construction only under very limited conditions (usually with verbs like choose, want, like, as in I'll invite who I want).

The above is from the 2005 textbook by Huddleston and Pullum, A Student's Introduction to English Grammar, page 192.

Also, there's this:

• As noted above, who, which, and how hardly occur in fused relatives other than as alternants of the -ever forms in the free choice construction.

The above is from the 2002 reference grammar by Huddleston and Pullum et al., The Cambridge Grammar of the English Language, page 1076.

Hopefully this will be enough for your teacher.

Solution 2:

You are correct.

In your case, "what" is being used in the eighth sense of Dictionary.com:

(used relatively to indicate that which): I will send what was promised.

A preposition-"which" construction is just like a normal "which" construction except that the noun phrase to which it refers (I used it here) is acting as the object of a preposition, rather than the subject:

Lithium is a metal which reacts easily.

or the object:

Lithium is a metal which chemists use.

Now consider the following sentence that uses "than which":

Lithium is a metal than which no other is lighter.

Here, the interpretation is "No other metal is lighter than lithium." In your case, "than which" is preceded by "better," which is not a noun phrase, so that simply makes no sense.