An example of a triple $(E,\pi,M)$ which is not a vector bundle

What is an example of a pair of finite dimensional $C^{\infty}$ manifolds $E$ and $M$, and a smooth function $\pi:E\rightarrow M$ such that $\pi^{-1}(p)$ has a vector space structure for each $p\in M\ $ (all of them with same dimension), but it is not a vector bundle?

The fact that $O(n)$ is a deformation retract of $\mathrm{Diffeo}(R^n)$ is a result of Tom Stewart and appears in his paper "On Diffeomorphism Groups" in the Proceedings of the AMS (Vol. 11, No. 4, Aug. 1960). I have made a pdf available here.

In Corollary 1 (page 561) Stewart draws the conclusion that a locally trivial fiber bundle with fiber $R^n$ is in fact a vector bundle.

I and others have given some silly counterexamples. Let me now try to explain why you won't find any good ones. Warning: This is very far from my expertise. Hopefully, some experts will tell me whether I am getting this right.

Let $\pi : E \to M$ be a map of smooth manifolds. Suppose that there is an open cover $U_i$ of $M$ such that $\pi^{-1}(U_i)$ is diffeomorphic to $U_i \times \mathbb{R}^n$, with $\pi^{-1}(U_i) \to U_i$ being the projection map. Theorem: Then we can put a vector bundle structure on $(E, \pi, M)$.

Let me make sure you see what is nontrivial here. We are not using the vector space structure of $\mathbb{R}^n$, only its smooth structure. In other words, we have two diffeomorphisms $\pi^{-1}(U_i \cap U_j) \cong (U_i \cap U_j) \times \mathbb{R}^n$, one from restricting the diffeomorphism $\pi^{-1}(U_i) \cong U_i \times \mathbb{R}^n$ and the other from restricting $\pi^{-1}(U_j) \cong U_j \times \mathbb{R}^n$. There composition gives an automorphism of $(U_i \cap U_j) \times \mathbb{R}^n$, which comes from a map $g_{ij}: U_i \cap U_j \to \mathrm{Diff}(\mathbb{R}^n)$.

The diffeomorphism group of $\mathbb{R}^n$ is much bigger than $GL_n(\mathbb{R})$. It is not at all necessary that our $g_{ij}$ will land in $GL_n(\mathbb{R})$. Nonetheless, our theorem states that, if we are given an $(E, \pi, M)$, we can choose our trivializations such that the $g_{ij}$ will land in $GL_n(\mathbb{R})$.

So, how do we prove this? As I say, this is far from my strength. But I think the point should be that $GL_n(\mathbb{R})$ is a deformation retract of $\mathrm{Diff}(\mathbb{R}^n)$. A proof of this fact is sketched in the Wikipedia article on diffeomorphism groups of manifolds. The idea should be to start with a cover $U_{i}$ with maps $g_{ij} : U_i \cap U_j \to \mathrm{Diff}(\mathbb{R}^n)$ as above. Take paths $g_{ij}(t)$ homotoping from the original $g_{ij}$, at $t=1$, to linear $g_{ij}(0)$'s. Show that all of these give valid gluing data, and that these are all diffeomorphic. There are a number of details that I don't see how to nail down here though. I'm leaving this answer CW, in hopes that people will improve it.

Let me make a point about the big picture here. There are several levels on which your claim could fail.

(1) You failed to ask that the vector space structure respect the topology. That's the point of Mariano's example. I assume you meant to include this.

(2) The map $\pi: E \to M$ might not be a submersion. Since every vector bundle is a submersion, that's an immediate failure. That's the point of user8268 and my counterexamples. In general, a submersion is probably much closer to your mental image of "smooth map" than a general $C^{\infty}$ map is.

(3) I'm not sure whether or not this could happen, but you might be able to have a submersion which isn't locally trivial. In complex algebraic geometry, this happens all the time: Think of a family of elliptic curves whose $j$-invariant varies. For proper submersions this can't happen by Ehresmann's theorem -- but your map isn't proper. (Proper maps have compact fibers.)

I thought I had a counterexample: Let $E = \mathbb{R}^2 \setminus \{ 0 \} \times [0, \infty)$ projecting onto the first coordinate. Then the fiber over every nonzero point is $\mathbb{R}$ and the fiber over $0$ is $(-\infty, 0)$, which is diffeomorphic to $\mathbb{R}$.

However, I eventually convinced myself that there probably was a diffeomorphism $E \cong \mathbb{R}^2$ commuting with the projections onto the first coordinate. It would definitely be interesting to have a non-proper Ehresmann's theorem to address cases like this.

(4) You could imagine that you had a submersion which was locally trivial, but where the gluing maps were not linear. The point of my (other) long CW answer is to explain that this doesn't.

Here is a more concrete example (which you still won't like): Say $p:E\to \mathbb{R}$ is a vector bundle (e.g. the trivial one of rank $1$) and $f:\mathbb{R}\to\mathbb{R}$, $f(x)=x^3$. Then $\pi=f\circ p:E\to \mathbb{R}$ has the same fibers as $p$, so they are vector spaces, but $\pi$ is not a vector bundle, as it is not a submersion.

That brings the following point. Suppose $\pi:E\to M$ is a submersion, and that $\cdot:\mathbb{R}\times E\to E$ and $+:E\times_M E\to E$ are smooth maps making the fibers of $\pi$ to vector spaces. Then $\pi:E\to M$ is a vector bundle. To see it, choose a basis $e_i$ of $\pi^{-1}(x)$ for a $x\in M$, then choose local sections $f_i:U\to E$ for some neighbourhood $U$ of $x$ s.t. $f_i(x)=e_i$. By passing to a smaller $U$ we can suppose that $f_i(y)$ is a basis of $\pi^{-1}(y)$ for every $y\in U$ - and that gives you a local trivialization of $\pi$.