Singular of "vertices"
The singular is vertex.
According to the Merriam-Webster dictionary entry for vertex, both vertices and vertexes can be used as the plural of vertex. Vertice is not listed as a word.
ver·tex
noun \ˈvər-ˌteks\
plural ver·ti·ces\ˈvər-tə-ˌsēz\ also ver·tex·es
The same happens with index, for example, where the plural can be both indices and indexes.
According to the same dictionary, vertex and index both originated from Latin:
Vertex
Middle English, top of the head, from Latin vertic-, vertex, vortic-, vortex whirl, whirlpool, top of the head, summit, from vertere to turn
Index
Latin indic-, index, from indicare to indicate First Known Use: 1561
Explanation for the two forms of plural from World Wide Words:
Index is one of those rare oddball words with two different plurals in English. English copied the original Latin plural at first, making indices. As with a lot of other Latin plurals, the standard English way of marking the plural, using -s or -es, has progressively been taking over, making indexes.
(..)
Other words retain their Latin plurals, but we have to work at remembering them because the English plural marker has otherwise so few exceptions: apices (of apex), corpora (of corpus), helices (of helix), matrices (of matrix), vertices (of vertex) and many others.
Notice that based on the last paragraph above, the author of the article does not consider vertexes a valid plural of vertex.
Additionally, vertices is much more used than vertexes: 12.600.000 occurrences of vertices vs. 26.700 occurrences of vertexes in Google. My Firefox's spell checker disagrees though. It doesn't know vertices but it does know vertexes.
As for vertex vs. vertice, there are 2.680.000 occurrences of vertex in Google and 130.000 occurrences of vertice.
These numbers also lead to a possible explanation for why people sometimes say vertice. The plural form vertices is more than 4 times more used than the correct singular form vertex. So it's possible that people derive vertice from vertices because they hear the plural form vertices much more often than the singular form vertex.
Neither Merriam-Webster nor Wiktionary have an entry for vertice, and I have never encountered it myself. Then again, I am heavily biased, working a lot with graphs.
The British National Corpus has 44 cites for vertex, 19 for vertices, but none for vertice (well, it does have exactly one, but judge for yourself).
All that being said, I think it is only natural for someone who encounters the plural form vertices for the first time in his life to derive the singular form as vertice. Looking at your location and its demographics, I think that Spanish influence could be yet another factor in your immediate environment, the Spanish word for vertex being vértice.
As to the plural form, vertices is much more common, but vertexes is not incorrect, see e.g. Merriam-Webster or Wiktionary. The BNC has 5 cites for vertexes (but again, 19 for vertices).
The singular of vertices is only vertex, although in colloquial language it may have developed into "vertice" as well, though this is not technically correct.
To answer your second question, vertexes is not technically correct, but again is used often in modern-day language and as a result has become an accepted term. However, to be completely correct, I would use vertices, and likewise vertex and NOT vertice for the singular form of this word.