What is the name of this castle part?

It's a battlement or crenellation. This consists of a parapet (a short wall on top of a roof) with cops or merlons (the solid parts) and crenels or embrasures (the parts you can look through or fire arrows through).

Those links are all to Wikipedia, which I know you might not consider a reliable source (although all of those articles contain references). So, here's the Oxford English Dictionary's definition of battlement:

An indented parapet at the top of a wall, at first used only in fortified buildings for purposes of defence against assailants, but afterwards in the architectural decoration of ecclesiastical and other edifices. The raised parts are called cops or merlons, the indentations embrasures or crenelles.

Also, you can read the entry for battlement in the 1911 edition of the Encyclopedia Britannica. You can also take a look at a page all about parts of castles in the Middle Ages for some related vocabulary.

The cutouts you highlighted are called battlements.

The definition: Fighting position on the top of towers and along the wall, with crenelated walls to protect the defenders.

The specific word for this would be merlon:

(in a battlement) the solid part between two crenels.

A battlement is made out of a:

consisting of a regular alternation of merlons and crenels; crenelation

A crenel is defined as:

any of the open spaces between the merlons of a battlement.

Thus, in a battlement, there are merlons (specifically, the parts sticking out), and crenels (the spaces in between these merlons).