What is the correct name to use with /give for Lapis Lazuli?

Lapis Lazuli is a dye. Each dye shares the same name and relies on the data value to determine what type of dye it is. To spawn it in, you need to specify that data value using /give's fourth argument:

/give <player> <item> [count] [datavalue]

To give a player one Lapis Lazuli, you need to specify a data value of 4:

/give <player> dye 1 4

These data values encompass other items too, like cocoa beans, bone meal, and ink sacs, which have data values 3, 15, and 0, respectively.

Source: Data value of Lapis Lazuli

From the wiki:

Lapis Lazuli has the ID name minecraft:dye, with a data value of 4. So do this:

/give player minecraft:dye 1 4

To get one of them.

1.13+ Update

Data values have been abolished in favour of separate item IDs. So now instead of dye with the data value for a colour, they now go by the item name. Lapis Lazuli's item name is minecraft:lapis_lazuli

/give @s minecraft:lapis_lazuli