Having a domain name use nameservers of the same domain
The specifics of how you do this will depend on who you registered the domain with, but you want to look into how to set the "glue" records for your domain.
is the domain "example.com" a real, valid on the internet domain?
If so, do you have access to the domain registrar that the domain is registered at?
If the answer to both questions is "yes" then you can create the three DNS servers, and then change the information at the domain registrar to specify that these new IP addresses are the authoritative domain name servers for example.com.
When a user does a query to resolve the address of a server on the example.com comain, the domain name severs that they ask should first ask one of the root name servers "Which IP addresses hold authoritative copies of the records for example.com?" This is the answer you really need to change... the root name servers should answer that YOUR IP addresses hold that authority, and the only way to do that is by changing the information through the registrar.
HTH,
Glenn
You can do that by getting a private name server, this allows you to set your nameservers using your domain name, example: ns1.yourdomain.com / ns2.yourdomain.com.