DDNS: Is a DIY solution possible? Better?

I'm trying to establish a personal mail/calendar server at my home (yes, I've heard that it's difficult, it's a lot of trouble, and so forth, but I'd still like to try). I have an ISP that doesn't offer static IP addresses, so it looks like some sort of Dynamic Domain Name Service (DDNS) is the solution.

However, I've been doing research, and I've found at least a couple of online resources which explain that you can do DDNS yourself: you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using for your home servers (I happen to have a domain parked with a hosting provider for just this eventuality, and as I understand it, I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically...someone let me know if I'm wrong on this and there's a simpler way).

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address). However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

Also, I have another concern: are there any security issues I may be overlooking with having a DDNS provider? Won't they be able to monitor all the traffic flowing through the domain name they provide? Does anyone have an informed opinion with regard to which method (paid vs. DIY) might be better?

I appreciate your time...thanks!

Solution 1:

I'm trying to establish a personal mail/calendar server at my home (yes, I've heard that it's difficult, it's a lot of trouble, and so forth, but I'd still like to try).

You probably won't have much luck with the mail part. See @Alex's answer.

you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using for your home servers

Pretty much that.

I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically

Yes, although if the company just provides a generic "host everything" service it might not have any DNS management API at all (focusing instead on web and mail) and you might need to move the domain elsewhere.

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address).

No. Only your DNS hosting provider's own systems have to be updated. The rest of the world doesn't keep a permanent record – it merely caches the results of individual lookups, for the duration indicated in each (sub)domain's "TTL" (Time To Live) field.

However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

I would guess that they allow configuring a very low TTL on the dynamic domains (down to a few seconds), which means it'll drop out of any caches very quickly, at the cost of the DDNS provider itself receiving many more requests (higher load on their DNS servers and databases, and an excuse to charge you more). That alone isn't something special and could be implemented with any DIY method.

Won't they be able to monitor all the traffic flowing through the domain name they provide?

No. The DNS server only provides you with an address (much like a phone book) and is not involved in any further communication.

(Unless the provider actually attempts to return false data, which would considerably shorten the company's TTL the moment news websites learn about it.)

That said, do pay attention at how the API works; of course you can't be sure that the service doesn't have any vulnerabilities, but if (e.g.) the API runs over unencrypted HTTP and transmits the API key in plain sight, then that's not something you want to rely on.

Solution 2:

If you don't have static IP then you should forget about mail server if you going with DDNS solution, most email servers either would reject emails from you or will tag email with highest spam level since all dynamic IPs are in PBL lists. (You can see more details in P.S. section why it isn't a good idea to have emails server on residential IP, but there are still a workaround way by using intermediate cheap VPS(virtual private server))

Regarding "DDNS yourself" - a good domain registrars providing for free IP updating via their API, all your program need to do is to verify periodically public IP and if it changed send new IP to registrar who will update A(AAAA) record. BTW, most nowadays routers already have such feature (watch for IP and report to DDNS provider)

I've read that it may take several hours to propagate throughout the system/world

It depend on DNS provider, respectful registrars allows to set TTL (time that tell others how often IP might be changed) equal to 5 minutes. Not all forwarding intermediate DNS servers following this to avoid high loading, but usually even if they won't follow domain owner TTL, that rarely go longer than a few hours. Most of forwarders will update their caches as you would set in domain TTL.

are there any security issues I may be overlooking with having a DDNS provider?

Going online it is already possible security issue. Isolate your server from local network to avoid unwelcome guests.

Does anyone have an informed opinion with regard to which method (paid vs. DIY) might be better?

You will throw your time and money to the air if you would go with DDNS. Nowadays you can get decent VPS (virtual private server) for 3-4 bucks per month. While web site (if you plan to have one) can be hosted directly on VPS since usually it doesn't take a lot of space, email server could be problematic if you expect to run your server for a long time or expecting high volume of emails. Usually 20GB space enough for small businesses up to 3-5 years even without deleting old emails. Even if you expecting huge amount of emails, you can use nginx feature to proxy email traffic to your home. So you can host primary email server at home on dynamic IP and VPS (that has static IP) will proxy incoming/outgoing traffic to your home. You can use your own VPS in such configuration without pain since there no need to worry about DNS propagation, domain will be always pointing to static IP of VPS. You still need to manage reporting your home IP changes to VPS, so VPS would know where to proxy traffic, but it is much easier, just query some URL on your VPS and parse in logs your incoming IP and adjust nginx, so it always know where you are.

P.S.

I can see that this topic is interesting for superusers, so I would add a little more details.

PBL lists holds database of IPs that generally are dynamic IPs, so PBL helps a lot to operators of emails servers. It isn't a technical issue or ISP are a bad guys to not allow email server on dynamic IPs, the issue is that most email traffic from dynamic IPs are coming from infected computers that sending spam or malware at huge volume that can easily DDoS receiving server if one is a target. Some ISP blocking outgoing connections to port 25 to prevent spreading of malware and DDoS, but some doesn't. Practically all corporate email's server simply dropping connections that comes from PBL list that reduce spam significantly.

The second one effective antispam solution is to drop connections from IPs that doesn't have reverse PTR record in DNS and doesn't match domain's DNS record. Even if connections comes from static IP that haven't PTR record, it usually either a badly misconfigured setup or mostly it comes from servers running by spam gangs (there could exclusions for some big (but careless) providers, but they can be added manually in whitelist). While it is a matter of a few minutes to set reverse PTR record on VPS, it isn't a case if static IPs obtained from ISP and process to set PTR is usually a PITA (one have to call them, submit a ticket after verification that you original owner of IP and wait for mercy of their sysadmin who need to set reverse PTR record, sometimes in a few hours, but sometimes a days)

Also, not a critical, but... to avoid email forging, most email servers owners using so called SPF(sender policy framework) that allows to specify most fast policy processing method if one set in DNS authorized IP addresses that allowed to send emails on behalf of domain. (one can specify authorized servers by FQDN as reference to MX record, but it extra round trips over DNS for connecting servers) So, managing floating IP in DNS wouldn't be a fun.

Solution 3:

I have an ISP that doesn't offer static IP addresses, so it looks like some sort of Dynamic Domain Name Service (DDNS) is the solution.

That's one solution. As an example of another solution, a HurricaneElectric.net IPv6 tunnel provides a static (IPv6) address with a movable tunnel endpoint. Granted, at this time, IPv4 would be nicer to support for such a functionality with the generality public, but if you can find a willing cooperative computer, you could technically do such a thing with IPv4 too.

you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using

This sounds like a technically solid plan.

I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically...someone let me know if I'm wrong on this and there's a simpler way).

The exact details would depend on the domain name registrar's choice of how they implement this feature. Some may use an API key of some sort, while others might rely on a web interface for automatic updates. In the old days, some ISP's provided such a service, but relied on manual changes in response to requests. So it's entirely up to whoever provides you with the service.

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address).

Bah humbug. DNS propagation has been known to take minutes or hours or days (e.g., 72 hours). However, when people have heavily analyzed things, they've found that much of that vague "propagation" time was simply from a DNS hosting provider being slow to update.

In better theory, you should just need to wait for the TTL value. Although, there is a problem with that theory...

However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

Okay, here's the reality: For your update to take full effect, you'll need to have the Internet flush its active cache of old information.

According to the standards, caching DNS servers may rely on their cache for the length of time specified by a TTL value that you can configure.

However, the reality is that at least some (and maybe even most?) very large ISPs have been known to run their own caching DNS servers which have been known to just completely ignore the TTL values. They do this because they feel like if they update their DNS caches less frequently, the overall effect will be less bandwidth (and maybe some less computing time).

So, any E-Mail server that relies on such a DNS server may be affected, and be unable to notice your updates until the DNS server gets updated. In some cases, that may take a day or two (or three?).

However, such effects have become increasingly rare. In actual practice, most DNS servers will have their caches flushed within an hour or two.

Since some caches won't be updated quite as quickly as some others, the effect is that some places on the Internet will work with the new address, while other places will still try to use the old address. Within a couple of hours, most computers will work just fine with the new information. (Many, many of them may work within minutes.)

The typical behavior of E-Mail software is to try to send the E-Mail. If that fails, then try again later. E-Mail servers will typically keep retrying (maybe about once an hour) for days before giving up. So what's likely to happen is that you won't lose E-Mail, but it will get delayed a bit.

Alex's comment "all dynamic IPs are in PBL lists" is clearly wrong, as this information is de-centralized (so the word "all" is inaccurate), but it is true that many dynamic IPs are in such lists, and so that may mean that some computers/devices related to E-Mail may decide to not cooperate with you.

Also, I have another concern: are there any security issues I may be overlooking with having a DDNS provider?

The biggest concern will be whether your updates are handled with a secure fashion.

Won't they be able to monitor all the traffic flowing through the domain name they provide?

No. The DNS server's job is to receive a request for a domain name, and provide a response. The traditional typical response is to provide one or more IP addresses. Other responses are possible, such as referring to another DNS server or domain name (e.g., with a CNAME), or other data (e.g., helping to provide security through the newer DNSSec standard).

Does anyone have an informed opinion ...

I'd like to point out that if you really want to run a serious E-Mail server, you may want to consider being compliant with modern E-Mail standards. That involves more than just being compliant with SMTP and DNS technical specifications. Many people use large providers, and those large providers may implement their own expectations.

For example, I know of an E-Mail server that was set up with Debian and Postgrey years ago. Postgrey is some software that provides "greylisting" anti-spam handling. However, the version of Postgrey that is used assumes that when an E-Mail server retries the E-Mail, the sending E-Mail server will use the same IP address when doing so. Office 365 E-Mail servers have been known to retry sending an E-Mail from a different IP address that is still within an IPv6 /64 subnet. Postgrey doesn't like that.

As more and more organizations have switched to Office 365, this has become more and more of a problem for people using that old E-Mail server. A newer version of the Postgrey software has been released, but the easy way to install such software is to use the official software repository for that operating system. So, in practice, the smart way to update that software will be to upgrade the operating system.

There are other conventions, such as having DNS names that start with "mail." which can cause your setup to be judged as being more or less trustworthy. This may impact whether devices treat you like a non-compliant spammer, or like a device that is worth communicating with.

Sure, maybe when speaking very strictly about official technical specifications, giant organizations are performing some actions that are different than minimum requirements demanded by the RFC documents that contain the technical specifications of the protocols being used. But if you want to communicate with the larger Internet community, there are some additional standards that get imposed by some significant/large players. Be prepared to meet those standards well, or be prepared to encounter some troubles.

I'm being a bit vague about exactly what all those standards are, because they can change over time.

Regarding that old E-Mail server that will need to upgrade its old Debian operating system, maybe people should be upgrading their operating system more frequently anyway. The point I'm making, though, is that a software setup that worked perfectly well for years is now broken, because of newer behavior that is commonly being used by many E-Mail addresses. If you try to do unusual things, like using Dynamic DNS on a slower Internet provider, you may be more likely to encounter some extra problems along the way. As you sound ambitious, maybe you can invest the efforts into that. I'm just warning you to prepare to need to do that.

... with regard to which method (paid vs. DIY) might be better?

As others have pointed out, paid will be much easier, and is pretty economical for most people. Large provides are likely to provide a stable IP address that you can have your MX record point to (so E-Mail goes there), and may provide notably better bandwidth.

DIY is better for gaining experience and learning how things work, and choosing not to rely solely on implementations from major corporations. Having more control over your implementation can also allow you to make significant custom changes much more rapidly.

Which is "better" will depend on your individual goals, so I leave such conclusions up to you.