SSL for devices in local network
Step 1: Make the device generate a self-signed certificate for its IP address and/or hostname on first setup or when the IP address is changed unless there is a customer-provided certificate in place. DO NOT sell devices with a common certificate (see the Let's Encrypt article you linked).
Step 2: Make your documentation describe how to recognize your specific self-signed certificates and accept them in the common browsers.
Step 3: Make your documentation describe how to replace the self-signed certificate with another one generated by the organization's own Certificate Authority or by a public CA.
I deal with this issue extensively for my current job - including high emphasis on the works-offline issues. I'm coming at the question with the precondition that enterprise-ish elements like adding a custom root CA are not appropriate or impractical.
First thing to remember is that if they do not have a DNS server available when offline - TLS for private (RFC 1918) IP addresses without manual modification and poking doesn't work. If this is a real concern that cannot be worked around - documentation and user training are your only real options.
If this is deployed beyond your physical control - wildcard certs are a scary proposition. A hacked-and-extracted wildcard cert can be used by a malicious party to do a lot of damage.
Also remember that more browsers do not accept long certificate expiration windows (see this for a recent Apple/iOS one - limiting a certs lifetime to a year-and-a-bit)
I'd recommend investing in infrastructure to automate deployments of certificates. We also have the luxury of controlling a small dnsmasq server on the device that provides the DNS and DHCP aspects required - again controlled by deployment automation that can ensure that it is up to date. Without knowing more about your solution and sales model its hard to tell if that would work for you. ie: our solution is not static and has a small 4G gateway - so internet losses are fine so long as it gets to talk once every few months.
IMO Puppet is good for internet constrained devices (as opposed the current dev-ops darling Ansible) as the agent aspect allows the device to attempt to deploy itself when it has an opportunity - and works well behind NAT'ted IPv4 gateways.
You're over thinking this. All you need to do is make HTTPS an available feature to be switched on should customers require it. In doing so, HTTP is disabled or better still, connections to the HTTP port of the device will be redirected to the HTTPS port.
To make HTTPS an available feature on your device it needs to be able to handle a certificate file or certificate chain and serve HTTPS with that cert. You don't need to generate your own certificates or handle anything in this area really. Just allow customers to be able to upload their own certificates to your device and reconfigure the device accordingly, then you're basically done. The DNS side of it is NOT your concern.
HTTPS (TLS) requires a DNS infrastructure to work properly, or as a minimum a correctly configured host file on every computer that connects to your device by HTTPS. Again, that's not your concern as every network is configured differently and that aspect of it is outside your control.
Just provide ample documentation on how to set it up if required. It would never be switched on by default.
This solution is exactly what we do for our device. And it works. In practice very few of our customers use it.
If you want to do some more research into how other devices handle it, look no further than the online help pages of various routers and managed switches.
I'm not sure about your environment, so will make some assumptions, please correct me if I am wrong.
Standard ways of doing this are pushing out a new root certificate with active directory, you can sign as many certs as you want, including for ip addresses, for a public example of a HTTPS IP check out https://1.1.1.1/
That is achievable locally.
If you are running that sort of set up, pushing DNS changes (so people use app.company.com instead) would be easy too, your DNS server should be able to be configured to respond to what ever query you like. Running a local DNS server which only returns internal DNS with no internet access is another part of infrastructure that you would need to maintain going forwards though (assuming its not already there, DCHP should provide you a DNS server, so please look at that first)
If people are using this to develop on, use https://github.com/FiloSottile/mkcert its really easy to use and gives trusted certificates (again by inserting them into all trust stores) - This is an automated version of the first option.
To answer the part about why private ips are excluded, its easier on a private network to re route traffic, if an attacker gets in, they can arp spoof the server, get all traffic to it and then control all inputs to the server (which may contain user logins), without the end user ever knowing. Having a valid cert is much harder to spoof, while the attacker could get the traffic, all users would be shown a "this is not secure" page instead as the TLS step would have failed. Local networks do have requirements to be secure too.
In one of our projects we faced a problem, that we wanted to connect to websocket on a local network from webpage, that was served from the Internet. In order to use secure web socket connection (wss://) we needed the SSL certificate, but you can't get one for local IP's. The bottomline was that we could not change client machines configuration (no /etc/hosts, no importing of own CA's).
Looking for the solution we got inspired by the service xip.io and came up with the idea, that we could use wildcard SSL certificate and slightly change the xip.io's method to put the local IP without dots into the URL - to get the subdomain that is complaint with the wildcard SSL. So we started our custom DNS service, which works like this:
dig 10-0-0-1.my.local-ip.co
;; ANSWER SECTION:
10-0-0-1.my.local-ip.co. 299 IN A 10.0.0.1
And we provide SSL certificates for domain *.my.local-ip.co for free on the website http://local-ip.co
The only downside is you still need internet connection to have it running, but on the other hand you don't have to install or configure anything to get it running...
Cheers, Marcin