Building your own DDOS protection and CDN will involve a lot of devops bandwidth in ensuring both low latency and high availability. You may need to negotiate good rates with your ISP/VPS/Cloud for network bandwidth. It will also involve keeping in sync with security fixes and the state of the art in terms of bot protection, etc. If this use case is not a core part of your business, it will be better to bite the bullet and go ahead with a 3rd party solution like Cloudflare / AWS Cloudfront + WAF + Route 53 / Google Cloud Armor / Fastly

Openresty with a few Nginx modules and Lua scripts can go a long way for many use-cases mentioned:

- ModSecurity for WAF: https://github.com/SpiderLabs/ModSecurity

- L7 Rate limiting: https://github.com/openresty/lua-resty-limit-traffic

- Cache: https://github.com/ledgetech/ledge . Or use varnish which gives you VCL for high configurability

- Load balancing: https://github.com/openresty/lua-resty-balancer

- CDN: https://github.com/taythebot/lightpath . This project seems to be a WIP which you can use as a starting point for your needs. You will also need to find good enough "edge" locations for your CDN.

Similarly, HAProxy does a lot of stuff with the correct config and is also extensible using Lua:

- Some basic DOS protection: https://www.sysbee.net/blog/haproxy-sysadmins-swiss-army-kni...

- L7 DDoS protection: https://github.com/mora9715/haproxy_ddos_protector

You can run your own authoritative DNS server using either djbdns or nsd. Or use AWS Route 53.

Ultimately, it will involve (a lot of) glue code/config depending on what solution you go ahead with.