There are a lot of considerations that our canary deployment server doesn't cover. We went over a number in the What aren't we doing? section earlier - things like more than two servers for the backend, and health checks.
Most of the concepts we glossed over are related to the load balancing aspect of this system. Ours is not designed to be an incredibly scalable or feature-full load balancer. That aspect could certainly be improved.
For example, if one needed this load balancer to have some aspect of reproducibility for requests, a new load balancing method would be necessary. It could potentially hash information about the request, like the IP or a combination of headers. This would allow it to remain fast, but also allow someone looking at the system to understand where a request would have been routed. That's something that can't be done with random per request, due to its randomness, or round robin, due to its statefulness.