PostHole
Compose
Login
SE Radio 621: Xe Iaso on Fly.io
Xe Iaso of Fly.io discusses their hosting platform with host Jeremy Jung. They cover building globally distributed applications with Anycast, using Wireguard to encrypt inter-service communication, writing custom code to handle load balancing and scaling with fly-proxy, why serving EU customers has unique requirements, letting users use docker images without the docker runtime by converting them to firecracker and cloud hypervisor microVMs, the differences between regular VMs and microVMs, challenges of acquiring and serving GPUs to customers. when to use Kubernetes, and dealing with abuse on the platform. Brought to you by IEEE Computer Society and IEEE Software magazine.
Xe Iaso of Fly.io discusses their hosting platform with SE Radio host Jeremy Jung. They cover building globally distributed applications with Anycast, using Wireguard to encrypt inter-service communication, writing custom code to handle load balancing and scaling with fly-proxy, why serving EU customers has unique requirements, letting users use docker images without the docker runtime by converting them to firecracker and cloud hypervisor microVMs, the differences between regular VMs and microVMs, challenges of acquiring and serving GPUs to customers. when to use Kubernetes, and dealing with abuse on the platform. Brought to you by IEEE Computer Society and IEEE Software magazine.
Show Notes
SE Radio Episodes
Transcript
Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.
Jeremy Jung 00:00:18 Today I am talking to Xe Iaso. Xe is the senior techno philosopher. I think I got that right at Fly.io and the last time we spoke it was about tail scale, but today we’re going to talk about Fly. So Xe, welcome back to Software Engineering Radio.
Xe Iaso 00:00:33 Thanks for having me back. Great to be here.
Jeremy Jung 00:00:35 The first thing we should start with is just defining what Fly.io for those who haven’t heard about it. So why don’t we start with that?
Xe Iaso 00:00:43 Fly.io is a way to run your app as containers across 35 data centers in one command the same way you would to deploy to one data center. You package your thing up as a docker program, configure its http port press button, receive server.
Jeremy Jung 00:00:59 There’s so many different hosting providers today. What would you say are the main unique parts about Fly? I know you just touched on the multiple locations.
Xe Iaso 00:01:10 Probably the BGP Anycasting and the private networking. BGP Anycasting is a technique that basically lets you use the entire internet as your load balancer or you declare you advertise the same IP range from multiple data centers and then the public BGP constellation will just intelligently route to whichever is the closest in internet geography speak for me, all of the stuff goes to the data center down in Toronto even though the Montreal data center is technically closer. BGP is weird. And the other big thing is private networking powered by Wireguard so that all of your internal traffic between regions is encrypted over the wire and there’s no egress charges going from region to region internally.
Jeremy Jung 00:01:54 And you were saying how Montreal was closer to you, but it went to Toronto. Is there some kind of, is it Anycast that’s responsible for finding what it thinks is the shortest or the quickest path?
Xe Iaso 00:02:06 Well, it’s less Anycast and more my ISP figure out in the BGP bid, which path is the shortest. I think that the connection from where I am in Ottawa to the Montreal Data Center might leave Bell’s network boundary. And I don’t know how Bell’s network works internally, but I’d be willing to guess that crossing a network boundary is more expensive than not having to, so that’s why it’s sending it down to Toronto.
Jeremy Jung 00:02:31 That’s sort of interesting. So it’s not even something that you control, it’s these ISPs may decide to route this traffic somewhere that’s not necessarily the fastest but might be cheaper to them.
Xe Iaso 00:02:43 Yes. BGP routing is an absolute nightmare. Don’t look into it. If you want to sleep well at night, let the experts handle it. That’s the Hakuna Matata way to deal with network engineering. Yeah.
Jeremy Jung 00:02:52 You mentioned really briefly about connections from region to region. So people who’ve worked with hosting services, they’re used to, let’s say Amazon, they have things like VPCs and ways of creating private networks. What is different about Fly’s particular implementation?
Xe Iaso 00:03:11 VPCs are very opinionated things. They let you basically have any kind of network topology you want for better and for worse. The way that Fly.ios private networking differs is that it’s very opinionated. It is IPV six only and you don’t get to control the subnet, but it’s in the IPV six unique local address space. So statistically you’re not going to be interfering with it. But you know how randomness works. I have it set up so that one of my machines in my home lab is a router from my other private network setup to my Fly.io private network and I can use it to poke things like my large language model inference server from basically anywhere without having to worry about encrypting traffic on the wire.
Jeremy Jung 00:03:53 If I understand correctly, your applications themselves don’t necessarily need to encrypt the traffic when you’re doing something from like your home lab to this instance in Fly.io’s data center because Wireguard or whatever infrastructure you have is taking care of the encryption.
Xe Iaso 00:04:12 Yes. It does mean that sometimes you have to tell applications, yes it’s okay, it’s actually encrypted, trust me bro. But it works out a lot better than you’d think. Removing problems from the application layer actually makes your life easier because it decreases the way that things can go wrong and makes it easier to tell where the things went wrong.
Jeremy Jung 00:04:31 And you’ve touched on how Anycast allows you to have a single IP address and routes you to potential different instances around the world. Maybe we can go into the details of let’s say you have an application and somebody connects to it. Maybe you could walk through what the steps are from there, if that makes sense.
[...]