Fly.io! Where the Web is! Totally Rad!

I built the most basic MCP server for flyctl I could think of. It took 30 minutes. MCP, for those unaware, is the emerging standard protocol for connecting an LLM (or an app that drives an LLM in the cloud, like Claude Desktop) to

It’s weird to say this out loud! For years, one of our calling cards was “developer experience”. We made a decision, early on, to be a CLI-first company, and put a lot effort into making that CLI seamless. For a good chunk of our users, it really is

We’ve spent too much time talking about security tokens, and about Macaroon tokens in particular. Writing another Macaroon treatise was not on my calendar. But we’re in the process of handing off our internal Macaroon project to a new internal owner

The basic idea of our service is that we run containers for our users, as hardware-isolated virtual machines (Fly Machines), on hardware we own around the world. What makes that interesting is that we also connect every Fly Machine to a global Anycas

A couple years back, we put a bunch of chips down on the bet that people shipping apps to users on the Internet would want GPUs, so they could do AI/ML inference tasks. To make that happen, we created Fly GPU Machines. A Fly Machine is a Docker/OCI

Question 1: Why, JP? Just why? LOL. When I looked at what I wanted to see from here in the next 3-4 years, it didn’t really match up with where we’re currently heading. Specifically, with our new focus on MPG [Managed Postgres] and [llm] [llm]. Edit

Over the last 5 years, we’ve done pretty well for ourselves writing content for Hacker News. And that’s mostly been good for us. We don’t do conventional marketing, we don’t have a sales team, the rest of social media is atomized over 5 different sit

This whole paragraph is just one long sentence. God I love just random-ass blogging again. This bit by Geoffrey Huntley is super interesting to me and, despite calling out that LLM-driven development agents like Cursor have something like a 40% succe

We’re interested in getting integrated into the flow VSCode uses to do remote editing over SSH, because everybody is using VSCode now, and, in particular, they’re using forks of VSCode that generate code with LLMs. ”hallucination” is what we call it

Let’s begin by introducing our cast of characters. Livebook is usually described as Elixir’s answer to Jupyter Notebooks. And that’s a good way to think about it. But Livebook takes full advantage of the Elixir platform, which makes it sneakily powe

Public cloud billing is terrifying. The premise of a public cloud — what sets it apart from a hosting provider — is 8,760 hours/year of on-tap deployable compute, storage, and networking. Cloud resources are “elastic”: they’re acquired and released

We just lowered the prices on NVIDIA L40s GPUs to $1.25 per hour. Why? Because our feet are cold and we burn processor cycles for heat. But also other reasons. Let’s back up. We offer 4 different NVIDIA GPU models; in increasing order of performanc

At the heart of our platform is a systems design tradeoff about durable storage for applications. When we added storage three years ago, to support stateful apps, we built it on attached NVMe drives. A benefit: a Fly App accessing a file on a Fly Vo

Let’s hypopulate you an app serving generative AI cat images based on the weather forecast, running on a g4dn.xlarge ECS task in AWS us-east-1. It’s going great; people didn’t realize how dependent their cat pic prefs are on barometric pressure, and

Picture this, if you will. You’re blind. You’re in an unfamiliar hotel room on a trip to Chicago. If you live in Chicago IRL, imagine the hotel in Winnipeg, the Chicago of the North. You’ve absent-mindedly set your coffee down, and can’t remember w

One of many odd decisions we’ve made at Fly.io is how we use WireGuard. It’s not just that we use it in many places where other shops would use HTTPS and REST APIs. We’ve gone a step beyond that: every time you run flyctl, our lovable, sprawling CLI,

Fly Kubernetes is the “blessed path"™️ to using Kubernetes backed by Fly.io infrastructure. Or, in simpler terms, it is our managed Kubernetes service. We take care of the complexity of operating the Kubernetes control plane, leaving you with the unf

There are three hard things in computer science: Cache invalidation Naming things Doing a better job than Amazon of storing files Of all the annoying software problems that have no business being annoying, handling a file upload in a full-stack a

GPUs are now available to everyone! We know you’ve been excited about wanting to use GPUs on Fly.io and we’re happy to announce that they’re available for everyone. If you want, you can spin up GPU instances with any of the following cards: Ampere

Serverless is great because is has good ergonomics - when an event is received, a “not-server” boots quickly, code is run, and then everything is torn down. We’re billed only on usage. It turns out that Fly.io shares many of the same ergonomics as s

There are many ways to delegate work in web applications, from using background workers to serverless architecture. In this article, we explore a new machine pattern that takes advantage of Fly Machines and distinct process groups to make quick work

Let’s implement an API token together. It’s a design called “Macaroons”, but don’t get hung up on that yet. First some throat-clearing. Then: Wrap text Copy to clipboard import sys impor

Hello all, and welcome to another episode of How I Fly, a series where I interview developers about what they do with technology, what they find exciting, and the unexpected things they’ve learned along the way. This time I’m talking with Yoko Li, an

Some people daydream about normal things, like coffee machines or raising that Series A round (those are normal things to dream about, right?). I daydream about commanding a fleet of chonky NVIDIA Lovelace L40Ss. Also, totally normal. Well, fortunate

Before proceeding, a caution. This is an engineering trade-off. Test carefully before deploying to production. By the end of this blog post you should have the information you need to make an informed decision. Introduction Alpine Linux is a Linux

Update, March 2024: FKS does more stuff now, and you can read about it in Fly Kubernetes does more now We’ll own it: we’ve been snarky about Kubernetes. We are, at heart, old-school Unix nerds. We’re still scandalized by systemd. To make matters mor

AI is pretty fly AI is apparently a bit of a thing (maybe even an thing come to think about it). We’ve seen entire industries get transformed in the wake of ChatGPT existing (somehow it’s only been around for a year, I can’t believe it either). It’s l

GPU hardware will let our users run all sorts of fun Artificial Intelligence and Machine Learning (AI/ML) workloads near their users. But, what are these “GPUs” really? What can they do? What can’t they do? Listen here for my tale of woe as I spell

Open-source self-hosted AI tools have advanced a lot in the past 6 months. They allow you to create new methods of expression (with QR code generation and Stable Diffusion), easy access to summarization powers that would have made Google blush a deca

Imagine if you could auto scale simply by wrapping any existing app code in a function and have that block of code run in a temporary copy of your app. The pursuit of elastic, auto-scaling applications has taken us to silly places. Serverless/FaaS

The topic of “AI” gets a lot of attention and press. Coverage ranges from apocalyptic warnings to Utopian predictions. The truth, as always, is likely somewhere in the middle. As developers, we are the ones that either imagine ways that AI can be use

Scaling discussions often lead to recommendations to add more memory, more CPU, more machines, more regions, more, more, more. This post is different. It focuses instead on the idea of decomposing parts of your applications into event handlers, sta

Previously when you ran fly launch, you got asked a bunch of hopefully relevant questions to help you get your app up and running. We’ve taken a lot of the guesswork out of the process and made it a lot more streamlined. It turns out that even though

I’m Xe Iaso. I’m a writer, technical educator, and philosopher who focuses on making technology easy to understand and scale to your needs. I use Fly.io to host my website and in nearly all of my personal projects now. Fly.io allows me to experiment

Fly.io has GPU Machines, which means we can finally run AI workloads with just a few API calls. This is exciting!

My favorite part about building tools is discovering their unintended uses. It’s like starting to write a murder mystery book but you have no idea who the killer is! History is filled with examples of these accidental discoveries: WD-40 was original

We’ve been using Sentry since the dawn of the internet. Or at least as far back as the discovery of the Higgs boson. Project to project, the familiar Sentry issue detail screen has been our faithful debugging companion. Today it’s no exception: All

When we started the LiteFS project a year ago, we started more with an ideal in mind rather than a specific implementation. We wanted to make it possible to not only run distributed SQLite but we also wanted to make it… gasp… easy! There were hurdle

You’ve done everything right. You are well aware of Murphy’s Law. You have multiple redundant machines. You’ve set up a regular back up schedule for your database, perhaps even are using LiteFS CLoud. You ship your logs to LogTail or perhaps some o

We built some little security thingies. We’re open sourcing them, and hoping you like them as much as we do. In a nutshell: it’s a proxy that injects secrets into arbitrary 3rd-party API calls. We could describe it more completely here, but that woul

Bun 1.0 comes out September 7th. Fly.io is making preparations. Previously,

Generate your own 90s page here!