You said something interesting about the preciousness of our development environments… And I’m with you that we’ve commoditized the servers, but we definitely have not commoditized dev, because it’s so intricate, it’s so set up… Sometimes it’s like “There be dragons. Please don’t touch my laptop, because it works right now, but I’m not sure if it’s gonna work tomorrow.” I do hate that. I think it’s almost a different skillset, of maintaining that. There’s overlap between development and the maintenance of a development environment in terms of things that you need to learn… But it’s almost a different task altogether. So I don’t like that about it, but it’s still very true that our development environments are precious to us, and they’re tweaked, and configured, and customized, and all the things. So I’m sure there’s probably lots of resistance to this…
[00:11:59.29">00:11:59.29] We talk about our setup - we have probably tens of thousands of lines of code, and very few dependencies in our stack, but GitHub is 14 years old, and there’s a million plus commits, and I’m sure the dependency list is very long… What kind of effort was this? Tell us the story of bringing it along.
It is. These are all very, very true points. You know, the last thing I wanted to do was kind of be the vessel that went out to GitHub and said “I wanna change your development environment”, because these things are so precious. Like, I’m an engineer, too. I think my environment is very much precious. And here I was, kind of the face in GitHub of saying “Well, we think we have a better way. Come join us over here.”
And I started off on this journey as a skeptic. I think I shared some of this, too… I didn’t think this would be a fruitful journey necessarily. I was just gonna go do my level best as an employee, see if I could make it happen, build moment etc. and see if I could find something out there. Now, on the other side of this journey, I feel like I’m completely on the other end now, where I’m just like “This is the future. This is the way that we will absolutely build software…”
But going back to the core of the story, it was literally just me out there, calling on my friends to begin with, inside of GitHub. I’d been there for five years, and the first few years were just me tapping into relationships, saying “Hey, can you give this thing a shot? Can you try this out? I wanna get your feedback and feelings about where this is at.” And no one could yet use it on our core repository. We call it github/github - the organization is GitHub, the repository is GitHub. We didn’t have this thing standing up in a Codespace yet, but we had other repositories that were compatible with Codespaces.
So I’d go out and ask favors of friends, and just be like “Can you try this out and give me some feedback?” And generally, the feedback I would get back was – first it was resistance, like “Why would I do this? It’s productivity lost; tax on productivity. I don’t trust HTTP. There’s gonna be lag”, that kind of feedback. But then people would try it and they’d come back and be like “Huh. That was maybe better than I thought.”
At the same time, as I hacked in this space too, I was starting to get some of that “Well, there’s something here.” The big a-ha moment for me was connecting VS Code into my Codespace out in the cloud and still retaining that local development experience. So it felt to me like it was still very local. The magic is the synchronization that’s happening between the local environment and the cloud. It feels totally transparent.
But that aside, it started with just a very small number of users. So we would go back to leadership in GitHub and talk about progress we were making… And the early days, the story was “I have five people that have responded positively to Codespaces.” So not much of a story, but starting to kind of make a little bit of progress. And then maybe it was ten people.
Then, the next iteration on this was like “Well, let’s go find a team. Let’s get a full team on Codespaces. How can we get a single team - 6 to 8 people - committed to using Codespaces, and stick in this thing?” At this point we’d had this other effort running on the side to get github/github, the core github.com repository, compatible with Codespaces. And we’d gotten it to a point – we detail how we did this in the blog post - where performance was mostly acceptable. So now we could go shop this with a team that worked primarily on GitHub.com and see what their experience was. And we’re making progress there. So we’re ramping in – I think y’all have talked to Kyle Daigle in the past. Kyle was the leader of that effort that got this team spun up inside of Codespaces on GitHub core. And again, it was somewhat retentive. People were sticking, and going like “Wow, this is not what I thought. It’s better than maybe what I thought.”
[00:15:59.11">00:15:59.11] But I think the real breakthrough moment came when we stopped calling this dogfooding. You hear this term all the time, dogfood… I think it actually originated – I looked up on Wikipedia; I think the term originated inside of Microsoft a number of years ago.
But GitHubbers, my colleagues don’t respond well to that term. Dogfooding doesn’t inspire anyone to go do anything. Just like “Eat the dogfood? Who feels good about that?” And so what we did was we launched what we called the GitHub Computer Club, and I would love to dedicate a full episode on this. It’s a really interesting concept, and something I hope to bring out to the industry. But we asked people to join the GitHub Computer Club. And in doing so, they took this commitment or oath. I wrote up this script, “I do solemnly swear to never – no shadow compute, not desktop compute. I’ll join this thing and forever be member of the elite, exclusive GitHub Computer Club.”
People straight up were just like, “This is great. Let me in. I want a membership card.” And in doing so – we had to give them something in return. So they would join the computer club, but we offered to our “exclusive” members what we call the concierge team. And this team was built to kind of support their productivity and success inside of Codespaces.
So the second these people had friction - you know, one of the requirements of entering the computer club was that you had to kind of raise your hand. You couldn’t disappear and go back to local desktop. You had to virtually raise your hand and say “I’m about to opt out of this, because Codespaces can’t keep my business right now.” And the concierge team that we had built could swoop in, respond to “What’s going on here? Let’s dig into it. Why can’t we keep your business in Codespaces?”
We continued to play that model back and forth between Computer Club and concierge team, until we had built the product and built enough momentum inside of GitHub that one day we kind of looked around and we were like “Wow, we have hundreds of people developing GitHub.com and GitHub Codespaces.” And I think the real story there is just commitment to make it happen. We want it to be successful with this, and not just go talk about it in the market, but actually show that this is a better tool for us. The computer club is still going strong. People are demanding that I give them satin and denim jackets; I’ll get around to that at some point.
All the C’s. I like this aspect because you treat this like a customer scenario. You built a product, and you have to retain customers. And you’re actually exercising a great principle for anybody building a product, which is “Talk to your users.” And when they have trouble - swoop in, as you had said, understand those problems and be committed to fixing them. I think that’s a great way, a great story for how Codespaces became powerful inside of GitHub, because that’s exactly how you build a product. Not just “Let’s just try this thing and hopefully our internal team adopts it by force.”
As you had said, you wanted to go along with your employee card and be able to see if Codespaces could work, and out the other end you became a believer. But you’re not forcing GitHub engineers to use it, you’re asking them to try it. In this case, the Computer Club, with the oath… And then as you said, you look up and you see hundreds now.
I think that’s right. The position was – no Fiat. We didn’t wanna lead with “You have to do this.” That’s the absolute wrong way to get adoption in your product. We wanted to literally win the business of our colleagues. We wanted to build such a fantastic experience in Codespaces that people would choose it. And yeah, I think the Computer Club probably boosted adoption a little bit, do doubt about it… But what made that work –
It had to have a soul. It needed some soul behind it, that was the idea. And the fact that we did respond to this – we actually did win business. When things didn’t go well and when people wanted to opt out, they could, and they would, for a week, or whatever… But the goal was “How do we get them back in here, kind of remove whatever that impediment is, and get them productive in Codespaces again?”
Well, you know, we leave that intentionally vague, so people can assume the worst. No, I don’t know that we’ve had any real regression there just yet, which is good. Codespaces is super-retentive. I think we have people from time to time use local desktop. We have a colleague – this is actually in the blog post maybe… A colleague of mine reported the other day, she said “I was using local development. My environment broke, so I switched over to Codespaces.” And she was like “I actually shipped my task in my Codespaces before my local development environment rebuilt.” I think everyone was like “Wow, that is such a good story.” And it’s so true. It’s kind of the experience we’re all having right now with Codespaces.
We talked about it, again, in the blog post - you click a button and the environment is live. So for every new engineers that joins GitHub, I think they all are probably fairly spoiled at this point, because day one they click a button and they’re able to run that entire GitHub.com environment. It’s just been really incredible to watch.
So Cory, the way you’ve explained the flow of this GitHub Computer Club seems a little smooth. I’ve gotta imagine you hit some friction. Can you share some of the struggle that you hit? Some opposing forces in the process of rolling this out.
Yeah. Basically, it started with a bunch of “No” throughout GitHub. I think people had seen previous iterations of Codespaces… We announced it, I think, in May of 2020, at GitHub Satellite.
It’s been out there for a while… And I think when people first try to use it inside of GitHub, there was a bit of friction. It didn’t work for them, and I think first impressions can sometimes be lasting impressions. So when I went out there, I’m just like “Use this thing. It’s great. It’s really evolved. We feel pretty proud of it”, and it was just a bunch of “No” left and right. So then it became “How are we gonna build this business?” And yes, the Computer Club was a big boost, and the concierge team certainly was a huge, probably the most high leverage practice we discovered along the way… But a lot of this was just like startup style practices. We’re building a business inside of GitHub, and I think that’s maybe a useful context for anyone that’s trying to build adoption of their own products in-house; you’ve gotta think of this sometimes as like “This is your own business. How are you gonna build it inside of GitHub, in what is a kind of very stubborn audience?” And I’m a developer, I can say that; we’re somewhat stubborn and we find the tools that work well for us, and if someone comes and says “I wanna change those”, your response is gonna be “Don’t.”
Yeah. And we’ll get to this in a second - one of the great parts about Codespaces is that we just commoditized the compute part of this. The environment is now running somewhere else. But dotfiles, VS Code setting sync, VS Code extensions - we bring those all to the environment. So you don’t lose your curated workbench. If you’ve got a dotfiles repo set up on GitHub right now, we bring that into the compute environment; we bring your environment and your personality, your expression of yourself captured in code into that environment. We bring your W out to your compute, which I think is a really nice touch. So you get the unburdened computer running in the cloud so you freed up your local machine, but you can still bring your preferences into that environment.
[00:23:54.17">00:23:54.17] I digress… Going back to building the business a little bit - it felt like startup tactics. So we had the concierge team, we had the Computer Club… We had effectively guerilla marketing. We were out on Slack every day, looking for opportunities to say “Have you tried Codespaces?” People were receiving M1 architecture Macs, and the github/github build just would not yet work. We had not put in the investment to make the github/github run on the M1 Mac, so we’d say “Hey, have you tried with Codespaces yet?” And people would be like “Well, I guess I’ll try. That feels like my only path right now.” And they’d click a button, they’d come back an hour later, or a day later, and just be like “What in the heck? This is incredible. How was this even possible?” And those people you just win for life. That’s their full mode of operating. So that was the guerilla marketing angle…
We did pairing sessions… So we were up in front of everyone all the time, saying “If you wanna get started, here we are. We’re gonna hold your hand through this and show you the ropes, show you how we’re doing.” Kind of social proof, I think, which is really valuable there. All hands – we’d get in front of the entire company and demo the thing, and be like “Look at this, it’s incredible” and just try to build hype.
We connected with the right people… I maybe loathe to call them influencers, but the people inside of GitHub that every engineer look up to. They look up to them and say like “This is the person that I aspire to be at some point.” We converted them. We want their business. They’re kind of like trendsetters and tastemakers internally. And then really it boiled down to ruthless prioritization. So we listened to our users, “What do you need?” and we demonstrated that we could follow through on those things. For some reasons, someone was trying to run some arcane karma test somewhere that wasn’t executing for them. It’s just like “Alright, great. Let’s figure out how to make sure that works in this environment.” That kind of thing. Even small tasks like that were important in building momentum.
And then I’ll say it again - one day we just looked up and we’d gone from a bunch of “No” to a bunch of super-fans inside of GitHub. We have cheerleaders. If you go out and look on Twitter right now, the day after we kind of announced Codespaces to the world, they were just like – GitHubbers were out there very enthusiastic about the thing, and it was a very genuine response. We didn’t ask anyone to go do that. People were just that enthused about what we built.
Yeah. I saw a tweet from Kelsey Hightower - again, I’ll mention Kelsey… I don’t know if this tweet was actually towards Codespaces or the announcement, but the timing - it was the same day, I believe, so I think it was a subtweet around it, but he said “Back in the day we wrote code on our own computers.” So I’d assume that he was reflecting on Codespaces and the announcement, but I wasn’t sure of that.
I saw that, too. I mean, you used to run your server in a grey tower, beige tower underneath your desk too, right? Those days are gone, it kind of feels like. This is this next wave - we’re now moving development environments out into the cloud. It just feels to me like two years from now we’re gonna see some incredible adoption in this space.
You know, Nat holds a very high bar. I remember, as we were trying to get GitHub running inside of Codespaces, I’d go back to Nat and we’d show him “Hey, now instead of 45 minutes it’s 20 minutes. We’ve made these changes.” And he was like “That’s super-cool. Not good enough.” And we totally agreed, we’re like “Yup, it’s not good enough, but I just wanted to show you progress.” We’d get that feedback, and then we’d come back again and say “We’re down at ten minutes.” “That’s great. It’s not good enough”, and everyone’s like “Yeah, you’re right, it’s not good enough. It’s gotta be seconds for it to be the experience we want.” That was kind of the iterative experience.
I think Nat has been a believer in where this thing could go, from kind of the outset of the journey. It’s just been a bit of a slug as we worked from the very early days of like “Look, we have all this tech orchestrated that can produce this effect of a Codespace”, maybe the early prototype, down to now the ten second story inside of GitHub. That didn’t happen overnight.
[00:28:08.25">00:28:08.25] But the good news is most of that - almost all of that now - has made it into the product itself. So the changes that we’ve discovered along the way didn’t just benefit github/github and the GitHub.com repository, it benefitted the entire product. I think Nat’s a super-fan now. I’ve got some screenshots from Nat that I look at from time to time, that keep me pretty enthused about the progress we’ve made.
![The GitHub Codespaces Story [Cory Wilkerson]](https://img.transistor.fm/ehfVRtOTdvanujsOR_PvWAZpieYtHwCRnQtepj-ExmI/rs:fill:800:800:1/q:60/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzE3NjMxLzE2MjY4/NDU3NzgtYXJ0d29y/ay5qcGc.webp)