Before working at Railway, I used to work at a large enterprise software company as a Product Manager. I liked that job but dreaded the moments where my team shipped a feature and then twiddled our collective thumbs while we waited for the user feedback to roll in. I don’t know about you, but I get energized talking to customers. As such, I would much rather be out and about (figuratively) being with our customers as they use the product to find out if we are truly solving their problems.

That is the reason why I prefer the way Railway works. For those of you who don’t know, our team chats exist alongside our users help chats. This lets us work with our users and break out immediately when they have problems. There are moments where I would accidentally ship a typo, and immediately have one of our wonderful Discord members gently identify where I introduced the error in a matter of minutes. Not only that, but there have been countless times where a customer would go through the effort sharing their issue and, after a brief discussion with the team, we could get their concerns resolved in a few hours with a Pull Request.

Shipping often and having a tight feedback loop is essential for high performing teams. Therefore, we would somewhat lose cadence when we would want to work on features that necessitated user feedback. Imagine you walking to work one day and finding that someone moved all the items on your desk to someone else’s liking, that would suck. Railway is an essential tool, and if you’ve seen the new Metro UI, we really wanted to know if we hit the mark.

However, when we would tease new features on Twitter and our weekly changelog- we didn’t get enough people to respond to our calls for feedback. Who could blame them, it’s so much mental energy imagining in your head how a feature would work? Then, after you have spent the time astral projecting into the world where the feature existed- only then you realize you had to jump into Discord or email or tweet at us to give us input. Too much hassle. Besides, that’s not our style either, we would rather build first and then ask questions later.

Although there is another blog post to be written on how we use Discord for our operations- we figured that it would be great to conditionally grant access to pre-release features to those who ask about them. Considering that most of our community members hang out on our Discord- we decided to create a Discord native way to handle the beta program to keep our customers close, and our feedback closer.

For those who know how we work, the team takes our feature pitches from the discussion channels to Notion where we write an Engineering Requirements Document. I find the process of writing helpful to think through clearly not only the What, but the Why. It was important for us to consider what we needed versus how the proposed feature helps our members.

When writing this document, it became clear that tying the Railway account to a user’s Discord account was the focus. I figured that adding a button that would allow someone to link their account to their Discord account would suffice. This would then prompt the user to log-in to Discord where we would then store the associated tokens. Then, we can query what roles they have in the server, invite them if they aren’t in our Discord server, and add roles from our monolith application.

The good news is that Discord maintains a fantastic API and OAuth flow that anyone can hook into.

We realized that we would need a few crumbs of data from the Discord API to maintain the notion of a User’s Discord account to make these actions possible.

1. The OAuth Access and Refresh Token
2. The Discord User Email
3. The Discord User Id

Initially, we wanted to use a Discord OAuth library but found that Discord and our application weren’t having a great time with some weird URI encoding issues when we added our state string (to protect against CSRF attacks) and went to implementing an OAuth compliant flow ourselves.

After we shipped the Discord Integration, we then focused on being able to read roles from the server. However, we were at a crossroads, naively, we could listen for all the role add events from the monolith and check against those events to see if it tied to the connected user account. Or on the other extreme, poll the Discord server with a connected user account. Neither approach seemed super attractive to us. What we ended up going with is a role caching method where the cache would be invalidated only if we ever needed to query the user’s roles to come up with a value like isBeta.

That way, when we wanted to gate access to a certain page, or show off an option in the command palette- we can lazily check if the user has the correct role. If there was no Discord account attached, then nothing would return when we checked Redis and users went on their merry way.

We then added a field on the User that we could check against a property called isBeta where the result would be computed when accessed. We could then also enforce that check at the Next JS Page level- for Metro, it was as simple as MetroOverviewPage.betaOnly = true;

Initially, we were very cautious handing out Priority Boarding due to expecting users to flood the beta channel with feedback. This would have made it harder for our small team to keep track of issues. When we announced the beta program, we asked users to manually request it. This resulted in situations over the weekend, the team would log in to Discord and add the role. This is bad for work life balance and my ability to climb through the Halo ladder uninterrupted. So an update to Percy, our Discord bot, was needed.

There is a whole blog post to be written about our Discord bot Percy and how essential it is to keep everything from derailing. Unfortunately, these blog posts don’t write themselves. The good thing is that we had a substantial reorganization of Percy’s codebase to help pay down some debt (read: Angelo’s rushed features) that made it more pleasant to extend the bot. We then added the /beta slash command where people could assign themselves the role if they felt adventurous. If users read an old changelog and manually asked about the beta program, Percy would gently DM you to ask if you meant to join the program. Made easy with a button so users can add roles to themselves.

We have had 115 people join Priority Boarding throughout the existence of the program and it’s only been around for a month. The feedback that we have gotten through our semi-exclusive #priority-boarding channel on Metro and the new Logs service has been invaluable. Having users directly talk to us about what’s working has helped us make better decisions on how to design for use cases large and small.

Because of the input from our users- our designer Jitachi organized and cataloged Discord messages and screenshots into items that turn into work items for the broader team. Greg’s new log service used the isBeta gate to build confidence before declaring general availability.

This is just the start: using Percy, we could help catalog pieces of feedback into Notion and Linear tickets. We could also have Discord roles be the source of truth on specific features to gate, and even have our Discord integration spin up help threads for users directly from the product if a user has an account linked. We are bullish on the broader Discord ecosystem and even more so on what our customers build.

From an implementation standpoint, we have a planned update to the implementation to where we can read from the Discord guild member intents to make it so we can optimistically update the user roles using the role additions rather ingesting all of the server events.

If reading this made you feel like you are left out of the fun, we saved a seat for you. See what the Railway community is all about, join us. We promise to be light on the pings.