Writing a New Story at the Intersection of Art and Technology

Storyboarder.ai is one of the coolest pieces of tech we’ve seen in a while. Storyboarder.ai can take any industry-standard screenplay or rough concept and with the power of machine learning generate high-quality storytelling assets like a script, a shot list, a storyboard, and soon, as you’ll hear, video.

We were excited to sit down with Storyboarder.ai Founder Zeyd Taha Candan to talk about how the company started, the tech stack, and what kinds of users Storyboarder.ai is serving today and where the product is headed.

Let’s dive in!

Storyboarder.ai helps turn ideas into high quality storyboards

Railway: Storyboarder.ai is one of the coolest pieces of tech we’ve seen in a while. We uploaded some slides and within a few minutes, Storyboarder.ai had generated a script, shotlist, and storyboard – it felt like magic! How does it work?

Zeyd: Basically we take whatever the user will give us, extract the texts and run some advanced LLMs in order to work with it.

You can start with a finalized screenplay in one of the industry-standard formats like Final Draft or Celtx. If you don’t have a screenplay that is properly formatted, we take any type of PDF concept. You can even type your rough idea directly inside of Storyboarder.ai. We then will take that as a starting point and in the first step generate a screenplay for you.

After it has finished generating, you can easily edit your screenplay or just go to the next step, which is generating a shot list. It’s as easy as clicking a single button to get the process started. Storyboarder.ai will take your screenplay, analyze all scenes and within all scenes all shots. It will automatically suggest not only how to split the shots, but also shot sizes, camera perspectives, focal lengths, camera types etc.

Finally, once the shot list is done, you can start generating your storyboard frames. Storyboarder.ai will generate a shot for each shot in each scene. In case you want to give it another try, you can easily hit the “retry” button which will give you a new version of the image, or the “variation” button, which will take what you already have and give you a slightly different version of that. You can also use our image-to-image option or use inpainting to fine-tune your images. Once you are done, exporting as PDF files is another single button click.

Storyboarder.ai generated a compelling script from some slides that we uploaded

Railway: How did the idea for Storyboarder.ai come to be? What can you say about your previous experience or interests that led to the founding of Storyboarder.ai?

Zeyd: We are filmmakers at heart. I have studied film production at a film school in Cologne and worked in the industry for quite some years at different projects ranging cinema, television, and commercials.

I was working as a producer when OpenAI’s ChatGPT was released. We instantly started experimenting with it. Its capabilities were blowing us away. From rewriting our texts, adding a scene here and there with the information we gave it, creating concepts for our clients quickly. We knew we had to implement this technology into our workflow more deeply.

I started researching these technologies and thanks to my dad, who is a SAP consultant, I already had some coding knowledge even though I was working as a film producer. My then-boss, now co-founder, Dustin Steinkühler, had the great idea to use not only the text capabilities of these models, but also add image generation capabilities to generate storyboard images and put everything into one single tool, to use ourselves and send to our clients as a more detailed concept before starting shooting the project.

The great thing about having a problem yourself instead of looking for other people's problems is that you know exactly what you would like to see in such a tool. I had developed a plan for what we needed in order to get things started and it was so easy to get feedback on this plan because everyone in the company was a filmmaker and everyone had the same struggles and needs. So through this feedback loop we were able to develop a detailed plan of how things needed to be in order to solve this need of storyboarding.

Storyboarder.ai also generated a list of shots to tell the story

Railway: One of the most impressive things about Storyboarder.ai is the ability to pull a couple of different kinds of ML models together – there’s some natural language processing, some generative text, and some generative art stuff going on. Can you tell us a bit about the models that you’re using?

Zeyd: Our approach is always to use state of the art technology to solve our users' needs the best way possible. However, since we were a film production company, we did not have the resources of developing these resource-intensive and complex models ourselves. Instead, we started using APIs of models that were available at that time. Today, there are many more models, of which many have mind blowing capabilities and are even open source.

For instance, when we started our research, Dall-E 2 was available. However, we quickly shifted to one of the models Stable Diffusion provides because of the flexibility we got there. These models are expanding so quickly, that we do not want to compete with them, but instead embrace them and make them accessible to filmmakers especially.

I personally know of no single filmmaker who is running their own instance of Stable Diffusion on their server to generate storyboard images. However, using our tool, we make the experience so easy for these creatives to simply be able to click a button to get some nice storyboard images for their project.

Railway: What can you tell us about your customers? Is this largely focused on storytelling creative work in advertising, film/media, etc?

Zeyd: We don’t like them to call our customers. Sure, they are paying us to access our tool, but they are more like colleagues in some broader sense. Most of them are filmmakers, just like us. We want to stand next to them, not in front of them. We want to develop the capabilities of Storyboarder.ai together with them, not against them. They are creatives, directors, producers, camera people, and even storyboard artists.

Since we know that the actual work they are doing can differ, we tailored different plans from which they can choose from when signing up for Storyboarder.ai. Some of the most well-known streaming services, film productions, and top advertising agencies worldwide are using our tool, which we are really proud of.

Storyboarder.ai also generated a full storyboard for our project as well

Railway: What’s next for the product? What do you envision the next couple quarters of development will bring to Storyboarder.ai?

Zeyd: The next big leap for Storyboarder.ai will be adding video. We are already working on making it possible not only to generate static images using our tool, but also to generate video.

Right now, the last step of the workflow using Storyboarder.ai is to generate your images and export them as a PDF file. But how cool would it be to generate videos for your shots and download them as a MP4 file? We are currently in the process of trying out different things and looking for what works and what does not.

Additionally, we are constantly working on expanding the technical limits of Storyboarder.ai, such as input and output limitations, image quality, and speed.

Railway: Can you tell us a bit about how Railway fits into your tech stack? We’d love to hear how Railway helps make Storyboarder.ai work.

Zeyd: When we started Storyboarder.ai, we had no idea where to deploy. We just wanted to make sure everything runs smoothly locally on our own devices first. Once the launch date came closer, we basically looked around and Railway was the coolest startup in the room. “No platform engineer required”? We are in!

Really, deploying Storyboarder.ai on Railway was a breeze. The best part is that you do not need a certification in order to understand what’s going on. Additionally, we absolutely love about Railway, that it feels like we are part of the team. We have direct access to everything we could possibly need on the platform, as well as really quick response times when we have a request or a question.

Railway: If readers would like to get started using Storyboarder.ai, what’s the best place to start? Are there any resources or starting points you recommend?

Zeyd: Anyone who would like to get started using Storyboarder.ai can head over to Storyboarder.ai and sign up for a 7-day free trial. We do have a product tour that will guide you through everything step by step. Once logged in, we also have a bunch of video tutorials prepared to make the best out of Storyboarder.ai and let your creativity flow.