Storyboards Matter: Three Insights for Using AI to Accelerate Product Design and Delivery

By Jeremy Romano, Managing Practitioner | Experience Design & Product Management

In software development, storyboarding is the essential blueprint for crafting exceptional user experiences. The product design phase and its artifacts are often the most misunderstood, underutilized, and notably absent components of the product and software development processes. Software development is more effective when guided by a design thinking methodology: Empathize, Define, Ideate, Prototype, Test, Implement, and Iterate. Products achieve optimal outcomes when they incorporate a user-centered process: Understand, Explore, Design, Evaluate, Refine, Implement, and Maintain. Despite these well-established frameworks, most organizations are constrained by resources, time, cutting corners, and making compromises that often leads to losing sight of the end-user’s needs and the business goals they aim to achieve. Even the most well-intentioned and successful teams face these struggles.

Over the past three decades, I have collaborated with top-tier brands to create award-winning products. I’ve witnessed product design, a core element of delivering great software that delights customers, become significantly less important to organizations. Now, with AI and Generative AI consuming the market, we believe a window of opportunity is wide open for organizations to embrace the power of AI for product design, specifically, storyboarding.

Why did we choose storyboards for our proof of concept (POC)?

We wanted to prove that one of the most valuable tools for building impactful products can be created faster than ever before and with a high degree of quality and accuracy. While most teams lack the typical skills, time, or resources to effectively produce high-quality design artifacts, using AI to storyboard will transform how organizations visualize and communicate the product vision from the user’s perspective. Taking it one step further and harkening back to Brian Woodring’s statement that “technology can be for everyone”, we want even the most artistically inexperienced people to be able to imagine and create impactful visualizations.

The hypothesis is that if AI can enable anyone to create visually compelling narratives without the associated cost and time challenges, with just the click of a button, the industry would see faster delivery, greater adoption, and happier, more productive, end-users.

Our AI Storyboarding Journey

Generative AI is powerful and challenging because it always generates a response. The scenes in our initial image generations were “awesome” but kind of useless, characterized by strangely abstract elements, wild metaphors, and a variety of artistic styles—from watercolor to oil painting, and from photorealistic to Disney-like. While the raw output from the language model was captivating, it often strayed from our intended narrative, and we watched as the age, race, and gender of our characters changed unexpectedly and with no prompt.

Insight #1: Developing our Cast of Characters

The most impactful learning from our POC was in character design. Drawing from cinema and traditional character design, we embraced the concept of using props. The language model required prompts with precise characteristics, so for us, ‘props’ included specifying ages, hair color, and even learning about various mustaches, beards, and hairstyles, such as the imperial mustache, curtain beard, afro puffs, and even the differences between a pixie cut, a pageboy, a bob, and a long bob. I also discovered the vast array of glasses styles available—rectangle, square, round, cat-eye, brow-line, and aviator (see Image #1). It was this level of detail that allowed us to refine and solidify our cast of characters in a more uniform fashion (see image #2).

Once we fine-tuned our approach, we were able to nail down PhoenixTeam’s cast of characters, each with a unique persona, and personality.

Insight #2: Defining the Scene Style

The second key insight was the importance of limiting the range of artistic styles for the AI. By narrowing down the color palette, specifying line weight and type, and providing a reference style for it to emulate, along with a clear focus for the “camera,” we achieved more consistent and repeatable results.

Insight #3: Finding a path to maintain a coherent story flow from start to finish.

The third breakthrough involved enabling our Value Engineers to select one of our characters and a series of user stories, which, when combined with our backend integration of artistic guidelines and camera settings, empowered the AI to autonomously generate consistent and engaging narratives. This ‘secret sauce’ has been a game-changer in maintaining a coherent story flow from the first panel to the last (see image #3), allowing us to weave seamless narratives that compellingly drive the story forward. We are now able to generate storyboards directly from user stories and acceptance criteria without any human intervention.

AI, for now, is not a magical solution that solves all problems; its role as an incredibly powerful assistant or as John Comiskey states “your ultimate wingman”, invites us to become active and engaged curators. Generative AI possesses the transformative power to break down longstanding barriers, enabling creativity and collaboration at unprecedented scale and speed. We can generate and share ideas more rapidly, pushing the boundaries of innovation and design thinking to make better software faster. This technology allows us to shape a future where AI and human creativity unite to create more meaningful and impactful design outcomes. I invite you to join this exciting journey, and experiment with AI in your projects. Comment below to share your experiences and insights with storyboarding or other image-generating results.