The joy of creating with DALL·E 2
An indie author's journey from human illustration to AI art for fantasy fiction
Looking for an in-depth discussion of the DALL·E 2 prompts used for the AI art discussed in this post? Subscribe and stay tuned! I’ll be sharing a post very soon that focuses specifically on the prompts and outpainting techniques I used for my book cover art and promotion images.
Dear reader,
While I was writing my first fantasy novel in 2011, the story world was so vivid in my mind that I couldn’t help daydreaming about illustrations to bring that world to life for readers. So, shortly after I self-published the novel in paperback, I approached my good friend and talented character artist Ian Steplowski to propose collaborating on an illustration project. Ian graciously agreed, and the color illustrated hardcover edition of Realms Unreel became a reality through a Kickstarter campaign in 2012, funded by 146 generous backers.
Illustrating Realms Unreel with Ian, and sharing the journey with my Kickstarter backers, was incredibly fun and rewarding. Ian utilized many innovative (for 2012) character modeling technologies in the illustration process, which was wonderfully interesting for me to observe and thematically appropriate considering the pervasive VR/AR technologies and virtual worlds that form the backdrop of Realms Unreel’s very-near-future world. Here are a few highlights of the end result.
Fast forward to August 2022. As I approached the end of writing The Voice in All (Book One of my new YA fantasy series The Artifex and the Muse), I once again found myself yearning for visual representations of the characters and scenes I’d spent so much time imagining over the last nine months of writing.
For me, writing a novel feels very much like traveling. By the time I’ve arrived at the final draft of a scene, I’ve visited that setting, those characters, their thoughts and speech and emotions so many times that the scene feels as real to me as my own memories. Unlike my “real” travel experiences, though, my writing travels are impossible to capture with a photo or video. Given my particular skillset as a writer, words are the only artifacts I can easily bring back from my solo journeys to imagined places.
Until now.
At just the moment I was most craving illustrations for The Voice in All, I received a very fortuitous email from the team at OpenAI. I was finally off the waitlist for DALL·E 2.
Imagine my delight, dear reader! At last, I (with a teensy bit of help from DALL·E 2 and the totality of all prior human-created art and technology) could paint using only my words.
So paint I did.
I’d been dabbling with AI-generated text, sound, and images prior to DALL·E 2, particularly enjoying AI illustration apps like Dream by Wombo and AI writing tools like Sudowrite. These—and many others—are amazing tools in their own right, and I’ve had so much fun playing with them over the last year and anticipating how much better they’ll all become in time.
However, until I tried DALL·E 2, I hadn’t experienced any AI collaboration that felt truly additive to my own creative process or worth incorporating into a finished product. Within hours of starting to play with DALL·E 2, I was producing beautiful images to represent characters, concepts, and scenes from my book. (Massive credit for this must be given to the free Dallery Gallery Prompt Book that distills learnings from the DALL·E 2 user community into helpful examples. I would have been completely lost without those example prompts and outputs.) I found collaborating with DALL·E 2 deeply satisfying and empowering as someone who is relatively unskilled in creating visual art.
My dear husband, who is a far more talented visual designer than I am, was surprised by the quality of the results I was getting from DALL·E 2. He expressed frustration with crafting prompts himself, saying he rarely found his own output images compelling. I thought this was fascinating, since DH is in every way more technologically adept than I am, as well as a much better visual thinker.
But then I started thinking more deeply about what’s happening when a writer collaborates with DALL·E 2, which could explain some of the differences between my results and DH’s. For me, writing isn’t a particularly visual process. Sure, there’s basic blocking and scene setting and character description going on in my head when I write a scene, but I spend a lot more time thinking in abstractions. What’s the mood of the scene, the key conflict, the pace of the action, the theme I’m developing, the mythic symbolism I’m using, the clue I’m planting for the next plot twist? Most of the work of writing is compressing all these sprawling abstractions into words, phrases, sentences, and paragraphs that get those ideas across as concretely and concisely as possible, in a sensible sequence, so that some human reader can later compile and render the scene in her own imagination with her own particular visual details, while still experiencing a near approximation of the story I meant to tell her. It’s a rather different skillset from painting. But it turns out, distilling abstract concepts into a few words is exactly the skill required to create good DALL·E 2 prompts. When I give DALL·E 2 a prompt, I’m telling the AI a little bit of a story. And unlike a human reader, DALL·E 2 can show me exactly what it’s imagining is going on in my story, so I can quickly recognize where my prompt is working, and where it needs work. I wish soliciting reader feedback from humans were this easy!
One of the images I produced while playing with DALL·E 2 this way was so eye-catching that I immediately started imagining what it might look like as part of my book’s cover art. Here’s the original tile that jumped out at me from a series of about 100 unique images I created with DALL·E 2 using a collection of 25 closely related text prompts. (I’ll be sharing much more detail on my prompts soon in a subsequent post.)
I continued to play with this image using DALL·E 2’s outpainting capability, correcting minor “defects” in the original image and eventually expanding outward into a mural image large enough to use as cover art for the ebook, paperback, and hardcover editions of The Voice in All.
Here’s the final front cover that’s gone to print for the first edition of The Voice in All. I’m personally delighted with the end result, and with the entire process of creating it with DALL·E 2.
Until now, I’ve been entirely reliant on visual artists to create illustrations and cover art for my books. I loved collaborating with dear friends on my prior cover art and illustrations, and as an indie author I could make the final call on every detail of what went to print. But it’s always been a little off-putting to me that every reader’s first impression of my book has nothing to do with the writing itself, but is instead a first impression of the work of another artist. (For traditionally published authors, the situation may be even worse, as many authors have minimal input on the final cover art that goes to print.)
I’ll allow that there are many good reasons writers aren’t usually responsible for creating their own covers. It’s a completely different art form, after all. However, there is something undeniably satisfying as the author to make your cover art a direct expression of your own vision for the book.
Still, as much as I personally love my cover, I had no idea what readers would think of it until I released the book into the wild. Does my book have a face that only a mother could love?
To my delight, I’ve discovered the answer (at least from fantasy readers) is a resounding no. Here’s just a sample of the dozens of comments I’ve received about the cover so far on my social media book tour.
@peschel_press: That is fabulous, wonderful cover art! So not generic and “I’ve seen this about 100 times.” … It isn’t easy to be genre-specific AND different AND attractive in its own right … A good cover is worth a thousand words.
@bearyintobooks: As soon as I saw the cover of this one I knew I had to read it. Like seriously, isn’t this cover beautiful??
@katieiscompletelyfine: this cover is really unique 😍
Over the course of working with DALL·E 2 on my cover, I’ve fallen in love with this way of illustrating. So much so that I decided to continue jamming with DALL·E 2 for my eight weeks of book promotion. To brighten up my previously un-loved social media presences for my social media book tour, I’ve been rapidly generating illustrations based on scenes from the book or recognitions the book is receiving. I’m tickled by what’s possible to produce in mere minutes (okay, occasionally more than mere minutes, but they’re fun minutes). I’m sharing some of these examples below, for your entertainment. Hope you enjoy!
Clearly we are entering a brave new world of content creation. A lot has been written and remains to be written on the legal, artistic, social, and many other implications of AI art. But I thought I’d share my personal story about the pure joy I’ve experienced playing with DALL·E 2 as part of my little indie writing project.
To all the talented thinkers, dreamers, and builders who’ve made DALL·E 2 possible, thank you!
And to all you indie authors out there who are eager to experiment with your own cover art, drop me a line! I’d love to see what you’re working on.
Happy reading,
AA