top of page

Generative AI Tool to Create Reference Images for Digital Artists

Team: Matthew Dress, F. Ria Khan, Daniel Robert Ritchie, Julie Salazar 


Course: IN4MTX 231


Date: Fall 2022 

Project Overview:

Overview of Problem

AI-based tools have the capacity to aid creative endeavors, such as digital drawing by providing source images to use for references when drawing particular subjects and providing general inspiration. For example, models like DALL·E can generate never-before-seen images that would be difficult to find through search engines. However, models like DALL·E are not often used as more than a novelty, with more advanced use of these models reserved for those with previous experience with AI. This motivated us to create an AI tool for generating novel reference images for digital artists in their creative process called ART-i. Through our app ART-i, we will enable digital artists to easily generate custom reference images using a generative AI model.

Target population
Our target population are young adults who have ongoing experience with creating digital art and do not participate in STEM fields/courses.

Design Process:

Formative Process

For the initial design intentions for this project, we wanted to help our target population learn more about the inner workings of Generative AI in a creative context by designing an educational, gamified experience. We took a qualitative approach by creating and administering individualized interviews. The team curated a semi-structured interview protocol to ask interviewees questions related to their background in college-level STEM course(s) taken and perceptions and experience with AI.


In the interviews, we found that interviewees avoided STEM courses due to a perceived irrelevance to their creative goals, which resulted in their lack of experience and interest in AI. This revealed to us that not every user may be interested in learning about the underlying mechanisms of AI, so we decided to focus on relevant tasks users want to accomplish in their creative endeavors (like digital art), and how those tasks might be accomplished with We identified the generation of reference images as a main pain point users were focused on. Thus, we narrowed our design scope to focus solely on digital visual art and to address the specific pain point from our original findings, using generative AI to create readily available, specific reference images based on the user’s creative endeavors. 

Paper Prototype and Class Feedback
We designed two versions of paper prototypes for our mobile app and presented them in an in-class evaluation. These initial prototypes were designed before fully scoping down our project to using generative AI for reference images. For the first version, users would use a bespoke Generative AI tool to help with providing prompt ideas for creating writing and the other, which took a narrative game approach to introducing DALL·E for generating digital visual art. Our evaluation found that users were confused in identifying the purpose of the app. This feedback led us to narrow down our scope and focus on a design that was more intentional and concise with what it can provide to the user.

prototype figure 2.png

Paper Prototype v1

proto 1.png

Paper Prototype v2

Mid-fidelity Evaluation

We scoped down our design based on feedback from our formative and paper prototype evaluations and created a low-fidelity digital prototype. We used a combination of pictures from our paper prototype and Google Slides to create the prototype. We implemented the following design changes: 1) We included prototypes for both mobile and tablet versions in order to address how to best support task, 2) we added a prompt input bar where the user can provide prompts to the AI model to generate reference images, and lastly, 3) we included image selection so that users can select images to display in the app.

For our evaluation, we used a “think-aloud” where users would vocalize their thoughts while using our application. This approach allowed us to get a deeper understanding of the user experience by capturing the users' thought process in real-time. 

After evaluations, we found that designing the app for tablets would be most advantageous through common tablet device features like split screen and the ability to have digital illustration apps like Procreate running on one side and our app on the other.  With these changes, we made a second, higher fidelity prototype of the tablet app using Figma.

prototype figure 3.png

Digital Paper Prototype Mobile Version

prototype figure 4.png

Digital Paper Prototype Tablet Version

prototype figure 5.png

Mid-Fidelity Tablet Version on Figma

High-fidelity Evaluation

For our high-fidelity prototype, we decided to add new elements such as a home screen that includes a “Learn More” providing the user a step-by-step overview of our system. We also replaced the dropdown color selection with a color selection map, finding that the initial dropdown was too discrete in nature, not allowing users the freedom to create art to their liking. Lastly, we included undo and redo buttons to our prototype to ensure that artists can undo any errors they may have made in their creative process. 

To evaluate our high-fidelity prototype, we created a Figma prototype of the app on tablet devices. We interviewed users on why they create digital art and of their creative process. We then had them test our Figma prototype through our “think aloud” approach while they navigated it. The evaluation then concluded with open-ended questions focusing on the users’ critique and feedback regarding the utility and workflow of ART-i.

Screen Shot 2022-12-05 at 6.24.06 PM.png

High-Fidelity Prototype on Figma

Future Recommendations:

Through our design process we came to certain takeaways about digital artists in regards to their needs and how ART-i can be supportive. We found that digital artists frequently used reference images in their initial process of creating digital art. While they utilized search engines to find reference images, they had certain limitations in sourcing reference images through their current means and applications. ART-i was able to accommodate these limitations, and our evaluations found that the features of our app were quite intuitive and overall easy to use.

We also received critiques on how to better the app further, and we plan to take these critiques into consideration if we wish to further develop this app in the future. These critiques included the lack of robust features of our app. Since our goal was to make the digital art process more seamless in using reference images (i.e. not needing to move between multiple apps like Google and Procreate), we neglected to ensure that the drawing component of the app included a wide variety of features similar to  existing drawing apps. We will also better consider the unique workflows of artists’ creative processes and design the app to holistically accommodate more diverse workflows.

bottom of page