Allen Ginsberg’s 1963 poem “Howl” is a literary work of art that juxtaposes the harsh reality of capitalism in America during the 1960s with vivid, almost tangible imagery. The poem is best experienced when read aloud, but what if we could visualize this abstract imagery? This was the inspiration behind AIlluminated, a project that uses AI and Python to create a visual representation of texts like “Howl.”

The Inspiration and Challenges

I was drawn to the poem’s concrete imagery and the power of stable diffusion and automation, which could potentially produce interesting video interpretations of “Howl.” However, the length of the poem, which spans 20 minutes, posed a significant challenge. Regardless, I gathered the necessary tools and assembled them in this repository to share the process and the output.

The AIlluminated Toolkit

AIlluminated is a Python-based video creation tool that elegantly combines images, audio, and subtitles into captivating videos with smooth crossfade transitions. It’s designed to be versatile and user-friendly, making it suitable for various applications ranging from artistic videos and educational content to memory preservation.

The key features of AIlluminated include:

• Timestamp-based image and subtitle integration
• Crossfade transitions between images
• Customizable frame rate and output format

The Process

The process begins by preparing the resources—images, audio, and a text file with timestamps and corresponding subtitles. A Python script, convert_srt.py, transforms an SRT subtitle file into a timestamp-based text file, while illustrate.py takes these resources and assembles them into a video.

Here’s a brief overview of the process:

1. Clone the repository and install the necessary Python libraries.
2. Prepare your resources: images, audio, and a text file with timestamps and subtitles.
3. Use convert_srt.py to convert the SRT file to a timestamp-based text file.
4. Run illustrate.py to create your video.

The versatility of AIlluminated is showcased in the variety of texts it can handle, including poetry, song lyrics, and comedy bits. Examples of this versatility are provided in the repository.

Examples

AIlluminated is an exciting exploration of the intersection of literature and AI, opening up new possibilities for experiencing texts. While the project started with a lengthy poem, the toolset developed can be used to bring to life any text, from poems and songs to speeches and more.

The project is open to contributions, and I invite others to use, adapt, and improve AIlluminated. I look forward to seeing the diverse ways in which it will be used and the unique creations it will inspire.