From Prompts to Moving Pictures: Designing with SD, FLUX & Video AI

Led by architect and AI researcher Fredy Fortich, currently a Technical Architect at MVRDV, Generative Architecture with AI 2.0 (SD+FLUX) was built on the foundations of his previous PAACADEMY studio, taking participants from diffusion model fundamentals to advanced multimodal workflows. Across two intensive sessions, attendees explored how to transform architectural ideas into refined images and cinematic videos by combining Stable Diffusion, FLUX, Photoshop, ComfyUI, and Runway into a cohesive creative pipeline.

Rather than treating AI as a shortcut, the workshop framed it as a precision tool, a system to be mastered, structured, and integrated into the architectural process.

Laying the Foundations: Diffusion Models and Controlled Image Generation

The first day opened with a clear introduction to diffusion model concepts, ensuring participants understood not just how to use the tools, but why they behave the way they do. Fredy covered prompt structuring, word weighting, seeds, and LoRA usage, giving attendees the vocabulary to speak AI’s language.

He then compared txt2img and img2img workflows, demonstrating how ControlNet can bridge the gap between a 3D massing model and a context-aware architectural render. A highlight of the day was Fredy’s segmentation-to-Photoshop method, which allowed participants to refine AI-generated outputs with precise control over materials, lighting, and composition.

By the end of Day 1, students had moved beyond random prompt experimentation, gaining a structured approach to image generation that would serve as the foundation for more advanced workflows.

Advanced FLUX Techniques, ComfyUI Mastery & Motion Workflows

Day 2 began with an in-depth look at FLUX prompting strategies, where Fredy compared CLIP and T5 models, explaining how each interprets text differently and how that impacts the creative outcome. This set the stage for exploring a range of advanced techniques. Participants learned FLUX Denoising to progressively refine imagery while preserving the core design intent, and discovered the FLUX Compositor, which brings Photoshop-style layer control directly into AI workflows, making it easier to manage complex compositions.

Fredy then introduced inpainting workflows, showing how targeted edits can be used to tell a visual story by gradually shaping specific elements without rebuilding the entire image. The group also dove into upscaling methods that combine LLM-assisted prompting with SDXL and FLUX to produce high-resolution, competition-ready visuals.

The session wrapped up with a full image-to-video pipeline, where still architectural renders were transformed into cinematic sequences using Runway’s Gen-2 and AnimateDiff with motion LoRAs. This final step tied the workflow together, enabling participants to move seamlessly from concept to polished moving imagery.

A New Workflow Mindset for Architects

Throughout both days, Fredy drew on his experience at MVRDV to encourage participants to think beyond simply creating forms and start viewing themselves as workflow designers. He showed how structuring tools like SD and FLUX into intentional pipelines can produce high-quality, context-aware visuals, while segmentation and inpainting techniques give designers precise control over refining their images. The sessions also demonstrated how to integrate 3D modeling, AI image generation, and cinematic motion into a single, multimodal process.

By the end of the workshop, participants had not only gained technical skills but also adopted a new creative mindset, one where architecture is approached as a dynamic, iterative process that evolves into visually compelling outcomes.

Join the experience! Discover PAACADEMY’s upcoming workshops and stay at the forefront of design and technology.