Recent image and video generation advancements have revolutionized content creation, offering unprecedented possibilities. While we rapidly approach seamless interaction with scenes and videos through text inputs, 3D scene generation remains a fertile ground for exploration. Over the past two years, significant progress has been made in 3D and 4D object generation, mainly through multiview diffusion models. More recently, the application of consistent video generation as a proxy for 3D spatial representation has emerged, paving the way for comprehensive 3D scene generation.

These developments present a unique opportunity to harness deep learning algorithms, allowing us to envision and create immersive 3D scenes, thereby unlocking a new dimension of creative expression.This workshop will explore the cutting-edge techniques of multiview diffusion models conditioned on camera paths for 3D scene generation. We will dive into the latest methods for representing 3D scenes and geometry and investigate how current research in 3D generative AI leverages pre-trained image and video models for a 3D generation. Participants will learn how to use text and image inputs to generate scenes, which will then be imported into Unreal Engine for post-production and to create a concept reel.

This workshop focuses on the automation of 3D scene generation from text and images. Through theoretical discussions and practical exercises, participants will explore the capabilities of 3D generative AI and its application in fast-tracking design exploration. Key topics include:

Introduction to Multiview Diffusion Models:

• Understanding the principles and techniques behind multiview diffusion models.
• Exploring case studies of successful applications in 3D scene generation.

3D Scene and Geometry Representation:

• Examining recent advancements in the 3D scene and geometry representation.
• Discussing the role of different representation techniques in enhancing 3D consistency.

Leveraging Pretrained Models:

• Learning how to adapt pre-trained image and video models for a 3D generation.
• Hands-on sessions with state-of-the-art algorithms for scene creation.

Text and Image Inputs for Scene Generation:

• Text and image inputs are used to automate the generation of 3D scenes.
• Practical exercises on integrating inputs to produce coherent scenes.

Integration with Unreal Engine:

• Importing AI-generated scenes into Unreal Engine for further development.
• Techniques for enhancing interactivity and immersion in Unreal Engine.

• Exploring 3D Generative AI: Understand and utilize advanced AI algorithms for scene generation.
• Scene Creation: Learn to automate the process of 3D scene creation using text and image inputs.
• Interactive Design: Develop skills in creating interactive and immersive experiences in Unreal Engine.
• Creative Exploration: Foster innovation and creativity in architectural design through AI tools.

By the end of this workshop, participants will have a solid understanding of how to leverage 3D generative AI to automate scene generation and create immersive architectural experiences, opening new avenues for creative expression and design exploration.

Inspirational Inputs:

• Text and Images: Use descriptive text and images to generate 3D scenes. These inputs will serve as the foundation for your scene’s development.
• Scene Development: Experiment with different inputs to see how they influence the generated scenes.

Scene Generation:

• Multiview Diffusion Models: Apply these models to generate consistent scenes across multiple views.
• Representation Techniques: Use various methods to represent geometry and scene details effectively.

Interactive Experience Creation:

• Unreal Engine Integration: Learn to import and refine your AI-generated scenes in Unreal Engine.
• Enhancing Immersion: Develop interactive elements to create a fully immersive experience.

Project Showcase:

• Concept Reel: Compile your generated scenes into a cohesive concept reel.
• Presentation and Feedback: Present your work and receive feedback from peers and instructors.