3D Gaussian Splatting: Hands-on Course for Beginners

By • 3D Gaussian Splatting 3D Reconstruction

This Tutorial targets 3D Reconstruction and 3D Real-Time Visualization leveraging 3D Gaussian Splatting. In the end, you will be able to generate 3D Point Clouds and 3D Renderings from a Set of Images and/or Videos.

3D Reconstruction with 3D Gaussian Splatting

In the rapidly evolving landscape of 3D modeling, innovations consistently push the envelope, enhancing our capabilities to create, visualize, and interact with digital objects.

One such groundbreaking technique that’s making waves is 3D Gaussian Splatting. Though rooted in decades-old principles, this method leverages modern computational power to transform how we handle and render 3D data from simple 2D images or videos.

In this Live Course Session, we explore the intricacies of this technique, highlighting its practical applications and advantages over traditional methods.

Resources and Materials 🍇

3D Gaussian Splatting: The Overall Workflow

The session begins with an introduction to 3D Gaussian Splating, outlining the process of transforming a set of images into a 3D Gaussian Splating experience. The workflow shows several steps as illustrated below:

The Process of Generation 3D Point Cloud with 3D Gaussian Splatting (Florent Poux)

Note: I emphasize the importance of having a CUDA-enabled graphics card for replicating the session and recommend specific models like the 3090RTX for efficient local testing. The session deliverables include a live recording, data set links, resource curation, and cheat sheets for efficient workflow. The mission involves starting with an image set and creating 3D Gaussian splits, point clouds, and 3D renderings.

Structure From Motion for 3D Gaussian Splatting

This historical context underscored the field’s continuous evolution, leading to the current focus on Gaussian Splatting. Unlike its predecessors, this technique promises a more streamlined and efficient workflow, particularly beneficial for those in the visual effects (VFX) and real-time rendering industries.

At its core, the concept of Structure from Motion (SfM), is key. It involves estimating the 3D structure of a scene along with the motion parameters (pose) of the camera that captured the images. SfM is commonly used in applications linked with 3D Reconstruction Tasks whenever we start from image-based reconstructions. SfM builds on extracted features from multiple images to reconstruct a 3D scene.

This process, from feature extraction and image matching to determining camera poses and sparse point clouds, is the first pass before moving onto the world of Gaussian Splatting. This allows for bypassing the computationally expensive dense matching phase of traditional photogrammetry, offering a faster and more efficient alternative.

Structure From Motion for 3D Gaussian Splatting

3D Gaussian Splatting Fundamentals

Gaussian Splating itself is a fascinating blend of rasterization and optimization.

The technique involves representing 3D data using Gaussians—each defined by a position, a covariance (which describes its stretch and orientation), color, and an alpha channel.

Combining these Gaussians with a differentiable rasterizer allows bi-directional communication between the 3D and 2D representations.

This enables real-time adjustments and optimizations, ensuring the rendered images closely match the ground truth images.

3D Gaussian Splatting: Principles

Hands-On Tutorial for 3D Gaussian Splatting

In order to extend the theoretical setup, we dive into a practical aspects of Gaussian Splating using the Post-Shot software.

This means that we have to go through the steps of importing an image set, configuring the training parameters, and launching the training process. Within minutes, we can obtain a rendering of detailed 3D scenes, allowing real-time adjustments and visualizations.

This demonstrates the technique’s accessibility and efficiency, making it a valuable tool for professionals in various fields.

3D Reconstruction Workflow to Generate 3D Gaussian Splatting

Conclusion

The session provided a comprehensive overview of 3D Gaussian Splating, showcasing its potential to revolutionize 3D reconstruction and rendering. Gaussian Splating offers a faster, more efficient, and highly detailed approach to 3D modeling by blending traditional photogrammetry principles with modern computational techniques.

As technology advances, methods like these will undoubtedly play a crucial role in shaping the future of digital content creation, offering new possibilities for innovation and creativity.

My 3D Recommendation 🍉

Generating 3D Point Clouds from a Set of Images is fantastic. It opens up a lot of horizons… For Creativity purposes only (at this stage). If you seek a more metrically controlled approach, I highly recommend getting expertise with Photogrammetry to unlock Multi-View Reconstruction. It is more constraining than single images but provides a much better geometric base layer. If you want to use it in your solutions, I recommend getting on the 3D Collector’s Pack.

If you want to get a tad more toward application-first or depth-first approaches, I curated several learning tracks and courses on this website to help you along your journey. Feel free to follow the ones that best suit your needs, and do not hesitate to reach out directly to me if you have any questions or if I can advise you on your current challenges!

Open-Access Knowledge

3D Online Courses

3D Learning Tracks

Scroll to Top