Tackling 3D Deep Learning, Gaussian Splats and Physics Simulation with NVIDIA Kaolin Library, a Hands-On Lab

3D Deep Learning often demands extensive boilerplate work such as managing data, camera conventions, and visualizing novel 3D representations. NVIDIA’s Kaolin Library, built on PyTorch, addresses these with tools like convenience APIs, reusable research modules, and GPU-optimized operations. The library’s updates are designed to address the evolving needs of the research community. Recent examples include supporting emerging representations like 3D Gaussian Splats (3DGS), and physics-based simulations for dynamic 3D modeling. Initially developed for internal use, Kaolin is shared externally under an open-source license. The tutorial will provide hands-on coding experience to equip attendees with practical skills for using Kaolin. In this session, we explore interactive tools 3DGS viewing in Jupyter, how to create optimizable physical simulations, and finally convert between common 3D representations to export results. GPU back ends will be provided. By the end of the tutorial, attendees will be able to utilize Kaolin’s features to streamline their research workflows and accelerate their projects.