Updated: January 26, 2023 RealityKit materials There are 6 types of materials in RealityKit 2.0 and RealityFoundation at the moment: SimpleMaterial UnlitMaterial OcclusionMaterial (read this post to find out how to setup SceneKit occlusion shader) VideoMaterial (look at this post to find out how to setup it) PhysicallyBasedMaterial CustomMaterial (Medium story) SwiftUI version Here I … Read more
Simple Ray-Casting, the same way as Hit-Testing, helps to locate a 3D point on a real-world surface by projecting an imaginary ray from a screen 2D point onto a detected plane. In Apple documentation (2019) there was the following definition of a raycasting: Ray-casting is the preferred method for finding positions on surfaces in the … Read more
Updated: October 08, 2022. TL;DR This post is not only about RealityKit, SceneKit & Metal, but also about related APIs and USD files. RealityKit 2.0 RealityKit (and RealityFoundation) is the youngest SDK in Apple family of rendering technologies. This high-level framework was released in 2019. RealityKit is made for AR / VR projects, has simplified … Read more
Updated: July 06, 2022. New Depth API You can improve a quality of People Occlusion and Object Occlusion features in ARKit 3.5….6.0 thanks to a new Depth API with high-quality ZDepth channel that can be rendered at 60 fps. However, for this you need iPhone 12 Pro or iPad Pro with LiDAR scanner. In ARKit … Read more
Scene Reconstruction Pity but I am still unable to capture model’s texture in realtime using the LiDAR scanning process. Neither at WWDC20 nor at WWDC22 Apple announced a native API for that (so texture capturing is only possible now using third-party APIs – don’t ask me which ones 🙂 ). However, there’s good news – … Read more
ARKit, RealityKit and SceneKit frameworks use 4 x 4 Transformation Matrices to translate, rotate, scale and shear 3D objects (just like simd_float4x4 matrix type). Let’s see how these matrices look like. In 3D Graphics we often use a 4×4 Matrix with 16 useful elements. The Identity 4×4 Matrix is as following: Between those sixteen elements … Read more
Updated: June 07, 2022. TL;DR ARAnchor ARAnchor is an invisible null-object that can hold a 3D model at anchor’s position in a virtual space. Think of ARAnchor just like it’s a transform node with local axis (you can translate, rotate and scale it) for your model. Every 3D model has a pivot point, right? So … Read more
Updated: March 31, 2022. usdzconvert command in Xcode 13, 12, 11 In macOS Monterey, Big Sur and Catalina USDZ converter has a rich set of commands and supports more input formats for conversion than previous version. But before using it you need to download USDZ Tools. And don’t forget to setup a global variables for … Read more