Get pixel value from CVPixelBufferRef in Swift

by

baseAddress is an unsafe mutable pointer or more precisely a UnsafeMutablePointer<Void>. You can easily access the memory once you have converted the pointer away from Void to a more specific type:

// Convert the base address to a safe pointer of the appropriate type
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// read the data (returns value of type UInt8)
let firstByte = byteBuffer[0]

// write data
byteBuffer[3] = 90

Make sure you use the correct type (8, 16 or 32 bit unsigned int). It depends on the video format. Most likely it’s 8 bit.

Update on buffer formats:

You can specify the format when you initialize the AVCaptureVideoDataOutput instance. You basically have the choice of:

  • BGRA: a single plane where the blue, green, red and alpha values are stored in a 32 bit integer each
  • 420YpCbCr8BiPlanarFullRange: Two planes, the first containing a byte for each pixel with the Y (luma) value, the second containing the Cb and Cr (chroma) values for groups of pixels
  • 420YpCbCr8BiPlanarVideoRange: The same as 420YpCbCr8BiPlanarFullRange but the Y values are restricted to the range 16 – 235 (for historical reasons)

If you’re interested in the color values and speed (or rather maximum frame rate) is not an issue, then go for the simpler BGRA format. Otherwise take one of the more efficient native video formats.

If you have two planes, you must get the base address of the desired plane (see video format example):

Video format example

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// Get luma value for pixel (43, 17)
let luma = byteBuffer[17 * bytesPerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

BGRA example

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)
let int32PerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
let int32Buffer = UnsafeMutablePointer<UInt32>(baseAddress)

// Get BGRA value for pixel (43, 17)
let luma = int32Buffer[17 * int32PerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

More Related Contents:

  1. Which is the best way to estimate measure of photographed things?
  2. Bounding Box from VNDetectRectangleRequest is not correct size when used as child VC
  3. iOS: Convert UTC NSDate to local Timezone
  4. Protocol can only be used as a generic constraint because it has Self or associatedType requirements
  5. What is the difference between ObservedObject and StateObject in SwiftUI
  6. How to segue programmatically in iOS using Swift
  7. A Swift example of Custom Views for Data Input (custom in-app keyboard)
  8. Transparent iOS navigation bar
  9. How to program a delay in Swift 3
  10. How do I set bold and italic on UILabel of iPhone/iPad?
  11. Dynamic cell width of UICollectionView depending on label width
  12. Module compiled with Swift 5.1 cannot be imported by the Swift 5.1.2 compiler
  13. How do you make a background image scale to screen size in swift?
  14. How to pass data from modal view controller back when dismissed
  15. fatal error: unexpectedly found nil while unwrapping an Optional value [duplicate]
  16. Detecting iOS UIDevice orientation
  17. UIAlertView/UIAlertController iOS 7 and iOS 8 compatibility
  18. Firebase with Swift 3 counting the number of children
  19. Can somebody give a snippet of “append if not exists” method in swift array?
  20. How to use NSLocalizedString function with variables in Swift?
  21. How to initialize/instantiate a custom UIView class with a XIB file in Swift
  22. Customize MKAnnotation Callout View?
  23. Prevent dismissal of modal view controller in SwiftUI
  24. Convert UIImage to NSData and convert back to UIImage in Swift?
  25. @Published property wrapper not working on subclass of ObservableObject
  26. Swift UITableView reloadData in a closure
  27. How can I generate a barcode from a string in Swift?
  28. How to use UnsafeMutablePointer in Swift 3?
  29. Gamecenter ios 9 GameCenter GKLocalPlayerListener methods not called
  30. Calculate all permutations of a string in Swift

Leave a Comment