Skip to content

Instantly share code, notes, and snippets.

@acj
Last active May 5, 2023 12:23
Show Gist options
  • Star 40 You must be signed in to star a gist
  • Fork 9 You must be signed in to fork a gist
  • Save acj/6ae90aa1ebb8cad6b47b to your computer and use it in GitHub Desktop.
Save acj/6ae90aa1ebb8cad6b47b to your computer and use it in GitHub Desktop.
Build a movie from jpeg images in Swift using AVFoundation

This code has moved

Please refer to the TimeLapseBuilder-Swift repository on GitHub from now on.

I will leave the original code here as a reference, but new comments may be removed. Please open an issue on GitHub if you have questions or would like to contribute.

Thanks!

//
// BuildTimelapseViewController.swift
//
// Created by Adam Jensen on 5/9/15.
//
import JGProgressHUD
import JoePro
import UIKit
class BuildTimelapseViewController: UIViewController {
@IBOutlet weak var resolutionSegmentedControl: UISegmentedControl!
@IBOutlet weak var speedSlider: UISlider!
@IBOutlet weak var removeFisheyeSlider: UISwitch!
var album: String?
var camera: JoeProCamera?
var timeLapseBuilder: TimeLapseBuilder?
init(camera: JoeProCamera, album: String) {
self.camera = camera
self.album = album
super.init(nibName: "BuildTimelapseViewController", bundle: nil)
}
required init(coder aDecoder: NSCoder) {
super.init(coder: aDecoder)
}
override func viewDidLoad() {
super.viewDidLoad()
}
@IBAction func buildButtonTapped(sender: AnyObject) {
if let camera = camera,
let album = album {
let progressHUD = JGProgressHUD(style: .Light)
progressHUD.textLabel.text = "Building your timelapse..."
progressHUD.indicatorView = JGProgressHUDRingIndicatorView(HUDStyle: .Light)
progressHUD.setProgress(0, animated: true)
progressHUD.showInView(view)
camera.listOfVideos(album) { (videos) -> Void in
self.timeLapseBuilder = TimeLapseBuilder(photoURLs: videos)
self.timeLapseBuilder!.build(
{ (progress: NSProgress) in
NSLog("Progress: \(progress.completedUnitCount) / \(progress.totalUnitCount)")
dispatch_async(dispatch_get_main_queue(), {
let progressPercentage = Float(progress.completedUnitCount) / Float(progress.totalUnitCount)
progressHUD.setProgress(progressPercentage, animated: true)
})
},
success: { url in
NSLog("Output written to \(url)")
dispatch_async(dispatch_get_main_queue(), {
progressHUD.dismiss()
})
},
failure: { error in
NSLog("failure: \(error)")
dispatch_async(dispatch_get_main_queue(), {
progressHUD.dismiss()
})
}
)
}
}
}
}
//
// TimeLapseBuilder.swift
//
// Created by Adam Jensen on 5/10/15.
//
// NOTE: This is the original Swift 1.2 implementation. For an updated version
// written in Swift 2.0, see https://gist.github.com/acj/6ae90aa1ebb8cad6b47b
import AVFoundation
import UIKit
let kErrorDomain = "TimeLapseBuilder"
let kFailedToStartAssetWriterError = 0
let kFailedToAppendPixelBufferError = 1
class TimeLapseBuilder: NSObject {
let photoURLs: [String]
var videoWriter: AVAssetWriter?
init(photoURLs: [String]) {
self.photoURLs = photoURLs
}
func build(progress: (NSProgress -> Void), success: (NSURL -> Void), failure: (NSError -> Void)) {
let inputSize = CGSize(width: 4000, height: 3000)
let outputSize = CGSize(width: 1280, height: 720)
var error: NSError?
let documentsPath = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)[0] as! NSString
let videoOutputURL = NSURL(fileURLWithPath: documentsPath.stringByAppendingPathComponent("AssembledVideo.mov"))!
NSFileManager.defaultManager().removeItemAtURL(videoOutputURL, error: nil)
videoWriter = AVAssetWriter(URL: videoOutputURL, fileType: AVFileTypeQuickTimeMovie, error: &error)
if let videoWriter = videoWriter {
let videoSettings: [NSObject : AnyObject] = [
AVVideoCodecKey : AVVideoCodecH264,
AVVideoWidthKey : outputSize.width,
AVVideoHeightKey : outputSize.height,
// AVVideoCompressionPropertiesKey : [
// AVVideoAverageBitRateKey : NSInteger(1000000),
// AVVideoMaxKeyFrameIntervalKey : NSInteger(16),
// AVVideoProfileLevelKey : AVVideoProfileLevelH264BaselineAutoLevel
// ]
]
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(
assetWriterInput: videoWriterInput,
sourcePixelBufferAttributes: [
kCVPixelBufferPixelFormatTypeKey : kCVPixelFormatType_32ARGB,
kCVPixelBufferWidthKey : inputSize.width,
kCVPixelBufferHeightKey : inputSize.height,
]
)
assert(videoWriter.canAddInput(videoWriterInput))
videoWriter.addInput(videoWriterInput)
if videoWriter.startWriting() {
videoWriter.startSessionAtSourceTime(kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = dispatch_queue_create("mediaInputQueue", nil)
videoWriterInput.requestMediaDataWhenReadyOnQueue(media_queue, usingBlock: { () -> Void in
let fps: Int32 = 30
let frameDuration = CMTimeMake(1, fps)
let currentProgress = NSProgress(totalUnitCount: Int64(self.photoURLs.count))
var frameCount: Int64 = 0
var remainingPhotoURLs = [String](self.photoURLs)
while (videoWriterInput.readyForMoreMediaData && !remainingPhotoURLs.isEmpty) {
let nextPhotoURL = remainingPhotoURLs.removeAtIndex(0)
let lastFrameTime = CMTimeMake(frameCount, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
if !self.appendPixelBufferForImageAtURL(nextPhotoURL, pixelBufferAdaptor: pixelBufferAdaptor, presentationTime: presentationTime) {
error = NSError(
domain: kErrorDomain,
code: kFailedToAppendPixelBufferError,
userInfo: [
"description": "AVAssetWriterInputPixelBufferAdapter failed to append pixel buffer",
"rawError": videoWriter.error ?? "(none)"
]
)
break
}
frameCount++
currentProgress.completedUnitCount = frameCount
progress(currentProgress)
}
videoWriterInput.markAsFinished()
videoWriter.finishWritingWithCompletionHandler { () -> Void in
if error == nil {
success(videoOutputURL)
}
}
})
} else {
error = NSError(
domain: kErrorDomain,
code: kFailedToStartAssetWriterError,
userInfo: ["description": "AVAssetWriter failed to start writing"]
)
}
}
if let error = error {
failure(error)
}
}
func appendPixelBufferForImageAtURL(url: String, pixelBufferAdaptor: AVAssetWriterInputPixelBufferAdaptor, presentationTime: CMTime) -> Bool {
var appendSucceeded = true
autoreleasepool {
if let url = NSURL(string: url),
let imageData = NSData(contentsOfURL: url),
let image = UIImage(data: imageData) {
var pixelBuffer: Unmanaged<CVPixelBuffer>?
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(
kCFAllocatorDefault,
pixelBufferAdaptor.pixelBufferPool,
&pixelBuffer
)
if let pixelBuffer = pixelBuffer where status == 0 {
let managedPixelBuffer = pixelBuffer.takeRetainedValue()
fillPixelBufferFromImage(image, pixelBuffer: managedPixelBuffer)
appendSucceeded = pixelBufferAdaptor.appendPixelBuffer(
managedPixelBuffer,
withPresentationTime: presentationTime
)
} else {
NSLog("error: Failed to allocate pixel buffer from pool")
}
}
}
return appendSucceeded
}
func fillPixelBufferFromImage(image: UIImage, pixelBuffer: CVPixelBufferRef) {
let imageData = CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage))
let lockStatus = CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.PremultipliedFirst.rawValue)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGBitmapContextCreate(
pixelData,
Int(image.size.width),
Int(image.size.height),
8,
Int(4 * image.size.width),
rgbColorSpace,
bitmapInfo
)
CGContextDrawImage(context, CGRectMake(0, 0, image.size.width, image.size.height), image.CGImage)
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}
}
//
// TimeLapseBuilder.swift
// Vapor
//
// Created by Adam Jensen on 5/10/15.
//
// NOTE: This implementation is written in Swift 2.0.
import AVFoundation
import UIKit
let kErrorDomain = "TimeLapseBuilder"
let kFailedToStartAssetWriterError = 0
let kFailedToAppendPixelBufferError = 1
class TimeLapseBuilder: NSObject {
let photoURLs: [String]
var videoWriter: AVAssetWriter?
init(photoURLs: [String]) {
self.photoURLs = photoURLs
}
func build(progress: (NSProgress -> Void), success: (NSURL -> Void), failure: (NSError -> Void)) {
let inputSize = CGSize(width: 4000, height: 3000)
let outputSize = CGSize(width: 1280, height: 720)
var error: NSError?
let documentsPath = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)[0] as NSString
let videoOutputURL = NSURL(fileURLWithPath: documentsPath.stringByAppendingPathComponent("AssembledVideo.mov"))
do {
try NSFileManager.defaultManager().removeItemAtURL(videoOutputURL)
} catch {}
do {
try videoWriter = AVAssetWriter(URL: videoOutputURL, fileType: AVFileTypeQuickTimeMovie)
} catch let writerError as NSError {
error = writerError
videoWriter = nil
}
if let videoWriter = videoWriter {
let videoSettings: [String : AnyObject] = [
AVVideoCodecKey : AVVideoCodecH264,
AVVideoWidthKey : outputSize.width,
AVVideoHeightKey : outputSize.height,
// AVVideoCompressionPropertiesKey : [
// AVVideoAverageBitRateKey : NSInteger(1000000),
// AVVideoMaxKeyFrameIntervalKey : NSInteger(16),
// AVVideoProfileLevelKey : AVVideoProfileLevelH264BaselineAutoLevel
// ]
]
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
let sourceBufferAttributes = [String : AnyObject](dictionaryLiteral:
(kCVPixelBufferPixelFormatTypeKey as String, Int(kCVPixelFormatType_32ARGB)),
(kCVPixelBufferWidthKey as String, Float(inputSize.width)),
(kCVPixelBufferHeightKey as String, Float(inputSize.height))
)
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(
assetWriterInput: videoWriterInput,
sourcePixelBufferAttributes: sourceBufferAttributes
)
assert(videoWriter.canAddInput(videoWriterInput))
videoWriter.addInput(videoWriterInput)
if videoWriter.startWriting() {
videoWriter.startSessionAtSourceTime(kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = dispatch_queue_create("mediaInputQueue", nil)
videoWriterInput.requestMediaDataWhenReadyOnQueue(media_queue, usingBlock: { () -> Void in
let fps: Int32 = 30
let frameDuration = CMTimeMake(1, fps)
let currentProgress = NSProgress(totalUnitCount: Int64(self.photoURLs.count))
var frameCount: Int64 = 0
var remainingPhotoURLs = [String](self.photoURLs)
while (videoWriterInput.readyForMoreMediaData && !remainingPhotoURLs.isEmpty) {
let nextPhotoURL = remainingPhotoURLs.removeAtIndex(0)
let lastFrameTime = CMTimeMake(frameCount, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
if !self.appendPixelBufferForImageAtURL(nextPhotoURL, pixelBufferAdaptor: pixelBufferAdaptor, presentationTime: presentationTime) {
error = NSError(
domain: kErrorDomain,
code: kFailedToAppendPixelBufferError,
userInfo: [
"description": "AVAssetWriterInputPixelBufferAdapter failed to append pixel buffer",
"rawError": videoWriter.error ?? "(none)"
]
)
break
}
frameCount++
currentProgress.completedUnitCount = frameCount
progress(currentProgress)
}
videoWriterInput.markAsFinished()
videoWriter.finishWritingWithCompletionHandler { () -> Void in
if error == nil {
success(videoOutputURL)
}
self.videoWriter = nil
}
})
} else {
error = NSError(
domain: kErrorDomain,
code: kFailedToStartAssetWriterError,
userInfo: ["description": "AVAssetWriter failed to start writing"]
)
}
}
if let error = error {
failure(error)
}
}
func appendPixelBufferForImageAtURL(url: String, pixelBufferAdaptor: AVAssetWriterInputPixelBufferAdaptor, presentationTime: CMTime) -> Bool {
var appendSucceeded = false
autoreleasepool {
if let url = NSURL(string: url),
let imageData = NSData(contentsOfURL: url),
let image = UIImage(data: imageData),
let pixelBufferPool = pixelBufferAdaptor.pixelBufferPool {
let pixelBufferPointer = UnsafeMutablePointer<CVPixelBuffer?>.alloc(1)
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(
kCFAllocatorDefault,
pixelBufferPool,
pixelBufferPointer
)
if let pixelBuffer = pixelBufferPointer.memory where status == 0 {
fillPixelBufferFromImage(image, pixelBuffer: pixelBuffer)
appendSucceeded = pixelBufferAdaptor.appendPixelBuffer(
pixelBuffer,
withPresentationTime: presentationTime
)
pixelBufferPointer.destroy()
} else {
NSLog("error: Failed to allocate pixel buffer from pool")
}
pixelBufferPointer.dealloc(1)
}
}
return appendSucceeded
}
func fillPixelBufferFromImage(image: UIImage, pixelBuffer: CVPixelBufferRef) {
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGBitmapContextCreate(
pixelData,
Int(image.size.width),
Int(image.size.height),
8,
CVPixelBufferGetBytesPerRow(pixelBuffer),
rgbColorSpace,
CGImageAlphaInfo.PremultipliedFirst.rawValue
)
CGContextDrawImage(context, CGRectMake(0, 0, image.size.width, image.size.height), image.CGImage)
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}
}
//
// TimeLapseBuilder30.swift
//
// Created by Adam Jensen on 11/18/16.
//
// NOTE: This implementation is written in Swift 3.0.
import AVFoundation
import UIKit
let kErrorDomain = "TimeLapseBuilder"
let kFailedToStartAssetWriterError = 0
let kFailedToAppendPixelBufferError = 1
class TimeLapseBuilder: NSObject {
let photoURLs: [String]
var videoWriter: AVAssetWriter?
init(photoURLs: [String]) {
self.photoURLs = photoURLs
}
func build(_ progress: @escaping ((Progress) -> Void), success: @escaping ((URL) -> Void), failure: ((NSError) -> Void)) {
let inputSize = CGSize(width: 4000, height: 3000)
let outputSize = CGSize(width: 1280, height: 720)
var error: NSError?
let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0] as NSString
let videoOutputURL = URL(fileURLWithPath: documentsPath.appendingPathComponent("AssembledVideo.mov"))
do {
try FileManager.default.removeItem(at: videoOutputURL)
} catch {}
do {
try videoWriter = AVAssetWriter(outputURL: videoOutputURL, fileType: AVFileTypeQuickTimeMovie)
} catch let writerError as NSError {
error = writerError
videoWriter = nil
}
if let videoWriter = videoWriter {
let videoSettings: [String : AnyObject] = [
AVVideoCodecKey : AVVideoCodecH264 as AnyObject,
AVVideoWidthKey : outputSize.width as AnyObject,
AVVideoHeightKey : outputSize.height as AnyObject,
// AVVideoCompressionPropertiesKey : [
// AVVideoAverageBitRateKey : NSInteger(1000000),
// AVVideoMaxKeyFrameIntervalKey : NSInteger(16),
// AVVideoProfileLevelKey : AVVideoProfileLevelH264BaselineAutoLevel
// ]
]
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
let sourceBufferAttributes = [
(kCVPixelBufferPixelFormatTypeKey as String): Int(kCVPixelFormatType_32ARGB),
(kCVPixelBufferWidthKey as String): Float(inputSize.width),
(kCVPixelBufferHeightKey as String): Float(inputSize.height)] as [String : Any]
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(
assetWriterInput: videoWriterInput,
sourcePixelBufferAttributes: sourceBufferAttributes
)
assert(videoWriter.canAdd(videoWriterInput))
videoWriter.add(videoWriterInput)
if videoWriter.startWriting() {
videoWriter.startSession(atSourceTime: kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = DispatchQueue(label: "mediaInputQueue")
videoWriterInput.requestMediaDataWhenReady(on: media_queue) {
let fps: Int32 = 30
let frameDuration = CMTimeMake(1, fps)
let currentProgress = Progress(totalUnitCount: Int64(self.photoURLs.count))
var frameCount: Int64 = 0
var remainingPhotoURLs = [String](self.photoURLs)
while videoWriterInput.isReadyForMoreMediaData && !remainingPhotoURLs.isEmpty {
let nextPhotoURL = remainingPhotoURLs.remove(at: 0)
let lastFrameTime = CMTimeMake(frameCount, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
if !self.appendPixelBufferForImageAtURL(nextPhotoURL, pixelBufferAdaptor: pixelBufferAdaptor, presentationTime: presentationTime) {
error = NSError(
domain: kErrorDomain,
code: kFailedToAppendPixelBufferError,
userInfo: ["description": "AVAssetWriterInputPixelBufferAdapter failed to append pixel buffer"]
)
break
}
frameCount += 1
currentProgress.completedUnitCount = frameCount
progress(currentProgress)
}
videoWriterInput.markAsFinished()
videoWriter.finishWriting {
if error == nil {
success(videoOutputURL)
}
self.videoWriter = nil
}
}
} else {
error = NSError(
domain: kErrorDomain,
code: kFailedToStartAssetWriterError,
userInfo: ["description": "AVAssetWriter failed to start writing"]
)
}
}
if let error = error {
failure(error)
}
}
func appendPixelBufferForImageAtURL(_ url: String, pixelBufferAdaptor: AVAssetWriterInputPixelBufferAdaptor, presentationTime: CMTime) -> Bool {
var appendSucceeded = false
autoreleasepool {
if let url = URL(string: url),
let imageData = try? Data(contentsOf: url),
let image = UIImage(data: imageData),
let pixelBufferPool = pixelBufferAdaptor.pixelBufferPool {
let pixelBufferPointer = UnsafeMutablePointer<CVPixelBuffer?>.allocate(capacity: 1)
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(
kCFAllocatorDefault,
pixelBufferPool,
pixelBufferPointer
)
if let pixelBuffer = pixelBufferPointer.pointee, status == 0 {
fillPixelBufferFromImage(image, pixelBuffer: pixelBuffer)
appendSucceeded = pixelBufferAdaptor.append(
pixelBuffer,
withPresentationTime: presentationTime
)
pixelBufferPointer.deinitialize()
} else {
NSLog("error: Failed to allocate pixel buffer from pool")
}
pixelBufferPointer.deallocate(capacity: 1)
}
}
return appendSucceeded
}
func fillPixelBufferFromImage(_ image: UIImage, pixelBuffer: CVPixelBuffer) {
CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(
data: pixelData,
width: Int(image.size.width),
height: Int(image.size.height),
bitsPerComponent: 8,
bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
space: rgbColorSpace,
bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue
)
context?.draw(image.cgImage!, in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))
CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
}
}
@jacobokoenig
Copy link

Lifesaver!!!!!

@seyhagithub
Copy link

hello everyone, my project is streaming jpeg frames from api, api response as image. so i want to recorded as video. How can I do with this source code because i see it accept string as path. please help me and thank you in advance

@acj
Copy link
Author

acj commented Feb 23, 2017

@seyhagithub Hi there. If you're receiving raw jpeg frames, you could adapt the appendPixelBufferForImageAtURL(url:pixelBufferAdapter:presentationTime:) method to take an NSData or UIImage instead. Most of the work would be removing the code that makes an NSData object for each URL.

@norsez
Copy link

norsez commented Mar 5, 2017

Thanks for this!

@seanmcneil
Copy link

seanmcneil commented Mar 9, 2017

If anyone is interested, I used acj's Swift 3.0 work as a basis to create a Cocoapod for writing images out to videos that you can install via pod Spitfire. Thanks to acj & others who contributed for helping me out. Figured this could be a good way to pay back :)

@Salman-Majid
Copy link

Guys, i used this project to build my application that takes some images and a music file and then convert them to video. But the images don't have cool animations. They just show and disappear. I want to apply animations to them. I have done a little research about CALayer class but i found nothing about adding animations to video in it. Can any body help me so that i can add desired animations to individual photos. That will be great. Thanks

@acj
Copy link
Author

acj commented Mar 12, 2017

@seanmcneil nice! Thanks for sharing.

@Salman-Majid I recommend looking at AVMutableComposition. You can use it to mix multiple tracks, including Core Animation layers. There are a few tutorials and WWDC videos that cover the details. Please refer any followup questions to Stack Overflow. Good luck!

@acj
Copy link
Author

acj commented Mar 12, 2017

This code has moved

Please use TimeLapseBuilder-Swift on GitHub instead.

Thank you, everyone, for your kind feedback, comments, and contributions. TimeLapseBuilder has outgrown this gist, and it's gotten difficult to manage the code and conversation. I've created a repository that should be used instead. Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment