This document describes my setup for taking notes from videos, using Emacs and org-mode.
My goals:
- Embed & playback video links into my notes
- Generate, embed & post process animated from my notes
A very common use case for me is note taking from BJJ instructionals. Such videos are long in duration and describe sequences that you need to iterate at least a couple of times in order to fully understand. Often it helps converting such sequences into:
- UML diagrams
- Flashcards
- Animated images or short clips
So, I needed a way to link a particular note to a particular point in the video and also capture parts of such videos as animated images so that I can embed in my notes.
I am often watching conference talks on my computer. Usually, those talks are downloaded locally (to make sure, I can find them again if needed). Often, I need to place a bookmark at a particular point in the talk, or even create a link (as a reference) in one of my talks, blogs etc. Since, all these are handled by org-mode, it was important to be able to link org entries to points in media files.
I am already using bongo as media player linked for things like elfeed, so it made sense to base my solution to that. If you want to learn more about the combination above do watch: Emacs podcast manager with bongo+elfeed.
This literate blog contains my whole setup and should be an autonomous unit that you can drop in to you configuration using org-babel.
I have this file under: `~/Documents/org/roam/video-notes.org`. The file is used both as a literate config file and also as the place where my notes are being stored.
I load this file using:
(org-babel-load-file "~/Documents/org/roam/video-notes.org")
Each time I playback videos using bongo, an instancee of `mplayer` is started behind the scenes, that communicates with Emacs in order to get information like the file being currently playing and the elapsed time.
From `org-capture` (usually bound to `C-c C-c`) I press `v n` to create a note. An org entry pointing to the particular point in the video will be created and stored in this file under `Notes`. Each time I need to visit that particular video, I just need to visit that particular entry and invoke the command `iocanel/bongo-play-org-entry-at-point`.
To capture parts of the video playing as an animation I need to mark the start and the end of the animation using `iocanel/bongo-mark-selection-start` (C-c b m s) and `iocanel/mark-selection-end` (C-c b m e) respectively. From `org-capture` (usually bound to `C-c C-c`) I press `v a` to create a note with an embeded animation. To fine tune the animation, you can modify the animation parameters, which are added as properties to entry and then invoke `iocanel/recreate-animation-at-point`.
The attached image, can be displayed inline using: `org-display-inline-images`. To watch the animation from within emacs you can follow this guide: Displaying animated gifs from within Emacs
Below you can find:
- templates
- functions
- configuration
- Place holder for the notes
The template below describes the structure of the enry created when I want to capture a point in a particular media file. It pretty much helps me create an entry with properties like:
- url
- timestamp
* %^{Video Name} :video: :PROPERTIES: :URL: %(iocanel/bongo-currently-playing-url) :TIMESTAMP: %(iocanel/bongo-currently-playing-elapsed-time) :END: %?
* %(iocanel/bongo-choose-animation-name) :animation:
:PROPERTIES:
:URL: %(iocanel/bongo-choose-selection-url)
:ANIMATION_START: %(iocanel/bongo-choose-selection-start)
:ANIMATION_END: %(iocanel/bongo-choose-selection-end)
:END:
[[%(iocanel/bongo-animation-output-file)]]
%(iocanel/bongo-create-animation)
%?
This section provides some helper functions for:
- media -> org entry
- org -> media
If not obvious, this code is used to jump from media to org entries and vice versa.
(require 'bongo)
;;
;; State
;;
(defvar iocanel/bongo-animation-name nil "Animation name")
(defvar iocanel/bongo-selection-url nil "Selection url")
(defvar iocanel/bongo-selection-start nil "Selection start")
(defvar iocanel/bongo-selection-end nil "Selection end")
;;
;; Utilities
;;
(defun iocanel/strip-file-scheme (url)
"Mark the start of the selection."
(interactive)
(if (and url (s-starts-with? "file://" url))
(substring url 7)
url))
(defun iocanel/bongo-choose-animation-name ()
"Choose selection url. This function is meant to be used from within templates so that selected value is used to update the selection url."
(setq iocanel/bongo-animation-name (completing-read "Animation name:" `(,(iocanel/strip-file-scheme
(file-name-sans-extension (file-name-nondirectory (iocanel/bongo-currently-playing-url))))) (lambda (a) t) nil)))
(defun iocanel/bongo-choose-selection-url ()
"Choose selection url. This function is meant to be used from within templates so that selected value is used to update the selection url."
(setq iocanel/bongo-selection-url (completing-read "Selection url:" `(,(iocanel/bongo-currently-playing-url)) (lambda (a) t) nil)))
;; Selection start
(defun iocanel/bongo-choose-selection-start ()
"Choose selection start. This function is meant to be used from within templates so that selected value is used to update the selection start."
(setq iocanel/bongo-selection-start (completing-read "Selection start:" `(,(iocanel/bongo-get-selection-start)) (lambda (a) t) nil)))
(defun iocanel/bongo-get-selection-start ()
"Return the selection start"
(cond ((stringp iocanel/bongo-selection-start) iocanel/bongo-selection-start)
((numberp iocanel/bongo-selection-start) (number-to-string iocanel/bongo-selection-start))
(t nil)))
(defun iocanel/bongo-mark-selection-start ()
"Mark the start of the selection."
(interactive)
(setq iocanel/bongo-selection-start (iocanel/bongo-currently-playing-elapsed-time)))
;; Selection end
(defun iocanel/bongo-choose-selection-end ()
"Choose selection end. This function is meant to be used from within templates so that selected value is used to update the selection end."
(setq iocanel/bongo-selection-end (completing-read "Selection end:" `(,(iocanel/bongo-get-selection-end)) (lambda (a) t) nil)))
(defun iocanel/bongo-get-selection-end ()
"Return the selection end"
(cond ((stringp iocanel/bongo-selection-end) iocanel/bongo-selection-end)
((numberp iocanel/bongo-selection-end) (number-to-string iocanel/bongo-selection-end))
(t nil)))
(defun iocanel/bongo-mark-selection-end ()
"Mark the end of the selection."
(interactive)
(setq iocanel/bongo-selection-end (iocanel/bongo-currently-playing-elapsed-time)))
;; Animation
(defun iocanel/bongo-create-animation ()
"Create an animation from the selection start to the selection end."
(interactive)
(let* ((url (or iocanel/bongo-selection-url (iocanel/bongo-currently-playing-url)))
(path (iocanel/strip-file-scheme url))
(dir (file-name-directory path))
(name (or iocanel/bongo-animation-name (file-name-sans-extension (file-name-nondirectory path))))
(output-file (concat dir name ".gif"))
(start (bongo-format-seconds (string-to-number (iocanel/bongo-get-selection-start))))
(end (bongo-format-seconds (string-to-number (iocanel/bongo-get-selection-end)))))
(iocanel/bongo-do-create-animation url start end name)))
(defun iocanel/bongo-do-create-animation (url start end &optional name framerate scale)
"Create an animation using the URL from the selection START to the selection END. Optionally, specify the output NAME the FRAMERATE and SCALE."
(interactive)
(let* ((path (iocanel/strip-file-scheme url))
(dir (file-name-directory path))
(name (or name iocanel/bongo-animation-name (file-name-sans-extension (file-name-nondirectory path))))
(output-file (concat dir name ".gif"))
(framerate (or framerate 5))
(scale (or scale "512:-1")))
(if (and path start end)
(progn
(message "Creating animated gif: %s for video: %s in dir:%s" output-file path dir)
(async-shell-command (format "ffmpeg -y -i %s -r %s -vf scale=%s -ss %s -to %s %s"
(shell-quote-argument path)
framerate
scale
start
end
(shell-quote-argument output-file))))
output-file
(progn
(message "Can't create animated gif for file: %s bounds (%s - %s)." path start end)
nil))))
(defun iocanel/bongo-animation-output-file ()
"Returns the path to the animation output file."
(let* ((url (or iocanel/bongo-selection-url (iocanel/bongo-currently-playing-url)))
(path (iocanel/strip-file-scheme url))
(dir (file-name-directory path))
(name (or iocanel/bongo-animation-name (file-name-sans-extension (file-name-nondirectory path))))
(output-file (concat dir name ".gif")))
output-file))
;;
;; Extract information from bongo player
;;
(defun iocanel/bongo-currently-playing-elapsed-time()
"Return the elapsed time of the media file currently playing."
(interactive)
(bongo-elapsed-time))
(defun iocanel/bongo-currently-playing-url ()
"Return the url of the media file currently playing."
(interactive)
(with-bongo-playlist-buffer
(cdr (assoc 'file-name (cdr bongo-player)))))
;;
;; Org entries
;;
(defun iocanel/bongo-play-org-entry-at-point ()
"Play the play the media file linked to the org-entry at point."
(interactive)
(let* ((p (point))
(url (org-entry-get nil "URL"))
(elapsed (org-entry-get nil "ELAPSED")))
(with-bongo-playlist-buffer
(bongo-insert-uri url)
(bongo-previous-object)
(bongo-play)
(when (stringp elapsed) (bongo-seek-to (string-to-number elapsed))))))
(defun iocanel/bongo-recreate-animation-at-point ()
"Recreate the animation that corresponds to the particular element. This allows tuning of parameters"
(interactive)
(let* ((p (point))
(url (org-entry-get nil "URL"))
(name (org-get-heading t t t t))
(framerate (org-entry-get nil "FRAMERATE"))
(scale (org-entry-get nil "SCALE"))
(start (org-entry-get nil "ANIMATION_START"))
(end (org-entry-get nil "ANIMATION_END")))
(message "Recreating animation for: %s using file: %s (%s-%s)" name url start end)
(iocanel/bongo-do-create-animation url start end name framerate scale)))
;;
;; Bindings
;;
(global-set-key (kbd "C-c b m s") 'iocanel/bongo-mark-selection-start)
(global-set-key (kbd "C-c b m e") 'iocanel/bongo-mark-selection-end)
(global-set-key (kbd "C-c b g") 'iocanel/bongo-create-animation)
;;
;; Advices
;;
(defadvice bongo-play-line (after bongo-play-line-around activate)
"Check if bluetooth is connected and use pulse audio driver."
(interactive)
(setq iocanel/bongo-selection-url (iocanel/bongo-currently-playing-url)))
For this setup to we work we need to configure org-catpure. I want all my video notes to land under `~/Documents/org/roam/video-notes.org` and from there I can move notes as I feel like.
(setq org-capture-templates (append org-capture-templates '(
("v" "Video")
("vn" "Notes" entry (file+olp "~/Documents/org/roam/video-notes.org" "Notes")(file "~/Documents/org/templates/video-note.orgtmpl"))
("vg" "Animation" entry (file+olp "~/Documents/org/roam/video-notes.org" "Notes")(file "~/Documents/org/templates/animation.orgtmpl")))))This is the place where captured notes are being added. From here notes can be refiled to where they belong.
The entries below are just examples:
This is an animated gif of how Mickey Musumeci performs the straight ankle lock. Original video can be found at: https://www.youtube.com/watch?v=JD3QucUWIwI
I'm looking for a workflow to take notes from Youtube videos on Emacs. Is this something that could be used?