Skip to content

Instantly share code, notes, and snippets.

@jethrokuan
Last active July 19, 2021 10:18
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jethrokuan/02f41028fb4a6f81787dc420fb99b6e4 to your computer and use it in GitHub Desktop.
Save jethrokuan/02f41028fb4a6f81787dc420fb99b6e4 to your computer and use it in GitHub Desktop.
Migration script
;;; THIS MIGRATION SCRIPT IS NOW DEPRECATED.
;;; PLEASE USE M-x org-roam-migrate-wizard, FOUND WITHIN THE ORG-ROAM REPO ITSELF.
@d12frosted
Copy link

This also should cover tags migration according to org-roam-tag-sources.

@d12frosted
Copy link

d12frosted commented May 10, 2021

Seems like (kill-line) should be (kill-line 1) to kill through line and not just the rest of line.

You should get aliases using (split-string-and-unquote (cadar (org-collect-keywords '("roam_alias")))) to avoid the described issue with multi-word aliases. And asumming that the alias value is valid, you can simply get it and set it as is - the same with ref I think.

This is what I used for migration (requires d12frosted/vulpea#92):

(defun vulpea-migrate-buffer ()
  "Migrate current buffer note to `org-roam' v2."
  ;; Create file level ID if it doesn't exist yet
  (org-with-point-at 1
    (org-id-get-create))

  ;; update title (just to make sure it's lowercase)
  (vulpea-buffer-title-set (vulpea-buffer-prop-get "title"))

  ;; move roam_key into properties drawer roam_ref
  (when-let* ((ref (vulpea-buffer-prop-get "roam_key")))
    (org-set-property "ROAM_REFS" ref)
    (let ((case-fold-search t))
      (org-with-point-at 1
        (while (re-search-forward "^#\\+roam_key:" (point-max) t)
          (beginning-of-line)
          (kill-line 1)))))

  ;; move roam_alias into properties drawer roam_aliases
  (when-let* ((aliases (vulpea-buffer-prop-get-list "roam_alias")))
    (org-set-property "ROAM_ALIASES"
                      (combine-and-quote-strings aliases))
    (let ((case-fold-search t))
      (org-with-point-at 1
        (while (re-search-forward "^#\\+roam_alias:" (point-max) t)
          (beginning-of-line)
          (kill-line 1)))))

  ;; move roam_tags into filetags
  (let* ((roam-tags (vulpea-buffer-prop-get-list "roam_tags"))
         (file-tags (vulpea-buffer-prop-get-list "filetags"))
         (path-tags (seq-filter
                     (lambda (x) (not (string-empty-p x)))
                     (split-string
                      (string-remove-prefix
                       org-roam-directory
                       (file-name-directory (buffer-file-name)))
                      "/")))
         (tags (seq-map
                (lambda (tag)
                  (setq tag (replace-regexp-in-string
                             ;; see `org-tag-re'
                             "[^[:alnum:]_@#%]"
                             "_"        ; use any valid char - _@#%
                             tag))
                  (if (or
                       (string-prefix-p "status" tag 'ignore-case)
                       (string-prefix-p "content" tag 'ignore-case)
                       (string-equal "Project" tag))
                      (setq tag (downcase tag)))
                  tag)
                (seq-uniq (append roam-tags file-tags path-tags)))))
    (when tags
      (apply #'vulpea-buffer-tags-set tags)
      (let ((case-fold-search t))
        (org-with-point-at 1
          (while (re-search-forward "^#\\+roam_tags:" (point-max) t)
            (beginning-of-line)
            (kill-line 1))))))

  (save-buffer))

(defun vulpea-migrate-db ()
  "Migrate all notes."
  (dolist (f (org-roam--list-all-files))
    (with-current-buffer (find-file f)
      (message "migrating %s" f)
      (vulpea-migrate-buffer)))

  ;; Step 2: Build cache
  (org-roam-db-sync 'force))

Update on 2021-05-18: the script doesn't override previous filetags anymore.

Update on 2021-06-09: the script handles tags much better by properly sanitizing names according to org-tag-re.

@junwei-wang
Copy link

Bugs when "#+roam_alias" exists:

$ git diff sha_2.org                
diff --git a/roam/sha_2.org b/roam/sha_2.org
index 1f07744..9155df8 100644
--- a/roam/sha_2.org
+++ b/roam/sha_2.org
@@ -1,8 +1,11 @@
+:PROPERTIES:
+:ID:       c20ceaae-7d3d-4f06-83a6-ba149b98f53f
+:ROAM_ALIASES: "Secure Hash Algorithm 2"
+:END:
 #+TITLE: SHA-2
 #+CREATED: [2021-04-23 Fri 22:06]
-#+LAST_MODIFIED: [2021-04-24 Sat 20:29]
-#+ROAM_ALIAS: "Secure Hash Algorithm 2"
-#+ROAM_TAGS: crypto hash
+#+LAST_MODIFIED: [2021-05-14 Fri 14:18]
+#+ROAM_ALIAS:#+FILETAGS: crypto hash

 * Overview

@junwei-wang
Copy link

I know org-roam-bibtex does not support v2 yet, but there is also a small bug in your migration script, for example

diff --git a/roam/biblio/AC:BCGHKR11.org b/roam/biblio/AC:BCGHKR11.org
index ece11a4..6e035e4 100644
--- a/roam/biblio/AC:BCGHKR11.org
+++ b/roam/biblio/AC:BCGHKR11.org
@@ -1,7 +1,10 @@
+:PROPERTIES:
+:ID:       3c8b23c9-1942-45d6-b1a1-ee66d24a98a6
+:ROAM_REFS: cite:AC:BCGHKR11
+:END:
 #+TITLE: 2011 - [AC:BCGHKR11] - Program obfuscation with leaky hardware -- Bitansky, Canetti, Goldwasser, Halevi, Kalai, Rothblum
 #+CREATED: [2021-01-21 Thu 17:37]
-#+LAST_MODIFIED: [2021-01-21 Thu 17:37]
-#+ROAM_KEY: cite:AC:BCGHKR11
-#+ROAM_ALIAS:
-#+ROAM_TAGS: paper crypto obfuscation
+#+LAST_MODIFIED: [2021-05-14 Fri 14:17]
+#+ROAM_KEY:#+ROAM_ALIAS:
+#+FILETAGS: paper crypto obfuscation
 #+PROPERTY: authors Bitansky, N., Canetti, R., Goldwasser, S., Halevi, S., Kalai, Y. T., & Rothblum, G. N.

@yosevu
Copy link

yosevu commented Jun 14, 2021

M-x-evaluate-buffer should be M-x eval-buffer.

@JonathanReeve
Copy link

Correct me if I'm wrong, but this won't work, since org-roam-db-sync appears to be a v2 function, and other functions, like org-roam--list-all-files are v1. Unless I'm missing something?

@rodelrod
Copy link

rodelrod commented Jul 10, 2021

@JonathanReeve: to make it work, I've ran step 1 in V1 and then steps 2 and 3 in V2.
Then I had to fix all of the files with ROAM_ALIASES by hand because they were borked. I didn't have many so I did it manually.
It was transforming something like:

#+title: PostgreSQL
#+created: [2020-04-03 17:22]
#+roam_alias: Postgres
- related :: [[file:databases.org][Databases]]

into something similar to:

:PROPERTIES:
:ID:       b6d673e3-d117-4bbe-806c-817effcb6050
:ROAM_ALIASES: [[file:databases.org][Databases]]
:END:
#+title: PostgreSQL
#+created: [2020-04-03 17:22]
#+roam_alias: 

Same thing with ROAM_KEY/ ROAM_REFS.

@rodelrod
Copy link

M-x-evaluate-buffer should be M-x eval-buffer.

Confirmed

@failable
Copy link

When I eval step 1, I got user-error: Before first headline at position 1 in buffer conditional_random_fields.org. Any idea?

@japhir
Copy link

japhir commented Jul 19, 2021

I get some errors with titles that use entities, such as \delta or \endash.

Here's the error message I get:

Debugger entered--Lisp error: (error "Invalid use of ‘\\’ in replacement text")
  vulpea-buffer-prop-set("title" "dosSantos2010: Glacial\\textendash interglacial var...")
  vulpea-buffer-title-set("dosSantos2010: Glacial\\textendash interglacial var...")
  vulpea-migrate-buffer()
  (save-current-buffer (set-buffer (find-file f)) (message "migrating %s" f) (vulpea-migrate-buffer))
  (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer))
  (while --dolist-tail-- (setq f (car --dolist-tail--)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (let ((--dolist-tail-- (org-roam--list-all-files)) f) (while --dolist-tail-- (setq f (car --dolist-tail--)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (dolist (f (org-roam--list-all-files)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)))
  vulpea-migrate-db()
  eval-expression((vulpea-migrate-db) nil nil 127)
  funcall-interactively(eval-expression (vulpea-migrate-db) nil nil 127)
  command-execute(eval-expression)

@japhir
Copy link

japhir commented Jul 19, 2021

(if I just replace the offending entity with the corresponding utf-8 symbol and run it again, it appears to continue nicely).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment