Migration script

migrate.el

;;; THIS MIGRATION SCRIPT IS NOW DEPRECATED.
;;; PLEASE USE M-x org-roam-migrate-wizard, FOUND WITHIN THE ORG-ROAM REPO ITSELF.

Thanks for the script. I had some issues that required manual intervention

1. Handling of links with backslashes in link text (e.g. for pretty symbols)

When migrating I got the error

progn: Invalid use of ‘\’ in replacement text

It seems the error was cause by roam file links with backslashes in the description, like the following:

 [[file:~/org/roam/20201208145501-b_to_mu_nu.org][B \to \mu\nu]]

Maybe it was also the target file #+title, which also had the same backslash string. I fixed both to be sure and it worked.

2. Wrong quotes in alias conversion

I e.g. get

:ROAM_ALIASES: "NN \"neural network\""

where the outer quotes were not present in the original string, it should be NN "neural network".

3. Aliases added to wrong property drawer

This happened in files where I already had a property ID drawer at the top of the file and the #+ROAM_ALIAS: definition below it. Then, the aliases were added to the property drawer in the next org headline instead of the property drawer at the top of the file.

This also should cover tags migration according to org-roam-tag-sources.

Seems like (kill-line) should be (kill-line 1) to kill through line and not just the rest of line.

You should get aliases using (split-string-and-unquote (cadar (org-collect-keywords '("roam_alias")))) to avoid the described issue with multi-word aliases. And asumming that the alias value is valid, you can simply get it and set it as is - the same with ref I think.

This is what I used for migration (requires d12frosted/vulpea#92):

(defun vulpea-migrate-buffer ()
  "Migrate current buffer note to `org-roam' v2."
  ;; Create file level ID if it doesn't exist yet
  (org-with-point-at 1
    (org-id-get-create))

  ;; update title (just to make sure it's lowercase)
  (vulpea-buffer-title-set (vulpea-buffer-prop-get "title"))

  ;; move roam_key into properties drawer roam_ref
  (when-let* ((ref (vulpea-buffer-prop-get "roam_key")))
    (org-set-property "ROAM_REFS" ref)
    (let ((case-fold-search t))
      (org-with-point-at 1
        (while (re-search-forward "^#\\+roam_key:" (point-max) t)
          (beginning-of-line)
          (kill-line 1)))))

  ;; move roam_alias into properties drawer roam_aliases
  (when-let* ((aliases (vulpea-buffer-prop-get-list "roam_alias")))
    (org-set-property "ROAM_ALIASES"
                      (combine-and-quote-strings aliases))
    (let ((case-fold-search t))
      (org-with-point-at 1
        (while (re-search-forward "^#\\+roam_alias:" (point-max) t)
          (beginning-of-line)
          (kill-line 1)))))

  ;; move roam_tags into filetags
  (let* ((roam-tags (vulpea-buffer-prop-get-list "roam_tags"))
         (file-tags (vulpea-buffer-prop-get-list "filetags"))
         (path-tags (seq-filter
                     (lambda (x) (not (string-empty-p x)))
                     (split-string
                      (string-remove-prefix
                       org-roam-directory
                       (file-name-directory (buffer-file-name)))
                      "/")))
         (tags (seq-map
                (lambda (tag)
                  (setq tag (replace-regexp-in-string
                             ;; see `org-tag-re'
                             "[^[:alnum:]_@#%]"
                             "_"        ; use any valid char - _@#%
                             tag))
                  (if (or
                       (string-prefix-p "status" tag 'ignore-case)
                       (string-prefix-p "content" tag 'ignore-case)
                       (string-equal "Project" tag))
                      (setq tag (downcase tag)))
                  tag)
                (seq-uniq (append roam-tags file-tags path-tags)))))
    (when tags
      (apply #'vulpea-buffer-tags-set tags)
      (let ((case-fold-search t))
        (org-with-point-at 1
          (while (re-search-forward "^#\\+roam_tags:" (point-max) t)
            (beginning-of-line)
            (kill-line 1))))))

  (save-buffer))

(defun vulpea-migrate-db ()
  "Migrate all notes."
  (dolist (f (org-roam--list-all-files))
    (with-current-buffer (find-file f)
      (message "migrating %s" f)
      (vulpea-migrate-buffer)))

  ;; Step 2: Build cache
  (org-roam-db-sync 'force))

Update on 2021-05-18: the script doesn't override previous filetags anymore.

Update on 2021-06-09: the script handles tags much better by properly sanitizing names according to org-tag-re.

Bugs when "#+roam_alias" exists:

$ git diff sha_2.org                
diff --git a/roam/sha_2.org b/roam/sha_2.org
index 1f07744..9155df8 100644
--- a/roam/sha_2.org
+++ b/roam/sha_2.org
@@ -1,8 +1,11 @@
+:PROPERTIES:
+:ID:       c20ceaae-7d3d-4f06-83a6-ba149b98f53f
+:ROAM_ALIASES: "Secure Hash Algorithm 2"
+:END:
 #+TITLE: SHA-2
 #+CREATED: [2021-04-23 Fri 22:06]
-#+LAST_MODIFIED: [2021-04-24 Sat 20:29]
-#+ROAM_ALIAS: "Secure Hash Algorithm 2"
-#+ROAM_TAGS: crypto hash
+#+LAST_MODIFIED: [2021-05-14 Fri 14:18]
+#+ROAM_ALIAS:#+FILETAGS: crypto hash

 * Overview

I know org-roam-bibtex does not support v2 yet, but there is also a small bug in your migration script, for example

diff --git a/roam/biblio/AC:BCGHKR11.org b/roam/biblio/AC:BCGHKR11.org
index ece11a4..6e035e4 100644
--- a/roam/biblio/AC:BCGHKR11.org
+++ b/roam/biblio/AC:BCGHKR11.org
@@ -1,7 +1,10 @@
+:PROPERTIES:
+:ID:       3c8b23c9-1942-45d6-b1a1-ee66d24a98a6
+:ROAM_REFS: cite:AC:BCGHKR11
+:END:
 #+TITLE: 2011 - [AC:BCGHKR11] - Program obfuscation with leaky hardware -- Bitansky, Canetti, Goldwasser, Halevi, Kalai, Rothblum
 #+CREATED: [2021-01-21 Thu 17:37]
-#+LAST_MODIFIED: [2021-01-21 Thu 17:37]
-#+ROAM_KEY: cite:AC:BCGHKR11
-#+ROAM_ALIAS:
-#+ROAM_TAGS: paper crypto obfuscation
+#+LAST_MODIFIED: [2021-05-14 Fri 14:17]
+#+ROAM_KEY:#+ROAM_ALIAS:
+#+FILETAGS: paper crypto obfuscation
 #+PROPERTY: authors Bitansky, N., Canetti, R., Goldwasser, S., Halevi, S., Kalai, Y. T., & Rothblum, G. N.

M-x-evaluate-buffer should be M-x eval-buffer.

Correct me if I'm wrong, but this won't work, since org-roam-db-sync appears to be a v2 function, and other functions, like org-roam--list-all-files are v1. Unless I'm missing something?

@JonathanReeve: to make it work, I've ran step 1 in V1 and then steps 2 and 3 in V2.
Then I had to fix all of the files with ROAM_ALIASES by hand because they were borked. I didn't have many so I did it manually.
It was transforming something like:

#+title: PostgreSQL
#+created: [2020-04-03 17:22]
#+roam_alias: Postgres
- related :: [[file:databases.org][Databases]]

into something similar to:

:PROPERTIES:
:ID:       b6d673e3-d117-4bbe-806c-817effcb6050
:ROAM_ALIASES: [[file:databases.org][Databases]]
:END:
#+title: PostgreSQL
#+created: [2020-04-03 17:22]
#+roam_alias: 

Same thing with ROAM_KEY/ ROAM_REFS.

M-x-evaluate-buffer should be M-x eval-buffer.

Confirmed

When I eval step 1, I got user-error: Before first headline at position 1 in buffer conditional_random_fields.org. Any idea?

I get some errors with titles that use entities, such as \delta or \endash.

Here's the error message I get:

Debugger entered--Lisp error: (error "Invalid use of ‘\\’ in replacement text")
  vulpea-buffer-prop-set("title" "dosSantos2010: Glacial\\textendash interglacial var...")
  vulpea-buffer-title-set("dosSantos2010: Glacial\\textendash interglacial var...")
  vulpea-migrate-buffer()
  (save-current-buffer (set-buffer (find-file f)) (message "migrating %s" f) (vulpea-migrate-buffer))
  (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer))
  (while --dolist-tail-- (setq f (car --dolist-tail--)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (let ((--dolist-tail-- (org-roam--list-all-files)) f) (while --dolist-tail-- (setq f (car --dolist-tail--)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (dolist (f (org-roam--list-all-files)) (with-current-buffer (find-file f) (message "migrating %s" f) (vulpea-migrate-buffer)))
  vulpea-migrate-db()
  eval-expression((vulpea-migrate-db) nil nil 127)
  funcall-interactively(eval-expression (vulpea-migrate-db) nil nil 127)
  command-execute(eval-expression)

(if I just replace the offending entity with the corresponding utf-8 symbol and run it again, it appears to continue nicely).