kotatsuyaki’s site

I found why my eshell is slow at rm-rf-ing stuff

Published on

It’s org-roam.

I had a long-standing problem with my Emacs eshell setup—it is noticably slower at rm -rf-ing directories with a large number of small files. When I do an rm -rf and it smells like Emacs is frozen, I usually do a C-g and resort to running a Bash process to do the job:

# In Eshell
$ bash -c 'rm -rf ...'

Now to debug this problem, let’s grab a tarball for the source of the kernel, extract it, and remove it recursively using Eshell:

# In Eshell
$ time rm -rf linux-6.12.33/
44.236 secs

44 seconds. That’s slow! But how slow? Let’s remove it again with Bash:

# In Bash
$ time rm -rf linux-6.12.33/
rm -rf linux-6.12.33  0.02s user 1.66s system 96% cpu 1.738 total

Less than 2 seconds, which is more than an order faster than Eshell, so there’s definitely something wrong in Eshell. Let’s profile it.

# In Eshell
$ (profiler-start 'cpu); time rm -rf linux-6.12.33/; (profiler-stop); (profiler-report-cpu)

And here’s the profiler report. Notice something wrong? The function org-roam-file-p takes up 28% of the CPU time while the actual delete-file takes up only 3%. The org-roam-file-p function is slow because it checks whether the file is within the org-roam directory (i.e. org-roam-descendant-of-p) before checking whether the file extension is .org. It turns out that checking org-roam-descendant-of-p is a time-consuming task.

43772  98% - ...
43772  98%  - #<byte-code-function F4F>
43772  98%   - eshell-do-eval
43772  98%    - eshell-do-eval
43772  98%     - eval
43771  98%      - eshell-named-command
43771  98%       - eshell-plain-command
43771  98%        - eshell-lisp-command
43771  98%         - eshell-exec-lisp
43771  98%          - apply
43771  98%           - eshell/rm
43771  98%            - eshell-remove-entries
43771  98%             - eshell-exec-lisp
43771  98%              - apply
43771  98%               - delete-directory
43771  98%                - #<byte-code-function F18>
43765  98%                 - delete-directory
43764  98%                  - #<byte-code-function A4D>
32392  73%                   - delete-directory
32329  72%                    - #<byte-code-function F10>
19632  44%                     - delete-directory
19487  43%                      - #<byte-code-function A4D>
16252  36%                       - delete-directory
16114  36%                        - #<byte-code-function F10>
14669  33%                         - delete-directory
14590  32%                          - #<byte-code-function FD9>
14036  31%                           - files--force
14031  31%                            - apply
14031  31%                             - delete-file
14031  31%                              - apply
12668  28%                               - org-roam-db-autosync--delete-file-a
12636  28%                                + org-roam-file-p
10231  23%                                 + org-roam-descendant-of-p
 2119   4%                                 + file-relative-name
  242   0%                                 + org-roam--file-name-extension
   16   0%                                  backup-file-name-p
   13   0%                                  auto-save-file-name-p
 1362   3%                                 #<native-comp-function delete-file>

Now that I know the cause of the slowness, I can write a patch for it. Let’s add an early check for the .org file extension to the function org-roam-db-autosync--delete-file-a. Here I use the el-patch library to re-define the function so that I get noticed when the upstream function changes, but a simple defun would also suffice.

(el-patch-defun org-roam-db-autosync--delete-file-a (file &optional _trash)
  "Maintain cache consistency when file deletes.
FILE is removed from the database."
  (when (and (el-patch-add (string-suffix-p ".org" file))
             (not (auto-save-file-name-p file))
             (not (backup-file-name-p file))
             (org-roam-file-p file))
    (org-roam-db-clear-file (expand-file-name file))))

The performance after the patch becomes acceptable again:

# In Eshell
$ time rm -rf linux-6.12.33/
6.643 secs