Most of the pre-processing work of my emails was already solved with a combination of a bash script, which uses the notmuch cli and the python package afew. However, I'm learning Guile and I need a real task to solve to practice the language.
In this series of posts I record how to use Guile as a scripting language and solve various tasks related to my email work.
When replacing a bash script, it always feels like too much work. Bash is so simple and for a quick task it is great. I only abandon bash when I need to do elaborate things. That means the benefits of leaving bash only show up way down the road.
Deleting files
This is a simple tasks, which I solve with one line of bash. The next code line queries notmuch for all messages tagged as deleted and lists all the matching files for those messages. The pipe then processes each line and applies the rm command to each.
notmuch search --output=files tag:deleted | xargs -I {} rm -v "{}"
Turning this into guile will only lead to more code, yet the purpose is to practice the language and get used to its tools.
(use-modules (ice-9 popen))
(let* ((port (open-input-pipe "notmuch search --format=sexp --output=files tag:deleted"))
(files-to-delete (read port)))
(for-each (lambda (file)
(display file)
(newline)
(delete-file file))
files-to-delete)
(close-pipe port))
open-input-pipe calls the command on a shell and captures its output, thus it becomes input for the running program. I directly pass the original command line command of notmuch to query for the deleted messages. I let the output be the filenames and ask to get everything formatted as a S-expression. That format is targeted to consume on Emacs, yet Guile as a lisp understands that too. read reads the first S-expression, which in this case is the entire list of files.
for-each lets me iterate over the files, it is aimed at procedures with side-effects and does not capture any result value. The lambda function writes to stdout which file is being deleted and then deletes it with the function delete-file. Finally, I close the port.
Tagging messages
This step is more involved. I tag my new messages according to a set of rules specified on a file. That file is piped to the notmuch tag command that processes the instructions in batch mode, usage is like this.
cat tags-rules | notmuch tag --batch
This is simple, yet I don't have any report of which tag is being applied. To generate such debug info, I would use afew. However, the way you configure tagging in afew is too verbose for my taste. Thus my next goal is to have a debugging log of the filters & tags and apply them directly.
A tag instruction in notmuch is composed of two parts. The first part corresponds to the tags to be applied or removed declared by a string like +mytag +inbox -new. The second part is the query for the messages to be tagged. Additionally, I want to optionally write a descriptive message of the tag for the info log. Thus my tagging rules will be configured like this, in a nested list form.
(define tag-rules
'(("+linkedin +socialnews -- from:linkedin.com" "Linkedin")
("+toastmasters -- toastmaster NOT from:info@meetup.com")))
I need to be able to modify the instruction, so that it only selects new messages that match the query and not all matching emails in the database. For that I extend the query to include the new tag and remove that tag when tagging the message. The next function covers that use case.
(define (tag-query rule new)
(let* ((split (string-contains rule " -- "))
(tags (substring rule 0 split))
(query (substring rule (+ 4 split))))
(string-append
(if new (string-append tags " -new") tags)
" -- "
(if new (string-append query " tag:new") query))))
(tag-query "+test -- from:ci" #t) ;; => "+test -new -- from:ci tag:new"
(tag-query "+test -- from:ci" #f) ;; => "+test -- from:ci"
The next code block does the work. open-output-pipe opens the notmuch-tag command on a shell and expects a batch input, the many lines with tagging instructions. I loop individually over each instruction, writing to stdout which tags are being applied and then send the tag instruction to notmuch.
(let ((port (open-output-pipe "notmuch tag --batch")))
(for-each (lambda (tag)
(let* ((rule (car tag))
(info (if (null? (cdr tag)) (car tag) (cadr tag)))
(query (tag-query rule #t)))
(display (string-append "[TAG] "
info
(if (string=? info rule) ""
(string-append " | " rule))
"\n"))
;; The next lines stream the rules to notmuch
(display rule port)
(newline port)))
tag-rules)
(close-pipe port))
Summary
There is a lot more code here compared to my bash script alternative. Yet I have won on features, logging in this case. Reaching the same result in bash would be less code, but probably unreadable after some time. Bash is not a language I use a lot, and having it out of my working memory makes it hard every time I need to use it. Guile has the advantage of not being that condensed on the instruction names, it uses full english words, so that reading the code is a lot easier. Though, I'm not insinuating I can't go out of practice on it too.
The opening and closing of the commands executed in a shell using open-{input,output}-pipe was annoying. Here, I miss the convenience of a context manager, as they are provided in python. I need to invest time on implementing those or find alternatives, especially to deal with exceptions. I experienced that this code stopped working when I retried after an exception. Well not really the code, but executing in on my interactive session at the REPL. Reason was that the pipe was not properly closed, after I ran into the exception[1] and notmuch places a lock on the database when tagging because it has to open it in READ_WRITE mode. That lock did not allow me to try the tagging again on a new execution. My Guile debugging knowledge is to limited to deal with that. I had no idea how to find the port to the command to close it and have notmuch close the database. Thus I ended restarting the REPL to try my code again.
I feel that the code is as nice as writing in Python for this simple task. The next challenge is to interface directly from Guile to the C++ notmuch library instead of going over the command line tools.
That happens in Python too, it is the way it must happen. Yet, in python I already know how to use try/catch blocks and context managers. ↩