Guile's Foreign Function Interface - linking to notmuch

On the previous post I covered how I can call external programs as new processes and interact with their inputs or outputs. Here I will use directly the foreign libraries written in C using Guile's foreign function interface.

In this series of posts I record how to use Guile as a scripting language and solve various tasks related to email work.

Guile dynamic Foreign Function Interface

This is where you really have to read foreign code and lots of it. You need to recognize how the library you are developing the bindings for works, and which parts of it you are going to need.

The next example is the simple case, just link the library using dynamic-link and then register a function. You need to give the return type and the parameter types, any pointer is declared with '*. The final line evaluates the function and calls pointer->string to convert the returned pointer into something that we can read.

(use-modules
 (system foreign))

(define nmlib (dynamic-link "libnotmuch"))

;; const char * notmuch_status_to_string (notmuch_status_t status);
(define nm-status
  (pointer->procedure '*
                      (dynamic-func "notmuch_status_to_string" nmlib)
                      (list int)))

(pointer->string (nm-status 5)) ;; => File is not an email

What is left to do is repeat this process for all the functions that you need exposed in Guile. That is terribly tedious and it doesn't scale. In the next post will present a way to do it automatically. In what is left, I explain more low-level elements I used to expose the notmuch library to Guile. I do this because, before going automatic on something, I need to understand how it works manually.

Connecting to the database

The easy example is always too simple to learn something about it, I always get annoyed on those tutorials that stay on the example and never show some real use. That is why I document here how I managed to get some things working.

Getting the rest of the interface is a lot more of work. Fortunately, the dynamic link is not that different from the python ctypes and notmuch already comes with a python interface implemented with ctypes. It was not a simple copy, but something I could orient myself to get things working.

(use-modules
 (rnrs bytevectors)
 (system foreign))

(define nmlib (dynamic-link "libnotmuch"))

;; const char * notmuch_status_to_string (notmuch_status_t status);
(define nm-status
  (pointer->procedure '*
                      (dynamic-func "notmuch_status_to_string" nmlib)
                      (list int)))

;; notmuch_status_t notmuch_database_open (const char *path,
;;     notmuch_database_mode_t mode, notmuch_database_t **database);
(define nm-db-open
  (pointer->procedure uint32
                      (dynamic-func "notmuch_database_open" nmlib)
                      (list '* uint32 '*)))

;; const char * notmuch_database_get_path (notmuch_database_t *database);
(define nm-db-path
  (pointer->procedure '*
                      (dynamic-func "notmuch_database_get_path" nmlib)
                      (list '*)))

(define (make-blob-pointer len)
  (bytevector->pointer (make-bytevector len)))

;; This db-pointer is of type **, it is the pointer to a pointer, because I
;; make a pointer to the byte vector, which is of a size to hold a pointer
;; address. That address is zero, because the byte vector is initialized to
;; zero. That means, I get a pointer to a null pointer to start with.
(define db-pointer (make-blob-pointer (sizeof ptrdiff_t)))
(format #t "DB pointer points to a null pointer: ~s~%"
        (null-pointer? (dereference-pointer db-pointer)))


;; The next codeblock has to be read from the bottom line to the top one to
;; understand what is going on, as each result is passed to another
;; function. When I pass db-pointer I'm passing the pointer by referecence
;; that the function notmuch_database_open requires.
(format #t "Open database: ~a~%"
        (pointer->string
         (nm-status
          (nm-db-open (string->pointer "/home/titan/.mail/") 0 db-pointer))))

;; To access the datastructure of the database I need to dereference the
;; pointer.  Here I verify that my byte vector has been changed to hold the
;; memory address of the database
(format #t "Open DB pointer is null?: ~s~%"
        (null-pointer? (dereference-pointer db-pointer)))

;; I can recover values from the database, like for example the path
(format #t "Database path: ~a"
        (pointer->string (nm-db-path (dereference-pointer db-pointer))))

Executing this small script gives the following result. I see each stage. First starting with a null pointer, then opening the database which returns a no error status. I verify that now I don't have a null pointer anymore, since now it points to the database address, which I verify by requesting the registered setup information.

DB pointer points to a null pointer: #t
Open database: No error occurred
Open DB pointer is null?: #f
Database path: /home/titan/.mail

I should close the database, that is part of managing memory. For the moment I don't want to continue describing/implementing this interface, since there is a way to do it automatically that I describe on the next post. I just let the script end and close, that frees resources.