The Curmudgeoclast

Thoughts, projects, and ramblings of Dave Astels

Wed 27 November 2019

uLisp Hacking

Posted by dastels in software   

This will get expanded as I get more code working.

uLisp

uLisp (i.e. micro lisp) is a lisp interpretor aimed at microcontrollers. There are versions for boards as limited as the Arduino UNO (8-bit, 16MHz, 32K flash, and 2K ram). More interesting is the 32-bit version for ARM microcontrollers. This runs nicely on some of the most advanced, yest still small, microcontroller boards. Boards like the SAMD51 products from Adafruit. Boards like the Grand Central with 1M flash and 256K ram running at 120MHz. That's plenty of power to run some decent sized lisp programms.

Being a lisp fan since the beginning, this was a natural thing for me to play with. So, based on a few recent from-scratch lisp implementations in various languages (Go, Ruby, and C++)

uLisp is easy to work with, as it is written using the arduino toolset.

Macros

Macros are one of the real power tools of Lisp, and basic macros are quite easy to implement.

Flash Filesystem Support

uLisp has SD card filesystem support as a feature that can be enabled in the build. Many microcontroller boards include exteral SPI flash which adds 2MB (and in some cases 8 MB) of flash storage. The TinyUSB library that's used in CircuitPython allows use of the flash as a USB drive: plug in your board and the flash shows up as a drive. This lets you copy files directly onto the flash drive. Most notably, this includes source code.

At the moment only one of SD or Flash filesystems can be enabled.

To support loading files from the flash, a load-file function is defined in LispLibrary:

(defun load-file (filename)
  (princ \"Loading \")
  (princ filename)
  (terpri)
  (with-flash-fs (s filename)
    (loop (let ((expr (read s)))
    (if (null expr)
        (return)
        (eval expr))))))

Additionally, to support automatic load on startup:

(dolist (fname (with-flash-fs (s \"/filelist.lsp\")
                 (read s)))
  (load-file fname))

You simply have a file named filelist.lsp that contains a list of names of files to be loaded. When the system starts up, that list is read and each file is loaded, in order.

As shown in the above code, the flash filesystem is used the same was as the SD filesystem.

Experimental auto-reload when flash filesystem changes

One tinyUSB feature that is still a work in progress, and will be available eventually, is the ability to have code reloaded automatically when anything on the filesystem changes. This gives the ability to restart the system everytime new code is copied to the drive. If you edit code directlyu on the USB/flash drive, every time you save it can be reloaded/rerun. This makes development very quick.

Formatting

Scheme provides a very simple yet capable string formating system, much like C's printf (but simpler). The following is edited from the MIT Scheme reference

(format destination control-string argument ...)

Writes the characters of control-string to destination, except that a tilde (~) introduces a format directive. The character after the tilde, possibly preceded by prefix parameters and modifiers, specifies what kind of formatting is desired. Most directives use one or more arguments to create their output; the typical directive puts the next argument into the output, formatted in some special way. It is an error if no argument remains for a directive requiring an argument, but it is not an error if one or more arguments remain unprocessed by a directive.

The output is sent to destination. If destination is nil, a string is created that contains the output; this string is returned as the value of the call to format. In all other cases format returns an unspecified value. If destination is t, the output is sent to the current output port. Otherwise, destination must be an output port, and the output is sent there.

A format directive consists of a tilde (~), an optional prefix parameter), an optional at-sign (@) modifier, and a single character indicating what kind of directive this is. The alphabetic case of the directive character is ignored. The prefix parameter is generally an integer, notated as optionally signed decimal numbers.

In place of a prefix parameter to a directive, you can put the letter V (or v), which takes an argument for use as a parameter to the directive. Normally this should be an exact integer. This feature allows variable-width fields and the like. You can also use the character # in place of a parameter; it represents the number of arguments remaining to be processed.

It is an error to give a format directive more parameters than it is described here as accepting. It is also an error to give an at-sign modifier to a directive in a combination not specifically described here as being meaningful.

~A

The next argument, which may be any object, is printed as if by display. ~mincolA inserts spaces on the right, if necessary, to make the width at least mincol columns. The @ modifier causes the spaces to be inserted on the left rather than the right.

~S

The next argument, which may be any object, is printed as if by write. ~mincolS inserts spaces on the right, if necessary, to make the width at least mincol columns. The @ modifier causes the spaces to be inserted on the left rather than the right.

~%

This outputs a #\newline character. ~n% outputs n newlines. No argument is used. Simply putting a newline in control-string would work, but ~% is often used because it makes the control string look nicer in the middle of a program.

~~

This outputs a tilde. ~n~ outputs n tildes.

~newline

Tilde immediately followed by a newline ignores the newline and any following non-newline whitespace characters. With an @, the newline is left in place, but any following whitespace is ignored. This directive is typically used when control-string is too long to fit nicely into one line of the program:

(define (type-clash-error procedure arg spec actual)
(format #t
        "~%Procedure ~S~%requires its %A argument ~
        to be of type ~S,~%but it was called with ~
        an argument of type ~S.~%"
        procedure arg spec actual))

(type-clash-error vector-ref
                  "first"
                  integer
                  vector)

prints

Procedure vector-ref
requires its first argument to be of type integer,
but it was called with an argument of type vector.

Note that in this example newlines appear in the output only as specified by the ~% directives; the actual newline characters in the control string are suppressed because each is preceded by a tilde.

Generating Symbols

Almost all Lisps have a way to programmatically generate unique symbols. This is typically the gensym` function.

(gensym [prefix])

gensym generates a sequence of unique symbols by appending a monotonically increasing integer to the prefix (GENSYM is used is the prefix is ommited). A separate count is maintained for each unique prefix.

> (gensym)
GENSYM-1

> (gensym)
GENSYM-2

> (gensym "foo")
foo-1

> (gensym)
GENSYM-3

> (gensym "foo")
foo-2

Interning Symbols

Interning provides a way to convert a string to a symbol that's present in the symbol table.

Interning a string is a simple matter of passing it to intern:

(intern "name")

Improve Comments

Comments in uLisp were handled in a way that was not compatible with any other Lisp I've used. So I fixed it. Comments not being with the first semicolon on a line (other than ones in a string) and extend to the end of the line. Furthermore, comments on multiple consecutive lines were not supported. Now they are.

Add Keyword Support

Keywords are symbols that evaluate to themselves. Taking a cue from Common Lisp, I've added keywords to uLisp. If you have a symbol that begin with a colon, it will be a keyword. E.g. :keyword .

The major feature of keywords is that, since a keyword evaluates to itself, they don't have to be quoted. They can make your code much less cluttered, and are very handy when writing macros.

Note that keywords are complete in themselves; they can not be used as a variable name. Also, they can't be used as a place for in-place operations such as setf, incf, and such. If you try, you'll get an error:

> (setf :key 7)
Error: 'setf' can't use a keyword

Basic Logging

Using some of the above additions I ported a simplified version of the VOM logging library (https://github.com/orthecreedence/vom) from Common Lisp. To be honest, the desire toport it was the impetus for looking into adding macros, format, intern, and keywords.

It's very simple for a logging framework. So simple that I'll include it here.

(defvar *LOGGER:LEVEL* 3)

(defun logger:set-level (level)
  (setf *LOGGER:LEVEL* level))

(defun logger:do-log (level-name log-level format-str &rest args)
  (when (<= log-level *LOGGER:LEVEL*)
    (let ((full-format (concatenate 'string "~A - ~A (~A): " format-str)))
      (dolist (arg (list log-level level-name (millis)))
        (push arg args))
      (eval `(format t ,full-format @args)))))


(defmacro logger:define-level (name level-value)
  (let ((macro-name (intern (format nil "log~a" name))))
    `(defmacro ,macro-name (format-str &rest args)
       `(logger:do-log ,,name ,,level-value ,format-str @args))))

(logger:define-level :emerg 1)
(logger:define-level :alert 2)
(logger:define-level :crit 3)
(logger:define-level :error 4)
(logger:define-level :warn 5)
(logger:define-level :notice 6)
(logger:define-level :info 7)
(logger:define-level :debug 8)
(logger:define-level :debug1 9)
(logger:define-level :debug2 10)
(logger:define-level :debug3 11)
(logger:define-level :debug4 12)

Then

> (load-file "/lib/logging.lsp")
Loading /lib/logging.lsp
nil

> (log:alert "hi")
2485938 - :alert (2): hi
nil

Proxy SWANK Server