Place-Based Programming—Part 2 - Functions

[lsusr]

In Part 1, we defined a place-of macro and a value-of function. The code from Part 1, as originally written, was not an importable module. I have modified the code from Part 1 to be portable.

;; Module from Part 1
;; Save this code into a file called part1.hy and then use it with:
;; (import [part1 [value-of]])
;; (require [part1 [place-of]])

(setv +place-dir+ ".places/")

(defmacro/g! place-of [code]
  `(do
     (import [hashlib [md5]] os pickle)
     (setv ~g!type-dir (os.path.join ~+place-dir+ (str (type '~code))))
     (if-not (os.path.exists ~g!type-dir)
             (os.mkdir ~g!type-dir))
     (setv ~g!place
           (os.path.join
             ~g!type-dir
             (+ (.hexdigest (md5 (.encode (str '~code))))
                ".pickle")))
     (if-not (os.path.exists ~g!place)
             (with [f (open ~g!place "wb")]
               (pickle.dump (eval '~code) f)))
     ~g!place))

(defn value-of [place]
  (import os pickle)
  (assert (= (type place) str)
          (+ (str place) " is not a place"))
  (if-not os.path.exists
          (raise (FileNotFoundError (+ "Could not find place " place))))
  (with [f (open place "rb")]
    (pickle.load f)))

The value-of function works fine. The place-of macro has no way to accept parameters. We will define a macro for constructing place-based functions, which can accept parameters.

defnp

Hy’s built-in function declaration macro is defn. We will call our place-based function declaration macro defnp. Our place-based function will hash its own code as before. We also need a unique identifier for its parameters. In data science, the values of our parameters are often gigantic. It takes a long time to hash a big data structure. Hashing big data structures takes many computations. The whole purpose of a persistent memoization system is to reduce how many computations we have to perform. Passing values to our place-based function is a wastes compute. Instead we pass places, which are always easy to hash. A place-based function takes places as parameters and then returns another place.

(import [part1 [value-of]])
(require [part1 [place-of]])

(import os [part1 [+place-dir+]])
(setv +funcall-dir+ (os.path.join +place-dir+ "funcall"))

(defmacro/g! defnp [symbol params &rest body]
  `(do
     (import [hashlib [md5]] os pickle)
     (defn ~symbol ~params
       (setv ~g!funcall-place
             (os.path.join
               +funcall-dir+
               (+ (. ~symbol code-hash)
                  "-"
                  (.hexdigest (md5 (.encode (str (list ~params))))))))
       (if-not (os.path.exists ~g!funcall-place)
               (do
                 (setv ~g!value
                       ((fn ~params ~@body)
                         #*(lfor ~g!param ~params (value-of ~g!param))))
                 (with [f (open ~g!funcall-place "wb")]
                   (pickle.dump ~g!value f))))
       ~g!funcall-place)
     (setv (. ~symbol code-hash)
           (.hexdigest (md5 (.encode (str ['~params '~body])))))))

;; Tests

(defnp plus [x y]
  (+ x y))
(assert (= (value-of (plus (place-of 1)
                           (place-of 2)))
           (+ 1 2)))
(assert (= (value-of (plus (place-of 3)
                           (place-of 4)))
           (+ 3 4)))

(defnp times [x y]
  (* x y))
(assert (= (value-of (times (place-of 1)
                            (place-of 2)))
           (* 1 2)))
(assert (= (value-of (times (place-of 3)
                            (place-of 4)))
           (* 3 4)))