1. tasks
1.1. core
Archived entries from file /home/ellis/comp/org/plan/tasks/core.org
1.1.1. DONE dot 0.1 [1/1]
lisp dot
- State "TODO" from
- State "DONE" from "NOPE"
- implemented in dat/dot.lisp
1.1.2. DONE migrate to sb-grovel [9/9]
lisp ffi
- DONE blake3
- State "TODO" from
- State "DONE" from "TODO"
:ID: 656c2683-8780-407c-b955-3defc4959595
- DONE bqn
- State "TODO" from
- State "DONE" from "TODO"
:ID: b4757746-525b-49e4-b8ac-677375867d8f
- DONE btrfs
- State "TODO" from
- State "DONE" from "TODO"
:ID: b3b411fe-6b2d-4338-9e23-83536336a487
- DONE k
- State "TODO" from
- State "DONE" from "TODO"
:ID: 8a463084-efa0-4535-8239-b1c4955bde4c
- DONE rocksdb
- State "TODO" from
- State "DONE" from "TODO"
:ID: 2d4d70af-228f-424c-8374-9fc8ed5c6f92
- DONE tree-sitter
- State "TODO" from
- State "DONE" from "TODO"
:ID: 0cdf3412-cb00-4069-8e1b-b49c736377cf
- DONE uring
- State "TODO" from
- State "DONE" from "TODO"
:ID: 45a52407-ccaf-4ea1-99c4-d6f7ba0b2731
- DONE zstd
- State "TODO" from
- State "DONE" from "TODO"
:ID: 6ade8dd8-83b1-470d-a0da-a352ce4c2d06
- DONE xkb
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
:ID: af778a5f-4834-4f0c-ae53-36d35cb98d8c
- State "TODO" from
1.1.3. DONE writebatch
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
1.1.4. DONE logger
- State "TODO" from
- State "DONE" from "TODO"
1.1.5. DONE merge-operator callbacks rocksdb
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
- Note taken on
still needs testing with actual data
1.1.6. DONE compactionfilter callbacks rocksdb
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
https://github.com/facebook/rocksdb/wiki/Compaction-Filter
(rocksdb-compactionfilterfactory-create nil (alien-sap (alien-callable-function 'rocksdb-destructor)) (alien-sap (alien-callable-function 'rocksdb-create-compaction-filter-never)) (alien-sap (alien-callable-function 'rocksdb-name)))
1.1.7. DONE comparator callbacks rocksdb
- State "WIP" from
- State "DONE" from "WIP"
1.1.8. DONE associative-merge implementation
- Refiled on
- State "DONE" from "TODO"
- need to figure out what delete-value requires
- simply passes state + value and value-length to a function
- when the value is null it doesn't exist in db (noop) ?
- unclear when the merge op actually runs in relation to deletion op - before or after?
- https://docs.rs/rocks/latest/rocks/merge_operator/index.html
1.1.9. DONE incf/decf-merge
- State "TODO" from
- State "TEST" from "TODO"
- State "TBD" from "TEST"
- Note taken on
use the STATE param for this. - State "DONE" from "TBD"
- Note taken on
see index-merge
1.1.10. DONE define-merge-operator macro
- Refiled on
- State "TODO" from "TODO"
- State "TEST" from "TODO"
- State "DONE" from "TEST"
- Note taken on
will need to do actual tests merging key/vals when we implement associative merge API
- define macro which eases the process of making custom merge operators
(define-merge-operator :incf nil :full nil :partial nil) (create-incf-mergeoperator)
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X75D288000C30 :TYPE (* (SB-ALIEN:STRUCT ROCKSDB::ROCKSDB-MERGEOPERATOR-T))>
1.1.11. DONE zstd
- State "PROJECT" from
- Note taken on
can probably improve error handling but don't need to address until we are well under way with IO/FLATE. - State "DONE" from "PROJECT"
- DONE zstd cstream tests
- State "DONE" from "TODO"
- v2 api
- DONE zdict
- State "TODO" from
- Note taken on
well this is annoying, there is one function which requires passing struct by value: ZDICTfinalizeDictionary - State "DONE" from "TODO"
- A few structs that we can't grovel
- A bunch of functions for pre-training a dictionary
1.1.12. DONE reference impls
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
- document in Commentary section
- also consider https://github.com/antirez/linenoise
- RLWRAP settings for SBCL · GitHub
- CIEL Is an Extended Lisp
- GitHub - sharplispers/linedit: Readline-style line-editor for Common Lisp.
1.1.13. DONE retire x.lisp lisp skel
- State "TODO" from
- State "TBD" from "TODO"
- State "TODO" from "TBD"
- Note taken on
actually let's move it to infra/scripts.. - Note taken on
on second look we really should start afresh lol - State "DONE" from "TODO"
with skel taking on more responsibility we no longer need a dedicated build tool.
- migrate all functionality to skelfile
- cleanup readme with installation instructions (with and without
infra
) - easy to break the skel binary and overwrite a LKG
version, preventing future builds.
just be more carefulsanity checks- repurpose x.lisp to recover from situations like these specifically
1.1.14. DONE implement globals *cli* *opts* *args*
etc
- State "TODO" from
- State "DONE" from "TODO"
1.1.15. DONE opt/arg parsing
- State "FIXME" from
- State "WIP" from "FIXME"
- State "TEST" from "WIP"
- State "DONE" from "TEST"
- Note taken on
took far longer than expected to track this down - root cause was thezerop
function. should be in much better shape now.
- do not need to handle multiple commands
- what do we want to do about subcommands?
- do not need to support 'global' option for commands
- use cli-parse to test.
1.1.16. DONE multipoint-images lisp bin sys
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
- multiple entrypoint lisp images
- create multiple links to core lisp image
- support multiple low-level methods?
- naive implementation is a single entry point with a switch dispatched on arg0.
- could also create a thin shim in Rust/Zig and call the appropriate entrypoint after loading the core.
- for now let's put this in lib/cli so that we can use
defmain
1.1.17. DONE make-sql-data-frame
- State "FIX" from
- State "DONE" from "FIX"
- finish implementation
1.1.18. DONE git vc git
- State "TODO" from
- State "DONE" from "TODO"
1.1.19. DONE hg vc hg
- State "TODO" from
- State "DONE" from "TODO"
1.1.20. DONE eval rules from CLI
- State "TODO" from
- State "DONE" from "TODO"
- need new command or flag for rules
- evaluate forms instead of looking in skelfile
1.1.21. DONE refactor system definition
- State "TODO" from
- State "DONE" from "TODO"
- remove package-inferred
- minimize number of systems
- use pkg.lisp files
1.1.22. DONE cleanup fu
- State "TODO" from
- State "DONE" from "TODO"
1.1.23. DONE CLTL2 impl
- State "TODO" from
- State "DONE" from "TODO"
1.1.24. DONE list-package-symbols, etc
- State "TODO" from
- State "DONE" from "TODO"
1.1.25. DONE std/tests refactor
- State "TODO" from
- State "DONE" from "TODO"
- a bit too large, time to shake it up
- need separation of concerns (files) for task-pool/threaded tests in particular
1.1.26. DONE tasks
- State "TODO" from
- Note taken on
currently includes jobs, supervisors, task-pool, kernel - a bit overloaded - State "DONE" from "TODO"
1.1.27. DONE more sb-alien utils ffi
- State "TODO" from
- State "DONE" from "TODO"
- refer to cffi and uffi
- cffi/src/types.lisp at 32c90d4a9a01e809f591276c058e5b3c7f39b154 · cffi/cffi · GitHub
1.1.28. NOPE uri net
- State "TODO" from
- Note taken on
the franz impl doesn't play well with SBCL, at least with our hacks. I'm not sure how faithfully I want to follow the RFCs anyhow.. maybe we go for the dead simple approach and focus on MOP for a path and URI protocol.
- mostly just need the tests and to finish porting the Franz implementation
1.1.29. DONE db db
- State "TODO" from
- State "DONE" from "TODO"
- Note taken on
XDB2 is no longer used - instead provide a simple abstract API which builds up to high-level systems like RDB
- ported from XDB2
1.1.30. DONE aggregate physical plan
- State "DONE" from "TODO"
1.1.31. DONE Physical Expressions
- State "TODO" from
- State "TODO" from
- State "DONE" from "TODO"
1.1.32. DONE crew proto
- State "TODO" from
- State "DONE" from "TODO"
1.1.33. DONE http proto
- State "TODO" from
- State "DONE" from "TODO"
1.1.34. DONE dns proto
- State "TODO" from
- State "DONE" from "TODO"
1.1.35. DONE tlv codec
- State "TODO" from
- State "DONE" from "TODO"
1.1.36. DONE osc codec
- State "TODO" from
- State "DONE" from "TODO"
1.1.37. DONE Woo/+
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
1.1.38. DONE fixture api
- State "TODO" from
- State "DONE" from "TODO"
1.1.39. DONE temporary files and directories
- Refiled on
- State "DONE" from "TODO"
1.1.40. DONE org graph edges
- State "TODO" from
- Note taken on
dblock will come later. for now we are going with the same strategy as org-super-links - using a drawer. instead of "BACKLINKS" our drawer is called simply LINKS. - State "WIP" from "TODO"
- State "DONE" from "WIP"
- Note taken on
initial implementation up and running
- list of links between other headings
- <= links to this item
- => links from this item
- == siblings
- ^= parent
- v= child
- how should it look when exported?
- org-graph-link-use-timestamp
- org-graph-link properties? edge properties?
- keyvals
:key val
- should probably align with whatever we use forCONSTRAINTS
, for now read as sxp
- keyvals
- EXPORTEDGES property?
1.1.41. DONE org-follow-location org
- State "TODO" from
- State "WIP" from "REFILE"
- State "DONE" from "WIP"
- follow LOCATION property of heading at point
- respect parent LOCATION, buffer local settings, etc
1.1.42. DONE org-set-location
- State "TODO" from
- State "DONE" from "TODO"
similar to org-set-effort, to be called automatically on capture.
should prompt for directory then file - skip file with C-u or with RET- simpler to just prompt for value with completion using
org-read-property-value
- simpler to just prompt for value with completion using
1.1.43. DONE flate [0/0]
- State "TODO" from
- State "PROJECT" from "TODO"
- State "DONE" from "PROJECT"
1.1.44. DONE review GNU stow API
- State "TODO" from
- State "DONE" from "TODO"
homepage 'symlink farm manager' rust impl: https://github.com/qboileau/rstow
1.1.45. DONE consolidate with skel skel
- State "TODO" from
- State "DONE" from "TODO"
1.1.46. DONE cli tests crashing
- State "FIXME" from "TODO"
- State "DONE" from "FIXME"
1.1.47. DONE :scripts regression infra
- State "DONE" from "TODO"
1.1.48. NOPE readline cli
- State "TODO" from
- Note taken on
we have removed readline in favor of the lisp-native cli/linedit.
1.1.49. DONE impl rocksdb/slicetransform callbacks
- State "TODO" from
- New deadline from " " on
- Refiled on
- New deadline from " " on
- State "TEST" from "TODO"
- State "DONE" from "TEST"
1.1.50. DONE OCI projects [4/4]
pod packy skel
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
- Note taken on
A bit too much for us to consider right now - ultimately I do not feel that OCI containers are a one-size-fits-all solution for image-based development.
1.1.51. DONE PMem.io
- State "RESEARCH" from "TODO"
- State "DONE" from "RESEARCH"
- consider use cases in Lisp images - pre-loading and coordinating shared fasls and core images
- no longer maintained by intel
1.1.52. NOPE encrypted btrfs luks/dm-crypt
- State "RESEARCH" from "TODO"
1.1.53. DONE XDP
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
A Gentle Introduction to XDP | Datadog GitHub - xdp-project/xdp-tools: Utilities and example programs for use with XDP
- xdp-paper/xdp-the-express-data-path.pdf at main · xdp-project/xdp-paper · GitHub
- BPF and XDP Reference Guide — Cilium 1.18.0-dev documentation
- will require linking to libbpf and libxdp, needs elf loader (or use xdp-tools/iproute2)
- blocked by bpf impl in lisp
1.1.54. DONE DPDK
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
Home - DPDK GitHub - DPDK/dpdk: Data Plane Development Kit
By leveraging DPDK libraries, developers can create optimized packet processing paths, manage timers for executing functions asynchronously, and utilize a wide range of drivers and libraries tailored for fast packet processing.
librteeal – Environment Abstraction Layer: Provides the foundational API for DPDK, facilitating access to hardware resources such as memory, timers, and logs.
librtemempool – Memory Pool Manager: Manages memory pools for efficient and speedy packet handling.
librtering – Ring Buffer Manager: Implements lock-free FIFO queues, enabling high-speed communication between various DPDK components.
librtembuf – Packet Buffer Management: Handles packet buffers, which are crucial for packet transmission and reception.
librteethdev – Ethernet Device API: Offers an API for configuring and querying Ethernet devices. It supports various operations, including sending and receiving packets.
librtenet – Network Helper Library: Provides helper APIs for dealing with network protocols.
librteipfrag – IP Fragmentation and Reassembly: Handles fragmentation and reassembly of IP packets, supporting both IPv4 and IPv6.
librtekni – Kernel Network Interface: Facilitates communication between DPDK applications and the Linux kernel networking stack, primarily used for debugging or interfacing with existing Linux network services.
1.1.55. DONE eBPF net
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
https://docs.kernel.org/bpf/standardization/instruction-set.html#bpf-instruction-set-architecture-isa https://mostlynerdless.de/blog/2023/12/31/hello-ebpf-developing-ebpf-apps-in-java-1/ https://github.com/iovisor/bcc GitHub - mikeroyal/eBPF-Guide bpftrace/tools/README.md at master · bpftrace/bpftrace · GitHub
- bpftrace seems quite powerful, will be useful for sysmon activities
where would ebpf support go?
probably io.. but maybe also log, net.. syn?
1.1.56. DONE batman-adv net
- State "RESEARCH" from
- State "DONE" from "RESEARCH"
1.1.57. NOPE libyear lisp skel
- State "TODO" from
Not that useful in our current environment (NIH)
1.1.58. DONE Level 1
- State "TODO" from
- State "DONE" from "TODO"
1.1.59. DONE Level 2
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
1.1.60. DONE Level 3
- State "TODO" from
- State "DONE" from "TODO"
1.1.61. DONE LAPACK (f77blas.h)
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
- Note taken on
this does NOT include the lapack.h functions
1.1.62. DONE LAPACK codegen
- State "TODO" from
- State "DONE" from "TODO"
- Note taken on
see lapack-ffi.lisp
lapack.h is massive and impractical to attempt lisp bindings from scratch.
instead, our current approach is based on tree-sitter:
(tree-sitter:load-tree-sitter) (tree-sitter:load-tree-sitter-alien) (tree-sitter:load-tree-sitter-c)
#P"/usr/local/lib/libtree-sitter-c.so"
(syn/ts::ts-file-query :c #p"/usr/include/openblas/lapack.h" "(declaration)")
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X71384C005800 :TYPE (* (SB-ALIEN:STRUCT TREE-SITTER:TS-QUERY-CURSOR))>
this segfaults:
(in-package :syn/ts) (sb-alien:with-alien ((m (* tree-sitter::ts-query-match) (sb-alien:make-alien tree-sitter::ts-query-match))) (sb-sys::with-pinned-objects (m) (tree-sitter::ts-query-cursor-next-match (syn/ts::ts-file-query :c #p"/usr/include/openblas/lapack.h" "(declaration)") m)))
(let ((tree (flatten (syn/ts::parse-file :c #p"/usr/include/openblas/lapack.h"))) (dx 0) (ret)) (loop with dx = 0 for di = (print (position :declaration tree :start dx)) if (null di) do (return ret) else do (progn (setf dx (incf di)) (let ((ei (print (position :declaration tree :start dx)))) (push (subseq tree di ei) ret)))))
(setq *lapack* (let* ((str (read-file #p"/usr/include/openblas/lapack.h")) (tree (flatten (syn/ts::parse-string :c str :consume t :produce-cst nil))) (dx 0) (ret)) (setq *lapack.h* str) (loop with dx = 0 for di = (position :declaration tree :start dx) if (null di) do (return ret) else do (progn (setf dx di) (incf dx) (let ((ei (position :declaration tree :start dx))) (push (subseq tree di ei) ret)))))) (list (length *lapack*) (length *lapack.h*))
1332 | 747544 |
(labels ((lastr (x start end) (apply 'subseq *lapack.h* (subseq x start end))) (lasym (str) (symbolicate (string-upcase (substitute #\- #\_ str)))) (lasym* (str) (let ((len (length str))) (list '* (lasym (subseq str 0 (if (char= (char str (1- len)) #\*) (1- len) len)))))) (lasyms (x start end) (symbolicate (string-upcase (substitute #\- #\_ (lastr x start end))))) (laparm (str) (let ((a (nreverse (split-sequence #\space str)))) (list (if (string-equal (first a) "T") (gensym "T") (lasym (first a))) (if (third a) ;; type param (could be X const* or const X*) (if (string-equal (third a) "const") (lasym* (second a)) (lasym* (third a))) (when (second a) (lasym* (second a))))))) (gen-lapack-args (p x) (let* ((pa (position :parameter-list x)) ;; param start (pb (position :preproc-ifdef x)) ;; param end (params (subseq x pa pb))) (loop with p1 = 0 for pos = (position :parameter-declaration params :start p1) while pos do (setf p1 (1+ pos)) collect (laparm (lastr params (+ pos 1) (+ pos 3))))))) (setq *code* (mapcar (lambda (x) (let ((p (position :function-declarator x))) (list* 'deflapack ;; routine name (lasyms x (+ p 5) (+ p 7)) ;; return-type (lasyms x (- p 3) (- p 1)) ;; list of function parameters (NAME TYPE) (remove-if (lambda (x) (eql (car x) 'fortran-strlen)) (gen-lapack-args p x))))) *lapack*))) (length *code*)
1332
(with-output-to-file (f file) (write-line ";;; lapack-ffi.lisp --- LAPACK Alien Routines -*- buffer-read-only:t -*-" f) (write-line "(in-package :lapack)" f) (mapc (lambda (x) (write x :stream f :case :downcase) (terpri f)) *code*))
1.1.63. DONE linedit path completion
- State "FIXME" from
- State "DONE" from "FIXME"
- Note taken on
turns out STD/OS:WITH-DIRECTORY-ITERATOR was the culprit - re-arranged a null-alien check and good to go.
currently segfaults - should be easy to fix
1.1.64. DONE sxp 0.2 lisp sxp
- State "TODO" from
- State "DONE" from "TODO"
1.1.65. DONE toml impl
- State "TODO" from
- State "DONE" from "TODO"
- Refiled on
the toml parser on quicklisp is based on esrap, make a comparable one using our parser module.
1.1.66. DONE split out skel rules
- State "TODO" from
- State "DONE" from "TODO"
- core/rule.lisp
1.1.67. DONE fix obj/query stack overflow on compile
- State "TODO" from
- State "DONE" from "TODO"
- Note taken on
soft fix - commented out shit
Control stack exhausted (no more space for function call frames). This is probably due to heavily nested or infinitely recursive function calls, or a tail call that SBCL cannot or has not optimized away. PROCEED WITH CAUTION. [Condition of type SB-KERNEL::CONTROL-STACK-EXHAUSTED] 849: ((:METHOD SB-PCL::COMPUTE-EFFECTIVE-SLOT-DEFINITION-INITARGS (SB-PCL::SLOT-CLASS T)) #<STANDARD-CLASS OBJ/QUERY:LOGICAL-QUERY-PLAN> (#<STANDARD-DIRECT-SLOT-DEFINITION OBJ/AST:AST> #<STANDARD-DIRECT-SL.. Locals: SB-PCL::ALLOCATION = :INSTANCE SB-PCL::ALLOCATION-CLASS = #<STANDARD-CLASS OBJ/QUERY:LOGICAL-QUERY-PLAN> SB-PCL::ALLOCP = T CLASS = #<STANDARD-CLASS OBJ/QUERY:LOGICAL-QUERY-PLAN> SB-PCL::DIRECT-SLOTDS = (#<STANDARD-DIRECT-SLOT-DEFINITION OBJ/AST:AST> #<STANDARD-DIRECT-SLOT-DEFINITION OBJ/AST:AST> #<STANDARD-DIRECT-SLOT-DEFINITION OBJ/AST:AST>) DOCUMENTATION = NIL SB-PCL::DOCUMENTATIONP = NIL SB-PCL::INITARGS = (:AST) SB-PCL::INITFORM = NIL SB-PCL::INITFUNCTION = NIL SB-PCL::INITP = NIL SB-PCL::NAME = OBJ/AST:AST SB-PCL::NAMEP = T 855: (SB-THREAD::CALL-WITH-RECURSIVE-LOCK #<FUNCTION (FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK :IN SB-PCL::UPDATE-CLASS) {739064260D1B}> #<MUTEX "PCL global mutex" taken owner=main thread>) Locals: GOT-IT = NIL MUTEX = #<MUTEX "PCL global mutex" taken owner=main thread> SB-C::THING = #<FUNCTION (FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK :IN SB-PCL::UPDATE-CLASS) {739064260D1B}>
1.1.68. DONE exfil from obj
- State "OUTLINE" from
- State "DONE" from "WIP"
- things lib/math will need from obj:
- plan
- store
- graph
- srv?
- schema?
- may go to std
1.1.69. DONE Ecosystem
- State "RESEARCH" from
- State "WIP" from "RESEARCH"
- State "DONE" from "WIP"
- Common Lisp Statistics · GitHub
- GitHub - Lisp-Stat/lisp-stat: Lisp-Stat main system
- GitHub - takagi/cl-cuda: Cl-cuda is a library to use NVIDIA CUDA in Common Lisp programs.
- GitHub - quil-lang/magicl: Matrix Algebra proGrams In Common Lisp.
- SRFI 231: Intervals and Generalized Arrays
- lisplab
- matlisp
- and next.. pola.rs
- use 3d-math patches as a way to learn more about what we will need
- we need a lot more than just matrix acceleration.
- llvm-project/mlir/docs/Dialects/Linalg/index.md · JuliaLang/llvm-project
- LAPACK? ye probably - be aware of symbol rewrites caused by FC
- solved in blas (wip)
What is the difference between Tensors and Matrices? · Learn Tensorflow
A matrix is a two-dimensional grid of size n×m that contains numbers: you can add and subtract matrices of the same size, multiply one matrix with another as long as the sizes are compatible((n×m)×(m×p)=n×p)((n×m)×(m×p)=n×p), and multiply an entire matrix by a constant.
A vector is a matrix with just one row or column (but see below), vector has direction, but tensor has no direction.
A tensor is often thought of as a generalized matrix. That is, it could be
a 1-D matrix, like a vector, which is actually such a tensor, a 3-D matrix (something like a cube of numbers), a 0-D matrix (a single number), or a higher dimensional structure that is harder to visualize. The dimension of the tensor is called its rank.
Any rank-2 tensor can be represented as a matrix, but not every matrix is really a rank-2 tensor. The numerical values of a tensor’s matrix representation depend on what transformation rules have been applied to the entire system.
1.1.70. DONE refactor libs into math
- State "TODO" from
- State "DONE" from "TODO"
- obj should NOT depend on math
- math depends on obj for tensor
- accumulators? hmm..
- need to be in math but also used by queries which is in obj
- may need to split out obj/query to Q
- done
- moved to std/seq
- math depends on ffi, depended on by dsp,net,apps,etc
- should probably have the most tests of all libs
- we want OBJ/TENSOR - see tensor
1.1.71. DONE symbol cleanup
- State "TODO" from
- State "DONE" from "TODO"
delete unbound non-anaphoric symbols
(let ((ret)) (do-external-symbols (s :std ret) (when (not (or (boundp s) (fboundp s) (cadr (multiple-value-list (typexpand s))) (find-class s nil))) (push s ret))))
TIME-REMAINING | PID-T | TIMESPEC | IT | LOFF-T | STAGES | %P | TASK | GID-T | .I | %THREAD | %A | MAYBE-INLINE | JOBS | TASKS | TIMEVAL | WINSIZE | UID-T |
1.1.72. DONE review previous state of std/tasks
- State "TODO" from
- State "DONE" from "TODO"
We were doing some things right - like our semaphore worker waking
- already added this to BROADCAST-WORK (comment suggestion by lmj)
- mailbox stuff not as useful (speed is a concern over more primitive cas queues and spinlocks)
1.1.73. DONE explain doc
- State "TODO" from
- State "WIP" from "TODO"
- State "DONE" from "WIP"
- generic done
- maybe simple version in std/fmt
1.1.74. DONE explore doc
- State "TODO" from "TODO"
- State "WIP" from "TODO"
- State "DONE" from "WIP"
- generic done
1.1.75. DONE define-alien-struct
- State "DONE" from "TODO"
- Refiled on
- convenience for defining alien structs, especially with the same name as lisp type
1.1.76. DONE remove makefile
- State "TODO" from
- State "DONE" from "TODO"
1.2. infra
1.2.1. DONE dictd
- State "TODO" from
- Note taken on
basically done - serving on compiler.company:2628 with route through https - State "DONE" from "TODO"
1.2.2. DONE reintegrate pod and box vc
- State "TODO" from
- State "DONE" from "TODO"
[X]
pod[X]
box[X]
.hgsub
1.2.3. DONE deploy [4/4]
lisp
- State "TODO" from
- State "DONE" from "TODO"
[X]
collect artifacts[X]
pack[X]
send output toDIST
[X]
Adapt toskel deploy
- assumptions
- host is qualified builder
- all dependendencies installed on system (not in stash)
- current user has access to DIST
- done in org skelfile
1.2.4. DONE publish [4/4]
lisp emacs
- State "TODO" from
- State "DONE" from "TODO"
- done in org skelfile
[X]
load dependencies (htmlize,default.el,babel.org)[X]
loadpublish.el
[X]
setpublish-directory
[X]
evalpublish
1.2.5. DONE tree-sitter-pack
- State "TODO" from
- State "DONE" from "TODO"
- pull-tree-sitter-langs.sh
- install-tree-sitter-langs.sh (OG, calls pull)
- make dist/tree-sitter
1.3. demo
2. roadmap
2.1. DONE 2024
- State "DONE" from
2.2. DONE migrate VC off of EC2 opsec vc code
- State "WIP" from
- Note taken on
removed dependency on heptapod, running plain ol hgweb behind proxy - State "DONE" from "WIP"
- Note taken on
next up: clonebundles,packy,git hosting
We no longer need to maintain an instance of our forge in AWS now that we have enough compute in-house.
vc.compiler.company will be migrate to rurik on a dedicated pod
(requires ubuntu).
2.3. DONE Wireguard VPN net opsec
- State "TODO" from
- State "WIP" from "TODO"
- Note taken on
all that is left is to configure for VM and containers - State "DONE" from "WIP"
Wireguard is a fast, secure, and easy to configure VPN library. All our primary devices have been configured and tested with two available endpoints. Our goal is to migrate all existing devices, then make appropriate accomodations for Podman containers, and finally KVM.
2.4. TODO IO lisp
- State "TBD" from
- State "TODO" from "TBD"
iouring is the Linux library we have chosen to spend some time getting comfortable with and for which we've written incomplete Common Lisp bindings - URING. This library will soon be developed in parallel to the higher-level IO library - similar to the development of ROCKSDB and RDB.
Over the next few months our IO ecosystem will slowly expand. As a short-term goal, we will implement simple binary streams backed by iouring.
2.5. DONE skel
- State "WIP" from "TBD"
- Note taken on
makefiles are now replaced by skelfile rules. next step is SKEL-VM for optimize rule execution plans. - State "DONE" from "WIP"
The pieces are there, just needs glue.
A usable SKEL system is a top priority for this year. To get there we
need to give love to many of the underlying core libraries too - most
of all cli
.
The two top priorities are VC commands and implementing a functional replacement for makefiles as used in the infra project.
2.6. DONE orgdb mvp
- State "TBD" from
- Note taken on
see graph.el, org-graph-init.lisp - Note taken on
renamed to org-graph-db (graph.el) see infra/scripts/org-graph-db-init.lisp and core/emacs/lib/graph.el - State "DONE" from "TBD"
- similar idea to org-roam
- stored in rocksdb columns
2.7. DONE infra
- State "TODO" from
- State "DONE" from "TODO"
It is time to move away from makefiles and start integrating our own
programs into the system bootstrap process. This iteration of the
infra project will have a few POSIX shell scripts for checking the
host and grabbing a pre-compiled lisp as well as a compressed
infra.core
image.