notes

1. On Infra

ID: 332145f8-50b4-46d1-8368-ddaf097c875f
CREATED: <2025-02-06 Thu 19:07>

Something that is missing from many organizations big or large, is an effective way to store and access information, even about their own org.

It can be difficult problem to solve - usually there's the official one, say Microsoft Sharepoint and then the list of unofficial sources which becomes tribal corporate hacker knowledge. Maybe the unofficial ones are more current, or are annotated nicely, but their very existence breaks the system. There's no longer a single source of truth.

My priority in this department is writing services which process and store information from a variety of sources in a distributed knowledge graph. The graph can later be queried to access information on-demand.

My idea of infrastructure is in fact to build my own Cloud. Needless to say I don't have an O365 subscription, and wherever possible I'll be relying on hardware I have physical access to. I'm not opposed to cloud services at large but based on principle I like to think we shouldn't be built on them.

2. WL vs X

ID: 9a398886-858a-450b-9029-2339cc551bc3
CREATED: <2025-02-06 Thu 19:07>
  • State "NOTE" from [2024-02-18 Sun 11:55]

In the past few months there has been drama regarding Wayland vs X. It seems to be on everyone's minds after Artem's freakout issue and the follow up YT vids/comments.

I admit that it made me reconsider the fitness of WL as a whole - there was a github gist that made some scathing arguments against it.

It's an odd debate though. I think there are many misunderstandings.

So first off, if we look at the homepage https://wayland.freedesktop.org/, Wayland claims it is a replacement for X11. It now has manifest destiny, which in my opinion is a great shame.

X-pros seem to agree that Wayland has manifest destiny - like if you are building softwares that look remotely like a window system, it's a successor to X. That's the model of doing things and there's no way around it.

The disagreement starts with how this destiny - of an X2 - should be fulfilled. X-pros want a fork of X, but it's too late for that. WL-pros want X to run on top of Wayland compositor: https://wayland.freedesktop.org/xserver.html.

Xwayland is a problem for me. From the project description: 'if we're migrating away from X, it makes sense to have a good backwards compatibility story.' Full disclosure: I have never done significant work on Xwayland, so perhaps my opinion is unwarranted. But I have no intention of attempting to maintain a computer system that uses Wayland and X clients at the same time.

To me, X is ol' reliable. Every distro has first-class X support, and it runs on most systems with very little user intervention. Where it doesn't, there is 20+ years of dev history and battle-tested workarounds for you to find your solution in.

Wayland is the new kid on the block, born just in 2008. It's a fresh start to one of the most difficult challenges in software - window systems. A re-write would be pointless though, and so the real value-add is in design. Wayland is designed as a protocol and collection of libraries which are implemented in your own compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this feels right even if the implementation is something very different (compositor vs compiler).

With X, it is assumed to be much harder to write an equivalent 'compositor'. Here's the thing though - with a significantly complex X client implementation, it is impossible to replicate in WL. This is really the crux of Artem's argument in his issue. He asked for a 1:1 equivalent X/WL comparison when no such thing exists, and in my opinion it is a waste of time.

The WL core team is fully aware of this dichotomy, but also that this is in no way a problem or weakness in either system. It means they're different systems, goddammit.

If it was up to me, Xwayland wouldn't exist. I understand why it does, and that it does make things easier for developers who need to support both, and users who have multiple apps with multiple windowing requirements. It's a bandaid though, and one that is particularly dangerous because it re-enforces the idea that Wayland is just X2 and that they're fully compatible.

What interests me in the Wayland world right now is the idea of a small, modular, full-stack Wayland compositor API. There are several 'kiosk' based compositors for single applications (cage), but these aren't complete solutions. It is possible to get much closer to the metal, and that's where I want to be so that I can build my own APIs on top - I don't want to live on top of X, and I certainly don't want to live on top of X on top of WL. I want a pure solution that hides as little as possible, exposing the interesting bits.

3. Intelligent Design in Software

ID: c2e682b3-d72a-41b3-b093-16614e7852ff
  • starting from a space where there are no external influences - a biome
  • answer questions regarding the nature of the software and its capabilities
  • incrementally adjust inter-dependencies
  • optimize
  • protect the biome at all costs
  • focus on composition
  • build applications
  • aggresively incorporate lessons learned

4. SBCL+Rust=?

ID: a8b0dd7e-741c-4f5f-9d66-40b26ef56b14
CREATED: <2025-02-14 Fri 19:04>

4.1. Overview

ID: 748ba6e4-60db-4ff7-9d78-12c3f67644d8
CODE
packy/stash/dysk

Lisp is powerful. Rust is consistent. I like these languages and have an interest in combining them - i.e. calling Rust code from Lisp and vice-versa. The level of interop we're after in this case is the simplest - FFI. In the future we won't be paying much attention to the FFI in particular, for reasons which will become obvious shortly, and instead opt for 'source2source' compilation.

There's an article about calling Rust from Common Lisp here which shows the basics and serves as a great starting point for those interested.

4.1.1. Rust ABI

ID: 6c24dc95-eea6-44fb-8c7f-bb49227ca7bf

The complication(s) with Rust I mentioned early is really just that it is not C. C is old, i.e. well-supported with a stable ABI, making the process of creating bindings for a C library a breeze in many languages.

For a Rust library we need to first appease the compiler, as explained in this section of the Rustonomicon. Among other things it involves changing the calling-convention of functions with a type signature and editing the Cargo.toml file to produce a C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used like the C ABI can.

abi_stable_crates is a project which addresses some of the ABI concerns, presenting a sort of ABI-API as a Rust library. Perhaps this is the direction the ecosystem will go with in order to maintain an unstable ABI, but for now there is no 'clear' pathway for a friction-less FFI development experience in Rust.

4.1.2. Overhead

ID: faf8fbcb-3403-4e2f-b881-ef1404ad9780

Using FFI involves some overhead. Check here for an example benchmark across a few languages. While building the core, I'm very much aware of this, and will need a few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular, I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).

4.2. basic example

ID: ab04fff7-51c9-4f1c-b61b-fa4739932343
header-args: :tangle no

4.2.1. Setup

ID: 08420b18-3856-4b62-9523-98fb3398afca

For starters, I'm going to assume we all have Rust (via rustup) and sbcl installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in this post).

  1. Cargo
    ID: cfeb1299-67c5-4a64-863e-f214e195e176

    Create a new library crate. For this example we're focusing on a 'skeleton' for dynamic libraries only, so our experiment will be called dylib-skel or dysk for short.

    cargo init dysk --lib && cd dysk && touch test.rs

    A src/lib.rs will be generated for you.

    The next step is to edit your Cargo.toml file. Add these lines after the [package] section and before [dependencies]: 

    [lib]
crate-type = ["cdylib","rlib"]
path = "src/lib.rs"
[[bin]]
name="dysk-test"
path="test.rs"

    This tells Rust to generate a shared C-compatible object with a .so extension which we can open using dlopen as well as an executable binary named dysk-test.

  2. cbindgen
    ID: 435d84d6-c959-4bf6-ad37-fb6e524b54ff
    1. install
      ID: 15115ab8-e7b4-4050-9567-bf026cc140d1

      Next, we want the cbindgen program which we'll use to generate header files for C/C++. This step isn't necessary at all, we just want it for further experimentation.

      cargo install --force cbindgen

      We append the cbindgen crate as a build dependency to our Cargo.toml like so: 

      [build-dependencies]
cbindgen = "0.24"
    2. cbindgen.toml
      ID: 21d776b7-9f8d-445c-83c2-8e0c28751827

      language = "C"
autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
include_version = true
namespace = "dysk"
cpp_compat = true
after_includes = "#define DYSK_VERSION \"0.1.0\""
line_length = 88
tab_width = 2
documentation = true
documentation_style = "c99"
usize_is_size_t = true
[cython]
header = "dysk.h"
    3. build.rs
      ID: 469dd142-bafa-4076-8ba9-baf088e60260

      fn main() -> Result<(), cbindgen::Error> {
  if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
    b.write_to_file("dysk.h"); Ok(())}
  else { panic!("failed to generate dysk.h from cbindgen.toml") } }

4.2.2. lib.rs

ID: e9d50e77-4d1e-4d38-90aa-9977d8c6c888

//! lib.rs --- dysk library
use std::ffi::{c_char, c_int, CString};
#[no_mangle]
pub extern "C" fn hello() -> *const c_char {
  CString::new("hello from rust").unwrap().into_raw()}
#[no_mangle]
pub extern "C" fn plus(a:c_int,b:c_int) -> c_int {a+b}
#[no_mangle]
pub extern "C" fn plus1(n:c_int) -> c_int {n+1}

4.2.3. test.rs

ID: 84b5dffe-7171-4d72-9788-f288e3185070

//! test.rs --- dysk test
fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::plus1(2 as core::ffi::c_int);}}

4.2.4. compile

ID: 30f4fb3e-0757-4df6-947f-8d1e11edabf9

cargo build --release

4.2.5. load from SBCL

ID: 69b517f2-648d-4e44-b956-7879b3dadf99

;;; dysk.lisp
;; (dysk:hello) ;; => "hello from rust"
(defpackage :dysk
  (:use :cl :sb-alien)
  (:export :hello :plus :plus1))
(in-package :dysk)
(load-shared-object #P"target/release/libdysk.so")
(define-alien-routine hello c-string)
(define-alien-routine plus int (a int) (b int))
(define-alien-routine plus1 int (n int))
  1. benchmark
    ID: 86e4195d-601b-4736-b852-1209a6d38bdf

    time target/release/dysk-test

    (time (dotimes (_ 500000000) (dysk:plus1 2)))

5. ops

ID: 68ab3b7d-272e-4807-b2e9-959f182b8966
CREATED: <2025-02-20 Thu 20:10>

5.1. Release

ID: 9376d6e5-fe52-41f3-a821-0c2286daf1e0
CREATED: <2025-02-20 Thu 20:10>

https://cl-net-snmp.common-lisp.dev/release/release.sh 

#!/bin/sh
set -x

rm -rf $1-$2/.git
rm -f $1-$2.tar.gz*
tar cvzf $1-$2.tar.gz $1-$2
rm $1_latest.tar.gz
ln -s $1-$2.tar.gz $1_latest.tar.gz
gpg -b -a $1-$2.tar.gz
rm $1_latest.tar.gz.asc
ln -s $1-$2.tar.gz.asc $1_latest.tar.gz.asc

6. Tree-sitter

ID: 026c796e-dcff-4eda-ad83-12e5d60f9971
CREATED: <2025-02-26 Wed 17:31>

[2025-02-26 Wed 17:31] – GitHub - tree-sitter/tree-sitter: An incremental parsing system for programming tools [2025-02-26 Wed 17:32] – Introduction - Tree-sitter [2025-02-26 Wed 18:45] – Syntax Highlighting - Tree-sitter

:END:

7. Get Off My Cloud

CREATED: <2025-02-26 Wed 20:38>
ID: 6dbb0064-1421-41bb-b2df-36f694efce0a
CUSTOM_ID: get-off-my-cloud
  • industry has moved away from client-side, distributed compute in favor of centralized server-side compute resources behind API gateways.
  • entire businesses are built on a single Cloud Provider and are fundamentally incapable of moving off that Cloud.
    • they think in terms of that Provider. The Provider influences all of their decisions.
  • users control very little compute power
    • personal computing hardware (consumer-grade) is limited in capability
      • non-servicable architectures, planned obsolescence, closed firmware
    • mainstream operating systems don't optimize for resource efficiency - they maximize for the volume of telemetry data they can collect and profit from

8. Death of the Programmer

CREATED: <2025-02-26 Wed 20:38>
ID: 138f4a50-1d52-4e13-bf67-f225ef9f6747
CUSTOM_ID: death-of-the-programmer
  • The role of the programmer is changing
  • programmers are no longer required to understand how computers work to have a successful career
    • Cloud Providers wrap all low-level details in their own proprietary vocabulary and APIs
    • To program on the cloud, you need to use the Cloud vocabulary and are discouraged from thinking of computers as they actually exist in the real world
  • Cloud Providers influence college cirruculums, replacing compute and systems theory with courses designed to teach you how to configure Cloud Services.

9. A Narcissist's Prayer

ID: 98e52e0e-104b-4dd8-b147-3a170163aeb9
CREATED: <2025-03-01 Sat 11:41>

[2025-03-01 Sat 11:41] – A Narcissist's Prayer | narcissistprayer

10. duangle texts

ID: 1c1bfaa5-53d9-4274-b4ef-621b7b52a959
CREATED: <2025-03-01 Sat 12:22>

[2025-03-01 Sat 12:23] – ~duangle/texts - devlogs, documentation, musings, pastes, etc. - sourcehut git

  • great list of resources from a small german indie-game studio
  • @paniq = Leonard Ritter
  • shared on Lisp Discord, via Mastodon
  • Main - Duangle
  • plan.md

11. Enshittification of the Internet

ID: aa7d66fb-d93a-49b5-9bb3-2425400f5100
  • State "NOTE" from [2023-11-04 Sat 12:20]
  • DEFCON 31 - An Audacious Plan to Halt the Internet's Enshittification - Cory Doctorow
  • "Five giant websites filled with screenshots of text from the other four" - Tom Eastman
  • lies, deceit, manipulation. you should be angry. furious.
  • another view - too many people. corporations are bad.
  • the crimes of the public company. the crimes of the stock owners.
  • Network effects - double-edged sword - once people leave they don't come back.
  • large companies hedge to keep competition out - buy competition, kinda suck themselves, sell below cost
  • Competitors hack interoperability layers to get over competitor high-switching costs.
  • we're hackers so there are artificially low switching costs.
  • now adversarial interoperability hacking is poopoo'd by FAANG
  • media giants have agreed to end history - chatbots instead of humans
  • felony contempt of business model = IP
  • adblocks in apps are now illegal
  • when companies do not face competition, are allowed to tweak backend laws to do 'Darth Vader shit'.
  • we can't pass laws against them.
  • The America Act - Ted Cruz, Elizabeth Warren
  • HUGE plug for the FTC and the good work they're doing (Activision, Blizzard)
  • interop is how we seize the means of computation
  • apply gig-economy
  • new API standards, platforms which maximize interop
  • easy to get wrong
  • these APIs can be hacked by big players.
  • restore right to reverse, mod, hack, tinker services
  • how do we keep the interoperators honest?
  • the alternative to good fire is wild fire
  • we could have a better tech sector
  • standard path is work a few years at FAANG, make shit startup and get hired back by your own boss
  • cory@eff.org
  • Ulysses is a hacker
  • I want to hear the siren's song
  • form a Ulysses pact - they exist in tech
  • anticipate your weakness - take steps while you are cognizent
  • Federated services are better. they fail gracefully.
  • can these services attain lift-off? hard equation
  • "If you wanted to do something I wouldn't start from here."
  • Norms, Laws, Codes, and Markets

12. ultimate traffic controller

ID: 02999b9e-d860-4608-8b63-80265d65f951

[2025-03-03 Mon 15:21] – The Ultimate Traffic Conditioner: Low Latency, Fast Up & Downloads

  • State "NOTE" from [2024-05-02 Thu 15:12]