You might like to know about SwitchNet. It uses the fast Walsh Hadamard transform to replace the one-to-all connectivity of a matrix of weights. It then uses parametric (switching based) activation functions as the adjustable system.
Then it uses a final fast transform as a set of set of statistical summary measures acting on the crude parametric function outputs.
https://ai462qqq.blogspot.com/2023/04/switch-net.html
Online code example:
https://editor.p5js.org/siobhan.491/sketches/RvqZfikaE