I’ve released a small library for parametric curves for PyTorch that are differentiable: you can backprop to the curve’s inputs and to its parameters. At this stage, I have B-Spline curves (efficiently, exploiting sparsity!) and Legendre Polynomials.
Applications include:
- Continuous embeddings for embedding-based models (i.e. factorization machines, transformers, etc)
- KANs. You don’t have to use B-Splines. You can, in fact, use any well-approximating basis for the learned activations.
- Shape-restricted models, i.e. modeling the probability of winning an auction given auction features x and a bid b. You have a neural network c(x) that predicts the coefficients of a function of b. If you force the coefficient vector to be non-decreasing, then if used with a B-Spline you will get a non-decreasing probability, which is the right inductive bias.
I hope some of you will find it useful!