Liquid Networks

Richard_Jenkins · July 12, 2023, 5:00am

I just watched an MIT discussion on Liquid Neurons and Liquid Networks. They took 10,000 neurons in an FC layer and replaced with 19 liquid neurons connected to 2 more layers with fewer liquid neurons each. The math function for the liquid neurons is defined in their work. Would a new nn class type be implemented as a LiquidNeuralNetwork as opposed to the Linear type for example? Would it be fairly easy to override existing classes and substitute the formulas?
Here is the video presentation on it Liquid Neural Networks, A New Idea That Allows AI To Learn Even After Training - YouTube

Richard_Jenkins · July 13, 2023, 5:18am

Is anyone working on implementing this today in PyTorch ?

J_Johnson · July 13, 2023, 6:32am

You would just need to translate this LTCCell definition from Tensorflow to Pytorch.

github.com

raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py

import tensorflow as tf
import numpy as np
import time
import os
from enum import Enum

class MappingType(Enum):
    Identity = 0
    Linear = 1
    Affine = 2

class ODESolver(Enum):
    SemiImplicit = 0
    Explicit = 1
    RungeKutta = 2

class LTCCell(tf.nn.rnn_cell.RNNCell):

    def __init__(self, num_units):

This file has been truncated. show original

ChatGPT or some other code LLM could probably handle most of the heavy lifting for you, and then some minor debugging.

banton · July 17, 2023, 6:33am

It seems like a pytorch implementation of liquid NN layers already exists: