Hi guys.
Is there specific tools/approach in PyTorch to organize incremental/cached model pass.
E.g. in cases when a model input could be purely differentiated to a stale / fresh (new) data still the separation is not constant in time as
input 1 [ █, █, █, █, █, █ ,█ ,█ ,█ ,█ , ░, ░, ░, ░, ░]
or
input 2 [ █, █, █, █, █, ░, ░, ░, ░, ░ , ░, ░, ░, ░, ░]
where █ - stale input unchanged from the previous pass
░ - fresh new data different for the current input
especially when such differentiation ratio could be significantly high is it possible to reuse calculations from the previous pass for the part that is still currently stale to save calculation resources.
How would the approach differ for trading and inference phases?
Thank you