Porting code from Thano/Lasagne to PyTorch


(Ergnoor Shehu) #1

Is there anything in PyTortch similar to theano.function()

import theano
x = theano.tensor.dscalar()
f = theano.function([x], 2*x)
f(4)
… array(8.0)


#2
>>> import torch
>>> def f(x):
...     return 2 * torch.DoubleTensor([x])
... 
>>> f(4)
tensor([8.], dtype=torch.float64)

(Ergnoor Shehu) #3

@Tony-Y thank you for answering my question, but your answer is not the one I was expecting and maybe because I used a simple example to present theano.function(). A more detailed explanation follows:

def function(inputs, outputs=None, mode=None, updates=None, …):
“”"
Return a class: callable object that will calculate outputs from inputs.

Parameters
----------
inputs : list of either Variable or In instances.
Function parameters, these are not allowed to be shared variables.
outputs : list or dict of Variables or Out instances.
If it is a dict, the keys must be strings. Expressions to compute.
mode : string or Mode instance.
Compilation mode.
updates : iterable over pairs (shared_variable, new_expression). List, tuple or OrderedDict.
Updates the values for Shared Variable inputs according to these expressions.


#4
>>> import torch
>>> import torch.nn as nn
>>> class F(nn.Module):
...   def __init__(self):
...     super(F, self).__init__()
...   def forward(self, x):
...     return 2 * x
... 
>>> f = F()
>>> a = torch.DoubleTensor([4])
>>> f(a)
tensor([8.], dtype=torch.float64)

(Ergnoor Shehu) #5

@Tony-Y thank you again for the answer, but what you give is a static solution while the theano.function() is a dynamic one in the following sense:

we have data and a list of expressions [‘expression_01’, ’ expression_02’, expression_03’, …]

for example:

expression_01 = 2*x
expression_02 = 3^x
expression_03 = 3x^2 - 5

and when there is given the following:

x = theano.tensor.dscalar()
list_of_expression = [‘expression_01’]
f = theano.function([x], list_of_expression, …)
f(2)
…array(8.0)

but in another scenario we have:

x = theano.tensor.dscalar()
list_of_expression = [‘expression_01’, ’ expression_02’, expression_03’]
f = theano.function([x], list_of_expression, …)
f(2)
…array(8.0, 9.0, 7.0)

this is how I understand the theano.function(), hence you just adjust the ‘list_of_expression’ according to your needs and theano.function() does the rest for you dynamically, and I hope it is a little bit more clear. for you too.


#6
>>> expressions = {
...   'expression_01': lambda x: 2*x,
...   'expression_02': lambda x: 3**x,
...   'expression_03': lambda x: 3*x**2 - 5}
>>> 
>>> import torch
>>> import torch.nn as nn
>>> class F(nn.Module):
...   def __init__(self, expression):
...     super(F, self).__init__()
...     self.expression = expression
...   def forward(self, x):
...     return self.expression(x)
...
>>> a = torch.DoubleTensor([4]) 
>>> f = F(expressions['expression_01'])
>>> f(a)
tensor([8.], dtype=torch.float64)
>>> f = F(expressions['expression_02'])
>>> f(a)
tensor([81.], dtype=torch.float64)
>>> f = F(expressions['expression_03'])
>>> f(a)
tensor([43.], dtype=torch.float64)

(Ergnoor Shehu) #7

@Tony-Y thank you very much for your time and help. This last answer is the closest one to what I was expecting. The only one requirement not fulfilled is that it is needed for the expressions to be fed one by one and not en block, but I believe that could be manged somehow in the init part with a ‘for’ loop, looping over all the expressions and assigning them in the following fashion:

for i, expression enumerate(list_of_expressions):
self.expression_0i = list_of_expression[‘expression_0i’]

and then at the return part we have:

     return self.expression_0i(x)

I mean something like this in general. Do you think it is doable?


(Thomas V) #8

The core difference between PyTorch and Theano you’re wondering about here is that in Theano you create a symbolic graph that you then feed into function to have it compiled to a function you can call while in PyTorch you write your calculation and PyTorch runs it as you write.

Modules are decidedly only there to hold learnable parameters / state - see Jeremy Howard’s recently added tutorial.

Now you could assemble lines of Python and then eval it to form your function, but quite likely, you’re not making the best use of PyTorch that way. One of the things people like about PyTorch is that you don’t have the create graph -> compile -> run workflow.

Best regards

Thomas


(Ergnoor Shehu) #9

@tom hi Thomas and thank you very much for reinforcing my knowledge about the differences between Theano/Lasagne and PyTorch.
I am an engineer and my main goal is to find a solution for my project. It is not that I am writing a code from scratches in PyTorch. As you have noticed my problem now is porting a code from Theano/Lasagne to PyTorch. I am doing this because in PyTorch it is easier to debug and there is more support, and I am experiencing this myself even communicating with you right now. In this porting procedure I would preferred to change the original code as less as possible up to that degree that gives me the possibility to debug it easier.
I understand that the workflow philosophy behind PyTorch is different from that of Theano/Lasagne. Telling you the truth PyTorch workflow is the one that I am used to, and it took me a while till I understood the workflow of Theano/Lasagne. Some times (I mean most of the times :-)) people do not have the luxury to be picky, they just have to float with the current. Even in my case I do not have the luxury of being picky in that sense that: no this is not exactly PyTorch, that is half Theano half PyTorch. With this I mean no offense for anyone else that is stringent in crossing the borders between the two libraries. My main goal is to complete my project with any reasonable mean possible.
I am trying to explain that I know this is not the best way to write a code.
Please know that I appreciate very much the help and advises offered by you guys all.
Cheers.
Ergnoor


(Thomas V) #10

Oh, sorry, I don’t want to make the impression to tell you how and how not to usw PyTorch. It was my impression that maybe you were looking for something more elaborate because the typical PyTorch transposition of that type of code can look suspiciously simple.
When I last did similar things, I tried to just write all the steps between the definition input variables later specified in of the Theano function call and the output in one regular Python function using PyTorch arithmetic. This looks a lot like Tony’s first example (except that the DoubleTensor constructor is probably not a good idea and you’d just use x there). In a way this should be very similar to what Theano does except that Python’s function declaration takes the place of Theano’s function call.
If there is code you find particularly difficult to translate, I’m sure we’ll try to help you out.

Best regards

Thomas


(Ergnoor Shehu) #11

@tom Hi Thomas and thank you very much for your understanding.
Cheers.
Ergnoor