Caffe2 + PyTorch

1

A few questions for core team about plans on Caffe2 + PyTorch1 co-existence:

  1. Are there plans for making Caffe2 kernels available to PyTorch? There are many vision-specific kernels in Caffe2, not currently available in PyTorch (like detectron etc).

  2. Are there plans to make Caffe2 and PyTorch kernels the same thing sharing the same ATen base ops? Or they are going to be independent and only share the Tensor format?

  3. Do Caffe2 and PyTorch have different optimizing compilers?

  4. Are there benchmarks on performance of Caffe2 versus PyTorch (jit / nojit)?

Thanks a lot :slight_smile: Maybe some of these were already addressed at PyTorch conference :slight_smile:

@smth @apaszke @fmassa

3 Likes
2
  1. They will be implicit, i.e. no you wont get all the C2 operators exposed in the torch.* namespace, but where PyTorch operators have equivalent Caffe2 operators, the backends will be merged to just use the Caffe2 kernel instead if it’s more efficient.
  2. Answered in 1. And as of last week, C2 and PyTorch have the same Tensor format (c10::TensorImpl I think)
  3. Not in the long term, yes in the short term
  4. No
3

Thanks @smth for clarifications! :slight_smile:

  1. Really hoping for more unification :slight_smile: Since many detectron and other vision ops get pushed out faster in Caffe2 world

Additional questions:
5. How tied is the TorchScript or torch.jit to Python? I mean what happens if Julia people write bindings for PyTorch, will they be able to make use of torch.jit? If you load a model trained somewhere else, will PyTorch be able to optimize its execution as well?

  1. Are there bigger plans for torch.contrib? Right now it seems to stay semi-secret :slight_smile:
4
  1. torch.jit is not tied to Python. C++ API can do tracing.
  2. Yes, once we get bandwidth that was all sucked up for 1.0 Preview and DevCon :slight_smile:
5

Thanks for the details @smth :slight_smile: More PyTorch goodness for everyone :slight_smile:

The original PyTorch 1.0 blog post was not very clear on these details, and I think it is important for users to understand general PyTorch directions, especially when TensorFlow develops eager mode, and heterogenous compilation technologies start off. Btw, any plans for TensorComprehensions / Halide tighter integration? (https://people.csail.mit.edu/tzumao/gradient_halide/gradient_halide.pdf)

6

Hi @soumith :slight_smile: Are there any news about a big picture view on Caffe2 and PyTorch codebases? https://github.com/pytorch/pytorch/tree/master/caffe2/operators seems to still have a lot of ops and many of them match what PyTorch does, e.g. sin() : https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sin_op.cu

1 Like