Hello! I’m trying to build pytorch cpp MNIST example from here. However, I can’t seem to build the example through README. I tried both building libtorch from source and a libtorch downloaded from the website.
When I build libtorch from source, I follow the example from this README. Basically, I did:
cd pytorch
mkdir build_libtorch && cd build_libtorch
python ../tools/build_libtorch.py
In the example I built it with
cd mnist
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=/home/pytorch/build_libtorch/build/ ..
make
I got a cmake error:
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /home/pytorch/build_libtorch/build/Caffe2Config.cmake:14 (include):
include could not find load file:
/home/pytorch/build_libtorch/build/public/utils.cmake
Call Stack (most recent call first):
/home/pytorch/build_libtorch/build/TorchConfig.cmake:40 (find_package)
CMakeLists.txt:4 (find_package)
CMake Error at /home/pytorch/build_libtorch/build/Caffe2Config.cmake:17 (include):
include could not find load file:
/home/pytorch/build_libtorch/build/public/threads.cmake
Call Stack (most recent call first):
/home/pytorch/build_libtorch/build/TorchConfig.cmake:40 (find_package)
CMakeLists.txt:4 (find_package)
CMake Error at /home/pytorch/build_libtorch/build/Caffe2Config.cmake:22 (include):
include could not find load file:
/home/pytorch/build_libtorch/build/public/gflags.cmake
Call Stack (most recent call first):
/home/pytorch/build_libtorch/build/TorchConfig.cmake:40 (find_package)
CMakeLists.txt:4 (find_package)
CMake Error at /home/pytorch/build_libtorch/build/Caffe2Config.cmake:24 (message):
Your installed Caffe2 version uses gflags but the gflags library cannot be
found. Did you accidentally remove it, or have you set the right
CMAKE_PREFIX_PATH and/or GFLAGS_ROOT_DIR? If you do not have gflags, you
will need to install gflags and set the library path accordingly.
Call Stack (most recent call first):
/home/pytorch/build_libtorch/build/TorchConfig.cmake:40 (find_package)
CMakeLists.txt:4 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/pytorch-examples/cpp/mnist/build/CMakeFiles/CMakeOutput.log".
Meanwhile, I also tried to build the example with libtorch downloaded from the website (nightly build).
cmake was through successfully but I got the following message
Unzipped /mydata/lexu/pytorch-examples/cpp/mnist/build/data/t10k-labels-idx1-ubyte.gz ...
-- Configuring done
CMake Warning at CMakeLists.txt:18 (add_executable):
Cannot generate a safe runtime search path for target mnist because there
is a cycle in the constraint graph:
dir 0 is [/mydata/lexu/libtorch/lib]
dir 1 is [/usr/local/cuda/lib64/stubs]
dir 2 is [/usr/local/cuda/lib64]
dir 3 must precede it due to runtime library [libnvToolsExt.so.1]
dir 3 is [/mydata/lexu/anaconda3/lib]
dir 2 must precede it due to runtime library [libnvToolsExt.so.1]
Some of these libraries may not be found correctly.
-- Generating done
-- Build files have been written to: /mydata/lexu/pytorch-examples/cpp/mnist/build
However, building with the nighly built cause the following error:
/home/libtorch/include/torch/csrc/api/include/torch/data/iterator.h:68:25: error: binding reference of type ‘torch::data::Example<>&’ to ‘const torch::data::Example<>’ discards qualifiers
return batch_.value();
^
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result]’:
/home/libtorch/include/torch/csrc/api/include/torch/data/detail/sequencers.h:93:39: required from ‘c10::optional<T> torch::data::detail::sequencers::OrderedSequencer<Result>::next(typename torch::data::detail::sequencers::Sequencer<Result>::ResultProducer) [with Result = torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result; typename torch::data::detail::sequencers::Sequencer<Result>::ResultProducer = std::function<c10::optional<torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result>()>]’
/home/libtorch/include/torch/csrc/api/include/torch/data/detail/sequencers.h:71:20: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:121:10: note: in call to ‘torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result& torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result::operator=(const torch::data::DataLoaderBase<torch::data::datasets::MapDataset<torch::data::datasets::MapDataset<torch::data::datasets::MNIST, torch::data::transforms::Normalize<> >, torch::data::transforms::Stack<torch::data::Example<> > >, torch::data::Example<>, std::vector<long unsigned int, std::allocator<long unsigned int> > >::Result&)’
struct Result : Sequenced {
^~~~~~
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::data::Example<>]’:
/home/libtorch/include/torch/csrc/api/include/torch/data/iterator.h:56:12: required from ‘void torch::data::detail::ValidIterator<Batch>::next() [with Batch = torch::data::Example<>]’
/home/libtorch/include/torch/csrc/api/include/torch/data/iterator.h:50:8: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::data::Example<>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/data/datasets/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/datasets.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/data/example.h:12:8: note: in call to ‘torch::data::Example<>& torch::data::Example<>::operator=(const torch::data::Example<>&)’
struct Example {
^~~~~~~
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<3>]’:
/home/libtorch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool3dImpl]’
/home/libtorch/include/torch/csrc/api/include/torch/nn/cloneable.h:81:8: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<3>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/nn/options/conv.h:6,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:7,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<3>& torch::ExpandingArray<3>::operator=(const torch::ExpandingArray<3>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<3, double>]’:
/home/libtorch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool3dImpl]’
/home/libtorch/include/torch/csrc/api/include/torch/nn/cloneable.h:81:8: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<3, double>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/nn/options/conv.h:6,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:7,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<3, double>& torch::ExpandingArray<3, double>::operator=(const torch::ExpandingArray<3, double>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<2>]’:
/home/libtorch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool2dImpl]’
/home/libtorch/include/torch/csrc/api/include/torch/nn/cloneable.h:81:8: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<2>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/nn/options/conv.h:6,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:7,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<2>& torch::ExpandingArray<2>::operator=(const torch::ExpandingArray<2>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
In file included from /home/libtorch/include/c10/core/ScalarType.h:6,
from /home/libtorch/include/c10/core/Scalar.h:9,
from /home/libtorch/include/ATen/core/TensorBody.h:7,
from /home/libtorch/include/ATen/Tensor.h:11,
from /home/libtorch/include/ATen/Context.h:4,
from /home/libtorch/include/ATen/ATen.h:5,
from /home/libtorch/include/torch/csrc/api/include/torch/types.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/data.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<2, double>]’:
/home/libtorch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool2dImpl]’
/home/libtorch/include/torch/csrc/api/include/torch/nn/cloneable.h:81:8: required from here
/home/libtorch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<2, double>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
In file included from /home/libtorch/include/torch/csrc/api/include/torch/nn/options/conv.h:6,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/libtorch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/libtorch/include/torch/csrc/api/include/torch/all.h:7,
from /home/libtorch/include/torch/csrc/api/include/torch/torch.h:3,
from /home/pytorch-examples/cpp/mnist/mnist.cpp:1:
/home/libtorch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<2, double>& torch::ExpandingArray<2, double>::operator=(const torch::ExpandingArray<2, double>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
CMakeFiles/mnist.dir/build.make:62: recipe for target 'CMakeFiles/mnist.dir/mnist.cpp.o' failed
make[2]: *** [CMakeFiles/mnist.dir/mnist.cpp.o] Error 1
CMakeFiles/Makefile2:72: recipe for target 'CMakeFiles/mnist.dir/all' failed
make[1]: *** [CMakeFiles/mnist.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Any hints on what’s going on?
Thanks!