Pytorch 0.2.0 Python 3.5 issue which seems quite common

clived2 · September 11, 2017, 8:09pm

Struggling away with pytorch 0.2.0. I am trying to run a Udemy deeplearning project on ubuntu 16.04 x64. Surprisingly enough the same project runs well on a Windows 10 laptop with an earlier version of pytorch on a conda python 3.5 env. So I created a conda python 3.5 env on ubuntu laptop to run things like tensorflow, pytorch, etc.
My problem is that after loading torch (import torch) at the top of my Python script, as soon as the script reaches the part where torch is enabled (torch.nn in this case) the script crashes, killing the kernel. I discovered that in a python window, i could “import torch” but when I do a torch.rand(4) as an example, i get an “illegal instruction, core dumped” so this is what is killing my script. I have a screenshot of this but am unclear as to how to attach it to my message. I hope this makes sense to any of the developers who are involved in the pytorch 0.2.0 project.

smth · September 12, 2017, 2:28pm

Clive, are you running this in a Virtual Machine? What is the output of cat /proc/cpuinfo?
I am looking to see if there is atleast SSE4 support. PyTorch binaries ship with SSE, SSE2, SSE3, SSE4 assembly instructions. SSE4 itself is quite an old standard with processors dating many years supporting it, so we didn’t think it was a problem. However, I wonder if your machine supports it.

clived2 · September 12, 2017, 6:44pm

hi there

I am actually running this on a desktop Ubuntu 16.04 x64 desktop running Python 3.6. but I did up to a day ago run my script in a PY35 env on the same computer and the results were the same. Here is the display of cat/proccpuinfo
clived@UbuntuGnome2:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Pentium® Dual CPU E2140 @ 1.60GHz
stepping : 13
microcode : 0xa1
cpu MHz : 1200.000
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm lahf_lm dtherm
bugs :
bogomips : 3189.63
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Pentium® Dual CPU E2140 @ 1.60GHz
stepping : 13
microcode : 0xa1
cpu MHz : 1200.000
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm lahf_lm dtherm
bugs :
bogomips : 3189.63
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

This computer does seem to support SSE.SSE2 and SSE3 but not SSE4. Both are older machines running I686 cpus
I hope this helps and thanks for your response

Clive

clived2 · September 12, 2017, 6:56pm

I might just add that I also ran the same code in another ubuntu 1604 x64 box with the same results

QuantScientist · September 12, 2017, 7:10pm

Can you build it from source like so:

github.com

QuantScientist/Deep-Learning-Boot-Camp/blob/master/day02-PyTORCH-and-PyCUDA/PyTorch/build_torch.sh

# PyTorch GPU and CPU
# If you dont have CUDA installed, run this first:
# https://github.com/QuantScientist/Deep-Learning-Boot-Camp/blob/master/docker/deps_nvidia_docker.sh

#GPU version
export PATH=/usr/local/cuda-8.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
export PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
export CUDA_BIN_PATH=/usr/local/cuda
export CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0


# Build PyTorch from source
git clone https://github.com/pytorch/pytorch.git
cd pytorch
git submodule update --init
#git checkout 4eb448a051a1421de1dda9bd2ddfb34396eb7287 

export TORCH_CUDA_ARCH_LIST="3.5 5.2 6.0 6.1+PTX"

This file has been truncated. show original

smth · September 12, 2017, 7:31pm

@clived2 your only option (if you dont have SSE4) is to build from source. Unfortunately our binaries dont work for machines with processors this far back.

See @QuantScientist 's advice or follow instructions from here: https://github.com/pytorch/pytorch#from-source

clived2 · September 12, 2017, 7:38pm

Thanks, I’ll try to build it from source. My graphics card is one of those internal Intel ones, is such graphic cards as this acceptable ?
I’ll follow QuantScientist’s email and see waht happens

clived2 · September 12, 2017, 7:42pm

Thanks, I’ll remove the version of torch that I installed and build it as you suggested here

clived2 · September 12, 2017, 9:13pm

WOW, guys it worked. I don’t have a nvidia graphics card, just one of those intel things on the motherboard, so I ignored any references to it in QuantScientist’s notes.
While my two linux boxes are sort of old, I’m running the latest Ubuntu distribution, and it took about 30 minutes to compile and it seems to be working just fine.
I tested it operationally on some of the Python scripts from my Udemy courses and they all seem to be working.
This has been quite the week for me, yesterday tweaking a tensorflow install to run on Python3.6 and today building my own version of Pytorch

Thanks guys

Clive

QuantScientist · September 12, 2017, 9:20pm

Great, happy it worked out for you, let me know if you need anything else.

clived2 · September 12, 2017, 9:25pm

Will do, QuantScientist. Your notes on the subject did the trick for me

Thanks a million