Copying files using shutil.copyfile gives "Permission denied error"

vision
1

Hi guys, I am trying to do a Cat & Dog CNN binary classifier network following this tutorial code from PacktPublishing:
https://github.com/PacktPublishing/Deep-Learning-with-PyTorch/blob/master/Chapter03/Image%20Classification%20Dogs%20and%20Cats.ipynb

However, you can see on the image below I am getting an “Permission denied error” despite I believe having the correct path to the folder set i.e. dogsandcats/valid

66e505230fd7f9b92530da525d8f4663
66e505230fd7f9b92530da525d8f46631136×436 25.8 KB

It should be basically going into the main directory and copying image files of dogs & cats into the ‘valid’ folder that was created earlier

I will appreciate any help, thank you.

2

either the files are not readable or not readable and writeable. I would recommed using chmod 644 -R foldername on that folder via command line (it’s a Linux/Unix command)

1 Like
3

Just run the notebook as Administrator in Windows or run with sudo in Linux and MacOS.

1 Like
5

OK, I solved the issue and found out what was going on. I will post here incase anyone follows the tutorial (Packt Publishing “Deep Learning with PyTorch”- Ch.3 “Training a Dog & Cat Classifier”):

Per this stack overflow post,python - Errno13 - Permission denied in documents folder? - Stack Overflow, I found that using “copy” instead of “copyfile” worked:

shutil.copy(files[i],‘…/chapter3/dogsandcats/valid/’)

I have already found alot of errors in this PyTorch book (Deep Learning with PyTorch), and it really makes following the tutorials difficult, especially when you are learning Machine Learning via PyTorch and following the tutorials in the book and even the tutorials have erroneous code.

Anyway, after fixing that, I still had an issue with this line of code:

os.rename(files[i],os.path.join(path,‘train’,folder,image))

I basically noticed after using some “print” statements to debug and see what variables/strings path, folder and image held, that it was double-cocantenating the directories thus my images were not being copied to the correct corresponding folders (i.e. the whole point of this code was to iterate thru a glob of 25000 images and pick out 2000 images at random to create a validation set. The images were string split between cat and dog and were to be subdivided into “cat” and “dog” folders corresponding to what the image was).

So basically it was concatenating something like:

“dogsandcats\train\valid\dogsandcats\train\cat\dogsandcats\cat\cat7.jpg”

So I was able to utilize os.path.basename such as the code below to properly extract the pieces of the directory/file name I wanted to properly copy and assign the cat & dog images to their correct corresponding folders for my validation set:

print (path)

for i in shuffle[:2000]:
shutil.copy(files[i],‘dogsandcats/valid’) # copies images into “valid” folder
folder = os.path.basename(files[i].split(‘/’)[-1].split(‘.’)[0]) # had to use os.path.basename to extract only folder name
#print (“The folder name is: %s” % folder) # used to debug to actual see what the image name was
image = os.path.basename(files[i].split(‘/’)[-1]) # had to use os.path.basename to extract only image name
#print (“The image name is: %s” % image) # used to debug to actual see what the image name was
#print (“The path name is: %s” % path)
os.rename(files[i],os.path.join(path,‘valid’,folder,image))

I do appreciate the quick replies and help from peoples on these forums, thanks again and hopefully this helps anyone following the similar tutorial and loading their data/creating a validation dataset for the very-popular and basic binary image classifier of cats and dogs.