Spec'ing a machine for Deep Learning

Hi all,

Apologies if this isn’t an appropriate post for the forum, but I figured there would be a wealth of knowledge to tap to.

What specifications would people recommend when buying/building a computer that will mainly be used for machine learning. The project I’m working on has some funding and I want to put together a reasonable machine without going overboard, but also one that won’t end up being underpowered and need updating again in a year.

Well it’s a tricky question, depends on the budget and what do you work on.
As everything in the life it’s a trade-off.

First think you should take into account is what do you want to work on.

I would go for threadrippers (they have plenty of cores at a good quality/price) to take advantage of multiprocessing.
A nvme hard disk, at least SSD, no hdd.
Realise PCIe 4.0 is out.

The delicated thing is how many of everything.
1st thing how many gpus you can afford.
You will need a power supply of 400*n + 200 more or less.

If you are gonna work with video I would get .3090s and at least 64 Gb of RAM.
But in the end there are no rules carved in stone, depends on your budget, how many ppl will use it, research field…

Let me know extra details for extra help.

1 Like

Thanks for the in-depth reply!

I’m currently waiting to find out how much of a budget we have to spend. I’m really hoping it’s at least <=£2k.

With the SSD - how much of a bottleneck will it create if it’s pulling the data from a HDD?

I’ll be working solely with multi-channel audio synthesis/multivariate regression - one of the reasons I’ve struggled trying to spec it myself is that most of the bench marks are done using computer vision models and I’m finding it hard to transfer it across into how they would perform on audio. Though I would assume most systems that perform well on image/video would be sufficient for audio.

How do the threadrippers stack up to the Intel stuff - I’ve no real experience of AMD?

I would assume that unless I go for older GPUs I’ll probably only be able to afford one.

I think at present I’d probably be the only person running stuff on it…however it would be ideal if I could run a couple of models at a time for time efficiency.

SSD is a must. Remember that majority of your work is preprocessing and transformations which benefit a lot from fast storage. I have 2.5TB of pci-e 3.0 Nvme storage and 3TB HDD. Every time I need to deal with the data on HDD it always bogs down my workflow to the rage inflicting point.

CPU is only marginally important these days as long as you got midrange and up. I have two desktop machines that I’m working on (a few generations old at this point), one is a total beast of it’s time i9-9900k with 5Ghz over clock on all 8cores, the other one is i5-10400 locked at 4.3 on 6cores. Both are hyper-threaded so double the amount for logical cores. I don’t really feel the speed difference whatsoever not in day to day activities nor crunching numbers.

If you can squeeze threadpipper into your budget - you won’t bottlenecking yourself on CPU whatsoever.

Now the main challenge these days is to get a GPU. Grab as many as you can, going 3090 is actually worth it cause it’s not as efficient for mining as lower cards, thus, the demand markups are less. Good luck

Okay so let me add some extra comments.

If you think more ppl will work with the computer, take into account that datasets may ocuppy a lot.
For working with audio the datasets I’ve played with are very contained (like MUSDB and so).
So you can start with 1 Tb nvme. If you go for audiovisual and need to store video you will lack hard disk.

I think you can go for 32 Gb of RAM in audio field.
Using HDD is a very strong bottleneck. Prob 2 times slower.
You can just check nvme specs vs sdd too and you will see nvme is way faster.

Given your budget, PCIe 4.0 is too expensive atm.
Threadrippers are from 12 to 16 cores (2 threads/core) thus between 24-32 threads.
Multiprocessing lauches 1 process per thread. Therefore the more cores the faster dataloading.

AMD usually offers more pci lanes than intel, where many of their cpus are capped. Although I didn’t try such config like @Anton suggest so cannot give u more hints. However, as he mentions, you don’t need to buy last gen necesarily.

For the motherboard you can look for something with 2 gpu slots so that you can buy 1 gpu when you have some extra money.

In terms of GPUs, 3090 is awesome for both, the amount of VRAM, and the speed.
It’s kinda 50% faster than previous gens but it’s very expensive too.

You need something with at least 12 Gb of VRAM as a U-Net already burns 9 or so.
Atm I find the market a bit ill. New gen GPUs are weirdly expensive and not really worth for deep learning due to the VRAM but 3090. Besides the market is kinda out of stock.

The 1080 ti used to be really worth in terms of quality/price. In fact Nvidia retired those due to that. I saw this in amazon USA

If that were the price, you can get 2 of these until you can afford.
Otherwise I wouldn’t know what to recommend you.

With a 3090 I guess it’s around 2k5 or so. Dunno if you have to pay taxes or not. W/o taxes can be 2k more or less.

Good luck

Unsurprisingly it seems to be the GPU area I’m struggling in. I think ideally I’d want at least 10GB VRAM which incurs a huge price hike compared to the 8GB cards currently available, and that’s the only thing which may push me over budget. But I’m not sure if there are many/any benefits with getting a machine with a single 8GB GPU over just using some of the cloud services. I’ve played around on Colab but the limited runtimes and fair usage policy means I don’t want to rely on it for my entire project.

So, my lab has a machine with the below specs, and looking at things like “compute capability” I’m not really sure what they mean.

Processor - dual Xeon e5-2670
RAM - 256GB
GPU - Quatro FX 5800

I’m aware this is all pretty old. But wondering if it’s not too out of date to still be useful.

Soo the GPU is not useful but it was a server cpu so it was very powerful back ago.
It’s prob DDR3 memory soo the bandwith is less but you can try to plug a modern GPU and do a benchmark to check.

Audio data loading is not really demanding so you may be fine if you get a ssd/nvme disk.

This website was recommended to me and also might be of use to others. It’s a pretty comprehensive guide to putting together hardware for ML/DL applications and where people commonly go wrong/spend money in the wrong places.