Level lower I mean that I don’t need to rely on Fastai to build Unet, Resent, Desnet, and more?
I’m sure you’ll get a better answer in the fast.ai forum or from @jphoward, as they have a much better overview what the course teaches and focuses on.
However, I always recommend the courses, as I’m learning a lot of useful techniques you can apply later in your research. I would also see it independent from using “lower level” PyTorch methods. 
Part 2 of the course shows how to build most of fastai from scratch using nothing but PyTorch tensors and autodiff. It’s not easy, but if you work at it and complete this part of the course, you’ll be a very strong PyTorch coder and DL practitioner at every level (except for stuff requiring CUDA coding). Many folks presenting at NeurIPS and working at top DL centers studied thru fast.ai.
4 Likes
The fastai courses do teach a lot of good things, but “good Python coding” without a lot of qualifications probably isn’t one of them.
Jeremy has been very outspoken that he doesn’t like Python and the fastai library clearly shows. There also are a lot of clever hacks around writing terse code that, shall we say, might not be universally seen as writing clear code for a library that can be built on. Now I do have a washlist of things I think are not so nice in PyTorch (e.g. spectral norm via hooks, default initialization, nondeterministic kernels…), but the structure of the API has been given quite a bit of thought and it shows.
The fast ai library versions 0.7, 1 and 2 seem to be really completely separate things with little bridges between them. The more mature PyTorch bits have been very stable since 0.4 or so (but if your code still uses 0.1.2/0.2/0.3-style Variable(..) a lot there is something to do…).
Don’t get me wrong, there are a lot of nice things to be found there and it is worth looking at, but – at least if you believe there is some truth to the Zen of Python – there are also lots of things that are really dubious fads in terms of how things are done.
As for the lectures, they are great when they do show how to get results with neural nets, but in my opinion they are weaker when they go into “how to design a library”. For this reason, I would recommend Part II 2018 over the 2019 version. (Starting with the outright bizarre “Python is 1000 times too slow” claim in the first lecture 2019 that has been qualified in the second to last lecture when it didn’t support the story anyways, but never made the errata list of the first lecture.) So lots of great insights and a refreshing style, but the frequency of dubious claims seems a bit higher than, say, what you get with e.g. the courses Stanford put online.
So if the university-style lectures you can find are not to your taste, fastai’s offering is one of the very cool ones out there and one that doesn’t dumb down things wile being very accessible. That is quite some achievement.
I should really write a more in-depth review, but the above was my impression of the main pros and cons.
Best regards
Thomas
6 Likes
That couldn’t be more wrong. The coding approach is built on far more years of research than Python has existed. The basic ideas go back to APL, developed in the late 1950’s.
For folks that are deeply immersed in just one way of doing things, this will be uncomfortable.
This is nearly universally accepted as being true.
2 Likes
Thank you for commenting on that.
I should indeed have said “idiosyncrasies” instead of “fads” and maybe “quite a few” rather than “lots”. What I was trying to express that in my observation some choices in the course seem to prefer terseness over concepts like explicit over implicit or sparse over dense and thus does not necessarily seem exemplary on the way to become a good Python coder as the question was put.
In the end it is your choice to disregard the things people generally to see as enhancing clarity in Python code. That is cool for you to do, but I view it as distracting form the value from the course.
Part of what strikes me as a weakness of the course that in between valuable takeaways there are passages similar to the above – it almost looks like a strong argument for your coding approach until one asks oneself what exactly (I paraphrase) “our coding approach is based on research that in part predates Python” actually says about the superiority of your coding approach over a more pythonic one.
The timing presented measured roughly “slowness of Python + slowness of creating lots of Tensors (relatively complex objects) in the hot loop” and it seems bizarre to conclude “Python is thousands of times too slow” in the context of a lecture emphasizing performance measurement as much as it did. Part of the disappointment might be that more controlled experimentation would have been quite easy to have. So in between other things, there is this bit presented with lots of emphasis that looks almost right but is, in reality, quite off.
So again, I do recommend the course, but the frequency of things like the above seems higher than in, to pick a random example, Chris Manning’s cs224n course.
Best regards
Thomas
4 Likes