Should you learn ML before starting with AI?
In this blog we will see why starting at the basics fails, and the order that takes a software engineer from RAG to the math.
In November 2022, OpenAI released ChatGPT.[3] Software engineers wanted to build on language models. Most of them opened the same courses, at the same first lesson.
Engineers are often asked to learn from the basics. But then, they end up learning useless information that is never used on the job.
The purists will swear by the fundamentals. Backpropagation and network bias. This is why most people lose motivation midway and give up.
A working engineer faced two problems:
- The courses started at ML and MNIST. The job started at RAG.
- A working professional had a few hours a week. The motivation ran out before the course reached the job.
In this blog we will see why starting at the basics fails, and the order that takes a software engineer from RAG to the math.
1. Where does the usual path end?
Where does a software engineer start with AI? Where the course starts.
One major issue I see with people who are starting off with AI is they start with the basics, the absolute basics, and they never reach the advanced concepts. For example, they go for ML, they go for MNIST, and by the time they've reached RAG, all of their motivation is gone. This is not going to work.
If a backend engineer opens a machine learning course, the first model they train reads a handwritten digit. The system they want to build is a RAG pipeline over their company's documents.
The evenings go into the digits. The course reaches RAG in its last lessons, if it reaches it at all. By then the evenings have run out.
The same thing happens outside AI.
I've seen this with DSA preparation also, where people start with binary search. At most, they reach arrays and then in the end they're gone. So don't do this.
That path starts at binary search, the first chapter of the algorithms book. The interview asks for dynamic programming, trees and graphs. Those are the last chapters.
A path that starts at the foundations and builds up to the job is called bottom-up learning. It spends the motivation on the foundations, and nothing is left for the job.
2. Take it like an onion
So where does the engineer start instead? At the job.
Take it like an onion, layer by layer. The core is what you work with on a daily basis. So RAG, agents, evals, that's where you want to start.
The core is what the engineer does at work. All three parts of it run on a model behind an API. None of them needs a line of math.
On Monday, the engineer splits the company's documents into chunks and fetches the chunks that match a question. The model answers from them.
Retrieval is the fetching of external context for a query, at the time the query is asked. With the model's answer after it, that is RAG.
On Tuesday, the model is given tools, and it decides which one to call for the question it was asked.
A system that uses a language model to plan and make decisions, while using tools to execute commands, is called an agent.
On Wednesday, the engineer runs the same 50 questions through Monday's pipeline and Tuesday's agent. The count says which one got more of them right.
That count is an eval. An eval is a test of an AI system's behaviour, run against a fixed set of inputs whose correct answers are known or judged.
So the engineer isn't training anything. The engineer is doing the job in the first week. The job raises the questions the next layer answers.
3. The deeper concepts
What happens when Monday's pipeline answers wrong? The engineer put 20 chunks in the prompt, and the model used the wrong one.
Then you want to look at the deeper concepts. Maybe you start with transformers, maybe you look at diffusion models, after which you can go for the deep ML side, like backpropagation, how is the math working out.
The second layer answers the question the core raised. Which chunk the model reads from is decided by attention, the mechanism inside a transformer, published by eight researchers at Google in 2017.[4]
Diffusion models are the same layer for images. An engineer who has called an image API has a reason to ask how the picture came out of noise.
The third layer is the math. Backpropagation, the procedure that sets a network's weights, was published in 1986.[5] By the time the engineer reads it, they've been calling those weights for months.
Starting at the working system and going down to the theory as it is needed is called top-down learning.
fast.ai teaches deep learning in that order. Its first lesson trains an image classifier, and the theory comes in the lessons after it.[6]
By the time the engineer reaches the math, every formula has a question waiting for it.
4. Don't waste your time
Couldn't the engineer do both, the math on weekdays and RAG on weekends? The evenings aren't the problem.
Don't waste your time. You're not going to have all the motivation in the world, especially if you're a working professional. You don't have the time to do all this stuff. So start with the important parts and then move on the sides.
Sources
- Should you learn ML before starting with AI?, Gaurav Sen, 3 June 2026.
- Gradient-based learning applied to document recognition, Yann LeCun and others, Proceedings of the IEEE, 1998.
- Introducing ChatGPT, OpenAI, 30 November 2022.
- Attention Is All You Need, Ashish Vaswani and others, 2017.
- Learning representations by back-propagating errors, David Rumelhart and others, Nature, 1986.
- Practical Deep Learning for Coders 2019, Jeremy Howard, fast.ai, 23 January 2019.
