7 questions to ask before you pick an AI engineering course
The Ship AI Engineer team · Published 2026-09-14
Anyone looking for an AI engineering course today faces hundreds of options: long video series, university programmes, bootcamps, free playlists. Prices and lengths vary wildly, but the descriptions are built from almost the same sentences: "from zero to expert", "hands-on", "the skills the industry needs".
Those sentences tell you nothing. The syllabus does. This post won't tell you which course to buy; it gives you seven questions to ask while reading any course's syllabus, and for each one, a concrete sign you can check before you pay. They apply to our own course too.
Which job does the course teach?
Check this first, because the label "AI" covers two separate jobs.
On one side is machine learning engineering, the job of training models: linear algebra, probability, gradient descent, neural network training. On the other is AI engineering, the job of building products on a model someone else trained: APIs, answering from your own data (RAG), agents that use tools, evaluation and shipping. Check: in the first third of the syllabus, do words like "derivative", "loss function" and "backpropagation" appear, or words like "API", "token" and "context"? The first set teaches the first job. If you're not sure which job you want, our roadmap post puts the two side by side.
What will you have when you finish?
If the answer is a certificate, be careful; what you want is working projects.
When people hiring in this field look at a candidate, they want to see what that person shipped first. Check: does the description list, one by one, the projects you'll have at the end? Does it say what each one does, which skill it shows, and where it ends up (in your own GitHub account, or on the course's platform)? The word "hands-on" on its own says nothing; the list of projects does. A certificate is at most one extra line for whoever reads that list.
Is the code actually run?
Code you write and run teaches you; code you watch doesn't.
Watching someone write code in a video gives you the feeling that you understand it. That feeling lasts until you try to write the same code into an empty file yourself. Check: does every lesson have an exercise you write, and is it checked by automated tests, or are you told to "copy the code from the video"? Ask one more thing: are the course's own solutions tested? An exercise whose reference solution doesn't work tells the learner, unfairly, that something is wrong with them, and that's one of the most common moments beginners quit.
Does it teach cost and failure?
It should, because those are usually what sink a real AI product.
Every call to a language model is billed per token, and one design decision (resending the whole chat history with every message, putting twenty document chunks into every question) multiplies the bill. Providers reject requests at busy hours, the model sometimes doesn't answer in the expected shape, a PDF turns into broken text. Check: does the syllabus have topics like "rate limits", "retries", "timeouts", "token budget" or "caching"? If none of them appear, the course only teaches you the day everything goes right. The cost game is a good place to see how an architecture choice changes the bill.
Does it teach evaluation?
This is the clearest question for telling a good course from an average one.
The only honest way to show an AI system "works" is to measure it against a fixed set of questions. You changed the prompt: did the share of correct answers go up, go down, or go up on one topic and down on another? Without measurement nobody can say. Check: does the syllabus cover how to build a test set, where having one model grade another model's answers (LLM-as-judge) misleads you, and how to compare scores? If that topic is missing entirely, the course is teaching you to say "I think it got better".
Does it teach shipping?
It should, because this stage is exactly the difference between a demo and a product.
A chatbot running on your laptop is a demo. A product needs sign-in, a database, rate limits against abuse, secrets stored safely, deployment and monitoring. Leaving an endpoint that calls a language model unprotected opens your bill to the internet. Check: at the end of the course, does your project run at an address on the internet, or only on your machine? If you want to see how real projects break after they ship, look at the failure cases.
When was the content last updated?
Check, because in this field last year's lesson can be wrong today.
Model names, prices and API details change every few months; a parameter that works in one version can return an error in the next. Check: is the information that will age (prices, model names, versions) given with a date, and is the last update date visible? A course whose description leans on one specific model version but shows no update date has most likely aged along with that version.
Ask our course these questions too
We built Ship AI Engineer around these seven questions: it teaches the job of building products, every module ends with a finished project, the exercises run in the browser and are checked by tests, and our own solutions are tested automatically on every change. Information that will age carries its date. But us saying so isn't enough; check the syllabus on the course page against the questions above yourself. The free knowledge map also shows the concepts the curriculum covers, no account needed.
Sources
- Hugging Face: Evaluation Guidebook: a practical guide to evaluating language models, including the limits of LLM-as-judge. (added 2026-09)
- Anthropic: Create strong empirical evaluations: how to design a test set. (added 2026-09)
- OWASP: Top 10 for LLM Applications: the risks a course's security section should cover. (added 2026-09)
- Google SRE Book: Monitoring distributed systems: the core text on what to watch in a running system. (added 2026-09)
Keep reading
2026-10-06
Resources worth following as an AI engineer: what to read, and when
A new post, model or tool appears in this field every day, and trying to follow all of it is the fastest way to learn none of it. A handful of resources that genuinely help, and the stage of learning at which each one is worth reading.
2026-10-05
Being an engineer in the age of AI: code got cheap, structure didn't
Writing code has never been this fast. But code produced quickly amplifies whatever structure sits underneath it. Why engineering matters more today, not less, and where to go deeper.
2026-10-02
Shipping an AI application: spending caps, caching, monitoring and model changes
In an ordinary web app every request costs roughly the same. In an app with a language model inside, it doesn't. Capping cost when you ship, using caching safely, watching the right things, and being ready when the provider retires a model.
2026-09-30
An AI engineer portfolio: what a project that gets you hired looks like
A chatbot demo impresses nobody anymore. Six signs that make an AI project credible, five project ideas that each show a different skill, and a README someone can read in two minutes.
2026-09-27
Prompt injection and abuse: why the server has to be the single source of truth in an AI app
An application with a language model inside carries every security problem of an ordinary web app and adds new ones. Why prompt injection can't be fully prevented, how to limit the damage, protecting the endpoint, and why the model's output can't be trusted.
2026-09-23
Building your first test set for an LLM application: a guide to evaluation
You changed the prompt and the answers look better. Do they? Building a first test set, taking a baseline, seeing what the average hides, the limits of one model grading another, and watching the system once it's live.
2026-09-19
When should you fine-tune, and when is it a waste of time?
Training a model on your own data is an attractive idea. Most of the time it's the wrong tool. What fine-tuning teaches and what it doesn't, how it compares with prompting and RAG, data quality, leakage, LoRA, and licences on open models.
2026-09-09
AI with image and audio input: the costs and failures your tests don't show
Handing a model a photo or a recording isn't the same as sending text. Request size, unreadable images, sentences invented in silence, generation that takes seconds, and an image paid for again on every turn.
2026-09-04
When don't you need an agent? Building and stopping a tool-using system safely
"Agent" is today's most requested and most easily misbuilt thing. The difference between a fixed workflow and an agent, why tool descriptions are critical, how to stop the loop, and how to protect irreversible actions.
2026-08-31
Why does RAG give wrong answers? Most failures are in retrieval, not the model
When a system that answers from your documents says something wrong, the model is the first suspect. Most of the time the culprit is the chunk it was given. Finding where the failure is, containing invention, citing sources and measuring properly.
2026-08-26
How to become an AI engineer: a roadmap that builds products, not models
Most roadmaps in the search results describe a different job. What the AI engineers being hired today actually know, the order to learn it in, and what makes each stage "done".
2026-08-21
When do you actually need a vector database? A practical guide to embeddings
The first step people take after deciding to build semantic search is usually picking a vector database. Most of the time it's an early step. What an embedding is, what a similarity score doesn't tell you, and the real reasons to move to a database.
2026-08-17
Managing prompts like code: versions, templates, schemas and tests
A prompt that works in a chat window quietly breaks inside an application. Keeping prompts in the repository, separating user input from instructions, validating the answer against a schema, and shipping a change only after measuring it.
2026-08-12
Why LLM APIs break under real traffic: rate limits, retries and timeouts
Three errors you never saw in development arrive together in the first busy hour. Telling 429 from 5xx, retrying properly, setting timeouts, and keeping the product up when the provider goes down.
2026-08-07
Do you need maths to become an AI engineer?
The honest answers to the six questions that stop people before they start: maths, degrees, machine learning, Python, hardware and money. Short answers first, reasons after.
2026-08-07
An AI engineer is not a machine learning engineer
Search "how to become an AI engineer" and the first page teaches linear algebra, CNNs and training loops. That is a different job. Here is what the work actually is, and what you can skip.