what does an ai software engineer do?

the actual weekly work

Sorted roughly by how much time it takes:

1. Data plumbing and retrieval. Getting the right context in front of the model. Chunking, embedding, indexing, ranking, and the endless work of noticing that the retrieved passage was almost right. This is where most of the quality lives.
2. Evaluation. Building the harness that tells you whether a change helped. Golden sets, scoring, regression runs. Teams that skip this ship on vibes and cannot tell an improvement from a regression.
3. Inference plumbing. Batching, caching, streaming, retries, timeouts, fallback between providers, and keeping cost per request under control.
4. Prompt and tool design. Less glamorous than it sounds. Mostly writing tool schemas the model does not misuse and reducing instructions it ignores.
5. Agent loops. Planning, tool calling, error recovery, and stop conditions, for the subset of teams building agents rather than assistants.

Notice what is absent: training models. That is a research engineering job at a lab, and it is a different hiring pipeline.

the two problems that make it different

Output is nondeterministic. The same input can give different results. Every habit built on deterministic systems, from unit tests to bug reproduction, needs adapting. You end up testing distributions and properties rather than exact strings, and you learn to be suspicious of a fix that worked once.

Every request has a price. Latency and cost are functions of how much text you send, which means architecture decisions that were free before now show up on an invoice. Caching stops being an optimisation and becomes part of the design.

Everything else is ordinary engineering: apis, queues, databases, deployments, observability.

the three job titles hiding under one

The label covers people doing quite different work, and pay tracks the tier rather than the title:

• Application tier. Builds product features on model apis. The largest and fastest-growing group.
• Platform tier. Builds the retrieval, evaluation, and serving infrastructure other teams use.
• Research tier. Trains, fine-tunes, and evaluates models. Smallest, best paid, hardest to enter.

When you read a job ad, find which tier it is by looking for training, inference at scale, and evaluation work. If none of the three is in the description, it is application engineering with an ai label.

the skills that actually transfer

Strong backend fundamentals, comfort with data pipelines, and a measurement habit. Almost every effective ai engineer I have worked with came from backend or data engineering rather than from a machine learning course. The scarce ability is deciding what to measure, not knowing what a transformer is.

If you want the pay bands, how much do ai software engineers get paid has them by market. If you want the route in, how do i become an ai software engineer is the practical version.