How to assess a software agency when you cannot read the code

You’re paying an agency every month. The invoices arrive, the demos look fine, and you have a feeling you can’t put a name to. You can’t read the code, so you can’t tell whether the feeling is right.

You don’t need to read the code. Most of what matters shows up in things you can check yourself.

1. Whose name is on the accounts?

Ask where the source code lives and who owns that account. Then the hosting, the domain, the app store listings and the database. If the answer to any of them is the agency, you don’t own your product. You rent it, and the landlord is the party you’re trying to assess.

This is the first check because it decides how much room you have on all the others. Get the accounts into your company’s name before you raise anything else.

2. Can you see the work as it happens?

Ask for read access to the repository and the task board. You aren’t going to review code. You’re looking for a pulse: is work committed in small pieces through the week, or in one lump the day before the demo? Steady work looks steady. A team that resists being seen has told you something too.

3. How does a change reach your customers?

Ask them to walk you through releasing a small fix. A healthy answer is short: the change is reviewed, tests run, it goes to a staging site, then live, and it can be rolled back. An unhealthy answer involves one named person, an evening, and the word “carefully”.

4. What happens when it breaks?

Ask about the last outage. How did they find out, how long did it take, what changed afterwards? If they found out from your customers, there’s no monitoring. If nothing changed afterwards, it will happen again.

5. Do estimates and invoices line up?

Take the last six pieces of work. Compare what was estimated with what was billed. Everybody misses sometimes. A team that misses in one direction every time is either unable to estimate or has learned you’ll pay.

6. Who is doing the work?

You met a senior person in the pitch. Ask who has written your code this month. Agencies staff from a bench, and the person who won the work may never have touched it. That’s normal. Not being told is the problem.

7. Could somebody else take it over?

Ask for the documentation a new developer would need to get the system running on a fresh machine. If it exists, ask when it was last followed. The test of a handover document is whether anyone has used it.

This is the question that tells you whether you have a supplier or a dependency. I write software for clients, and I hold myself to it: you’re buying a system you could take elsewhere, and the day you can’t is the day the relationship stops being a choice.

8. Do they ever say no?

Look back over the last year. Did they ever tell you a feature was a bad idea, or that the cheaper route was to build nothing? An agency that builds whatever is asked is pleasant to work with and expensive to keep. The specification is where the money is won or lost, and a supplier who never challenges it isn’t looking after yours.

What the answers mean

One or two weak answers is ordinary, and a conversation fixes it. A good agency will be glad you asked, because a client who understands the work is easier to serve.

If the answers are weak on ownership, on releases and on handover together, you have a structural problem and no amount of chasing will fix it. That’s the point to get an independent technical read of what has been built: what state it’s in, what it would cost to move, and whether moving is the right call at all. It’s a bounded piece of work with the days agreed up front, and it’s a lot cheaper than another year of the feeling.