I Don't Care What You Call Your Tests

The below article, while focused primarily on software testing practices, contains many attempts at quips or jokes, including some not-so-subtle political jabs. Reader discretion is no longer my problem, it’s yours.

I don’t care what you call your tests

• Unit tests
• Integration tests
• Emotional support tests

I don’t care. What I do care about is that your tests are meaningful, reproducible and simple. I want your tests to be enforcers; that is to say that they enforce expectations around intended and exceptional behaviors, not to be confused with a societal enforcer - you don’t want your tests getting stuck behind the impenetrable wall that is an elementary school door.

So allow me to talk to you about why I don’t care… actually, now that I think about it that title is a lie. The real title is:

I don’t care what you call your tests (before I started talking) *

• And you and I worked together *
  • And I had the time *
    • And I actually wanted to maintain your code in the first place *

So, “What do you think I should be doing differently?”, I say as if to project the question I want asked upon you, thanks for that. Personally I believe this comes down to a few key goals:

• Simplicity
• Practicality
• Third thing here

Test Simplicity

You ever pop open a file in your code-editor-of-the-week just to be met with incomprehensible noise, references dangling and mutating across hundreds of lines and think to yourself: “I can’t right now, this is too much.” Yeah? Well tests can’t be like that, or else they fail at their most important job: simplicity.

Tests need to be simple. A person must be able to pick up a project, look at a test and instantly know what is being tested for. Simplicity is the biggest reason I advocate to mock as little as you can get away with. Mocks are a powerful tool that obfuscate behavior, and tests that use mocks in the wrong way can even hide bugs, broken integrations and crashes oh my!

To be clear I believe there is no always in code. Mocks are a wrench in your proverbial toolbox, and just like my less-than-proverbial toolbox: I’ve got a hammer, a wrench and tens of tools people have gifted me that I barely know the name of. When you need a mock, reach for a wrench- er, proverbially speaking, but be willing to consider when simpler solutions, such as more closely sticking to a real use-case like an API call with mock data. Trust me, the results when you pull this off is some delightfully readable, maintainable code.

When testing backends, I strongly prefer to write and run tests against a real database. I’ll run migrations before all my tests, seed data per-test and even reset said data after each test. I do this both locally and in CI, where I will run all these dependencies as part of my test suite. Friendly tip: even if you don’t use docker in production, just plop a docker-compose.yml in the root with some blank template and a small comment that explains this is here to run any service dependencies needed for local development or testing. Other developers will thank you later, silently; we are a timid sort - but alas, I’m projecting again.

Take a look below at this Jest setup. I use a describe wrapper block for a group of tests specific to MyService. It’s largely just normal setup, but you can see I’m making sure that I clear the database before each test and after they all finish, resulting in a consistent test state when leveraging my database as part of my tests.

Before you test

// Describe my feature group, which in this case is a service
// or perhaps this could be a specific slice of domain logic.
describe('MyService', () => {
  let prisma: PrismaService
  let service: MyService

  // before every test...
  beforeEach(async () => {
    // Prepare the service and any dependencies before each test.
    prisma = new PrismaService()
    service = new MyService()

    // clear database before each test. This is usually a helper
    // function I either dangle off of my wrapped service class or
    // just as a function which accepts my database as it's argument.
    await prisma.truncateAll()
  })

  // after *all* tests finish...
  afterAll(async () => {
    // it's good to do some final cleanup
    // so we don't leave any orphan records.
    await prisma.truncateAll()
  })
})

”But Madison,” I hear you begin, my fingertips braced against my temples, “that would make your tests unsound and volatile to bugs and crashes from your database!” Why yes, attentive reader, that would be the case, but allow me to postulate this: why is that a bad thing?

If your database, or any dependency for that matter, has buggy behavior, or perhaps the way you used your database was incorrect - as detected by your tests - then your tests have caught a problem that would appear in a live environment. Your tests did their job, and you should make sure to thank them with a short template email provided by HR and that donut somebody left at the office and never claimed by weeks end. (Don’t actually do this. I want the donut.)

Further aligning with the goal of simplicity, tests written against a real database are also just easier to read and write. You can rapidly write tests where the only thing that you mock is data, usually via seeding.

Here’s a test for an API I’m working on that does all of these things. I’ve commented thoroughly to explain the code, but consider just how simple this is!

Tests that are Simple

// Setup the tests with Jest. This would live inside of 
// the above describe block and be an inner layer.
describe('byUserUUID', () => {
  // @faker-js/faker is used to mock data; it's wonderful!
  const user_uuid = faker.datatype.uuid()

  it('should return an array of MyType', async () => {
    // Factories are a rapid seeding tool I've stolen directly from Laravel.
    // https://laravel.com/docs/9.x/database-testing#defining-model-factories
    //
    // I had to define these functions myself, but they generate some `count`
    // of mocked data, relationships included, and send it to the database
    // all in one convenient little call! You can see here that I even choose
    // to support pasing in a Partial<T> where `T` is the record struct to mock.
    // These are taken as absolute overrides, which is super handy to have.
    const expected = await myTypeFactory(prisma, {
      user_uuid,
      count: 10
    })
    // Prior to this test definition I've defined `service` as the actual
    // underlying code being ran. If this was a pure function you could
    // just call that.
    //
    // A pattern I like for pure functions is returning a new function:
    // Accept your services/dependencies first and return a 
    // new function that makes the request and writes the response.
    const actual = await service.find({ user_uuid })

    // Expectations are dead simple. The count should be what I expect
    // because I know that these are the only items in the database.
    // That's the power of truncating _before_ each test!
    expect(actual.length).toBe(expected.length)
    // You can see here that I even get a little fancy. I expect my data
    // to be sorted in descending order, and I also expect some data overrides
    // to happen at the API level.
    expect(actual).toEqual(
      expected
        .sort((a, z) => z.created_at.getTime() - a.created_at.getTime())
        .map((s) => ({
          ...s,
          title: s.title ?? s.fallback.title,
          company: s.company ?? s.fallback.company,
          description: s.description ?? s.fallback.description,
          location: s.location ?? s.fallback.location,
          history: s.history.sort((a, z) => z.created_at.getTime() - a.created_at.getTime())
        }))
    )
  })
})

Test Practicality

Testing more directly against your dependent services actually makes your tests more practical too. To be clear when I say practical here what I’m referring to is how similar your tests are to a real life scenario. Writing tests that run real SQL against a real database may reveal a bug that mocking would obscure, like how your database is actually MongoDB.

I certainly wasn’t involved in that conversation.

Writing practical tests is about testing the actual behavior that a user of your software would enact. In a web service this would be your API, in a library that’s your public and exported types and functions. Generally speaking tests that assert against I/O will serve you well in this regard; it’s a safe rule-of-thumb you can fall back on.

Practicality also means knowing what not to test. I don’t test my private and un-exported members. I’ve met developers who curse my name under the moon, burning essential oils into a binding talisman to dispel this idea - alright even I admit that was a bit of a cheap shot. Any devs I know don’t remember what the night sky even looks like.

So once again I will postulate: what are you actually testing? Are you trying to maximize code coverage, a number which even at 100% can still miss bugs and, ever frequently, unhandled exceptional behavior? Are you testing because it felt right, and you never bothered to consider if “good feels” might not be the kind of confidence you want to handoff to your client or businesses at go live?

Quality tests are written with thoughtful intent about the behaviors they are testing. Testing methodologies - a topic I won’t soon break into for fear of attracting the attention of internet zealot’s looking for the first chance to strike me down with their mighty word stabbing pitchforks and “very hot do-not-touch” torches - exist to try and provoke some engaging and conscious thought when writing tests.

Let me reiterate, because that bit went on so long that even I forgot the original point by the time I reached the end of my run-on sentence: testing methodologies exist to ask you to think before you write both code and tests. Remember: the value of Test Driven Development is what comes out of it; if you’ll achieve that value without the process then the routine becomes nothing more than a formality.

Oh boy here I go projecting again!

Let’s face it: none of us want to write tests. It’s a chore, and like all chores the reward only comes when the task is complete. Writing good tests is it’s own skillset, in fact it could even branch into its own career path: intern- I mean QA!

Writing good tests is what a strong QA team are best at. If you have a QA team, great! Rely on the them, and strive to meet their standards and recommendations. The last thing you want is to end up in a situation where the inequal power of the engineering department agrees to overturn QA’s longstanding rights to declare standards upon the tests and the software, a scenario made worse when the head of each department within the engineering umbrella can choose to either protect that privilege or ban it entirely for their representative QA, forcing QA to work through disadvantaged and oppressive environments where their expertise on the very matter you hired them for is ignored by developers who’s skillset is, largely, something else entirely, and further expanding the inequality gap between the two independent, capable groups of highly skilled and respectable individuals.

Apropos of nothing at all, of course.

So in conclusion:

• Call your tests whatever makes you happy
• Make an effort to keep tests simple to read, simple to write, and practice practicality.
• Third point here…

And please, get to it before I pull my hair out trying to help someone mock another damn ORM.

Thanks for reading ❤️