Embracing Pragmatic TDD

There are numerous discussion and debates around different testing strategies, and it still feels hard to find consensus on what makes unit test. That makes software teams confused and somehow start applying TDD for the sake of it. They usually focus on tooling / testing libraries and test coverage number, but less on what actually need to be tested and how to write maintainable tests.

So how should we apply TDD to gain real value? Ultimately it should enable the team to ship code with more confidence without slowing down. I realised this more clearly until I read this tweet:

People love debating what percentage of which type of tests to write, but it's a distraction. Nearly zero teams write expressive tests that establish clear boundaries, run quickly & reliably, and only fail for useful reasons. Focus on that instead.

https://twitter.com/searls/status/1393385209089990659

This ticks all the box for what I'd like to achieve towards pragmatic TDD. Let me explain with some more details.

Solitary or Sociable? All about Boundary

Solitary basically means we test each individual component completely in isolation, it won't rely on any external infra when running tests, they assume external infra services are properly configured to communicate with.

Sociable however requires some infra set up, e.g local db or cache etc. When running tests, it actually pulls data from db and verify logic based on these result. From this sense, it's more like integration test for me.

This slides summarises pretty well:

Sociable	Solitary
Cross some boundaries	Never cross boundaries
One or more concrete classes during test	Only One concrete class during test

A boundary is "a database, a queue, another external system, or even an ordinary class if that class is 'outside' the area testing objects are responsible for"

Let's continue with previous checkout api scenario as an example. I'll show how we can write solitary tests effectively with clear boundaries for each layer. It should also give you better understanding why we follow DIP to structure code which makes testing easier.

Test DB in Repository layer with sqlmock

What do we test here?

Test SQL generated via any SQL builder or ORM is expected
Test returned entity structure is expected based on mocked DB rows

Let's assume we have ProductRepo to pull a product by its id and the related supplier from DB. Here we use GORM as an example.

type ProductRepo interface {
  GetById(id int) (entities.Product, error)
}

func NewRepo(db *gorm.DB) *productDBRepo {
  return &productDBRepo{db}
}

// specific model where ORM pulls data into
type Product struct {
  ID    // Standard field for the primary key 
  Name  // Convention mapping to name column
  Supplier models.Supplier // declare relation to suppliers table
}
// domain entity is infra agnostic type to return
type ProductEntity struct {
  Id   int
  Name string
  SupplierName string
  Contact string
  ...
}

func(r *productDBRepo) GetById(id int) (entities.Product, error) {
  p := Product{}
  res := r.db.Preload("Supplier").Take(&p, "id=?", id)
  if err != nil {
    return entities.Product{}, err
  }
  // transform internal model to domain entity
  return toEntity(p), nil
}

How to test without crossing the DB boundary with sqlmock.

sqlmock is a mock library implementing sql/driver - to simulate any sql driver behavior in tests, without needing a real database connection.

import (
    "testing"
    "github.com/DATA-DOG/go-sqlmock"
    "gorm.io/gorm"
    gormmysql "gorm.io/driver/mysql"
)

func OpenTestDB() (*gorm.DB, sqlmock.Sqlmock) {
    // creates sqlmock database connection and a mock to manage expectations.
    db, mock, err := sqlmock.New(sqlmock.QueryMatcherOption(sqlmock.QueryMatcherEqual))
    if err != nil {
        log.Fatalf("failed", err)
    }
    // initiate gorm wrapped db session with mock db connection 
    gormDB, _ := gorm.Open(gormMysql.New(
      gormMysql.Config{Conn: db}
    ))
    return gormDB, mock
}

func Test_GetById(t *testing.T) {
  gormDB, mock := OpenTestDB()
  rows := mock.NewRows([]string{"id", "name"})
  supplierRows := sqlmock.NewRows([]string{"id", "supplier_name", "contact"})
  expectedSQL := "SELECT id, name from products where id=?"
  expectedSupplierSQL := "SELECT * from suppliers where `suppliers`.`id` = ?"

  t.Run("should return product entity", func(t *testing.T)) {
      id, supplierId := 1, 10
      rows.AddRow(1, "Product 1", supplierId)
      supplierRows.AddRow(10, "SupplierA", "1234")
      // set up expectation
      mock.ExpectQuery(expectedSQL).WithArgs(id).WillReturnRows(rows)
      mock.ExpectQuery(expectedSupplierSQL).WithArgs(supplierId).WillReturnRows(supplierRows)

      expectedEntity = ProductEntity{
        Id: id, 
        SupplierName: "SupplierA"
        Contact: "1234"
      }
      repo := NewProductRepo(gormDB)
      rs, err := repo.GetById(id)
      t.NoError(err)
      t.Equal(expectEntity, rs)
      // check all queued expectations were met in order
      mock.ExpectationsWereMet()
  }

  t.Run("should return error when query failed", func(t *testing.T)) {
      mock.ExpectQuery(expectedSQL).WithArgs(id).WillReturnError(errors.New("query failed"))
      repo := NewRepo(gormDB)
      rs, err := repo.GetById(id)
      t.Error(err)
  }
}

Here as you see we don't connect to external db for testing data. And our testing subject productDBRepo is the only concrete instance.

💡

You probably already noticed we do have concrete gorm.DB instance passed into ProductRepo, but its underlying db connection is mocked via sql.mock. This is the special part when testing DB interaction, where we mock at low level sql driver provided by Go db package. This makes sqlmock suitable for any third-party SQL builder or ORM libraries.

Test domain service layer with testify/mock

Let's move to the upper layer: domain service. If we look at CheckoutService in previous post.

// domain service only knows the contracts via interface
type CheckoutService struct {
  productRepo  ProductRepo
  orderRepo    OrderRepo
  paymentProvder PaymentProvider
}

As we follow DIP principle, "Not crossing boundary" means we can easily mock out these dependencies without actually issuing query or communicating with external payment gateway. Let's see how we do it via testify/mock. Comments in the code block to help understand.

package repomocks // under repo/mocks

type ProductRepoMock struct {
   // embed Mock struct to "inherit" all utilities from it
   mock.Mock
}
func (r *ProductRepoMock) CheckAvailability(n []entities.Product) (bool, error) {
    // track Check method is called. Panic if call is unexpected
      args := r.Called(n)
    // return whatever passed in mock call, see in test cases
    return args.Get(0).(bool), args.Error(1)
}
// same as other OrderRepo and PaymentProvider mock
// ...

💡

It can be tedious to write if we have a bunch of methods to implement for an interface. We can use mockery to auto generate these mock code.

Now let's look at domain service tests.

func NewTestObject() CheckoutService {
  productMock := new(ProductRepoMock)
  paymentMock := new(PaymentProviderMock)
  orderMock := new(OrderRepoMock)
  // see how we can use mock objects to instantiate real CheckoutService
  // as they all satisfy interfaces CheckoutService depends on
  return CheckoutService{
    productRepo: productMock,
    paymentProvider: paymentMock
    orderRepo:  orderMock
  }
}
func Test_DoCheckout(t *testing.T) {
  t.Run("should not charge and return error if product is not available", func(t *testing.T) {
     svc := NewTestObject()
     // First set call expectation to return false which means not available in our domain logic
     productMock.On("CheckAvailability", entities.Product{}).Return(false, nil)
     // other method calls should not be invoked according to our logic
     paymentMock.AssertNotCalled(t, "Charge")
     orderMock.AssertNotCalled(t, "UpdateStatus")
     res, err := svc.DoCheckout(entities.Order{...})
     assert.NotNil(t, err)
  })

  t.Run("should not update status if charge failed", func(t *testing.T) {
    svc := NewTestObject()
    // mock order entities as testing input
    order := entities.Order{}
    // call expectation to return true 
    productMock.On("CheckAvailability", order.Products).Return(true, nil)
    paymentMock.On("Charge", order).Return(errors.New("charge failed"))
    orderMock.AssertNotCalled(t, "UpdateStatus")

    res, err := svc.DoCheckout(order)
    assert.NotNil(t, err)
  })

  ..
}

You can see that we have only one concrete instance CheckoutService, it does not care how external parts are implemented, but only focus on the input/ouput. If we need to change payment gateway (e.g to Stripe), we just need to provide a new implementation to satisfy PaymentProvider, our CheckoutService tests should still work as domain logic does not change.

This is important to write stable tests ("Only fail for useful reasons") as well in this article: "Not testing implementation details": focus on testing use cases from users' perspective. For backend service, the user could be caller of this service method. This makes test cases resilient to implementation details change.

Test Application Handler via built-in httptest

The final part moves to the API handler using mocked CheckoutService. What to test in this layer:

Test received request is valid with mocked request payload
Test successful response with expected shape
Test unsuccessful scenarios, make sure status code and message are expected (however avoid comparing hard-coded message)

func newReq() *http.Request {
    reqBody := `{
        "order": "",
        "basket": ""
    }`
    req, _ := http.NewRequest("POST", "/api/checkout" , strings.NewReader(reqBody))
}
func serv(h *CheckoutHandler, req *http.Request) *httptest.ResponseRecorder {
    rtr := mux.NewRouter()
    rtr.Path("/api/checkout").Methods("POST").HandlerFunc(h.HandleCheckout)

    w := httptest.NewRecorder()
    rtr.ServeHTTP(w, req)
    return w
}

func Test_HandleCheckout() {
  s.Run("should return 200 with successful response", func() {
    h := NewHandler(new(CheckoutSvcMock))
    checkoutMock.On("DoCheckOut", entities.Order{}).Return(res, nil)
    // invoke handler func via mock request
    w := serv(h, newReq())
    // unmarshal based on API response struct
    resp := parseResp(w)

    s.Equal(http.StatusOK, w.Code)
    s.Equal(expectedId, resp.OrderId)
  })

  s.Run("should return 400 if request is invalid", func() {
    h := NewHandler(new(CheckoutSvcMock))
    w := serv(h, newBadReq())

    s.Equal(http.BadRequest, w.Code)
  })
}

Again we only have one concrete handler to test, it does not care complex CheckoutService logic, but only successful or fail use cases.

Closing Thought

You can see how this solitary tests aligned with the beginning quote.

First we have clear boundaries across data access/infra layer, domain service and API handler layer, each layer depends on the contract, this gives us a good design as well. Solitary make bad code design stink -) So if you feel pain to write tests, it's time to review your code.

It's obviously very fast without the need to set up any external dependencies. In my experience my whole test suites only takes a few seconds to run, enable quick iterations over time. I can ship large features with complex logic or doing large refactoring confidently, and rarely need to do time consuming debugging and fix after deploying, it just works straight away.

This is the real benefit we need to gain from TDD.