New post - 'Scaling front end codebases'

The core of my skillset is front end and I’ve added over 5 years’ solid experience in Node.js. I have a good handle on the array of developments in the Javascript space, plus the experience to know which of them to benefit from. I’m passionate about web performance and know how to measure and achieve it.

I understand every angle of a successful agile team including DevOps, cross-disciplinary collaboration, user-centred testing and design, lean UX, early & continuous delivery, TDD, test automation and how to funnel business requirements through user stories.

I have skills in:

React

scalable CSS including CSS-in-JS

Node.js

Javascript

Typescript

page load performance

accessibility

micro-services

micro-frontends

progressive enhancement

TDD & unit tests

BDD & end-to-end tests

responsive / mobile web

CI / CD

agile

rapid prototyping

Scaling front end codebases with micro-frontends

Front end code can proliferate quicker than in backend APIs / services. It can be harder to test than pure units of backend logic, eg it's tricky to automate tests for visual regressions; so when faced with making an intentional breaking change to a component, people will often cautiously decide to extend or copy instead.

When pressure to pump out new features is unrelenting (ie any startup, ever), the codebase for a single web app will balloon rapidly. If our team stays small over time, we might stick with this setup. The code is cluttered and deployments might get slower over time as the number of tests increases; but with a single group of developers, it's workable.

We could just allow the monolith to continue to grow, if we decide it's not that painful. We can just keep hanging onto the rising balloon and hope it doesn't sail too high to jump off later..

Anyway, some things are just easier with one codebase.

  • a single Javascript build lets us chunk your code easily for efficient delivery to the browser
  • a single router makes it easy to add slick client-side navigation
  • one CI pipeline to maintain and monitor
  • easy to co-ordinate releases with a single deploy

And other things just seem easier, at least to the squad that grows and owns the monolith

  • we know exactly where everything is and how / where to make new changes
  • there's an increasing list of gotcha's and pitfalls, but we have a handle on them, even if they aren't all documented yet
  • It's easy to gatekeep a single codebase - standards, contributions and even releases can be regulated by the team

The difficult teenage years

We've piled on users! Ambition for the product scales in line; now additional dev teams are being hired.

A different set of growing pains emerge.

  • newly-hired developers don't know the labyrinth or the pitfalls; they want to prove themselves useful quickly. Is it not possible to make and release a change simply? Many top-drawer software teams (famously, Facebook) pride themselves that new devs can get their first commit into production within days or even hours of arriving.
  • the gatekeeper model, or even any division of skillsets, emasculates developers who are ready to take responsibility for code, quality and monitoring.
  • inevitably, any codebase that was scaled quickly (or in fact, any codebase at all) will have made wrong turns and accrued tech debt; if you helped build the monolith your inside knowledge will carry you over the bumps every day; and it's always harder to objectively see how much impetus is slowed by the tech debt we created ourselves

The new dev teams don't have the ownership and autonomy which is the secret sauce to turbo-charge motivation and productivity.

Some autonomy for each squad doesn't mean the wild west.. we can have an agreed core tech stack and approach. But if we don't let responsible developers innovate, we're not going to improve anywhere near as quickly. And we'll have a hard time keeping hold of the best ones.

It's natural and unavoidable for humans and teams to seek control over what they're doing, as per Conway's law which constantly re-proves itself in software teams.

What does good look like?

How do several teams contribute to the same product in a co-ordinated way?

  1. Any developer, whether experienced or new, can quickly and safely make and release a change to a system. Ideally, this shouldn't require any of:
    • reading copious documentation (if it's copious, it's almost certainly out of date)
    • extensive consulting with in-house experts on the pitfalls and how to avoid them
    • manual regression testing - there has to be good coverage at some level, preferably both unit and end-to-end tests
    • waiting a long time (days rather than hours) for gatekeepers to allow the feature through to production, which might involve pull request approvals, extensive manual testing, troubleshooting build failures
  2. Each individual feature development team can deploy their stuff independently to production, at the times of their choosing. They shouldn't be blocked in a single CI pipeline behind other features that are being tested or are having a problem investigated, and the pipeline should be fast, predictable and stable
  3. Developers have the space to innovate (and if the new library, tool or technique works, it can be propagated to other similar systems in a controlled way; if not, it's quick to back it out).
  4. Code releases to production are small, focussed and frequent. If something goes wrong we can quickly work out what caused it - because the released commit, and the affected service are both narrow in scope, issues are faster to debug.
  5. Many of the advantages of the monolith are retained:
    • Common Javascript libs are shared not duplicated, to enable caching and good performance
    • The user's experience of navigating around the website is slick and coherent. The UI looks consistent and uses async page navigation where possible.
    • Conventions and code are easily shared, whether it's linting rules, config, utilities or UI components
    • It should be quick and simple for a developer to run the site locally (or at least the parts they're interested in)

Is it enough to break the monolith up into modules?

The modules can then be NPM installed into a single codebase. We've gained modularity; developers can more easily see where to make a change. Good enough?

Nope - having independent deployment pipelines for each part of the system is absolutely key, for reasons 2 - 4 above.

Micro-frontends

Micro-frontends are one solution to the problem of monolithic front end codebases. They also come with their own set of challenges and overheads. I'll cover these, with an example setup and repo, in the next post.

jonnywyatt2 (at) gmail.com