The ideal WebApp Architecture

Designing the architecture that can stand the test of time

Assume you were to design a new web application today. This application would need to be scaled to millions of users, would have a lot of complex business logic, is intended to be a long-term investment and would be a high revenue generator. How would you design it?

I started thinking about the same a while ago and here is my answer to the question. Let’s get started!

Factors that Matter

Before designing anything, I want to be concrete about the factors that matter and have influenced the design.

1. EtC Principle: The definition of a good design is that it makes the code easier to change in the future. We should not be stuck with a dependency or a part of the code just because change requires a lot of efforts. This is the main bane of a monolith; and it’s THE most important thing when designing an architecture, as far as I see.
2. Maintainability: Making loosely coupled and highly coherent modules with all the relevant information that make the application easy to maintain in the long run.
3. Testing: Testing, a lot of it and on multiple levels is paramount to deliver a reliable product. (One optimization in this architecture is fast testing by separating UI and E2E tests, as discussed below).

The Architecture Design

Now, let’s start creating the architecture with these principles in mind, but first thing first, please see the image below, you can view a larger version on Miro.

The ideal WebApp Architecture

1. JSON Request/Response: Our backend is only concerned with JSON. It doesn’t take care about the “V” in “MVC”. The benefits of having separated views is that our backend is platform agnostic. It doesn’t care if the JSON request comes from a website, mobile application, desktop application or CLI. It also helps us in keeping separation of concerns, promoting good documentation & testing of our API. We can avoid any additional friction or maintenance cost due to separation of views by using specification standards like OpenAPI.
2. Front-end Components Module: The component module will consume our API and styles library to create the view for the user. Creating a separated component library will allow us to focus on performance & accessibility of the website without being concerned about the rest of the shebang.
3. Styles Library: We separate out the styles our application to yet another repository where designers can create the elements down the very specification of their imaginations, and examples to explain the look and feel. The front-end developers would only need to use the classes / components created to get the styles in the front-end components module. The advantage of keeping the styles separated is that we can do a complete lift and shift of the UI with minimum efforts and again, separation of concerns would promote having good documentation for usage.
4. Common Components (CC): This is a list of the additional components that all the repositories must contain, including documentation, versioning, linting etc. It’s mentioned separately to avoid repeating it everywhere.

Testing

If our codebase doesn’t have tests, the only thing we can be certain about is bugs in production.

Pyramid of Testing

1. Unit Testing: This stage of testing is to verify that the functions meet their requirements in isolation. We check all the states of a function to get confidence in our code. A unit test is the first user of our code.
2. Integration Testing: In this stage, we test the practical use cases of our application. It helps us verify that the isolated functions work well together to solve the problem they were designed to solve.
3. UI Testing: E2E is traditional, but it’s resource hungry and a slow process, we can separate out the UI testing by having a separate web server that just provides the data required for testing. In separating E2E tests from UI tests, we can get confidence on all the states of our front-end application without spending exorbitant about of resources on massive E2E testsuites. The Simple Web Server is just an application to simulate our backend, it would return JSON responses based on request headers and requested API endpoint.
4. E2E Testing: We test the main use cases of our application end-to-end with all the services & requirements to ensure that we meet the performance, security & functional SLO of the project.
5. User Testing: While nobody wants their users to be testing their application, it’s naive to assume bugs will never seep to the users with all the testing we have done. Hence, we should have a quick and easy way for the users to be able to report bugs.

Documentation

Your code is not complete without documentation, no matter how well it works, if the users can’t use it, it’s a waste of efforts.

1. User API documentation: As the name suggests, this documentation should empower the user to be able to build their own application over our API.
2. End-user UI documentation: While a good UI is intuitive and doesn’t require the user to learn the quirks of using the application, it’s still a good idea to invest in the UI documentation to explain the features provided to the end-users to enable them to make the most out of the application.
3. Architecture documentation: A document containing the flow diagram to explain how each module connects with each other and what problem the module is solving.

All the documentation must be in text format using languages like Markdown or reStructuredText and parsed to format of requirement, fancy formats like pdfand doc are not valid primary documentation formats.

Deployment

People can’t be “fixed” but processes can be fixed.

1. Automation: The only correct way to deploy & create infrastructure is to automate the entire process. Humans will always make mistakes, but machines can follow the deployment steps to the beat with 100% repeatability. Docker, Terraform, Ansible and Chef are an integral part of the modern Architecture.
2. Logging & Monitoring: Your testing will never get all the bugs and cases. Things will fail, but a failure should only happen once. To ensure that, we need heavy logging and monitoring in the application running in production.

Much more on deployment can be found in the Google Site Reliability Engineering Book.

Conclusion

Change is a part of life, what’s a best practice today would be listed in bad practices tomorrow, but since this architecture is borne by micro-services, embracing the change, one module at a time, is practical for even the largest projects; hence it stands the goal of meeting EtC principle.

Since the modules have to be highly coherent and loosely coupled for the developers to be able to make changes quickly, the architecture provides orthogonality out-of-the-box.

Finally, the code is tested at multiple levels, providing a peace of mind when releasing.

What’s Next?

That’s not all, of course this is not all! This is just the code at rest, but the code is never at rest! It’s always moving, moving from localhost to production, from severe security releases to A/B releases, but release sanity & release cycle management are a whole different rabbit hole. We’ll come to that soon, in another blog.

For now, I am interested to hear what you think, would you create an architecture similar to mine or would your ideal architecture look completely different? I want to hear about your additions, removals and the reasons behind your decisions. I’ll share this blog with folks who work on completely different type of architectures and publish any updates with various opinions soon, until then, have fun pondering about problems!

Hope this got you thinking about your ideal architecture, good luck with your next design.