Last month, I had to interview around 30 candidates for a Senior Front End Engineer position at Shiftgig, since we are about to start several big projects, I was expecting to hire someone who I can rely on to help me to create a solid foundation architecture that can scale in terms on the number of people working on it at the same time, complexity and a lifetime around 3 to 5 years; I've realized something, we (as a community) have not clear understanding of what 'a big project' is neither what Front End Architecture is about.
It's also good to mention that I work in a distributed team, this means, we were looking for candidates in 2 different countries, different cultures and why not, different realities. After going on this interviewing process I've also found that people (no matter the country) really like to talk about things including but not limited to: Why React is really cooler than Angular and vise versa; how the diff algorithm is even way more cooler than the Angular's digest cycle, MVC is dead in Front End; Angular 2...4...#n is super performance and my favorite Javascript Fatigue and how we are forced to change technologies each 6 months (mmm yes sure), also if you search Front End Architecture on google, you will get a lot of post about how to organize your folder, content for SEO and one or two about AMD modules.
Chances are that if you have been in the industry for quite a long time, you are familiar with the following scenario; startup starts looking for developers, they have to save every cent (no founding rounds have arrived yet), they like a developer, they cannot afford it, they move on, repeat last step several times, the mvp concept, maybe we do not need a really good one for now, mvp, mvp, mvp, the mvp finally arises, they got the money, we need new features, it's complicated, technical debt shows up, they can deal with it for a few months, ok maybe not that long, the front end app needs to be rebuild, and maybe, the developers leave the job.
[IMAGE front end each 6 months]
I really love the concept of Technical Debt and after more than 10 years in the industry I can define it as:
It's basically when a person of a group a persons make a bad decision and someone else has to pay for it. Basically, it like the Government.
Sometime these are financial persons (cause they understand the technical debt very well) but most of times, we are these persons, the developers, the software engineers, UI architects, you name it. We are those who rush to start a project using Angular, React, Rxjs or whatever top of the line new library that you can see all recruiters asking about. Cory House once defined us as CV Driven Architects, hell yes this dude is right.
Last year I was invited to Minneapolis to a Front End Masters workshop, I had the chance the meet really cool and smart people there and I remembered someone asked the Kyle Simpson what was his opinion about new technologies like Angular 2, React, among others, he answer was something like we are developing tools to solve problems what we don’t have, end of the conversation. I took the chance to make sense of this statement based on my reality and helped me a lot to figure out a solution I came up with.
I got inspiration for several sources, once is Minko Gechev (@mgechev) | Twitter who made a really cool presentation about how to scale SPA applications, his solution was really cool, however, I don’t like the fact to couple to the Angular framework again, main problem I had to solve is how to create something that do not depend of any frameworks. Martin Fowler (@martinfowler) | Twitter was another key point of inspiration, this book Patterns of Enterprise Application Architecture: Martin Fowler: 8601300201672: Amazon.com: Books if a first class citizen in my library and help me a lot to solve the problems I’ve found when creating architectural patterns. Last but never least, once of my professors in my Software Engineering career Diego Fontdevila (@dfontde) | Twitter and his pepper Software Architecture in the Agile Life Cycle which gave me the enterprise touch for this.
This solution works good for us but I cannot tell it will be a good fit for you, we set a list of several problems we wanted to solved and came out with this, this is not a magic wand but a good foundation.
This is just and abstract and recommendations how a combination of patterns placed together with clear definition of what a Object Service and a Object Value are can work together t provide a solid architecture for a SPA.
Architecture is all about Quality Attributes and trade-off, this is not something that you can read on a blog post, or mimic the solution someone else did, we really need to involved the business, each vertical is different, based on this you can select the stack of technology that you can use. We’ve found that we really need to pay attention to these:
- Maintainability: The SPA is the core of the company, thousands users interact with this application on daily basis, the solution needs to consider a lifetime of 3 to 5 years as well as a lot of developers working on the same codebase.
- Testability: The Application is big in complexity and number of features, we need to ensure the solution can grow without braking previous functionality.
- Time to Market: Yes! there are other companies doing the same as we do, we need to deliver features fast keeping the quality in order to compete.
These three are very generic but important, a lot of people mentioned Performance, but in my experience, performance in Front End works in a very different way, unless you work with some complex animation of super complex graphs, I don’t even consider this as something that makes me choose between one technology or another. I think performance is about how well you can be aware of profiling and optimize your asynchronous operations.
Said all this, a very minimalist but solid architecture can looks like this:
[IMAGE OF ARCHITECTURE DIAGRAM]
Let’s go piece by piece.
One of the problem we have previously had with other solution like Angular was the fact they are opinionated frameworks, it works really good but it’s really easy to use it bad and a after a few months you can easily end up coupled and you can easily need to re-build the hole app at some point if you want to escalating.
Notice I won’t talk anything about performance, I don’t really consider you should changes from a library/framework to another just because the community say the performs in better, chances are, the problem it’s not performance but we made a lot of bad decisions when coding unleash of course a few cases when really you can blame the digest cycle for all you misery (I really want to hear about that).
Great! after a few rounds of iteration, Angular # was out o the equation you because I need to have control over how to interact with my architecture and even when there are a lot of smart people behind of it, it doesn’t feel good for me anymore, again, IMHO.
Vue, Inferno, React, Preact… what I really like about them was the fact that they do one thing, UI Components. This will server the purpose of not violating The Demeter Law anymore. I ended up with React, not because I think it’s cooler but because the fact that on an enterprise world I really need to trust on a solution it’s really well supported by the community and the fact that Facebook team has almost all their app is running of it gave me a big level of trust that will have support for a few more years.
Said this I chose React because:
- Really cool developer experience / ecosystem.
- It fits the purpose of not Violating the Demeter Law.
- I can just “plug and play” it without braking my all hole application core.
All the component will use it owns state and will pass processed information to the store, this means, I totally support he use of setState in fact, we use it a lot, and each component will be responsible to keep it owns state until an action is trigger to pass the information to the model. Also, it will use mapStateToProps to keep in sync with the store.
A regular container can looks like,:
https://gist.github.com/2639fc13f9618b133dbc51f15f7e28b4
We also separate all the Containers for Presentational components and pass the props down unless it really need to keep a own state, we have a lot of case for this. Also, in case we need to load information from different sources, we can play around with the React lifecycle hooks, specially with componentWillReceiveProps , I will talk later about how we manage different sources of async data.
Notice how I used the create method to trigger the action, React component should never use mapDispatchToProps because this creates a dependency of Redux, that is task will be handle by the models. I support the use of mapStateToProps because it has really good checking methods that can easily tell if we should or not update the component, I rather to take advantage of this than having to create it myself, the goal is not set communist architecture, but setting rules that will make us good.
- React components will hold its own state for UI iterations like Modals, UI Events, Forms among others.
- Components will read information directly of the store through Containers via
mapStateToProps. - Components will never directly update the state of the store, they will delegate what task to the Models which can be passed through the Container via props.
The Models are the only piece of the architecture that understand the UI, in fact, not always we receive all the information ready to be processed, we need to shape it in that is easy to manage into the React Components, and also re-shape it before we put it back to the state, this Model in combination with an Data Service are the responsible for this.
All Models extend from a Model class which expose an asyncOperation method, which it’s only function is to orchestrate all the computed operations required when an action it’s trigger, I took this idea from Minko’s proposal.
https://gist.github.com/72c8dbe002709b7d48795d5c13545b4c
In combination with a strong naming convention for out actions methods works really great decoupling the services from the implementation, allowing use to isolate each execution, let’s see an example:
https://gist.github.com/7d73b3585efaf8e30ec7c84041e8cc81
This can be a little confuse so let’s explain what it’s going on here; on my requirements I need to be able to:
- Save the Order to pouchDB.
- Trigger and action to Firebase to push a notification.
- Log the event on mix panel.
- Make a optimistic update to refresh the UI.
I’ve inject all the dependencies to the Orders model, this way we don’t brake any SOLID principle and also it’s was to test because we can mock all the dependencies for convenience. I’m exposing a singleton because I don’t need any other instance of the model in this case. I’m using all the Object Services needed to perform the actioned to satisfy the requirements, I provide the object value as we as the store in case the need it.
The Data layer only expose static methods which take case of happing the data to be handle by the UI and the way back to the Store.
Main idea is that the create method normalizes the data and trigger an create event on each of service, which means, executer the create method of each. Again, this is only possible using a solid naming convention. I case the service doesn’t have a method called like the action, it will just ignore it. The implementation of this methods will be explained on the Services section.
- Models inherited a method called
asyncOperationwhich is responsible to orchestrate the actions to executed when there is a UI Input and/or call to action. - Models expose the redux store to the services to used when needed and also all the information required for the service to execute its operations.
- We use Data layer when needed which is the one to shape the data in a way that is understandable to the UI and the store.
- The Model is the only one who understand the UI Component.
- This is a key layer for the business logic.
This Layer task is to execute all the operations necessary by the business in order to achieve a goal. They are just Service Objects which encapsulate all the logic to set between the app state, the back end and the UI.
Let’s see some samples:
https://gist.github.com/10b26a5b5bc148527bbf026cfd548d37
We have 3 different services in this case, they work independently, they just response to the asyncOpertion call of action executing their respective comments. That’s basically it, a single way to abstract all the service logic into a Object Service.
Notice that we are returning a single instance, this is the only entry point we serve to the app, a singleton for each unless we need several instance of this, this is not the case.
Important to note is how different the implementation of each service can be, we can easily isolate each implementation and don’t mix any complexity respecting the Open/Closed principle.
Finally, the PouchDB’ create method, only triggers and action, this means, it requires several operations to be executed, we will talk more about this on the Async Operation layer.
- We isolate each service complexity into its own implementation.
- We can expose a single point of entry
createand use several private class to orchestrate an operation. - The services and the models are not coupled, we respect the Open/Closed principle.
- We only talk with the closest friend, we don’t ask, only tell. ( Law of Demeter - Wikipedia
There are several ways to manage asynchronous operations, Promises, Callbacks, Observables among others. When designing this I needed a way to manage several operations at the same time, as well as, execute one after other and in this process do not force the user to wait a lot until the operations are finished to release the UI.
Depends of you Use Cases and the Business, you can find a lot value using Observables, or maybe if it’s not complicated a single Promise/Callbacks operations are ok. In my case, I needed something more robust, my business logic is complex and we depends on a lot of services to be notified when some actions happens.
Finally, I needed a solution that fit not only for the purpose but for the use, something that is easy to read and do not take a lot of time to learn, said this, Rxjs was out of the equation, I needed the hole team to learn this new technology and time market is this case is more important.
Again, It’s all about trade-off and quality attributes. After reviewing several patterns, I ended up using Redux-Saga. I like the way how it implemented the functions generators and make some complexes async calls to looks ver synchronous.
Let’s see an use case for the sample:
- The order will be send to a pouchDb and return a uniqued id.
- This ID will be send to the Back end API.
- Once an order is place, we need to notify all the market place that an order is up (3 different calls).
- Use the their 3 markets posts id to send an email to the Account Manager notifying about this event.
This is a common use case in a large and complex application, our main goal as Front-end engineers is to deliver this operation successfully as well as provide a smooth user experience, I mean, we cannot just make the user pay the price for all out operations, right? this is when optimistic updates and redux-saga works really well together.
Our saga for this case can looks like:
https://gist.github.com/fa38b6b1f1505f1fd43db3b82e954ee2
As you can see, this is really easy to read, to maintain, to implement and to test. I won’t spend time explaining how Redux-Saga works, however, if you are interested I wrote a post about it, show me some love and take a look a it. (Async operations using redux-saga – freeCodeCamp.
A service and be really easy and really complex an the same time, but, does the user need to pay the price for this? Nope.
Notice that marketThree receive two arguments instead of one even if they appear to be called the same method, we’ll see more about this in the Gateway section.
- I use redux saga to manage my async operations cause it fits all my use case very smoothly.
- We never let the user pay the price for our business rules, we need to provide a really cool user experience, again, this is really easy to achieve using sagas/generators.
- We need to put some time planning how the async calls can play well and do not block user operations.
Gateways is the layer that understand the backend, this can be as complex as the business logic want to be, since, we can work with several backend teams, I needed a way to decouple this implementation and expose a single point of entry and also give me the flexibility to modify the implementation as needed. This is where the ( Command pattern - Wikipedia fit like a great candidate.
This objects extends an exec method, this is only point of the entry to the gateway, even we you can call each method as separate, I highly recommend to enforce the convention, this way, you can encapsulate all the implementation hiding it as an event, let’s see a sample:
https://gist.github.com/43b9d5db67dd7ea63333b0657e2b9983
Just like a that, this allow us to implement the gateways like marketplace.exec(‘marketOne’) where marketOne is the name f the object as pass it as must arguments as needed.
https://gist.github.com/a40d45bf956ab7490931ddce8c0ef1e1
Implementation is totally decoupled and you can pass all the arguments you need to exec , the command will take only those it cares about and ignore the rest. This allow of to do really cool tricks like this:
https://gist.github.com/66e6ce1ea297273addfbc93bc3e8ebc7
In this particularly sample, I’m saving all the operations send to the gateway in case I need to re-send them if the XHR call fails, send logs to logging service, etc, all this just decorating the point of entry, pretty neat eh?
- Gateways is the layer that knows the Backend.
- This is the only place where XHR operations are done.
- We take advantage of the Command Patter to expose a single point of entry.
I won’t spend to much time on this section, nowadays, Redux is really popular around and you can easily find resources about how to use it, however, I keep a few rules with the team that allows to work really well, let’s see a few of team:
- The store should be a pristine as possible, not UI interaction will be manage in the store.
- Data needs to be normalized.
- Most of reducers will be associated with a Model, this allow us to measure the level of coverage per feature (we have a testing plan in place).
- There is a layer in between the Store and the UI, the Data Service due to a normalization. React component won’t never NEVER prepare data to be handle. Remember… ask, don’t tell.
- Services and Async Layers are the only place to trigger actions.
- Small piece of deleatable code can increase your team velocity.
- Making this decision at first will let you to sleep better.
At first it can be like too much boilerplate up front and I’m 100% agree with this, however, the real value is how easy is to plan a migration to others UI Technologies, yes… React won’t last forever, or at least the way it is right now, at some point we will need to support other devices, VR, etc, and technology evolves really quickly, but we can do it as smoothly as possible but also there a few other benefit of this architecture:
- New team members can become productive really quickly working on the the UI layer.
- You can quickly isolate your issues, this way you know the boundaries by layer, we spend a really short amount of time solving issues.
- New features are really easy to implement, we can isolate any new feature and control the iteration between other parts of the application.
- As you can see, most of the architecture is pure Vanilla JS, so the re-use is really easy.
- Quality attributes are very well defined by technologies ex: Lazy loading with React Router, Testability with Layers model, etc.
Front end application development has evolved a lot in the last years, SPA are first class citizens on any company and the first point of entry for current and potentials clients; talking about FE architecture needs to be a easy topic for any Senior Developer, it’s not about how well you manage a technology, we need to include the business and think in terms of how expensive is keep a technical debt all around, that’s our job.
We need to able to response to changes in the economy and the market, this is why software architecture is important, Backend engineers have known this from a lot of time, now it’s time for us to get into the game, if we don’t keep good architecture, if we don’t put an effort on the internal quality, we are in the end deceiving that customers of in fact stealing from out customers because we're slowing down their ability to compete.
Again, this worked for us at Shiftgig