Everyone loves a good software engineer. Their skill and competence makes managers and stakeholders trust them, and young engineers look up to them.
To build this reputation, here are five critical subjects you must master.
1. Distributed Systems
Distributed code is very hard to understand compared to conventional “void main” code. The code might be called by an external party, so the flow does not start in a clearly defined function/class. You have to expose methods and objects that will be used by external services, which brings the concept of client packaging and export.
It's best if you try to write an HTTP server for starters. It exposes us to APIs and how services interact. Once you are done with this, think about using multiple computers for handling requests. Consider the following.
a) What happens if one computer crashes?
b) How do I load balance my requests?
c) What happens to cached or in-memory data? How do I handle stale data in caches?
2. Dependency Injection
While creating objects, you want to:
a) Provide convenient methods to create them.
Eg. Customer.setCity('Hyderabad')
=> State set to Telangana
=> Country set to India
b) Limit ways of instantiation.
Eg. Customer.age = 173 🧓🏽
=> IllegalStateException 💥
c) React to the event.
Eg. Customer c = new Customer()
=> members.add(c)
=> welcomeEmails.add(c)
Dependency Injection is a way to handle object creation through a framework. It helps meet the above objectives while avoiding code duplication.
What should you do?
Try implementing a factory pattern, dependency injection and package management in your project.
a) Guice (DI tool)
b) Maven (Package management)
c) Jenkins (CI/CD tool)
The more problems you face on the way, the better you'll get at StackOverflow 😜
3. Concurrency
"Something that helped me progress early in my career, was that I could imagine multiple copies of my code running in parallel." - Thuan Pham, former CTO at Uber
The programs you write in college are single threaded. A single thread is like a hole in the kitchen sink. If one request gets stuck, everyone else has to wait on it to finish. ⏱
Instead, the operating system breaks time into chunks, and processes requests concurrently. It allows us to poll the status of a request more frequently, which in turn reduces latency (wait times).
Sharing OS time between threads is called #concurrency.
But it can cause issues. For example:
a) You start with a bank balance of 5.
b) You send a request to update your bank balance to 10.
c) The database takes time to complete the write.
d) Meanwhile, you send a new request asking for your current balance 😊
e) The read operation is quick, and completes before operation C. The response shows balance = 5 😱
f) You buy a mini-gun to confront the bank.
This is called "out of order processing". What should you do?
Try to make:
a) Objects immutable
b) Database and network calls asynchronous
c) Resource schedulers use #thread pools
4. Data Structures and Complexity Analysis
The number of applicants for a software developer role are numerous. Before investing in long interviews on design, companies like to filter applicants with short tests. These tests are designed to evaluate algorithmic and problem solving abilities.
Even after the interview, data structures are critical tools in an engineer's arsenal. Here are some real life war stories:
Exhibit A:
Our app just went viral. We must vertically scale.
My teammate updates some code to use a HashMap instead of a list. He knows HashMaps run in O(1) time. But his hash function is bad, which means O(N) time.
Result 📉: We had a 1 hour argument, which escalated to my manager.
Cost 💵: Unknown. The app never made it big.
Exhibit B:
We have a real time analytics engine. Usually, you use a streaming architecture for this.
But the initial developers couldn't solve the problem algorithmically. They took a shortcut to process the data in batches instead.
Result 📉: The 'real time' engine would take 2 hours to update.
Cost 💵: After one year of dev effort and bug fixes, which spanned across infra teams, we move to the right streaming architecture using Merkle trees. The processing time is now 5 minutes.
Exhibit C:
I am working on an API. The front end engineer and I brainstorm on what we need to do. I classify one of his approaches as infeasible: O(N^2).
He looks at my calculations: 1+2+3+...+N.
And changes it to: O(1)+O(2)+O(3)+...+O(N).
All terms are constant except N.
He claims that's O(N). 😳
Result 📉: My team had to stop me strangling him.
Cost 💵: Nothing. It was a frustrating moment though.
5. Understand databases
Databases are the most critical part of an engineering solution. A company's data is so critical to it that system admins will grant you data access till you have gained some experience or have a specific requirement.
Knowing how to read and persist data is not enough. When thinking of a data design, think of the following.
a) What are the indexes in my database?
b) How do I keep application logic away from the database?
This is the most common beginner mistake. You want to just pipe data in and out of the database.
Write stored procedures only if they save you a ton of bandwidth. Debugging is hell there.
c) How do I avoid manipulating data?
Stuff like delete and update should only be done after discussing with your manager. How do you delete a user? Set the status as deleted. How do you update something? Usually by doing another insert.
Reading books for the above is useful, but most of the learning happens during implementation. Real world systems are built on practical considerations and trade-offs.
You can check out more system design content on our YouTube Channel or our video course at InterviewReady.