Startup Stories — Deciding architecture
This is a small fictional story about a startup named EventsAreAwesome.io, and their struggle to find the optimal software architecture for their product.
This is a small fictional story about a startup named EventsAreAwesome.io, and their struggle to find the optimal software architecture for their product.
Michael is the CEO of the company, in charge of product decisions and team psychotherapy.
Johnny is the most experienced developer in the team, but not an Architect (does that role exist in startups anyway?). Johnny is a Ruby aficionado.
Marc and Andrew are the other developers in the team. They are both true startup hackers that can work with different technologies and both front-end and back-end.
Seed funding is in! The team is formed! let’s build stuff!
5th of January
Johnny has scheduled a brainstorming to kickstart the architecture. Startups are flat and decisions are best made together!
“Ok Guys, we are building a competitor to EventBrite. A Ticketing system with a smart algorithm that defines the prices of event tickets with Machine Learning, and tries to optimize the price over time like a real-time bidding system to offer the optimal price at any given time.” — Michael
Michael hasn’t slept for days because of the excitement. He was conceptualizing the product in his dreams and the designer put together some mockups based on his direction. The UI looks awesome and the users are gonna love it!
During the brainstorming, after some random drawings on the whiteboard Johnny suggests to build the system in Rails, the lingua franca of the team.
The team plans to throw away the MVP when they figure out what they are doing. Just the way they learned from Lean Startup.
The CEO Michael approves. Time to market is essential.
Marc from the team suggested to consider Node.js which he deems more scalable than Rails because of Async IO, but he was ignored. What is Async IO anyway? Rails has scaffolding and the team knows how to use it. Speed is essential and quality comes later. Move fast and break things!
Andrew from the team mentioned Elixir and a super-cool prototype that he built to share Cat Gifs online with hot-code reloading and high scalability with OTP. Hmm.. let’s skip it for now, the team is not familiar with it.
Johnny thought for a moment about GoLang, but it seemed too hipster with the GoRoutine thingy, and there is no time to waste experimenting.
Someone mentioned micro-services as the way to go, so the team members can use their favorite language in different parts of the system, but hey let’s start with a lean Rails app monolith and figure things as we go. Michael suggests the team to stop analyzing! Just fucking do it.
Johnny argues the team is going to be more efficient with a familiar language and the team decides to build the app with Rails. Not everyone is super-happy. The CEO will do the psychotherapy later.
20th of June
The system is out in production for a few months, and it started getting traction! There are already thousands of active users on the app, and it seems that the system has found a product-market fit in the market for sport events but the company is almost out of money, and looking into an A-round to get some juice.
The system is working relatively well, but some users are complaining about occasional slow downs of the system.
Andrew found some issues with CPU resource utilization happening in New Relic. Johnny has ideas how to optimize the app, and to fix the problem quickly he decides to throw a few more servers in the web farm behind the ELB in Amazon to handle the load. This is what AWS was built for right? horizontal scaling baby!
The AWS bill is getting bigger, but the A-Round is gonna fix it. Michael, as a conscious CEO, is getting more and more nervous about money.
Johnny brags at a dinner with friends the system is using already 20 web servers in production. It’s great to build a big software system!
13th of October
Marc left the company for a Node.js job. Nobody knows why he left.
The system is at 50 servers in production. The bill in AWS is skyrocketing. Michael asked about ways to cut the costs.
Andrew setup some elaborate monitoring and tracing tools to track everything that’s going on with resource utilization and application performance. He thinks he found the bottleneck: Blocking IO.
The team tries to fix the problem by re-thinking the rails architecture, with caching and by using NGinx in front of the Rails deployment.
The A-Round is getting closed. More millions in the bank should help cover the costs!
12 of December
Michael got an advice from A-Round VC investors to take care of the architecture, because the team is spending too much on hardware.
Michael quickly hired Tom, an experienced cloud architect to help with the system performance problems.
Tom run a thorough analysis of the system and the code-base and draws the component and deployment diagrams for the system, and accompanying documentation with a performance, scalability, reliability, security and maintainability analysis.
He has a hypothesis about the system problems and he runs performance and other measurements for each section to validate his hypothesis.
Tom is baffled the team hasn’t defined non-functional requirements and expectations for the system before they choose the technology. Michael thinks there wasn’t a need to care about non-functional requirements in MVP phase.
Tom concludes Rails was a wrong choice for the system. Johnny does not approve. Rails was clearly a great choice, since the team was familiar with it and it got the team to the A-Round. Michael looks at Johnny with suspicion.
Tom claims that they can cut 50 servers down to 4 by moving to GoLang or Node.js based on the measurements on the parallel prototypes he has built in the cloud.
Tom run more tests to measure the efficiency of different implementations in a parallel prototyping fashion, and after looking at the overall business context, functional and non-functional requirements he suggested to rewrite the system to GoLang, since the quantitative measurements prove that Async IO with GoRoutines is much more efficient than the current web architecture in Rails for this specific context. His hypothesis has a very solid proof .
Tom argues the whole day with Johnny to prove the point choosing Rails because of familiarity and speed was a wrong decision. Johnny handed his copy of “Lean Startup” to Tom for reading.
3rd of March
Most of the system is rewritten, and running in production with GoLang. The efficient async architecture utilizes CPU, Memory and Disk in a perfect way as Tom expected based on his earlier measurements and prototypes.
The web farm was reduced from 50 to 5 servers and the system was never more performant. Users are very happy, and the AWS bill is down 80%.
Michael pops a bottle of finest champagne to celebrate the technical excellence with the team.
A story was written on http://highscalability.com/ about the architectural experience and the team is happy that their architecture can inspire others. Michael is happy he has hired an architect to do the job.
4th of March
Marc, stumbles upon the article, and he recalls past memories from the company, and the fact they ignored his advice to use Node.js. After reading the article he nods approvingly, but still feels a bitter taste in his mouth and thinks: “Could we have removed our personal bias and measured things from the beginning?”
….
This story was inspired by the common approach to architecture in Startups, and also by the fact that most software companies ignore non-functional requirements, analysis, prototyping and data/measurement/analytics as important information for architectural decision making, and instead favor familiarity and short-term speed over long-term consequences.
Real Stories about the cost of wrong choices:
High Scalability
The Iron.io Blog
Originally published on Medium.
Discussion
No comments yet. Start the conversation.