Scaling of High-load system

This article describes challenges of software scalability. The case relates to IT product in EdTech that has evolved from startup to a full-scale IT company. At startup phase it utilized a single database which was good enough to support several thousands of users. The goal of the company was to transform its software product into a high load international enterprise system with database supporting hundreds of thousands users and further scaling up to millions of users. This process took several years of development and will be described in a chronological order.

Within several months after project has started our development team has discovered an issue as having too many logic in form of stored procedures. Though it could be quite neutral by itself, it led to even bigger issue that any change introduced to a database schema could potentially break stored procedure.
As a result, the application became not functional. Developers had to spend significant amount of time just to check database schema integrity manually each time a change has been introduced. This was an obstacle to writing a good quality code in a reasonable time, so we had to solve that concern as quick as possible.

At that moment MS already issued several versions of SSDT framework which

introduced an improved support of database projects into the VS. It wasn't production ready yet, but it was good enough to solve our database schema matter. In less than a week we cleaned up our database and made the whole database project consistent and compliable.

This database project led to stabilized development during expansion of development team from 2 to 6 developers in total.

Architecture is the core of every software system. Once designed it is quite complicated to introduce changes. Therefore, when there are preset requirements, architectural approach is the best.

ENBISYS experts advise on optimal architecture for every product

There are 3 main architecture approaches nowadays:

• SOA
• Microservices
• Event-driven architecture

SOA is an approach when system is built as a set of services. Every service covers particular part of functionality and might have several different APIs for different users. For example you might have billing service and it could provide RestFul JSON API for mobile clients (iOS/Android) and SOAP (Web Services) for 3rd parties systems.

Microservices approach takes SOA to the next level. In SOA all services belong to single information system which shares data (common database). In Microservices approach all services are independent systems. Every microservice has its own database and its own backend. Usually in this approach there is additional orchestration layer between microservices and frontend.

The benefits of Java Microservices are:

• High system availability / Robust — if malfunction happens only small part of functionality will be temporarily unavailable
• Hot swap in production — Upgrade any parts of Java system in production with no downtime
• Highly effective scaling — Possible to scale each service considering its usage/load
• Each microservice could be owned by dedicated team with its own release calendar
• Each microservice could have own technology framework (can mix Java, .Net, Ruby ...)

Approaching the process of migration from legacy technologies, like PL/SQL, ENBISYS takes functional blocks of the system one by one and rewrites each in Java. Then this new Java modified piece is integrated into existing system seamlessly.

This allows end-users to continue using old system while the Client is introducing technological Java updates to running system.

Migration to Java makes the application layered. From now business logic runs on dedicated server which request from database data via SQL-queries only.

It is important to be aware of possible degradation of response time due to:

• Penalty time in communication between application and database servers via network
• Inefficient algorithm requests too much unnecessary data
• Inefficient algorithm too talkie (100 small request per single user operation)

Our programmers are aware of this possible pitfalls and in most cases avoid them. In some cases it turns impossible to avoid response time degradation without changing database scheme. Changing database scheme is expensive part but it is worth to use it for future performance and scalability.

Once migration of functional part to Java is done we execute stress-tests. Stress-tests also know as load testing is a special tests where load to system is increased over time.

During this time period all system parameters are monitored and drawing on the graph. For example we could run stress test simulating from 1 to 100 concurrent users doing random term searching scaling up (ramp-up period) in 10 minutes. We execute the same test on new Java code and old PL/SQL code. After this we compare performance on low and high load as well as resource consumption. Usually Java code is highly optimized and response time decreased in 2-3 times. On some parts we reach 10 times faster response. We compare database resource consumption and there we gain with the same magnitude
as in response time.

Therefore, legacy technology migration to Java leads to optimized code and it will be possible to serve 2-3 times more users with the same database hardware and same licenses. Still dedicated application server for Java code is needed. But it is not as expansive as database server.