Systems of Qiuzziz and Details !!
At Quizziz our mission is to engage teachers by giving locks in apparatuses to upgrade understudy learning results. A key include of our stage is the capacity for instructors to conduct live, Qiuzziz gamified tests in classrooms. Given the real-time nature of these exercises, it is pivotal to guarantee a consistent and continuous involvement for each student.
Our current system of Qiuzziz
As we’ve developed and picked up more clients, the recurrence of DoS assaults has moreover expanded. In our early days, we actualized fast and cost-efficient arrangements to oversee these assaults. We depended on AWS WAF (Web Application Firewall) and custom application-level rate limiters for each service.
However, Qiuzziz as the recurrence and complexity of these assaults developed, along with our progressively complex foundation. We required a more lasting arrangement. Our starting strategies still permitted a few DoS assaults to enter. Primarily since the WAF’s 5-minute discovery window was as well long to caught. Aassaults some time recently they come to our administrations. Besides, as our activity expanded, the taken a toll of WAF got to be restrictively expensive.
Problems
Failure to piece all DoS attacks
We have been managing with Qiuzziz DoS assaults for a few time presently, and AWS WAF (Web Application Firewall. A overseen DoS security benefit by AWS) hasn’t been completely viable in ceasing these assaults. Numerous assaults still overseen to invade our framework. Causing our administrations to scale up, tall asset. CPU utilization on person holders, and cascading impacts all through our whole framework. This for the most part come about in precariousness for our users.
Application-level rate limiters and speedy fixes demonstrated incapable, as aggressors would adjust and begin focusing on diverse endpoints and administrations. We realized we required a vigorous, centralized arrangement to handle DoS assaults. This arrangement required to rate restrain based on different qualities. Such as login state, IP address, HTTP strategy, way, headers, and more, to viably ensure all our services.
Duplication of rationale in numerous services
While this approach rearranges the engineering of our stage, it has a noteworthy drawback. We have negligible control over how demands are directed. We need a centralized put to actualize common commerce rationale, such as authentication.
As a result, person microservices started to actualize their possess forms of the same trade rationale. This driven to a situation where changes to common layers required comparable upgrades over all administrations, possibly presenting bugs and Qiuzziz expanding specialized obligation. These bugs can have extreme results, such as security vulnerabilities.
The arrangement is to build up a common passage point—a door service—that serves as middleware for all our demands. This portal benefit permits us to centralize and streamline our trade rationale, upgrading security and diminishing the hazard of bugs and specialized debt.
Heavy dependence on AWS confining customizability, swelling costs
We utilize AWS Application Stack Balancers for different stage highlights that improve the engineer encounter, empowering them to test and test with their code without discharging it to genuine clients. This methodology has worked well so distant, but as we proceed to develop quickly, we’ve experienced issues with AWS benefit quantity limits and rising costs.
To maintain our development, give speed to engineers, and guarantee Qiuzziz remains a solid and profoundly accessible stage whereas keeping up fetched productivity, we realized the require to move parts of our stage absent from AWS administrations towards cloud-native solutions.
We chosen to move forward by building a custom portal benefit with our possess rate limiter usage, sending it on Kubernetes, and utilizing a cloud-native tech stack. This approach permits us to superior control our framework, move forward versatility, and keep up fetched efficiency.
Gateways, Kong: What are these?
An application door is a organizing component that acts as an middle person between clients and web applications, giving secure and versatile get to by advertising functionalities such as stack adjusting, SSL/TLS end, and web application firewall (WAF) capabilities. It serves as a central section point, directing approaching demands to the suitable backend servers, and can too offer highlights like session partiality and URL-based steering. In general, an application door upgrades execution, unwavering quality, and security for web applications by overseeing activity and implementing approaches at the application layer.
Kong Door is an open-source, cloud-native API door that serves as a central control point for overseeing and securing API activity. It empowers proficient API administration by giving capabilities such as verification, rate constraining, request/response changes, and activity control. Acting as a middleware layer between clients and backend administrations, Kong Portal guarantees dependable and secure communication. With its broad plugin biological system, it offers adaptability and extensibility to customize and upgrade API usefulness. By disentangling API administration assignments, Kong Portal permits engineers to center on building vigorous and adaptable applications whereas keeping up tall execution and security for their APIs. There is much more to say approximately Kong, and if you’re interested, this is a incredible start.
Why we built our possess rate limiter
Kong gives a default rate limiter, but it didn’t fill our utilize case totally. There were numerous highlights we needed that required custom execution, such as:
Redis Cluster Support
Quizizz encounters top activity of roughly 750k RPM, whereas DoS assaults can surge into the tens of millions of RPM. To handle this volume, depending on a single Redis hub wasn’t doable for ensuring tall accessibility and execution. Hence, we actualized a expansive Redis cluster to guarantee soundness and anticipate crashes amid DoS attacks.
Configurable Time Windows
The default rate limiter offers as it were a restricted set of time windows for designing rate limits. For our utilize case, we found that a one-minute window is as well long; inside that time period, an assailant seem cause critical hurt to our framework and disturb the client involvement. On the other hand, a one-second window, whereas vital, is not adequate as we require to piece assaults for a longer length. Subsequently, we required configurability on n-second windows to viably moderate these attacks.
IP Blacklisting/whitelisting
Blacklisting and whitelisting IP addresses are pivotal components of our rate limiter, empowering numerous highlights. For occurrence, we require to whitelist our possess custom IPs and once in a while whitelist IPs for particular clients with uncommon cases that require briefly loose rate limits. Also, we are improving our framework to show a captcha instep of an mistake when a specific IP is rate constrained. If the IP completes the captcha, it will be whitelisted for a brief period. Another important include is boycotting known malevolent IPs to anticipate attacks.