Tactical platform remediation for Performance, Security and Scaling
Hungry Hungry leverages the flexibility and modern services of Amazon Web Services (AWS) to perform tactical platform remediation - and achieve Performance, Security and Scalability
HungryHungry is a digital menu and ordering platform that unlocks the evolving dining experience for restaurant customers.
HungryHungry tells its customers what their diners want next. The platform provides instant, discreet guest feedback. This curcial feedback allows retaurants to stay ahead of new menu creation, staff training, and insights into what customers need and want. The platform leverages real-time insights on customers behaviour, when they’re ordering, what they're eating and drinking and where they come from, to ensure restaurants continue to provide the ultimate experience for diners.
The platform allows for the creation of stunning digital menus, special diet filters and upsell options to menu items. It can take instant online payments, and pre-ordering to satisfy a variety of business models.
HungryHungry (HH) were facing performance issues with their existing Drupal architecture - a single App server in Production with Drupal and MariaDB running on the same server. HungryHungry used an AWS Marketplace product that they continually scaled up vertically until they’ve reached allowed limit for this product - m4.10xlarge.
This problem was exacerbated by the lack of detailed performance visibility on the database side. They had some visibility of the DB logs (errors, slow queries etc.) and Application logs but could not pinpoint the actual problem or performance bottleneck. All HungryHungry infrastructure and applications were manually configured.
Mantalus went through a deep analysis of the Database and Application setup. Since Production Application was affected quite badly during peak hours, it was paramount to buy some time for a proper design implementation. MariaDB was migrated to RDS but performance was still affected. We have discovered that it was caused by a bug in the unpatched Drupal Application version. Application and Database were split into separate instances at this stage. Drupal patch was applied a few problematic slow queries were updated by HH team.
The Mantalus “Tactical Solution” (remediatory) approach was to use AWS Drupal Reference Architecture as a base adding additional security in the form of Amazon WAF and automation for all stacks.
HungryHungry advised that they would like to utilise AWS Certificate Manager to manage client certificates and cater for auto renewal of those. Due to this requirement Mantalus team came up with 2 options for a “Tactical Solution” as per the diagrams below:
Option 1: CDN in front of the Origin (To cater for clients with Hosted Zones managed by HungryHungry).
Option 2: Global Accelerator with ALB and OpenResty containers in front of the Origin (To cater for Hosted Zones not managed by HungryHungry).
In the end HungryHungry elected Option 1 as it enabled them to migrate their clients that use externally managed Hosted Zones, to HungryHungry’s generic hosted zone. This solution utilises Cloudfront Distribution (With AWS WAF) → Application Load Balancer (With AWS WAF) → AutoScaling Group (With Memcached Cluster, RDS and EFS).
An ECS Cluster was used (Utilising capacity providers + EFS for Fargate containers as persistent storage) to migrate HungryHungry’s cronjobs from the single App server to per-environment containers. HungryHungry performed load tests with up to 100K orders to test this new “Tactical Solution” with positive outcome.
The Tactical Solution provided by Mantalus to HungryHungry allowed them to remediate performance bottlenecks and provided path for cloud uplift according to industry’s best practices. Unlike previous HungryHungry setup - Mantalus provided the development team with infrastructure as a code via Cloudformation and automation, as a first step in uplifting HungryHungry’s cloud capabilities. Cloud uplift should include at the very least - properly configured Landing Zone, CI/CD and new Application architecture.