Netflix migrated most of its batch workloads onto Kueue, an open-source cloud-native batch job execution system that has outgrown its homegrown solution over the years. The company mapped the capabilities previously created in-house to Kueue’s functionality and also benefited from new features that would have been costly to incorporate into its homegrown solution. Engineers used API parity with the existing system to derisk the project and allow for a gradual and seamless migration.
The IT industry is changing before our very eyes. With AI entering the software architecture and development landscape, whether we like it or not, we are set to be taken on a roller coaster, the likes of which we haven’t seen since the advent of microservices and cloud-native computing. If the last decade was spent mostly on mastering and optimizing the cloud and micro- or nano-scale service-oriented architecture, the next decade will likely be spent figuring out how to master AI for building a new generation of IT systems.
In this eMag, we try to establish agentic AI architecture as a new type of software architecture that will likely dominate the industry for years to come. The articles, written by industry experts, cover various elements and aspects of agentic AI architecture. We aim to present the latest trends and developments shaping the new type of architecture as it enters the mainstream.
Monzo, the UK digital bank, has redesigned its fraud prevention platform to keep pace with increasingly sophisticated scams and a growing volume of payments. The reactive system is built to detect fraudulent transactions in real time, enable rapid deployment of new controls, provide detailed performance monitoring, and maintain minimal latency on the payment hot path.
Agoda started utilizing ChatGPT to optimize SQL stored procedures (SP) as part of their CI/CD process. After introducing the automated LLM-assisted step, the company observed shortened stored procedure optimization times, which lightened the load on DB developers. Agora works on making ChatGPT more accessible for SP optimization outside of the CI/CD pipeline.
Instacart has redesigned its search infrastructure by replacing Elasticsearch with PostgreSQL, combining keyword and embedding-based retrieval in a single system. By consolidating catalog and search data into Postgres, the company aimed to simplify operations, reduce synchronization overhead, and improve precision and recall in search results.
Allegro shared the details of the process it uses to manage code migrations at scale. The company combined GitHub’s Dependabot and OpenRewrite projects into a custom solution that helps developers perform mundane code migration tasks automatically across numerous source code repositories. The company tackled many edge cases to ensure the process operates smoothly, relieving initial trust issues among developers.
Datadog combined structured metadata from its incident management app with Slack messages to create an LLM-driven functionality assisting engineers in composing incident postmortems. While working on this solution, the company dealt with the challenges of using LLMs outside of the interactive dialog systems and ensuring that high-quality content was produced.
Stripe replaced its observability platform, which used a third-party vendor solution, with a new architecture utilizing managed services on AWS. The company made the move due to scalability limits, reliability issues, and increasing costs while transitioning to microservices. The migration involved dual-writing metrics, translating assets, validation, and user training.
Netflix extended its prioritized load-shedding implementation to the individual service level to further improve system resilience. The approach uses cloud capacity more efficiently by shedding low-priority requests only when necessary instead of maintaining separate clusters for failure isolation.
PostNL shared insights and guidance from its transition from outsourced IT project delivery to an in-house product delivery capability. By embracing cloud-native technologies, with an emphasis on serverless services, the company achieved significant gains in productivity and market responsiveness while reducing operational costs.
Cell-based architecture has emerged as a response to many challenges associated with distributed systems. It employs the bulkhead pattern to isolate failures to a fraction of the affected infrastructure footprint and prevent widespread impact. Cells can also help organize large architectures into domain-bound deployment and delivery units, which provides essential sociotechnical benefits.
Monzo products are supported by an extensive microservice-based platform of over 2800 services. The company relies on planning and heavy automation to drive code migrations at scale and leverages config service to support gradual roll forwards and quick rollbacks in case of issues. Migrations are managed by a central team rather than service owner teams to avoid delays and inconsistencies.
Netflix shared details on the evolution of Pushy, a WebSocket messaging platform that supports push notifications and inter-device communication across many different devices for the company’s products. Netflix’s engineers implemented numerous improvements across the Pushy ecosystem to ensure the platform's scalability and reliability and support new capabilities.
Incurring Technical Debt (TD) is a way to learn things, and to avoid over-investing in solutions to problems you may not yet fully understand.
The idea that releases have to be perfect gets in the way of running experiments. TD reduces the cost of learning and the time it takes to get feedback.
TD represents a deviation of your implemented solution from your ideal solution, but that ideal may not turn out to be correct. In other words, what you think is TD may not actually be TD.
Using a Minimum Viable Architecture (MVA) approach helps teams decide when and if they need to address specific TD items by considering whether failing to address them will impair the success of the MVP/MVA.
TD is, ultimately, a good thing because it reduces the likelihood that a team will over-invest in architecture.
AWS shared how to deliver frugal architecture using Well-Architectured Framework guidance. The authors outline areas of alignment between the WAF and the Frugal Architect laws Dr. Werner Vogels introduced in his re:Invent 2023 keynote. By implementing the frugal approach, organizations can achieve sustainability and cost optimization goals for their cloud architectures.
Figma migrated its compute platform from AWS ECS to Kubernetes (EKS) in less than 12 months with minimal customer impact. The company decided to adopt Kubernetes to run its containerized workloads primarily to take advantage of the large ecosystem supported by the CNCF. Additionally, the move was dictated by pursuing cost savings, improved developer experience, and increased resiliency.
Canva evaluated different data massaging solutions for its Product Analytics Platform, including the combination of AWS SNS and SQS, MKS, and Amazon KDS, and eventually chose the latter, primarily based on its much lower costs. The company compared many aspects of these solutions, like performance, maintenance effort, and cost.
Cell-based architecture has emerged as a response to many challenges associated with distributed systems. First and foremost, it employs a bulkhead pattern to isolate failures to a fraction of the affected infrastructure footprint and prevent widespread impact. But that’s not all it can offer. Cells can also help organize large architectures into domain-bound deployment and delivery units, which provides essential sociotechnical benefits.
Uber migrated all its payment transaction data from DynamoDB and blob storage into a new long-term solution, a purpose-built data store named LedgerStore. The company was looking for cost savings and had previously reduced the use of DynamoDB to store hot data (12 weeks old). The move resulted in significant savings and simplified the storage architecture.
Yelp reworked its data streaming architecture by employing Apache Beam and Apache Flink. The company replaced a fragmented set of data pipelines for streaming transactional data into its analytical systems, like Amazon Redshift and in-house data lake, using Apache data streaming projects to create a unified and flexible solution.
Reddit consolidated its media metadata storage into a new architecture using AWS Aurora Postgres. Previously, the company sourced media metadata from various systems, including directly from AWS S3. The new solution simplifies media metadata retrieval and handles 100k+ requests per second with latency below 5ms (p90).
Decathlon established the Backend For Frontend (BFF) architectural pattern as a company-wide recommendation and provided guidelines for its adoption among engineering teams. The four-part series introduces the pattern and explores its benefits and potential pitfalls. The company also shares alternatives to using the BFF pattern and reviews architectural considerations.
Expedia made flight search faster by up to 52% (page usable time) by applying a range of optimizations to web and mobile applications. To support these improvements, the company improved the observability of its applications. Expedia Flights web application has been migrated to Micro Frontend Architecture (MFA) to allow flexibility, reusability, and better optimization.
AWS optimized the stack creation process to make it faster and achieved significant improvements. The company split the resource creation process into two phases (creation and stabilization), which allows for creating other resources in the stack earlier. The change is available in all regions and doesn’t require any user action.
Hashnode created a scalable event-driven architecture (EDA) for composing feed data for thousands of users. The company used serverless services on AWS, including Lambda, Step Functions, EventBridge, and Redis Cache. The solution leverages Step Functions' distributed maps feature that enables high-concurrency processing.
CNCF announced the graduation of Falco, a tool designed for Linux systems and a de facto Kubernetes threat-detection engine. The project successfully met all graduation requirements, including undergoing the due diligence process, completing a third-party security audit, and obtaining the software licensing approvals.
Uber replaced the legacy architecture built using the WAMP protocol with a new solution that takes advantage of GraphQL subscriptions. The main drivers for creating a new architecture were challenges around reliability, scalability, observability/debugibility, as well as technical debt impeding the team’s ability to maintain the existing solution.
CNCF published the results of its latest microsurvey report on cloud-native FinOps and cloud financial management (CFM). Kubernetes has driven cloud spending up for 49% of respondents, while 28% stated their costs remain unchanged and 24% saved after migrating to Kubernetes. Respondents listed overprovisioning, lack of awareness and responsibility, and sprawl as the main factors for overspending.
Instacart moved their Apache Flink workloads from AWS EMR to Kubernetes to meet the high demand for data processing use cases using Flink within the organization, as using EMR became problematic for many teams with different requirements. As a result, they made the platform easier to use and reduced their operational and infrastructure costs.
Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs.