In modern system architectures, understanding data flow is crucial. Two fundamental concepts are north-south and east-west communications. Although both deal with APIs and services, they solve fundamentally different problems and require distinct tools. Below, we’ll explore when to use API gateways or a service mesh for north-south and east-west traffic. We’ll consider when each approach is appropriate and when it is a disadvantage for your architectures.

North-South Communication

North-south communication refers to all traffic that enters or leaves your infrastructure. It’s the conversation between an external client (such as a mobile application, a web browser, or a partner system) and your backends.

For these external requests to reach the correct services and return a response, they need to be managed by an entry point. This flow is classically handled by components like load balancers and reverse proxies.

However, for robust management, the most suitable solution is the combination of an API manager with an API gateway. These components are ideally responsible for managing and governing these exposures, handling security, rate limiting, authentication, and routing at the edge of your architecture.

East-West Communication

East-west communication, on the other hand, is the traffic that happens within your infrastructure. It’s the interaction between different distributed services that need to collaborate, such as in:

• Microservices-based solutions
• Distributed applications with high integration needs
• Autonomous interfaces within the same network

In this scenario, the communication medium is even more critical. The internal network, even though more reliable than the internet, is never 100% stable. Failures can and will occur, compromising data exchange between services.

Strategies for East-West Communication

To ensure this internal communication is resilient, especially in microservices, several strategies can be adopted:

• Service mesh: This became the standard for managing communication during the rise of the microservices era. Historically, this was achieved using sidecar proxies attached to each service. Recent innovations, such as Ambient Mesh, provide a ‘sidecar-less’ alternative to reduce overhead. Both architectures focus on securing and managing east-west (intra-domain) traffic through features like mTLS and advanced traffic routing.
• API gateway: A robust solution proven for centralized security and governance. It provides observability and, most importantly, a strong solution that focuses on boundary control across domains, teams, and sub-organizations. This is enabled by the maturation of the Kubernetes Gateway API, providing a role-oriented specification that allows platform teams and developers to manage services with a consistent standard.
• Asynchronous communication (messaging brokers): Here, data is shared asynchronously using tools like RabbitMQ or Kafka. Services publish events or messages without waiting for an immediate response. This approach offers native resilience — if a consuming service goes down, the message simply stays in the queue, eliminating synchronous dependencies.
• Smart client libraries: Before service mesh, this was the main strategy. The application itself includes libraries that handle network complexity. The Spring Cloud ecosystem is a classic example, providing features (based on libraries like Resilience4j) that implement essential patterns like circuit breakers and service discovery directly in the application code.

The Evolution of the Kubernetes Gateway API Specification

As Kubernetes became the foundation for managing microservices and APIs on the cloud, organizations started facing the challenge of efficiently managing and securing traffic between services. This is where the Gateway API comes into play.

The Kubernetes Gateway API is an official, role-oriented specification designed to modernize and standardize service networking by evolving beyond the limitations of the legacy Ingress resource. It introduces a decoupled architecture, using resources like GatewayClass, Gateway, and protocol-specific routes like HTTPRoute or GRPCRoute that allows platform teams and developers to manage traffic independently.

By unifying both north-south (external) and east-west (internal) communication, it provides consistency for implementing enterprise-grade features such as advanced traffic splitting, zero-trust security, and built-in resilience policies like retries and circuit breaking.

Improvements from the Kubernetes Gateway API Specification

The Kubernetes Gateway API, particularly through the GAMMA initiative, transforms east-west communication from a “wild west” of internal calls into a highly governed and secure environment. By standardizing how we define internal traffic, it moves security and governance into the hands of the Kubernetes native API.

Governance Benefits: Control and Accountability

Governance in a microservices environment is about ensuring that the right teams have the right level of control without creating bottlenecks.

Clear Roles for Better Speed

The Gateway API uses a role-oriented model to divide work among three groups. Infrastructure providers set up the networking types, cluster operators manage global security (like requiring encryption), and developers control the specific routing for their own apps. This separation gives developers the freedom to move fast without the risk of accidentally breaking the entire cluster’s security.

Plug-and-Play Security Policies

Instead of writing custom code for every service, you can use Standardized Policy Attachment. This allows teams to easily “plug in” features like rate limiting or traffic throttling to a specific service or namespace. It ensures that every team (like finance or HR) follows the same corporate rules, making it much easier for security teams to audit and manage the entire organization.

Smart Conflict Solving

The Gateway API has built-in conflict resolution rules that act like a traffic cop. If two different teams try to claim the same route, the system uses clear, automatic logic to decide what happens. This prevents the silent failures common in older systems, ensuring the network stays stable even when hundreds of different teams are working in the same environment.

Security Benefits: Zero Trust and Identity

The Gateway API (via GAMMA) shifts security from network-based (IP addresses) to identity-based (service accounts), which is a core pillar of zero-trust architecture.

Explicit Permission

In standard Kubernetes, any service can usually talk to any other service. ReferenceGrant changes this by requiring a service owner to opt in. For example, for a service in the frontend namespace to reach a service in the payment namespace, the owner of the payment service must explicitly grant permission. This prevents unauthorized access between different parts of your organization.

Identity-Based Encryption (mTLS)

Through the GAMMA initiative, the Gateway API provides a standardized way to handle mutual TLS (mTLS). This ensures that services only communicate if they can prove their identity to one another. It automatically encrypts traffic and prevents man-in-the-middle attacks, even within your own trusted network.

Secure Last Mile

This feature allows operators to define exactly how the network should connect to a backend pod using TLS. It ensures the last mile of communication is always encrypted, so data is never sent in plain text, even when moving between the gateway and your actual application.

Detailed Control

Because the API understands specific details like URL paths and headers (layer 7), you can write very precise rules. Instead of just allowing a connection, you can say something like: “The Reporting service is allowed to access /metrics on the Database service, but it is blocked from everything else.”

The Sidecar-less Movement

Another movement from the industry is the shift from traditional sidecar-based meshes to Ambient Mode (often called “sidecar-less” mesh). Here is a simplified breakdown of why this shift happened:

• Cost efficiency: Replaces individual sidecar proxies in every pod, significantly reducing CPU and memory usage.
• Zero disruption: Since the mesh is no longer inside the application pod, you can patch or update security proxies without restarting your services or interrupting developers.
• Flexible adoption: Allows you to “start small” with basic Layer 4 security (mTLS and encryption) and only add complex Layer 7 routing (waypoints) where specifically needed, reducing initial complexity.
• Broad compatibility: By moving the proxy logic to the node level, the mesh stays out of the application’s internal networking, making it compatible with more protocols and software types.

Using the Right Tool at the Right Time

The correct architecture depends on where you draw your lines of trust and who is responsible for managing the traffic. For organizations where compliance and cross-departmental auditing are paramount, API gateways or Kubernetes Gateway API are the superior choice.

For example, in a banking environment, the payments team and the mortgages team operate as separate entities. An internal API gateway acts as a hard boundary. It treats internal services as products, enforcing standardized security, protocol translation (like SOAP to REST), and centralized logging for auditors.

The key advantage is that it allows for a federated model where a central platform team provides the infrastructure guardrails, but individual business units manage their own API contracts and lifecycle.

Otherwise, for a lean team with deep Kubernetes expertise that prioritizes shipping fast, a service mesh (specifically in Ambient Mode) may fit.

For example, consider an AI startup that operates as a single, cohesive domain. They may start without the governance of an API product portal for every internal call and need an observability and security fabric that works out of the box. Ambient Mesh provides mTLS and retries without the sidecar tax, allowing them to scale their distributed systems without manually configuring a gateway for every new microservice.

In this way, service mesh provides invisible infrastructure, allowing developers to focus purely on code while the mesh handles the networking plumbing transparently.