The failure is usually between platforms
At 18:07, a payment API began timing out. The Kubernetes pods in AWS Cape Town were healthy, the database dashboard showed normal CPU, and no deployment had taken place. The useful clue appeared only after correlating three signals:…
Unified Monitoring for Multi-Cloud Ecosystems: A Practical Control Plane for African Infrastructure
At 18:07, a payment API began timing out. The Kubernetes pods in AWS Cape Town were healthy, the database dashboard showed normal CPU, and no deployment had taken place. The useful clue appeared only after correlating three signals: rising latency on the Johannesburg-to-eu-west-1 path, a full NAT gateway connection pool, and packet loss at a branch network still running through an unstable last-mile link.
That is the operational gap Unified Monitoring for Multi-Cloud Ecosystems must close. Multi-cloud monitoring is not putting every provider’s dashboard on one screen. It is building a consistent telemetry model that lets engineers follow one customer request across cloud accounts, regions, clusters, networks and on-premise systems.
The failure is usually between platforms
Cloud-native estates often have good local visibility. AWS exposes CloudWatch metrics, Azure provides Azure Monitor, and Google Cloud has Cloud Monitoring. The problem starts at the boundaries: a workload in Cape Town calls a service in London; a managed database depends on a private link; an on-premise identity system authenticates users in the cloud; and a third-party DNS provider controls the first step of the request.
Each platform reports health in its own vocabulary. One calls a request a transaction, another calls it an operation, and a Kubernetes application may expose an entirely different set of labels. The result is a dashboard that says each component is green while the user experience is red.
A unified design begins with labels and identity rather than panels. Every signal should carry stable attributes such as:
service.name,service.namespaceandservice.versioncloud.provider,cloud.regionandcloud.account.iddeployment.environment,clusterandavailability_zoneteam,cost_centreanddata_classification
Keep cardinality under control. A request ID belongs in logs and traces, not as a Prometheus label. Customer, pod and route labels need explicit limits, otherwise a central metrics system becomes an expensive index of every possible combination.
Use the right backend for each signal
Prometheus remains the local collection and alerting workhorse. Run it close to workloads so a temporary WAN outage does not erase visibility. Prometheus instances in EKS, AKS, GKE and on-premise Kubernetes can remote-write selected metrics to Grafana Mimir for long-term, horizontally scalable storage. Mimir’s multi-tenancy is useful when platform teams need central retention without giving every team unrestricted access to every environment.
Loki should hold logs with a deliberately small label set. Store high-value routing labels such as cluster, namespace, application and region; keep stack traces, user IDs and request IDs in the log body. This makes cross-cloud searches practical without turning every field into an index.
Tempo provides the missing connection between an HTTP timeout and the infrastructure underneath it. Instrument services with OpenTelemetry, propagate trace context across queues and HTTP calls, and link Grafana panels from a trace span to the relevant logs and metrics. A trace should answer whether a slow request spent its time in application code, DNS, a database call, a service mesh or a cross-region network hop.
Grafana then becomes the investigation surface rather than another source of truth. A regional overview can show SLO status, traffic and saturation; a service view can pivot to Mimir, Loki and Tempo without forcing an engineer to learn three query languages before starting an incident.
Make the telemetry path resilient to South African realities
Telemetry is itself a distributed system. A collector that cannot reach a central endpoint during load-shedding or a transit failure should buffer and retry rather than block application traffic. Run Grafana Alloy or another OpenTelemetry Collector close to the workload, apply batching and memory limits, and use separate queues for critical metrics and lower-priority debug logs.
A basic collector pattern might look like this:
receivers:
otlp:
protocols:
grpc:
http:
processors:
memory_limiter:
check_interval: 5s
limit_mib: 512
batch:
timeout: 5s
send_batch_size: 1024
resource:
attributes:
- key: deployment.environment
value: production
action: upsert
exporters:
otlphttp:
endpoint: https://telemetry.example.africa/v1/otlp
sending_queue:
enabled: true
num_consumers: 4
queue_size: 10000
service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, resource, batch]
exporters: [otlphttp]Do not page because a collector cannot reach the central backend for five minutes. Page when the queue is approaching exhaustion, when local disk buffering is failing, or when critical SLO metrics are no longer being exported. Those alerts distinguish “the monitoring path is delayed” from “the service is down”.
For sites with unreliable connectivity, retain a short local window and forward compressed data when the link returns. Keep the local Prometheus and Loki deployments small and purposeful; the goal is survivability, not accidentally operating a second full platform in every branch.
Alert on user impact, not provider noise
Cloud-provider alarms are valuable inputs, but they should not define the incident model. A provider may report instance health while a South African customer experiences a failed checkout because the application’s dependency in Europe is slow.
Use recording rules to create consistent service-level signals across providers:
- record: service:http_request_error_ratio5m
expr: |
sum by (service, region) (
rate(http_requests_total{status=~"5.."}[5m])
)
/
sum by (service, region) (
rate(http_requests_total[5m])
)
- record: service:http_request_latency_p95
expr: |
histogram_quantile(
0.95,
sum by (service, region, le) (
rate(http_request_duration_seconds_bucket[5m])
)
)Page on a burn rate or a customer-facing symptom, then attach diagnostic context: affected region, cloud account, dependency, deployment version and recent change events. Route alerts by service ownership rather than by provider. “AWS alarm” is rarely an actionable team boundary; “checkout SLO burning in production” is.
Load-shedding-aware alerting needs a similar distinction. If a site loses power, infrastructure availability may be expected to fall briefly while battery-backed networking remains online. Suppress duplicate host alerts during a declared maintenance or power event, but continue alerting on data loss, unsafe shutdown, failing failover and customer-facing SLO breaches.
Control cost, sovereignty and access from the start
Centralising everything can create a ZAR cloud bill that grows faster than the applications. Metrics with excessive label cardinality, verbose Kubernetes logs and unsampled traces are the usual culprits. Set retention by signal and purpose: short local metrics for rapid troubleshooting, longer aggregated metrics for capacity and SLO reporting, and selective trace retention for high-value services.
Use tail sampling to retain errors, slow traces and selected business-critical routes. Drop health checks and known noisy endpoints before they cross a region. Mimir recording rules can store useful aggregates without retaining every raw series indefinitely.
Data location also belongs in the architecture review. Logs and traces may contain email addresses, identity claims, payload fragments or internal hostnames. Classify telemetry under the organisation’s POPIA controls, redact at the collector, restrict tenant access, and decide whether raw data must remain in South Africa. A dashboard hosted in another region is not automatically a sovereignty problem, but exporting sensitive raw events without a documented basis is an avoidable one.
Use Grafana folders, teams and data-source permissions to separate platform, application and security views. Engineers need enough access to resolve incidents; they do not need unrestricted access to every customer log.
Test the control plane before the next outage
Unified Monitoring for Multi-Cloud Ecosystems is complete only when it works during failure. Run a quarterly exercise that removes one telemetry route, delays the Johannesburg-to-Europe link, stops a collector, and rotates a cloud credential. Confirm that local dashboards remain useful, queues drain safely, alerts do not flood the on-call channel, and traces still connect to logs after reconnection.
Measure the monitoring system itself:
- Telemetry delivery latency by region and signal type
- Percentage of critical services with valid traces, logs and SLO metrics
- Remote-write rejection and collector queue depth
- Alert precision, duplicate rate and mean time to acknowledge
- Telemetry cost per service, environment and retention tier
The strongest implementation is not the one with the most panels. It is the one that lets an engineer answer, quickly and with evidence: which customers are affected, where the request slowed down, which dependency changed, whether the problem is regional or global, and what action is safe while connectivity, power and cloud providers remain imperfect.