← Back to Architecture Cloud · Streaming · Ingestion

Azure Event Hub
Streaming telemetry

The telemetry ingestion backbone that connects Azure IoT Hub with the analytics layer. It receives millions of events per day from the whole fleet, distributes them to multiple consumers in parallel and feeds Azure Data Explorer with no data loss.

What it is

Azure Event Hub is a high-capacity, managed and fully elastic event streaming service. Its abstraction model is a distributed, partitioned event log: producers write events to the end of the log; consumers read at their own pace from any point in time, within the configured retention period.

In the IN-SIGHT architecture, the Event Hub receives telemetry from all fleet vehicles through Azure IoT Hub's built-in endpoint, with no need for any intermediary service. From the Event Hub, three independent consumer groups process the data in parallel for three distinct purposes.

Guaranteed decoupling: The Event Hub acts as a buffer between the production rate of the IoT Pods and the consumption rate of ADX. If ADX experiences latency due to heavy ingestion, messages accumulate in the Event Hub without loss, and are processed when the consumer catches up.

Role in IN-SIGHT

The Event Hub is the single telemetry entry point to the cloud layer. Three consumer groups read the same stream for differentiated purposes:

  • Consumer group: adx-ingest — Azure Data Explorer consumes the events directly via a native data connection (no intermediate ETL). Telemetry lands in ADX in under 5 seconds from emission by the Pod.
  • Consumer group: realtime-alerts — An Azure Stream Analytics function evaluates each event against the alert rules configured in Portal IN-SIGHT. If a value exceeds the EKF thresholds, it generates a classified alert (CRITICAL / WARNING / INFO) in under 2 seconds.
  • Consumer group: archive — Azure Blob Storage receives a compressed copy of all events for long-term retention (minimum 12 months), independently of the Event Hub's retention policy.

End-to-end telemetry flow

The complete journey of a telemetry packet from the sensor to the dashboard:

IoT Pod  (CM4 on board the vehicle)
    │  Compact JSON · MQTT over TLS 1.3
    ▼
Azure IoT Hub
    │  Routes D2C messages to the built-in endpoint
    │  Device Twin synchronised in parallel
    ▼
Azure Event Hub  ← distribution point
    │
    ├── [adx-ingest]       → ADX (latency <5 s)
    │                         KQL queries · cloud EKF
    │
    ├── [realtime-alerts]  → Stream Analytics
    │                         Rules → Portal alerts
    │
    └── [archive]          → Azure Blob Storage
                              12 months+ retention

Partitioning scheme

Events are partitioned by vehicleId, which guarantees that all messages from the same vehicle are processed in order by the same consumer, preserving the temporal sequence required for the EKF analysis.

Partition key: vehicleId

Partition 0 → TMB-5000-01, TMB-5000-02, ...
Partition 1 → TMB-5000-05, TMB-5000-06, ...
Partition 2 → RC-447-201, RC-447-202, ...
Partition 3 → RC-447-205, RC-447-206, ...

With 4 partitions by default in IN-SIGHT's standard configuration, the system can process up to 4 MB/s of telemetry sustainably, with automatic scaling up to 20 MB/s through additional throughput units.