Azure Event Hub Architecture

Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters : https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about

Key Architecture Components

With Azure Event Hub, several communication protocols are possible :

  1. AMPQ (Advanced Message Queuing Protocol : https://www.amqp.org/ )

  2. HTTPS

  3. Apache Kafka (Version 1.0 or later - https://kafka.apache.org/protocol ; )

Azure Event Hub supports for real-time and batch processing : Using a partitioned consumer model, it enables apps to process the stream concurrently and lets control the speed of processing.

SAS Tokens

Event Hubs uses Shared Access Signatures, which are available at the namespace and event hub level. A SAS token is generated from a SAS key and is an SHA hash of a URL, encoded in a specific format. The SAS Token mechanism includes Authorization Policies (Azure Service Bus access control with Shared Access Signatures - Azure Service Bus | Microsoft Docs)

Use SAS at HTTPs Level

After creating the SAS, we can perform an https POST :

POST https://<yournamespace>.servicebus.windows.net/<yourentity>/messages Content-Type: application/json Authorization: SharedAccessSignature sr=https%3A%2F%2F<yournamespace>.servicebus.windows.net%2F<yourentity>&sig=<yoursignature from code above>&se=1438205742&skn=KeyName ContentType: application/atom+xml;type=entry;charset=utf-8

 

Use SAS at AMQP Level

can access Service Bus using the Advanced Message Queuing Protocol (AMQP) that is the preferred protocol to use for performance reasons, in many scenarios.

Before starting to send data to Service Bus, the publisher must send the SAS token inside an AMQP message to a well-defined AMQP node named $cbs : Azure Service Bus access control with Shared Access Signatures - Azure Service Bus | Microsoft Docs

The publisher must specify the ReplyTo field inside the AMQP message.

/// <summary> /// Send claim-based security (CBS) token /// </summary> /// <param name="shareAccessSignature">Shared access signature (token) to send</param> private bool PutCbsToken(Connection connection, string sasToken) { bool result = true; Session session = new Session(connection); string cbsClientAddress = "cbs-client-reply-to"; var cbsSender = new SenderLink(session, "cbs-sender", "$cbs"); var cbsReceiver = new ReceiverLink(session, cbsClientAddress, "$cbs"); // construct the put-token message var request = new Message(sasToken); request.Properties = new Properties(); request.Properties.MessageId = Guid.NewGuid().ToString(); request.Properties.ReplyTo = cbsClientAddress; request.ApplicationProperties = new ApplicationProperties(); request.ApplicationProperties["operation"] = "put-token"; request.ApplicationProperties["type"] = "servicebus.windows.net:sastoken"; request.ApplicationProperties["name"] = Fx.Format("amqp://{0}/{1}", sbNamespace, entity); cbsSender.Send(request); // receive the response var response = cbsReceiver.Receive(); if (response == null || response.Properties == null || response.ApplicationProperties == null) { result = false; } else { int statusCode = (int)response.ApplicationProperties["status-code"]; if (statusCode != (int)HttpStatusCode.Accepted && statusCode != (int)HttpStatusCode.OK) { result = false; } } // the sender/receiver may be kept open for refreshing tokens cbsSender.Close(); cbsReceiver.Close(); session.Close(); return result; }

It's important that the connection is created with SASL authentication mechanism set to ANONYMOUS (and not the default PLAIN with username and password used when you don't need to send the SAS token).