This article is going to describe how to create a trigger/continuous checking on a data set using an Azure function app, this article has used Azure CosmosDB as a data set with multiple documents. Sample application shows how to get an alert or a notification when a vehicle is going to violate the speed limit in a road or in a highway. Let's dig into deep and see how to use Azure function app to find a solution to this scenario
↑ Return to Top
If you are interested in Azure and its features, check these articles
Go to Azure Cosmos DB and navigate to Azure portal, Click on Add button, It will show Azure Cosmos DB account creation page. Add account name and select an API from list. In this example, we want to work with a simple document collection, so choose SQL as the data API
You can select a subscription for billing of the account as below. You can create a new resource group or use an existing one. Select account location as a nearest region to your clients. You can see a option to enable geo redundancy. At this point we are not going to replicate this data into another region, so don't tick that option. You can later replicate your data into different places to achieve high availability of your data.
You will be redirected to Quick Start page, you can create a collection in various platforms, in this case let's use .NET Core. You can get an idea what is the default storage of your collection, what will be the throughput and estimated hourly billing of the collection.
Let's try to create a document in Azure Cosmos DB collection, Go to Data Explorer, your new database account is created. Create a new collection, We can create a document in that collection. Lets add some test data and create a document as below.
{
"id"
:
"replace_with_new_document_id"
,
"firtsName"
"Hansamali"
"lastName"
"Gamage"
}
Click on your collection and click on New SQL Query, In the New Query window, write the simplest query and execute it. You can see inserted document as a result, and it has required 2.28 request units to run the query
We created a document in Azure Cosmos DB collection using Azure portal, Let's see how we can create a document in Azure Cosmos DB programatically
Open Visual Studio, This example has used Visual Studio 2017 with .NET Core 2.0.3 version to create the application. Select .NET Core and Console App (.NET Core), You can give a name and click on OK button
We need to add necessary references to access Cosmom DB, Go to Tools -> NuGet Package Manager -> Manage NuGet Packages for solution, Search for Azure Document DB, it will show as below. This version of Cosmos DB references are not compatible with .NET Core 2.0. So we have to find another one
Search for Microsoft.Azure.DocumentDB.Core, then you can find a reference that compatible with .NET Core. This is a special version of Document DB library designed for .NET Core application
Create a class named DBConnector and inside that, create a method as follows First we need to create a DocumentClient to connect to the database account, we have to pass endpoint url and auth key
static
class
DBConnector
DocumentClient client;
public
async
Task Create ()
try
string
endPointUrl =
""
;
primaryKey =
We have to instantiate a DocumentClient object to connect to Azure Cosmos DB account, in order to do that, we have to pass service endpoint URL and auth key or a token, Let's see how we can find them
Go to Keys section in Azure Cosmos DB account, Get the URI and Primary Key to connect to Azure Cosmos DB account
"https://highwaytraffic.documents.azure.com:443/"
"jSuZYTAvpOiw65P1O1TEPD7pAmjTTDUd5I9pudCkABUz97f3fxcnjg259o7jqDNCJXDyRAQrjOm6H2k24ZjyrQ=="
Create a DocumentClient by passing endpoint URL and key
client =
new
DocumentClient(
Uri(endPointUrl), primaryKey);
Create the database by passing database name
Task CreateDatabase (
database)
var db = await client.CreateDatabaseIfNotExistsAsync(
Database { Id = database});
Create a document collection by passing database name and collection name
Task CreateCollection(
database,
collection)
var databasecollection = await client.CreateDocumentCollectionIfNotExistsAsync(UriFactory.CreateDatabaseUri(database),
DocumentCollection { Id = collection });
In the main method, pass database and collection name and call necessary methods
databaseName =
"trafficdata"
databaseCollection =
"vehiclespeed"
await CreateDatabase(databaseName);
await CreateCollection(databaseName, databaseCollection);
call database create method from main program as below,
namespace
highwaytraffic
Program
void
Main(
[] args)
DBConnector.Create().Wait();
Let's run the application and see database is created in Azure account, Go to Overview tab in Cosmos DB account and check whether database is created. In Collections tab, it shows database and collectionid
In order to create a document, we should have a object, In this example we try to find vehicles who exceeds certain speed limit, so need to create a class to store vehicle speed, Add a class called VehicleSpeed into the solution,
VehicleSpeed
[JsonProperty(PropertyName =
"vehicleNumber"
)]
VehicleNumber {
get
set
; }
"speed"
double
Speed {
"city"
City {
Create a document by passing database and collection ids, And specially you should pass an instance from a VehicleSpeed class
Task CreateDocument (
collection, VehicleSpeed vehicleSpeed)
await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database, collection), vehicleSpeed);
Create an instance from VehicleSpeed class, and pass it to the document create method
VehicleSpeed vehicleSpeed =
VehicleSpeed { VehicleNumber =
"KJ -7788"
City =
"Colombo"
, Speed = 100.78 };
await CreateDocument(databaseName, databaseCollection, vehicleSpeed);
Let's run this latest code and see whether document is created, Click on Data Explorer, then it shows document inside the collection, Click on Documents, you can see created document with few other meta data
We have to create multiple documents to demo this application, let's see how we can do that, Create a for loop and insert few documents with a random speed value as below
Task CreateMultipleDocument(
int
documentCount = 500;
for
(
i = 0; i < documentCount; i++)
speed =
Random().NextDouble() * 100;
, Speed = speed};
await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database,
collection), vehicleSpeed);
Let's call this method and check whether it works,
await CreateMultipleDocument(databaseName, databaseCollection);
Let's run the application and see how it works, Go to Documents, it shows inserted documents as follows,
Click on New SQL Query and Get a count of documents as follows,
SELECT
COUNT
(1)
FROM
c
We created required no of documents for this demo, You can find the full code from below,
catch
(DocumentClientException de)
Exception baseException = de.GetBaseException();
Console.WriteLine(
"{0} error occurred: {1}, Message: {2}"
, de.StatusCode, de.Message, baseException.Message);
(Exception e)
Exception baseException = e.GetBaseException();
"Error: {0}, Message: {1}"
, e.Message, baseException.Message);
Let's see how we can create a azure function app in the portal, Click on New icon, In next screen, click on Compute and it shows Function App, click on it
You can see the following create screen,
You will be prompted to this screen when function app created, You can find App URL, status of the function app from this screen
We have to create a new function inside this Function app, Click on + icon as below,
You will see this screen, Click on Custom Function to view available templates to create a Function App
You can see available templates as below,
We are going to create a trigger on Azure Cosmos DB and call a function based on that, So select CosmosDBTrigger - C# template, Then you can see what are the options you want to provide, In Azure Cosmos DB trigger section, You can see Azure Cosmos DB account connection, we have to provide a Cosmos DB account to proceed with the trigger, click on new link
You can see available database accounts as below, Let's select our existing Azure Cosmos DB
Give a name to your function, You can see new Cosmos DB account connection has been created as below,
Click on show value in front of Azure Cosmos DB account connection, You can see endpoint URL to Cosmos DB account and the auth key as below
Provide Database name & Collection name to the data set we are going to work on, You can see Collection name for leases textbox, Azure Cosmos DB Triger use Cosmos DB Change feed to listen to the changes, This trigger needs a secondary collection to store leases over the partitions, Database collection and leases collection both should be available for the trigger to work
Go to trafficdata database and expand the collections, You can see leases collection has been created automatically, You can see available documents as follows, Those documents contains metadata about data partitions
If you go into the Function App, you can see new function is created, Let's Run this application and see what happens, You can see Logs windows appears as below, it shows Function start and completion, And it logs no new traces available in each minute,
Go to Visual studio and modify Create() method to insert a new document into monitored collection, Let's see this function app can trigger that
You can see logs window, Since collection got changed, Function has been started, and it has gone inside the if condition and printed document id as below,
Let's change function app logic to detect high speed of the vehicle as follows, Add a for loop in Run method, its going to detect a high speed value and print it in the screen If you want, you can copy high speed vehicles into another collection, may be can store it in a blob
Run(IReadOnlyList<Document> input, TraceWriter log)
if
(input !=
null
&& input.Count > 0)
log.Verbose(
"Documents modified "
+ input.Count);
"First document Id "
+ input[0].Id);
i = 0; i < input.Count; i++)
speed = input[i].GetPropertyValue<
>(
);
(speed > 100)
vehicleNo = input[i].GetPropertyValue<
city = input[i].GetPropertyValue<
.Format(
"High speed detected in {0}, Vehicle No {1} and Speed {2}"
, city, vehicleNo, speed));
Change the speed of the vehicle as 112.78, and run the solution
New document is created in the Cosmos DB collection and you can see the log window as below It shows 'High speed detection' message. We have created a trigger on Azure Cosmos DB and it fires when a change occur in a document, Cosmos DB has a new feature called Cosmos DB change feed, it can continuously read batch of changed entities, it can track list of changed documents, not only documents, graphs, mongo db documents as well When you query, as a result you get list of changed documents, list is going to depend on the throughput of the collection Let's say you got a fatal error in the middle of the process, you can restart and continue from the recent change token, it's not going to miss any change
You can download sample application from this location, https://gallery.technet.microsoft.com/Create-Cosmos-DB-Documents-e4d8ae00
You can clone the source code from here, https://github.com/hansamaligamage/cosmoscollection-create
This article explains how to create a Azure Cosmos DB database, collection & documents using Azure Cosmos DB SDK. We can track changes on a collection by using Azure Cosmos DB change feed. Application explains how to run a trigger on Azure Cosmos DB data set and perform an action using a Azure Function app.