Skip to main content

Text Analytics API With Azure Machine Learning

Scope
This article is about demonstrating the use of Machine Learning to do Sentiment Analysis on texts.

Introduction
Sentiment Analysis is the process of detecting the feeling or the mood of a person when writing a text (technically called contextual polarity). In other words, it determines whether a piece of writing is positive, negative or neutral.

Uses of Sentiment Analysis
  1. Product reviews
    • Is the review positive or negative
  2. Analyzing customer emails
  3. Social Media Analytics
    • What do customers think about my company
If one has about 10 mails or products, this will be quite a simple task, but what if some store has thousands of products each of them with  hundreds of reviews daily.

Then, automating this process makes sense. And this is where Text Analytics comes into play.

Text Analytics API
Text Analytics API is a suite of text analytics web services built with Azure Machine Learning. The API can be used to analyze unstructured text for tasks such as sentiment analysis and key phrase extraction.

The API returns a numeric score between 0 & 1. Scores close to 1 indicate positive sentiment, while scores close to 0 indicate negative sentiment.

The advantage of this API is that a new model need not be designed and trained, the user only needs to bring the data and call the service to get the sentiment results.

However, because this is only the initial release of this service only English is supported right now.

How it works
Text Analytics API does not simply use a lexicon approach to map words such as “good or “bad” to return the results.

Instead, it uses advanced natural language processing techniques under the hood.

For even more details of the service, consult the documentation here.

Signing up for the Text Analytics Service
  1. Go to Text Analytics
  2. Select the required package. 

    For testing purposed, up to 10,000 transactions are available for free.

    The following are the prices as on October 10, 2010.

    prices  
  3. Once the registration is done, you shall be provided with an account key. Keep it safely, this shall be used in the applications.
Sample Chat Application
One cool application of this API can be to use it in a chat application which analyse the sentiment of each chat and automatically places a smiley for the user.

Expected Outcome



High Level Application Design
When a user sends a message, the message is first broadcast to all the connected clients. Then, an asynchronous call is made to the Web Service to get the sentiment. After the sentiment score is obtained, the chat is updated with the corresponding smiley.

High Level Application Design

Diving in the Codes
This is an ASP.NET MVC Application which makes calls to the Web Service asynchronously and place the smiley only after the result is obtained.

Calling the Text Analytics Service
  1. Declare the Service Base URI and Account Key
    1. string ServiceBaseUri = "https://api.datamarket.azure.com/";  
    2. string accountKey = "xxxxxx";  
  2. Connect to the Service and pass connection details
    1. using(var httpClient = new HttpClient()) {  
    2.         string inputTextEncoded = HttpUtility.UrlEncode(inputText);  
    3.         httpClient.BaseAddress = new Uri(ServiceBaseUri);  
    4.         string creds = "AccountKey:" + accountKey;  
    5.         string authorizationHeader = "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes(creds));  
    6.   
    7.         httpClient.DefaultRequestHeaders.Add("Authorization", authorizationHeader);  
    8.         httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));  
  3. Get the sentiment score.
    1. string sentimentRequest = "data.ashx/amla/text-analytics/v1/GetSentiment?Text=" + inputTextEncoded;  
    2. System.Threading.Tasks.Task < HttpResponseMessage > responseTask = httpClient.GetAsync(sentimentRequest);  
    3. responseTask.Wait();  
    4. HttpResponseMessage response = responseTask.Result;  
    5. Task < string > contentTask = response.Content.ReadAsStringAsync();  
    6. string content = contentTask.Result;  
    7. if (!response.IsSuccessStatusCode) {  
    8.     return -1;  
    9. }  
    10.   
    11. SentimentResult sentimentResult JsonConvert.DeserializeObject < SentimentResult > (content)  
The UI
The UI is mainly in 2 parts; a form to capture and a div to display the result.
  1. The form
    1. <div class=" col-sm-4" <form class="form-horizontal">  
    2.     <div class="form-group">  
    3.         <div class="col-sm-10">  
    4.             <input type="text" id="name" placeholder="name" class="form-control" /> </br>  
    5.         </div>  
    6.     </div>  
    7.     <div class="form-group">  
    8.         <div class="col-sm-10">  
    9.             <textarea class="form-control" id="msg" placeholder="Message" rows="3"></textarea>  
    10.         </div>  
    11.     </div>  
    12.     <div class="form-group">  
    13.         <div class="col-sm-10">  
    14.             <br />  
    15.             <input type="button" id="send" value="send" class="btn btn-primary btn-lg" />  
    16.         </div>  
    17.     </div>  
    18.     </form>  
    19. </div>  
  2. The div to display the result.
    1. <div id="message" class=" col-sm-8 container" style=" height: 50%; max-height:300px; overflow: scroll;">  
    2. </div>  
The JavaScriptThere are 2 main functions on the JavaScript part, to send a message and to append messages to the div.
  1. Append the messages to the div

    Here, 2 actions are defined. Action 1 is when a broadcast happen and action 2 is when the result from the Web Service is obtained and it appends the smiley to the existing div.
    1. var chat = $.connection.chatHub;  
    2. // Create a function that the hub can call back to display messages.  
    3. chat.client.addNewMessageToPageChat = function(message, id, action) {  
    4.     //broadcast  
    5.     if (action == 1) {  
    6.         var text = document.getElementById('message').innerHTML;  
    7.         document.getElementById('message').innerHTML = message + '<label id =' + id + '> </label> ' + '<br/>' + text;  
    8.     }  
    9.     //update  
    10.     if (action == 2) {  
    11.         document.getElementById(id).innerHTML = message;  
    12.     }  
    13.     //update the emoticon  
    14.     $('#message').emoticonize();  
    15. };  
  2. Send a message

    This method captures the input and sends it to the server.
    1. $.connection.hub.start().done(function() {  
    2.     $('#send').click(function() {  
    3.         var name = $('#name').val();  
    4.         var msg = $('#msg').val();  
    5.         var tosen = name + ': ' + msg;  
    6.         chat.server.send(tosen);  
    7.         $('#msg').val('').focus();  
    8.     });  
    9. });  
C# Code
This method will be called when the send button is clicked.

This will firstly broadcast the message to all the clients by calling the JavaScript addNewMessageToPageChat function.

Then, it will call the method UpdateEmoticon which will connect to the service, get the sentiment score and asynchronously update the div on the front end to update the emoticon.
  1. public async Task Send(string message)  
  2. {  
  3.    string guid = Guid.NewGuid().ToString();  
  4.    Clients.All.addNewMessageToPageChat(message, guid, 1);  
  5.    await UpdateEmoticon(guid, message);  
  6.   
  7. }  
The method UpdateEmoticon will then determine which smiley to use based on the Sentiment Score obtained and update the chat on the front end.
  1.   public async Task UpdateEmoticon(string guid, string message)  
  2.   {  
  3.    //Calls the web service  
  4.    //The codes in the method Process Text is described above in section Calling the Text Analytics Service
  5.       double result = await ProcessText(message);  
  6.       string output = "";  
  7.       if (result >= 0.6)  
  8.       {  
  9.             
  10.               output = "   :) ";  
  11.       }  
  12.       else  
  13.        {  
  14.               output = "   :| ";  
  15.   
  16.       }  
  17.       Clients.All.addNewMessageToPageChat(output, guid, 2);  
  18.   }  
References

Comments

Popular posts from this blog

Creating and Querying Microsoft Azure DocumentDB

DocumentDB is the latest storage option added to Microsoft Azure.
It is a no-sql storage service that stores JSON documents natively and provides indexing capabilities along with other interesting features.

This article is available available on theMicrosoft Technet Wiki. This article was highlighted in theTop Contributor awardson the 12th of October 2014. This article was highlighted in the TNWiki Article Spotlight. This article was highlighted in the The Microsoft TechNet Guru Awards! (October 2014).


DocumentDB is the latest storage option added to Microsoft Azure.
It is a no-sql storage service that stores JSON documents natively and provides indexing capabilities along with other interesting features.
This wiki shall introduce you to this new service.

Setting up a Microsoft Azure DocumentDBGo to the new Microsoft Azure Portal. https://portal.azure.com/ 


 Click on New > DocumentDB


Enter A Database ID and hit Create!



Query Unstructured Data From SQL Server Using PolyBase

Scope The following article demonstrates how unstructured data and relational data can be queried, joined and processed in a single query using PolyBase, a new feature in SQL Server 2016. Pre-RequisitesIntroduction to Big Data Analytics Using Microsoft Azure Big Data Analytics Using Hive on Microsoft Azure Analyze Twitter Data With Hive in Azure HDInsight Running Hadoop on Linux using Azure HDInsight  Introduction Traditionally, Big Data is processed using Apache Hadoop which is totally fine. But what if the result of this needs to be linked to the traditional Relation Database? For example, assume that from the analysis of tons of application logs, marketing needs to contact some customs that faced problems in an application following a failure in the application.
This problem is solved with PolyBase. PolyBase allows you to use Transact-SQL (T-SQL) statements to access data stored in Hadoop or Azure Blob Storage and query it in an ad-hoc fashion. It also lets you query semi-structure…

Creating and Deploying Microsoft Azure WebJobs

Azure WebJobs enables you to run programs or scripts in your website as background processes. It runs and scales as part of Azure Web Sites.
This article focuses on the basics of WebJobs before demonstrating an example where it can be used.

This article is also available on the Mirosoft TechNet Wiki.
This article was highlighted in the The Microsoft TechNet Guru Awards! (October 2014).


Introduction
What is Microsoft Azure WebJobs?
Azure WebJobs enables you to run programs or scripts in your website as background processes. It runs and scales as part of Azure Web Sites.

What Scheduling Options is supported by Microsoft Azure WebJobs? Azure WebJobs can run Continuously, On Demand or on a Schedule.
In what language/scripts are WebJobs written?
Azure WebJobs can be created using the following scripts:  .cmd, .bat, .exe (using windows cmd).ps1 (using powershell).sh (using bash).php (using php).py (using python).js (using node)In this article, the use of c# command line app shall be demonstrated.
Cr…