Humongous amounts of data are being generated each moment today. Artificial intelligence and machine learning services are needed to handle and analyse this Big Data. This is where the AI and ML services of Microsoft Azure come in. Read on to find out how they are helping developers and businesses today.
artificial intelligence (AI) is the traditional model devised to mimic human brain functions by computers using a technology-driven approach along with mathematical and statistical models (e.g., probability). Machine learning (ML) was conceived later to make programming models of brain functions. It aimed to improve the behaviour of AI models using various learning approaches and storing the results of machine learning models for future predictions. This was inspired by the neural network — the core storage layer of the human brain that stores data using the input layer, the hidden layer and the output layer.
There are many popular machine learning frameworks available in the market, many of which are open source. To choose a machine learning tool or framework, one has to apply the problem scenario, data requirement, data volume to process, training data availability, trainability, algorithm support, visualisation and reporting facility, capability to auto-build the knowledge or training model, processing speed required (CPU/GPU), to name a few.
In ML, building the model is a time consuming task as a lot of training is required to get accuracy for the given problem. For example, for text recognition from images, a lot of training needs to be done in order to build the neural network of data sets for better recognition accuracy. This requires higher compute resources and hence greater scalability. This can be well achieved with cloud based platforms and, hence, machine learning services in cloud platforms are very popular today.
Azure machine learning services, including ML Studio, are being widely used for various business scenarios like anomaly detection, sentiment analysis for marketing research, and predictive analysis for customer relationship management, to name a few. But there are more practical business use cases now being experimented using Azure ML services in the banking industry. These include biometric identification, cheque processing using biometric analysis like face detection, and signature detection using pattern matching.
Microsoft Azure Machine Learning (MAML) Studio is an online, collaborative, drag and drop tool for building machine learning models. It helps developers to write common machine learning algorithms as modules rather than write code in languages like Python or R, and build models graphically using data sets. Once the model is built, developers can fine-tune the hyper parameters of the algorithms. And on finalising the model, machine learning models are published as Web services to support custom apps or BI tools like Excel.
The Azure ML Studio menu is divided into three main sections: Author, Assets, and Manage. Author deals with the creation of the code and set up of machine learning processes. Assets are the resources that are created and stored within the Author section, such as the pipelines created in the Designer. They control the entire resources workflow, right from inputting data sets over a pipeline to the endpoints for the output (e.g., connection to real systems via REST API). The Manage section is for the system behind the scenes. It encompasses the computation clusters and instances, data stores on which the data sets are stored, and the integrations into other systems. It is an invisible layer.
Azure Machine Learning Studio creates a machine learning model in seven steps:
a. Data gathering: Gathering of data starts the process, as data correction leads to perfect prediction models. MAML Studio provides a number of sample data sets, and additional collections can be downloaded from archives.ics.uci.edu/ml/dataset.html.
b. Data preview: After uploading the data, the data is previewed to validate the data input.
c. Modules integration: After viewing, modules are added by a simple drag and drop from the panel on the left.
d. Data preprocessing: This involves adjusting the available data as per nodes. Statistical data is computed from the available data.
e. Defining features: In this step, the features of the data set that are most relevant to the result for prediction are determined.
f. Applying algorithms: The algorithm is applied to train the model.
g. Predicting: The prediction is tested and data is visualised in this step.
AI based solutions in Fintech enable greater security, improved customer experience and return on investment, and faster time to market. A few AI driven solutions being used in the Fintech industry are listed below.
Azure Cognitive Services are cloud based services with APIs (application programming interfaces) and SDKs (software development kits) which allow developers to integrate cognitive intelligence into applications even if they don’t have a good knowledge of AI. With Azure Cognitive Services, various features like face recognition, speech recognition, emotion detection, and speech and language based advanced features can be added to applications.
These are available as REST APIs, client library SDKs and user interfaces. Azure Cognitive Services have tons of useful benefits for developers like:
a. Easy integration of pretrained and customised AI models in applications
b. Strong deployment of cognitive services from cloud to edge
c. Improved customer experience with simple and intuitive GUI in applications
Azure has introduced a new service called Form Recognizer, which is built on cognitive services and OCR methods. It enables character extraction from pre-printed forms like invoices and bills. Currently, it works for printed character extraction and can be used to extract text from PDFs, office documents, and images.Feature enhancement is in progress for recognition of handwritten forms/invoices/bills.
Azure ML Studio based OCR APIs is a supervised approach, where training and manual correction is required to improve the recognition rate of extracted text. Forms Recognizer is an unsupervised model, which improves the overall efficiency of the text recognition process.
Anomaly detection is a Big Data processing solution that aims at finding rare event patterns from a collection of patterns by creating pattern groups based on occurrences. For example, it can identify unusual login behaviour from a collection of audit logs, or unusual server failures from a server log.
In an enterprise, these logs are ever growing in size and have various patterns. Identifying which of these are valid events and which are suspicious ones is a big task.
In an enterprise, these logs are ever growing in size and have various patterns. Identifying which of these are valid events and which are suspicious ones is a big task.
Azure provides a sophisticated AI based service called Azure Anomaly Detector which uses API calls to use time series data like logs, enabling AI based detection of unusual event patterns. Before Azure introduced the Anomaly Detector, the Azure Machine Learning service was used to identify patterns of events, and use these for training the system to identify unusual behaviours.
With the introduction of anomaly detection, the system can self-train to build the knowledge base of patterns
In an enterprise, these logs are ever growing in size and have various patterns. Identifying which of these are valid events and which are suspicious ones is a big task.
Azure provides a sophisticated AI based service called Azure Anomaly Detector which uses API calls to use time series data like logs, enabling AI based detection of unusual event patterns. Before Azure introduced the Anomaly Detector, the Azure Machine Learning service was used to identify patterns of events, and use these for training the system to identify unusual behaviours.
With the introduction of anomaly detection, the system can self-train to build the knowledge base of patterns of events. The configuration can also be customised for fine-tuning the sensitivity and ranking of behaviour.
Microsoft Machine Learning for Apache Spark comprises tons of tools aimed at expanding the distributing computing framework Apache Spark in diverse directions. It adds deep learning and data science tools to Spark and even integrates Spark machine learning pipelines with the Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. This helps application designers to create powerful, scalable, predictive and analytical models for large images and text based data sets.
Some of its features are:
Azure Language Understanding (LUIS) allows an application to understand what a person wants. It uses machine learning to help developers design applications that accept the input in natural language and extract the meaning from it. The application interacts with end users to pass the input into the LUIS app and translate it into relevant information to fetch the results back
Bot framework: This chatbot based API talks with the end user via a text input.
Bing Speech API: This converts the natural language into text and then LUIS processes it.
There are three key concepts on which LUIS is based.
Intents:An intent represents the actions a user wants to perform. The intent is a purpose or goal expressed in a user’s input, such as booking a flight, paying a bill, or finding a news article. You define and name intents that correspond to these actions. A travel app may define an intent named ‘BookFlight’.
Utterances: An utterance is text input from the user that your app needs to understand. It may be a sentence, like ‘Book a ticket to Paris’, or a fragment of a sentence, like ‘Booking’ or ‘Paris flight’. Utterances aren’t always well-formed, and there can be many utterance variations for a particular intent.
Entities: An entity represents detailed information that is relevant in the utterance. For example, in the utterance ‘Book a ticket to Paris’, ‘Paris’ is a location. By recognising and
LUIS has two ways to build a model — the Authoring APIs and the LUIS.ai Web app. Both methods give developers and their collaborators control of the LUIS model definition. One can use either LUIS.ai or the Authoring APIs or a combination of both to build a model. This management includes models, versions, collaborators, external APIs, testing, and training. Once the model is built and published, the utterance is passed to LUIS and the JSON object results
Microsoft Azure has introduced a range of AI and ML services in recent years to help developers and businesses tackle the large amounts of data being generated. Do take some time out to learn more about them (check the References below) to keep up with these fast changing times.
document.getElementById( “ak_js_1” ).setAttribute( “value”, ( new Date() ).getTime() );