F-Secure Policy Manager's data mining feature explained This article gives you an introduction into the upcoming Data mining feature for F-Secure Policy Manager.
Note: This feature is part of the Policy Manager version 15.00 release.
Policy Manager acts as a focal point for the Business Suite solution, accumulating various data from the managed endpoints. The data includes information about blocked malware, malicious sites, and other incidents. In addition to that, each host reports telemetry statistics on the endpoint protection state, platform information, missing software updates, and so on.
Due to the amount and variety of data, administrators need tooling to help search and browse the data so that they can resolve issues, verify concerns, and create queries for custom reports and exporting data to react in a timely and smart fashion. Also, if you ever wanted to dynamically group and analyze your managed hosts independent of the domain structure then read on.
The concept
There are a few underlying concepts behind the flows of browsing and analyzing the data.
Firstly, everyone is familiar with the concept of searching for goods in online shops. The typical interface shows a number of filters that are relevant for the searched items, so you can start applying them one by one to gradually narrow down the set of results.
A set of items of the same type can be considered a polyhedron where each facet represents an item property. Using the same abstraction, the procedure of searching through the data set can be mentally visualized as manipulating the polyhedron facets by applying property filters and observing the effect.
- The polyhedron represents a data set that is a collection of items of the same type.
- Property filters are applied to the polyhedron facets.
The next concept is based on the relations between the data types, thus you can work with multiple data sets in the Data mining view. Policy Manager defines a number of data types to work with, for example alerts can be linked to the hosts that they belong to.
As an example, the following image visualizes the steps to get a list of missing Adobe updates for Windows Server 2019 hosts where malware detections have been recently triggered.
Data mining view
Data mining works with four data set types: hosts, alerts, software updates, and deleted hosts. These are selected using the Data set filter. Only the relevant filters remain visible after you select the data set type.
A full-text search is available in addition to property filters that search through all relevant item properties, including those that have unique values such as GUID, WINS, and IP addresses.
Items from one data set typically have related items in another data set, for example hosts can be linked to alerts. Use the Show related links to add new data sets for analysis.
You can also save search queries for later use either in the Data mining view or for publishing a custom report or API data export endpoint.
Property filters
Filter popup dialogs only show values that are applicable to a current subset of items. For instance, the Product filter for hosts with Windows 7 only lists the workstation family of products.
It can be useful to pin certain filters so that you can see their values in a separate widget. This way, you can observe the correlations while applying other filters and also include the filter values as part of the query results.
Filters support a few features to make data mining and correlation discovery between data set items easier. First of all, filter values are dynamically updated to reflect changes made to other filters. Secondly, filters with the same property value for all items in a given data set are highlighted with the property value shown.
Popup filters show the 100 most frequent values. Use manual input controls if a search query needs to define a value that is not on the list.
Relations between data sets
One search query can work with multiple related data sets. For example, after filtering a subset of hosts with Windows 7, you can link them to missing software updates to analyze them. Or you can filter only servers and check the subset of real-time protection alerts, then go to missing software updates to verify if any similar patches are missing.
Custom reports
The new Web Reporting introduced in Policy Manager 15.00 uses search queries as a basis for reports, which allows you to create custom reports.
All published queries are automatically available as custom reports in Web Reporting. Custom reports now only support table presentations and can include search query outputs as well as widgets.
The expansion state of item outputs and widgets defines whether the output is included in a report or not.
Data export
Search queries are also used for exporting data when integrating Business Suite with a monitoring or reporting ecosystem. Policy Manager introduces support for running search queries using REST API.
You have to publish a query to make it accessible over the API. You can copy the URLs from the My queries dialog. Search query results are in JSON format.
For example:
curl.exe -o query-result-test.json -k --header "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ" "https://pm_server:8080/fspms/api/v1/search/queries/saved/test/results?domainId=1"
dXNlcm5hbWU6cGFzc3dvcmQ is a Base64-encoded UTF-8 username:password string
