Issue:
When an WithSecure endpoint product is installed on a computer or server, there is high CPU usage and applications are experiencing performance issues. The connectivity of some applications can also be slow or blocked completely.
Issue affects all WithSecure clients:
- Elements Endpoint Protection EPP for Computers
- Elements Endpoint Protection EPP for Servers
- Business Suite Client Security
- Business Suite Server Security
- Business Suite Email and Server Security
Resolution:
Performance issues can for example be the result of:
- Connectivity issues to the Security Cloud
- Misconfigured Application Control
- Server Share protection
Connectivity issues to the Security Cloud
What is Security Cloud?
When Security Cloud is enabled on WithSecure endpoint products, it connects to WithSecure Backend to check reputation and other objects. WithSecure endpoint products have database updates which can detect the malware without connection to cloud, but, to check the reputation we need cloud connection. There is the local cache, but it comes first from the cloud, where the whitelisting of false positives is done.
Disabling Security Cloud is not recommended as there are many features that are dependent from Security Cloud, like:
- DeepGuard with cloud connection:
Unsigned processes reported as trusted by ORSP will be excluded from deep monitoring by DeepGuard
- DeepGuard with no cloud connection:
Big performance loses in operations of 3rd party applications
Unsigned files will not be excluded
- Application control with cloud connection:
Rules depending on prevalence rating and reputation will be working fine
- Application control with no cloud connection:
Unsigned files reported as trusted by ORSP will be excluded from scanning by local engines
Rules depending on prevalence and reputation will not work
Feature is partially not operational
- File scanning with cloud connection:
Unsigned files reported as trusted by ORSP will be excluded from scanning by local engine
- File scanning with no cloud connection:
Unsigned files will not be excluded
Some performance loses on file access
- Browsing protection with cloud connection:
Works without restrictions
- Browsing protection with no cloud connection:
Will not work at all as features fully depend on Security Cloud
Feature is partially not operational
- Web Traffic Scanning with cloud connection:
For URLs reported as trusted and prevalent by ORSP content returned by the server will be scanning
- Web Traffic Scanning with no cloud connection:
WTS will scan all responses of all URLs intercepted causing big performance issues
Big performance issues on web browsing
How does WithSecure Security cloud work?
The Security Cloud collects information about unknown applications and websites, malicious applications and malicious activities that exploit the information of users of websites. When you subscribe to Security Cloud, we collect important information so that we can provide you with the security services you subscribe to and enhance the security of our other services. For this reason, and for the operation of our services, we need to collect security information about unknown files, suspicious device activity or visited URLs.
Security Cloud does not monitor your Internet usage and does not collect information about websites that have already been analyzed or about unsafe applications installed on your computer.
How do I troubleshoot connectivity issues related to Security cloud?
When you enable Security Cloud, you also need to whitelist the following domains on your Firewall, as the endpoints need to communicate to Security Cloud.
- *.f-secure.com
- *.fsapi.com
Note: The domains mentioned above needs to be whitelisted to your firewall or proxy. In case your have enabled some proxy in your environment, the client reads it via discovery service and tries to connect to *.fsapi.com through it.
Client writes that information in registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\F-Secure\Ultralight\Settings\proxy]
"value"=(REG_SZ):http://proxy.example.intern:3128
"access"=(REG_DWORD):1
Example when network queries fail to connect to WithSecure back-end, from fsscorplug.log you will see how the client tries to connect to one of our backend servers and fails:
2021-11-12 17:40:30.152 [15c0.1d1c] .W: CurlQuery::completeWithStatus: failure on handle 0000023C45E27C50 5 Could not resolve proxy: proxy.example.intern
2021-11-12 17:40:30.152 [15c0.1d1c] .W: fs::xrssdk::HTTPQueryTask::update_http_stats: http error 111 (5) for http task 0000023C45E9AC20, time 4 ms
2021-11-12 17:40:30.152 [15c0.1d1c] .W: ipc_impl::on_async_complete_ex: winrpc call completed err 111
2021-11-12 17:40:31.751 [15c0.1d1c] I: fs::xrssdk::DoormanCache::update: doorman cooldown is off, ttl: 15, fserr: 0
2021-11-12 17:43:02.424 [15c0.1d1c] .W: CurlQuery::completeWithStatus: failure on handle 0000023C468C1D90 28 Operation timed out after 1006 milliseconds with 0 bytes received
2021-11-12 17:43:02.424 [15c0.1d1c] .W: fs::xrssdk::HTTPQueryTask::update_http_stats: http error 201 (28) for http task 0000023C45E76790, time 1006 ms
2021-11-12 17:43:02.424 [15c0.1d1c] .W: ipc_impl::on_async_complete_ex: winrpc call completed err 201
2021-11-12 17:48:02.964 [15c0.1fe8] I: ipc_impl::stopRpcServer: MSRPC Server stopped
The log can contain fserr 101 or 218 which are actual network failures.
The log shows some results from cache, as the queries are stored for 2 hours in cache, meaning if you just allowed our domains in firewall, client will still use cache queries for another 2 hours. Cache cleanup is for faster results to test the connectivity. you can clean the cache directly from client as follows:
- Open a Command prompt with administrator priviledges
- Stop the network hoster: net stop fsulnethoster
- Remove all files from "C:\Windows\ServiceProfiles\NetworkService\AppData\Local\F-Secure\fsscor"
- Start network hoster: net start "fsulnethoster
If issue remain, you shall clear the Doorman cache too:
- Open a Command prompt as an administrator
- Stop the services by running commands:
- net stop fsulhoster
- net stop fsulnethoster
- Open the Windows Registry editor (regedit), backup and clear values under
- HKEY_USERS\S-1-5-20\SOFTWARE\F-Secure\Ultralight\Doorman
- Remove all files from "C:\Windows\ServiceProfiles\NetworkService\AppData\Local\F-Secure\fsscor"
- Start the services by running commands:
- net start fsulhoster
- net start fsulnethoster
If you have allowed *.f-secure.com and *.fsapi.com in your firewall, you can test the connection in two ways:
- Opening the URLs on Browser and they should respond with ok
https://baseguard.doorman.fsapi.com/doorman/v1/healthcheck
https://doorman.sc.fsapi.com/doorman/v1/healthcheck
https://a.karma.sc2.fsapi.com/healthcheck
- Use WithSecure Connectivity Tool, which is available in the installation folders of Elements Endpoint Protection (EPP for Computers and EPP for Servers), Business Client Security and Business Suite Server Security. With the tool you can view the list of addresses the product connects to and check the connectivity towards them.
Note: For Client Security the tool is available in 15.20 and later versions, and for Server Security 15.10 and later.
The tool is located in the following folder:
- Client Security: C:\Program Files (x86)\F-Secure\Client Security\ui\fsconnectionchecker.exe
- Server Security: C:\Program Files (x86)\F-Secure\Server Security\ui\fsconnectionchecker.exe
- Elements EPP for Computers and EPP for Servers: C:\Program Files (x86)\F-Secure\PSB\ui\fsconnectionchecker.exe
For older Client Security and Server Security releases, you can download the tool from here: https://download.sp.f-secure.com/connectivitytool/ConnectionChecker.exe
What logs do should be checked in case of such behaviour?
fsscorplug.log
.W: fs::rs::WinSocket::Impl::waitForConnection: Wait failed: 258
.W: fs::rs::WinSocket::Impl::connect: Conection timeout: doorman.sc.fsapi.com
CcfPluginState.log
.W: Filter2::ContentFilter2State::ReplyDriverMessage: Failed to reply message 2222
orspplug.log
.W: fs::rs::WinSocket::Impl::waitForConnection: Wait failed: 258
.W: fs::rs::WinSocket::Impl::connect: Conection timeout: doorman.sc.fsapi.com
DeepGuard.log
.W: SecurityCloud::Query: ORSP failed for 0dac68816ae7c09efc24d11c27c3274dfd147dee (0, 0)
.W: SecurityCloud::Query: Too many successive ORSP failures. Further failure logs will be suppressed
.W: SecurityCloud::Query: ORSP query took 3016ms
transportAgent.log (Email and Server Security only)
.W: FSecure.Ess.Fsscore.Client: FSSCORE query for URL('http://schemas.microsoft.com/office/2004/12/xxxx') Failed, error=Timeout
.W: FSecure.AntiVirus.Exchange.Transport.FSMessageScanner: Can't get a response from FSSCORE. The following URLs will not be scanned
Misconfigured Application Control
If you have a premium subscription of Business Suite or Elements Endpoint Protection, it will include the Application Control feature.
If the product is using high amounts of CPU performance, make sure you have not set the Application Control Global rule as Allow and monitor all applications. This setting should be used only during testing to find out which applications need exclusion rules, since it will affect the performance of devices.
Also make sure that you have not created Application control exclusion rules which only include a SHA1 as a condition, since the calculation of the SHA1 will require some CPU performance. We recommend to use other conditions in conjunction with the SHA1 condition.
Server Share Protection
Elements Endpoint Protection for Servers has a Server Share Protection feature. If you have enabled it on your Elements EPP for Servers installation, try disabling Allow and report mode for it:
- Log in to the Elements Endpoint Protection portal
- Go to the Profiles page
- Go to the For Windows Servers tab
- Select the profile you want to edit
- Go to the Server Share Protection settings page
- Disable Allow and report mode
- Click Save and publish
Restart the server after disabling the feature and see if the CPU usage has decreased. If not, try disabling Server Share Protection feature off completely.
Article no: 000030468