Event 4625 Audit Failure NULL SID failed network logons

In 3 separate systems, the following event is being logged many times (between 30 to 4,000 times a day depending on the system) on the domain controller server:

An account failed to log on.

Subject:
    Security ID:        SYSTEM
    Account Name:       %domainControllerHostname%$
    Account Domain:     %NetBIOSDomainName%
    Logon ID:       0x3E7

Logon Type:         3

Account For Which Logon Failed:
    Security ID:        NULL SID
    Account Name:       
    Account Domain:     

Failure Information:
    Failure Reason:     Unknown user name or bad password.
    Status:         0xc000006d
    Sub Status:     0xc0000064

Process Information:
    Caller Process ID:  0x1ec
    Caller Process Name:    C:\Windows\System32\lsass.exe

Network Information:
    Workstation Name:   %domainControllerHostname%
    Source Network Address: -
    Source Port:        -

Detailed Authentication Information:
    Logon Process:      Schannel
    Authentication Package: Kerberos
    Transited Services: -
    Package Name (NTLM only):   -
    Key Length:     0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

This event is slightly different to all of the others that I've found during research but I have determined the following:

Event ID: 4625. "An account failed to log on".
Logon Type: 3. "Network (i.e. connection to shared folder on this computer from elsewhere on network)".
Security ID: NULL SID. "A valid account was not identified".
Sub Status: 0xC0000064. "User name does not exist".
Caller Process Name: C:\Windows\System32\lsass.exe. Local Security Authority Subsystem Service (LSASS), is a process in Microsoft Windows operating systems that is responsible for enforcing the security policy on the system. It verifies users logging on to a Windows computer or server, handles password changes, and creates access tokens. It also writes to the Windows Security Log.
Workstation Name: SERVERNAME. The authentication request is being submitted by or via the domain controller itself.

Affected systems' similarities:

Server Operating System: Windows Small Business Server 2011 or Windows Server 2012 R2 Essentials
Desktop Operating System: Windows 7 Professional (generally)

Affected systems' differences:

Antivirus
Active Directory-integrated Internet filtering
Desktop cached logons
Roles (Exchange, backup, etc)

Some interesting things I've noticed in the most severely affected system:

We recently started synchronizing Active Directory and Office 365 user account passwords via Windows Server 2012 R2 Essentials' Office 365 integration. The integration requires an Office 365 administrator's password and the security policy to be escalated. The synchronization requires each user account to be assigned to the corresponding Microsoft online account which requires the account's password to be changed on next logon. We also added their primary email domain as a UPN suffix in Active Directory Domains and Trusts and changed all user accounts' UPN to their email domain. Effectively, this allowed them to logon to the domain and Office 365 using their email address and password. However, since doing this the number of events logged per day has increased from ~900 to ~3,900. Note: none of the administrative or job-based (backup, scanner, etc) user accounts have been modified and no users are having issues accessing any parts of the system.
The bulk of the events seem to be logged at regular intervals usually every 30 or 60 minutes except for ~09:00 which is when the users arrive at work: 2015/07/02 18:55
2015/07/02 19:25
2015/07/02 19:54
2015/07/02 20:25
2015/07/02 20:54
2015/07/02 21:25
2015/07/02 22:24
2015/07/02 23:25
2015/07/03 00:25
2015/07/03 01:24
2015/07/03 01:55
2015/07/03 02:24
2015/07/03 02:55
2015/07/03 03:55
2015/07/03 04:55
2015/07/03 05:54
2015/07/03 06:25
2015/07/03 07:25
2015/07/03 08:24
2015/07/03 08:27
2015/07/03 08:49
2015/07/03 08:52
2015/07/03 08:54
2015/07/03 08:56
2015/07/03 08:57
2015/07/03 09:00
2015/07/03 09:01
2015/07/03 09:03
2015/07/03 09:06
2015/07/03 09:08
2015/07/03 09:10
2015/07/03 09:12
2015/07/03 09:13
2015/07/03 09:17
2015/07/03 09:13
2015/07/03 09:25
2015/07/03 10:24
2015/07/03 11:25

The following event is logged on the terminal / remote desktop services server though nowhere near as many times:

An account failed to log on.

Subject:
    Security ID:        NULL SID
    Account Name:       -
    Account Domain:     -
    Logon ID:       0x0

Logon Type:         3

Account For Which Logon Failed:
    Security ID:        NULL SID
    Account Name:       %terminalServerHostname%
    Account Domain:     %NetBIOSDomainName%

Failure Information:
    Failure Reason:     Unknown user name or bad password.
    Status:         0xC000006D
    Sub Status:     0xC0000064

Process Information:
    Caller Process ID:  0x0
    Caller Process Name:    -

Network Information:
    Workstation Name:   %terminalServerHostname%
    Source Network Address: %terminalServerIPv6Address%
    Source Port:        %randomHighNumber%

Detailed Authentication Information:
    Logon Process:      NtLmSsp 
    Authentication Package: NTLM
    Transited Services: -
    Package Name (NTLM only):   -
    Key Length:     0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

So, in summary, it definitely seems to be related to network access from desktop computers using staff user accounts but I can't see how.

Update 2015/08/25 08:48:

In the most severely affected system I have done the following to isolate the issue and after each reverted the change:

Shut down the terminal / remote desktop services server and the generic failed logons did continue.
Disconnected the domain controller server from the network and the generic failed logons did continue.
Rebooted the server into Safe Mode with no networking and the generic failed logons did not continue.
Stopped and disabled all "unnecessary" services (monitoring agent, backup, network filtering integration, TeamViewer, antivirus, etc) and the generic failed logons did continue.
Stopped and disabled Windows Server Essentials services (WseComputerBackupSvc, WseEmailSvc, WseHealthSvc, WseMediaSvc, WseMgmtSvc, and WseNtfSvc) and the generic failed logons did not continue.
Eventually, stopped and disabled the Windows Server Essentials Management Service (WseMgmtSvc) and the generic failed logons did not continue.

I have double-checked that the Windows Server Essentials Management Service (WseMgmtSvc) is responsible for these generic failed logons by disabling it for a few days and there were no generic failed logons and enabling it for a few days and there were thousands of generic failed logons.

Update 2015/10/08 09:06:

On 2015/10/07 at 16:42 I found the following scheduled task:

Name: "Alert Evaluations"
Location: "\Microsoft\Windows\Windows Server Essentials"
Author: "Microsoft Corporation"
Description: "This task periodically evaluates the health of the computer."
Account: "SYSTEM"
Triggers: "At 08:54 on 28/10/2014 - After triggered, repeat every 30 minutes indefinitely"
Actions: "Start a program: C:\Windows\System32\Essentials\RunTask.exe /asm:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\AlertFramework\v4.0_6.3.0.0__31bf3856ad364e35\AlertFramework.dll" /class:Microsoft.WindowsServerSolutions.NetworkHealth.AlertFramework.HealthScheduledTask /method:EvaluateAlertsTaskAction /task:"Alert Evaluations""

This timeframe almost exactly matches the behaviour above so I disabled it to see if it affects the issue.

On 2015/10/08 at 08:57 I found that only 47 of these generic failed logons were logged since at irregular intervals.

So, I have narrowed it down even further.

This Event is usually caused by a stale hidden credential. Try this from the system giving the error:

From a command prompt run: psexec -i -s -d cmd.exe
From the new cmd window run: rundll32 keymgr.dll,KRShowKeyMgr

Remove any items that appear in the list of Stored User Names and Passwords. Restart the computer.

It seems that the problem was caused by the scheduled task "Alert Evaluations".

Event 4625 Audit Failure NULL SID failed network logons

Related

Recent Posts