When you log into a Windows machine, whether you know it or not, a negotiation happens behind the scenes. Windows decides which authentication protocol to use — Kerberos or NTLM — based on the situation. Most of the time, you never notice. But which protocol gets chosen has significant security implications.
Kerberos and NTLM are the two primary authentication protocols in Windows environments. They represent two different eras and two different philosophies of authentication. Understanding how they work, when Windows chooses one over the other, and why both still exist is essential for anyone managing Windows security.
Kerberos: The Ticket-Based System
Kerberos is the default authentication protocol for Active Directory domains. It was developed at MIT in the 1980s and named after Cerberus, the three-headed dog of Greek mythology that guards the gates of the underworld. The name is apt — Kerberos guards access to network resources.
The protocol works using a system of encrypted tickets:
**The Key Distribution Center (KDC):** The KDC is the heart of Kerberos. In Windows Active Directory, the KDC runs on every domain controller. It has two components: the Authentication Server (AS) and the Ticket Granting Server (TGS).
**Initial Authentication (AS Exchange):** When a user logs in, their computer sends a request to the AS. The AS responds with a Ticket Granting Ticket (TGT), encrypted with the user's password-derived key. Only someone who knows the password can decrypt the TGT. This proves the user's identity to the KDC without sending the password over the network.
**Service Tickets (TGS Exchange):** When the user wants to access a network resource (like a file share), their computer presents the TGT to the TGS and requests a service ticket for that specific resource. The TGS issues a service ticket encrypted with the service's password-derived key.
**Accessing the Resource:** The user's computer presents the service ticket to the resource server. The server decrypts it using its own password-derived key. If decryption succeeds, the server knows the KDC vouched for the user and grants access.
The beauty of Kerberos is that the user's password never goes to the resource server. The KDC vouches for the user through encrypted tickets. The resource server trusts the KDC, not the user directly.
Kerberos also supports mutual authentication — the server proves its identity to the client, preventing man-in-the-middle attacks. And tickets have a limited lifetime (typically 10 hours), after which they must be renewed, limiting the window for ticket theft.
NTLM: The Challenge-Response System
NTLM is the older protocol, and it works very differently. Instead of tickets, NTLM uses a challenge-response mechanism:
**Challenge:** The server sends a random challenge to the client. **Response:** The client uses its NTLM hash (MD4 of UTF-16LE encoded password) to compute a response to the challenge and sends it back. **Verification:** The server (or domain controller) computes the expected response using its stored copy of the user's hash and compares it to what the client sent.
The critical difference from Kerberos: in NTLM, the hash itself is the credential. The server verifies the response by recomputing it with the stored hash. This means anyone who possesses the hash can authenticate as the user — the basis of pass-the-hash attacks.
NTLM doesn't support mutual authentication by default. The server doesn't prove its identity to the client, which makes NTLM vulnerable to relay attacks. And NTLM doesn't use a central authority like the KDC — the server does the verification itself (or forwards it to a domain controller).
You can see the hash that powers NTLM authentication using our NTLM Hash Generator. That hash is what the protocol uses to compute responses to server challenges.
When Windows Chooses NTLM Over Kerberos
Windows prefers Kerberos in domain environments, but it falls back to NTLM in several situations:
**Non-Domain Authentication:** When authenticating to a machine that's not part of the domain (like a workgroup machine or a non-Windows system), Kerberos isn't available. Windows uses NTLM.
**Domain Controller Unreachable:** If the domain controller can't be contacted (network issue, DC down, or the client is on a remote network without DC access), Windows falls back to NTLM. This is a common scenario for remote workers.
**IP Address Instead of Hostname:** Kerberos relies on Service Principal Names (SPNs) that are associated with hostnames. If you connect to a resource by IP address instead of hostname, Kerberos SPN resolution can fail, and Windows falls back to NTLM. This is a common misconfiguration that weakens security.
**Application-Specified Protocol:** Some applications are hardcoded to use NTLM. They call the NTLM authentication APIs directly, bypassing the normal negotiation. Legacy applications are common culprits.
**Missing SPN:** If the SPN for a service isn't properly registered in Active Directory, Kerberos authentication fails and Windows falls back to NTLM. This is one of the most common causes of unexpected NTLM usage.
**Cross-Domain or Cross-Forest Scenarios:** In complex Active Directory environments with multiple domains or forests, Kerberos may not be configured properly for cross-domain authentication, leading to NTLM fallback.
Why NTLM Can't Be Fully Removed
Given its security weaknesses, why hasn't Microsoft removed NTLM entirely? Several factors prevent this:
**Application Compatibility:** Countless applications rely on NTLM authentication. Removing NTLM would break these applications, potentially causing massive disruption. Many organizations can't identify all their NTLM dependencies without risking outages.
**Interoperability:** NTLM is used for authentication with non-Windows systems in some scenarios. SMB authentication to older NAS devices, authentication to Linux systems via certain configurations, and other cross-platform scenarios may require NTLM.
**Fallback Necessity:** NTLM serves as a fallback when Kerberos can't be used. Without NTLM, authentication would fail entirely in scenarios where the domain controller is unreachable. This could lock users out of resources during network issues.
**Workgroup Authentication:** For machines not in a domain (home machines, small business networks), NTLM may be the only option. Kerberos requires a domain infrastructure that not all environments have.
Microsoft is working toward reducing NTLM usage. Windows 11 includes NTLM auditing policies that log when NTLM is used, helping organizations identify dependencies. NTLM restriction policies can block NTLM in specific contexts. And Microsoft has published guidance for migrating away from NTLM.
But the reality is that most organizations still have NTLM somewhere in their environment, and removing it completely is a project, not a setting.
Security Recommendations
Given that both protocols exist in most Windows environments, here are practical security recommendations:
**Prefer Kerberos:** Ensure Kerberos is working correctly. Register SPNs for all services, use hostnames instead of IP addresses for connections, and ensure domain controllers are accessible. Monitor for NTLM usage and investigate why it's happening.
**Restrict NTLM:** Use Group Policy to restrict NTLM where possible. Start with auditing to identify where NTLM is used, then gradually restrict it. Set "Network security: LAN Manager authentication level" to require NTLMv2 and refuse LM/NTLMv1.
**Protect Against Pass-the-Hash:** Since NTLM enables pass-the-hash attacks, implement mitigations. Enable Windows Defender Credential Guard, protect LSASS with PPL, use LAPS for local admin passwords, and implement tiered administration. Read more in our article on pass-the-hash attacks.
**Monitor Authentication:** Use Windows event logs to monitor authentication. Event ID 4624 shows successful logons, including the authentication protocol used. Watch for NTLM authentication from unexpected sources or to unexpected destinations.
**Plan for NTLM Removal:** Start the journey toward eliminating NTLM. Audit NTLM usage, identify dependencies, remediate applications, and gradually restrict NTLM in your environment. Microsoft's NTLM auditing and restriction guidance is the roadmap.
**Secure Kerberos Too:** Kerberos isn't perfect. Watch for Kerberoasting attacks (where attackers request and crack service tickets), ensure service account passwords are strong, and consider Group Managed Service Accounts (gMSAs) for automatic password management.
The Bigger Picture
Kerberos and NTLM represent a fundamental tension in security: the ideal versus the practical. Kerberos is the ideal — strong, ticket-based, mutual authentication. NTLM is the practical — simple, compatible, always available. Windows needs both because the real world isn't ideal.
Understanding both protocols helps you make better security decisions. You can identify when NTLM is being used unnecessarily, push for Kerberos where it's possible, and implement appropriate mitigations where NTLM can't be eliminated.
For more on the specific hash format that powers NTLM, read our NTLM hash explained article. And for the protocol-level differences between NTLM versions, see our NTLM vs NTLMv2 comparison.
Windows authentication is a layered system built on decades of evolution. Understanding the layers — from the MD4 hash at the bottom to the Kerberos tickets at the top — gives you the knowledge to secure your environment effectively.
