Top Ten Things You Should Know About IPSEC in Order to Pass a Microsoft Exam
Date: Jun 7, 2011
It will quickly become apparent once you begin studying for your chosen Microsoft accreditation that security plays a major role within all topic areas. One key weapon in Microsoft’s security armoury is IP Security (IPSEC), which is a joint Microsoft/Cisco venture designed to protect network communications between clients as they traverse both the private and the public network infrastructures. However, in terms of studying IPSEC for your Microsoft exams, it can be a little tricky to grasp all of its settings and acronyms. In order to hopefully make the topic a bit clearer, I have listed the top ten things you should keep in mind when studying IPSEC for your Microsoft networking exams.
1. When to choose tunnel mode or transport mode
Before you begin setting up an IPSEC policy, you need to establish which mode of IPSEC transport is required. There aren’t any guarantees on what may or may not come up as a question in a Microsoft exam, but if an IPSEC-related question appears without making reference to transport mode or tunnel mode, I would be amazed. Microsoft is very keen to test whether you know which method you should apply to a given scenario.
The transport mode is the default mode and should be used on local network deployments. A common exam scenario is that you have an accounting server on the local network (LAN) that requires all client communications to be authenticated and/or encrypted. It is appropriate here to mention that you have the option when creating an IPSEC policy to ensure a packets integrity, using the Authentication Header (AH) or integrity and encryption using the Encapsulating Security Payload (ESP). Without trying to confuse matters, the AH and ESP operate differently according to whether you use transport or tunnel mode. I will explain the main differences next.
The tunnel mode option is used for external connection types—for example, site-to site-connections over the internet or client-to-site connections over a VPN. The main reason that tunnel mode is more suitable is that like transport mode, it uses AH and ESP for encapsulating the IP packet but then encapsulates the entire packet header and trailer again in tunnel mode for additional security.
The simplest way to differentiate between the two modes is to know transport mode is for the LAN and tunnel mode is for external connections. And for your exams, it is essential to know the finer points of how each IP packet header is encapsulated with the required headers and footers, and the AH and ESP methods used. To a lesser extent, you should know the specific authentication protocols (SHA1 and MD5) and the encryption protocols (DES and 3DES) and the varying levels of security they provide to a policy.
2. What makes up an IPSEC policy
The key facts to remember here is that an IPSEC policy is made up of five variables. These are the filter list, the filter action (sometimes referred to as the filter rule), the authentication method, the tunnel endpoint, and the connection type. As you begin to build up your IPSEC policy, you pass through each of these areas and set them on your policy accordingly. The IP security console isn’t short of a wizard or three when it comes to setting each sub-area of the policy up, but for the purpose of the exam, it’s worth knowing how to navigate your way around the policy setup wizard-free as well.
By default, Microsoft supplies three IPSEC policies, which is nice of them! You should know these default policies very well for the exam. The client (respond only) policy is designed to respond to IPSEC requests and reply as required. So for example, if the next level IPSEC policy was assigned to a server, the request security policy, then the client policy will attempt to negotiate a security method with the server; but if no agreement is made, they will continue to communicate unsecured. If the server was assigned the require security policy, the client policy will respond securely with the server if a security method is negotiated. If an agreement isn’t made, the communications will stop as the server required that security was used. As I mentioned earlier, you should be very familiar with these policies, including the authentication methods used, and the granular security settings they have in regards to the encryption and integrity. Quite often, at least one if not all these policies are referred to in the exam.
3. What makes up an IPSEC policy, part two
I feel that I should give you a bit more detail on the five variables I touched on earlier. The amount of options and detail available to you here is too much for me to cram into a single paragraph... but I’ll try. Your filter list establishes what exactly is going to have a filter rule applied to it. In most cases, this is a destination IP or subnet, but there are other variables too, and the policies can be applied to inbound and outbound traffic. The filter rule or action establishes whether traffic is permitted, blocked, or whether security negotiation needs to take place. The first two in that list need little explanation, but negotiating security has a number of additional security options, briefly covered above in regards to the three default policies. You should note that when multiple filters are applied, they are applied in order of the most specific first. I discuss authentication in more detail below. The connection type refers to whether you are applying the policy to the LAN or to a dial up connection. I have obviously limited the detail required here, but your Microsoft materials will cover the more granular detail required.
4. Coming to grips with Main mode and Quick mode
In order to understand the workings of an IPSEC policy further, the second stage is to know it is made up of two parts: the Main mode and the Quick mode. Make sure you know that Main mode uses a three-stage negation process—stage one is the negotiation of the security suites to be used, stage two is referred to as the Diffie-Hellman key exchange (Diffie-Hellman is explained in more detail later), and stage three is the authentication stage between the clients using the chosen authentication method (also mentioned later). An important fact to remember is that the strength of the Main mode connection will then dictate the strength of the quick mode negotiations within it once the connection is established.
The Quick mode phase of the connection is used to conduct the actual transfer of data, creating a separate security association (SA) from within the Main mode connection. As a result, the lifespan of Quick mode is much shorter and by default will timeout after just five minutes (3600 seconds) or when the data limit is reached, which by default is 100mb. After this point, the session is renegotiated and the process starts again. Although this isn’t a common topic area covered in the exam, you should be aware of two issues when IPSEC is deployed on a large scale. The first is that the processing and negotiating of policies does take its toll on the computers involved, so either limit your deployment or consider a network card that allows IPSEC processing to be offloaded. Secondly, if you are not using PFS (perfect forward secrecy—which would also add to your processing load), the Quick mode negotiations will use the Main mode keys to generate its session keys. Any attacker that is monitoring your network could use the Quick mode keys to build up a picture of your Main mode session key.
5. Authentication methods
The choice you make here is dictated by what type of IPSEC connection you have in place. However, this choice is further narrowed when in the context of a Microsoft exam, as you need to apply best practice as well. If we were to do it by the book, then you should use Kerberos when you need to provide authentication on the local network, usually when operating a tunnel mode connection. This makes sense, as no further configuration is required provided that the clients and servers you are looking to secure communications between authenticate on the same domain. If you are using IPSEC over the internet using IPSEC tunnel mode, then you would need to use an external authentication method, namely a Public Key Infrastructure (PKI). This could be in the form of a third-party certification provider or via the Microsoft certification authority built into Microsoft Server operating systems. The bottom of the pile in terms of authentication methods is the pre-shared key. If we were to take the real world stance on this, we would likely adopt this method for authentication for budgetary or practicality reasons. However, this isn’t a preferred method within Microsoft’s best practice, as it is the weakest in comparison to PKI and Kerberos because it uses symmetric key encryption, making it more vulnerable to being compromised. However, that being said, the topic of pre-shared keys is a common exam question, mainly to point out its weaknesses. It is for that reason that you should make sure you know when you would use a PSK over the more secure authentication methods, and the extra lengths you should go to to make your PSK is a strong one (using varying case sizes of alpha-numerics and special characters, and making it at least 15 characters long).
6. IPSEC interoperability
When it comes to Microsoft exam questions, you can bet your last dollar that there will be at least one question which tests your knowledge on how different Microsoft products work or—as is sometimes the case—don’t work, on their different operating system packages. Within regards to IPSEC, there are a few cases where you need to be aware of some interoperability issues. First of all is the deployment of IPSEC via the command line. Within Windows Server 2008 and 2003 editions, you would use the netsh utility. This is a very powerful tool within Windows servers that is not reserved for IPSEC alone; but one of its many features is using it to setup and deploy server-based IPSEC policies. In regards to client operating systems, if you are using any version from XP onwards, you will be using the IPSECCMD. In regards to the Windows Server 2003 examination track, it is also worth knowing that the Windows Server 2000 command line tool of choice for IPSEC deployment is IPsecpol.
There are also further considerations when it comes to choosing the top levels of encryption. The use of 3DES (pronounced triple-DES) is only available to Windows Server 2003 operating systems upwards (including Windows XP upwards for the client operating systems). If a question arises where you have a number of Windows 2000 clients on a network communicating with a server that is using 3DES encryption strength, then without at least Service Pack 2 and the High Security Pack installed, they will only communicate at the DES level, which is much less secure.
Finally, it is worth noting for that possible sneaky exam question that may include a home-based operating system on your network that is failing to communicate using IPSEC. Non-domain clients do not support the use of Kerberos, and so any IPSEC policy that is deployed with Kerberos as its authentication method will fail here.
7. Deploying IPSEC
As with any network deployment, it is usually the scale that dictates the deployment method you use. For the Microsoft exams you will need to know the three main methods of deployment—locally using the IPSEC management console, using the IPSECCMD or netsh (usually in a batch file), and finally through group policy.
You should familiarize yourself with the IPSEC management tools, as they are a likely exam question area. The IP security management console is made up of two snap-ins: the IP security policies and the IP security console. The latter contains the monitoring tools required to observe an IPSEC policy once it is deployed, whereas the IP security policies snap-in lists the three default IPSEC policies supplied by Microsoft (mentioned in point 2). The two console snap-ins are together by default in group policy computer security settings, but you have to make up the console yourself if you are looking to set this up locally, or perhaps set up the policies first before exporting to the group policy template.
8. Command line tools
There are always command-line tools to master at this level of Microsoft configuration, but within IPSEC you do have your work cut out, for depending on which operating system you are using will in turn dictate which command-line tool you use (as described above).
- Netsh—Used within server operating systems, this is a very powerful tool which can do much more than just IPSEC. However, for the exam, you should know that there are two further sub-commands: netsh ipsec static creates the policies before applying, whereas the netsh ipsec dynamic applies changes to policies which are already in effect. Using the /? switch will give you the further configuration options available
- IPSECCMD—Like netsh, you can use IPSECCMD.exe in dynamic mode to apply policies or changes to policies already applied. This applies to Windows XP SP2 clients onwards.
- IPsecpol—You are unlikely to come across this tool any longer in the real world, but just knowing that it is the IPSEC command line tool for Windows 2000 systems is enough for the exam
- Netdiag—More of a testing tool, the netdiag /test:ipsec command allows you to view the status of any applied policies on Windows Server 2003 systems.
- Ping—The tool which makes it into pretty much any networking command-line tool kit. When you deploy an IPSEC policy and you want to ensure your clients are still talking, regardless of the IPSEC policy you have just applied, then the ping [IP address] is always the first port of call.
9. Logging and monitoring
There are plenty of logging and monitoring tools to make use of in IPSEC. Within the graphical tools, there is the IP security monitor, which has a very in-depth console-based tool for monitoring your active connections. From here you can see the active policies applied, and then the statistics for both the Main mode and Quick mode for your policy. As painstaking as it may be, it is worth spending the time getting to know what each monitoring statistic means as an exam question with a snapshot of these statistics is quite likely.
The event viewer is a stalwart for any troubleshooting in a Microsoft environment. The security event log lets you see whether or not a policy has been applied as expected. However you must consider the effects of any audit logging on a server, as this will cause a security event log to fill up and by default this will cause the server to shutdown.
For granular logging of IPSEC events, you must know about IKE (Internet Key Exchange) tracing. This is not enabled by default, and in truth you won’t be quizzed on it very hard in the security exams as the logs it produces are beyond the boundaries of knowledge required on the Microsoft network security track. However, you should know how to enable it in the registry on a client PC and the relative netsh command thay is required to turn it on for Server operating systems. The logged files are then retrievable from the %systemroot%\debug\oakley.log file. As an aside, you may wonder what the term Oakley is, as it appears in most exam materials without explanation. In short, Oakley, is the protocol which dictates which security levels within the Diffie-Hellman group will be used during the Main mode phase of negotiations.
10. Troubleshooting IPSEC
We have covered the monitoring and the logs provided for IPSEC; furthermore, we have covered the multitude of command line tools for setting up and managing IPSEC. The first step in troubleshooting an applied IPSEC policy is to stop the IPSEC service in the services console or using the net stop command.
When you are faced with a question that involves tunnel mode communication issues between two networks over a router, this is normally related to NAT issues. Make sure you know why you need a NAT-T (the “T” stands for traversal) when routing IPSEC traffic and the port numbers for NAT-T (UDP 4500), Internet Key Exchange (IKE—UDP 500), AH (TCP 51), and ESP (TCP 50).
Other areas of IPSEC troubleshooting you will be expected to know are related to authentication methods. As mentioned earlier these are Kerberos, PKIs and PSKs. If you are having issues with Kerberos, then this will most likely be down to domain issues such as a home-based client operating system on your network, or having applied an IPSEC policy to a domain controller which is stopping the authentication requests between clients and the domain controller. You should always consider whether the question has followed Microsoft best practice when troubleshooting a Microsoft product. In terms of PKI troubleshooting, you should check that the certificate is valid and that it isn’t on a certificate revocation list (CRL). Ironically, the best way of checking whether your authentication method is working correctly is to use the pre shared key (PSK) authentication method to see if this allows for a successful connection...for testing purposes only, of course!