VMware Cloud Expert

VMware Cloud Expert  /  VMware Cloud Expert: VMC on AWS (4-day) - VMCEXPERT1  /  Day 2

Lab 05 - L7 Security - L7 FW, FQDN Filtering & IDPS

Updated on

Introduction

VMware Cloud on AWS provides VMware’s enterprise-class SDDC software on AWS cloud. It includes a robust set of networking and security capabilities that enable customers to run production applications in the cloud. Every SDDC is provisioned with the Gateway Firewall to protect the perimeter of the SDDC, and the Distributed Firewall to secure lateral communication across workloads inside the SDDC. Powered by the proven security capabilities of VMware NSX-T, Gateway and Distributed Firewall provide enterprise-class Layer 4 security for applications in VMware Cloud on AWS:

  • Gateway Firewall enables customers to selectively allow and deny traffic from and to applications deployed in the SDDC. It also controls access to management infrastructure, such as vCenter and NSX manager
  • Distributed Firewall is built into the hypervisor and automatically scales across every host in the SDDC. Enabling micro-segmentation at the workload level, Distributed Firewall policies migrate with the VM when they move from host to host in the SDDC.

NSX Advanced Firewall features take the network security capabilities of VMware Cloud on AWS SDDC to the next level, allowing customers to define security policies at Layer 7 and enabling deep packet inspection across all vNICs within the SDDC.

With the NSX Advanced firewall add-on to your VMC on AWS SDDC(s) you can deliver enhanced security for your VMC on AWS workloads in any of these scenarios:

  • Distributed IDS/IPS (Detect and prevent threats to your workloads) - Enterprises are constantly reminded of threats to their applications by a never-ending stream of news about exploits on the internet. With NSX Distributed IDS/ IPS, customers gain protection against attempts to exploit vulnerabilities in workloads on VMware Cloud on AWS. Distributed IDS/ IPS is an application-aware deep packet inspection engine that can examine and protect traffic inside the SDDC. Customers can detect and prevent lateral threat movement within the SDDC using the intrinsic security capabilities of Distributed IDS/IPS.
  • L7 (Context-aware) Firewall - With L7 (Context-aware) firewall you can go beyond simple IP/ port level layer 4 security to complete stateful layer 7 controls and filtering. Deep packet inspection (DPI) built into the Distributed Firewall enables you to allow only the intended application/protocols to run, while denying all other traffic at the source. This enables you to isolate sensitive applications by creating virtual zones within the SDDC. Distributed Firewall (DFW) layer 7 policies are enforced at the hypervisor (vNIC) level and can migrate with the VM when they move from host to host in the SDDC, ensuring there are no gaps in enforcement.
  • User Identity Firewall (IDFW) - You can create groups based on User ID and define DFW rules to control access to virtual desktops and applications in the SDDC. Per user/ user session access control limits the amount of time and exposure users have to desktops or applications. Integration with Active Directory / LDAP enables the DFW to continuously curate user access to applications. UserID-based rules are enforced by the DFW at the source, delivering pervasive, intrinsic security throughout the SDDC.
  • FQDN Filtering - Applications that communicate outside the SDDC also gain layer 7 protection using Distributed Firewall FQDN filtering capability. Customers can define specific FQDNs you can define specific FQDNs that are denied access to applications in the SDDC. The DFW maintains the context of VMs when they migrate. Customers increasingly rely on application profiling and FQDN filtering to reduce the attack surface of their applications to designated protocols and destinations.

TASKS

Task 1 - Enable the NSX Advanced Firewall Add-on

Before any of the NSX Advanced Firewall features can be used, you must first enable the add-on onto your SDDC. In this task, we'll walk through the steps of enabling the NSX Advanced Firewall functionality onto your SDDC.

  1. From the VDI Desktop open the Google Chrome Browser or Firefox
  2. Launch the VMware Cloud SDDC bookmark or browse to https://vmc.vmware.com/console/sddcs
  3. Login as
    • User name: vmcexpert#-xx@vmware-hol.com (Where # is your Environment ID and xx is your student number)
    • Password: VMware1!
  4. On your SDDC tile, click View Details
  5. Click the Integrated Services tab
  6. In the NSX Advanced Firewall Tile, click Activate
  7. In the Popup, click Activate
  1. Click OPEN NSX ADVANCED FIREWALL (This will take you to the Networking & Security Tab)

At this point, The NSX Advanced Firewall Add-on has been enabled. To make use of the functionality it provides, you must configure them individually. In the upcoming tasks we will configure and test each of these features.

Task 2 - Configure Context-Aware Firewall rules.

NSX Context-Aware Firewall  Rule (L7) enhances visibility at the application level and helps to override the problem of application permeability. Visibility at the application layer helps you to monitor the workloads better from a resource, compliance, and security point of view.

Firewall rules cannot consume application IDs. Context-aware firewall identifies applications and enforces a micro-segmentation for EAST-WEST traffic, independent of the port that the application uses. Context-aware or application-based firewall rules can be defined by defining Layer 7 service objects. After defining Layer 7 service objects in rules, you can define rules with specific protocol, ports, and their application definition. Rule definition can be based on more than 5-tuples. You can also use Application Rule Manager to create context-aware firewall rules.

With Context-Aware Firewalling you can enable enforcement of security protocol versions/ciphers reduce attacks by only allowing traffic matching APP Fingerprint, and enforce port-independent rules

Task 2.1 - Create an L4 Firewall rule for SSH

We will begin by creating a standard L4 Distributed firewall rule for SSH, doing so will allow us to better understand the power L7 rules bring to securing applications running in your SDDC.

  1. In the VMware Cloud on AWS portal click the OPEN NSX MANAGER button
  2. Click ACCESS VIA THE INTERNET to connect to NSX Manager UI
  3. Click on Security tab
  4. Click Distributed Firewall
  5. Click on Actions 
  6. Choose General Settings
  7. Click the slider for Identity Firewall Status to enable it
  8. Click Identity Firewall Tab
  9. Click the slider for Cluster-1 to enable it
  10. Click SAVE
  1. Expand the Web-Tier Firewall Policy
  2. Move the slider at the far right of the Block Web-to-web rule to the left to disable the rule.
  3. Click PUBLISH

Doing this will allow communications between the web servers.

We created this rule in lab 2 and set it to block all traffic between the web servers.

  1. Click on Open vCenter in the upper left. (if you no longer have it opened)
  2. Log into the SDDC vCenter as:
    • [email protected]
    • <copy the cloudadmin password from the settings tab or your worksheet>
  3. In the vCenter Inventory, select webserver02 and take note of its IP address (10.10.X.X)
  4. In the Inventory select webserver01 
  5. click LAUNCH WEB CONSOLE
  1. In the browser tab for webserver01, login as:
    • root
    • VMware1!
  2. attempt an ssh session to webserver02
ssh {webserver02_ipaddress}
Click to copy
  1. If prompted with he SSH Thumbprint type Yes, then press enter. You should be prompted to log into webserver02
  2. Press ctrl+c on the keyboard to cancel the ssh session without logging in

Let's go back to the SDDC NSX Manager UI, let's modify the firewall rule to block SSH between the web servers

  1. In the Services field of the Block web-to-web rule, move your mouse to the right of ANY and click the blue pencil that appears
  2. In the dialog find and select SSH
  3. Click APPLY
  4. In the far right move the slider to the right to enable the rule
  5. Click the Action Field and Set the rule action to DROP
  6. Click PUBLISH
  1. Return to the console tab for webserver01 and once more attempt an SSH session to webserver02
  2. You can use the Up Arrow key to recall the previous command
ssh {webserver02_ipaddress}
Click to copy

Notice the amount of time it took before the ssh session timed out.  If you go back and change the rule to REJECT, you will see an immediate failure.

This time notice that the connection timed out and was never accepted by webserver02, this is because the DFW blocked the session and it never made its way to webserver02

Task 2.2 - Enable SSH on a different port and test

In task 2.1, we created a standard L4 rule to block SSH traffic between the web servers and it worked as expected. This is because, at L4 the DFW evaluates the Source, Destination, and Port of the packet. In this case, the source was webserver01, the destination was webserver02 and the destination port was 22.

What if SSH was listening on another port, however? What if some nefarious person (knowing SSH on port 22 is being blocked) changed the port the server listens on and attempts to SSH to the server against this new port, what happens then? Let's find out in this task and also see how an L7 rule can protect against this type of activity.

We are going to first modify the SSH server to listen on port 2222 instead of 22

  1. In the vCenter Inventory, select webserver02
  2. Click LAUNCH WEB CONSOLE
  3. In the browser tab for webserver02, login as:
    • root
    • VMware1! 
  1. Type the following commands in the console of webserver02 to create a copy of the ssh configuration file
cd /etc/ssh
ls
cp sshd_config sshd_config.orig
ls
Click to copy
  1. Type sudo nano sshd_config to open the configuration file in nano
  2. look for the line with Port 22 and change the port to 2222 (Important - remove any # sign at the beginning of the line so that the line is not commented out)
  3. Press ctrl+o to save the file
  4. Press enter to confirm the save
  5. Press ctrl+x to exit nano
  1. restart the ssh service by typing the following command
sudo systemctl restart sshd.service
Click to copy
  1. Return to the console tab for webserver01 and once more attempt an SSH session to webserver02 but this time on port 2222 by typing the following command
ssh -p 2222 {webserver02_ipaddress}
Click to copy
  1. Confirm that you get a password prompt
  2. Press ctrl+c to exit ssh

This time the connection does not timeout, the DFW doesn't block it. As mentioned earlier, the firewall is looking for SSH on port 22, not port 2222. We can therefore bypass the firewall policy

Task 2.3 - Create an L7 Context rule to block SSH regardless of port

We will now see what happens when we apply port-independent context awareness to the firewall rule. We want to so if we can still bypass the Firewall and access SSH on the default port or a port other than the default port.

  1. In the SDDC NSX Manager UI click Security tab
  2. Click Distributed Firewall
  3. Expand the Web Tier Policy
  4. In the Services field of rule Block web to web mouse over SSH and click the blue pencil
  5. In the dialog remove SSH from the list of selected service
  6. Click APPLY
  7. In the Context Profile field, mouse over None and click the blue pencil
  8. Select SSH 
  9. Click APPLY
  10. Click PUBLISH
  1. Return to the console tab for webserver01 and once more attempt an SSH session to webserver02 on both ports 22 and 2222
ssh {webserver02_ipaddress}
ssh -p 2222 {webserver02_ipaddress}
Click to copy

This time both attempts (on the standard ssh port of 22 and the modified port of 2222 is not allowed. This is because the DFW now assesses the packet at layer 7 and identifies the heuristics of the packet to be ssh and does not allow the traffic through.

Task 3 - FQDN Filtering

VMC on AWS can allow users to only access specific domains by whitelisting and/or blacklisting FQDNs. In many high-security environments, outgoing traffic is filtered using the Distributed firewall. When you want to access an external service, you usually create IP-based firewall rules. In some cases, you don't know which IP addresses hide behind a domain. This is where domain filters come in handy.

You must set up a DNS rule first, and then the FQDN allowlist or denylist rule below it. This is because NSX-T Data Center uses DNS Snooping to obtain a mapping between the IP address and the FQDN. SpoofGuard should be enabled across the switch on all logical ports to protect against the risk of DNS spoofing attacks. A DNS spoofing attack is when a malicious VM can inject spoofed DNS responses to redirect traffic to malicious endpoints or bypass the firewall

We will start by creating a couple of Groups we will use in the Distributed firewall, followed by a context profile for FQDN filtering and finally, we will define the firewall Policy.

Task 3.1 - Create Security Groups

We'll need to create two security groups.  One for our Desktop VMs, to specify the source of our DNS traffic, and one for all possible DNS servers (Public and internal) to pick up the DNS query/response and send them to the Deep Packet Inspection (DPI) Engine, so that the distributed firewall can learn IP addresses that are matched with domain names for all lookups by the Desktop VM.

  1. In the SDDC NSX Manager UI click Inventory tab
  2. Click Groups
  3. Click Compute Groups
  4. Click ADD GROUP
  5. Name: Desktops
  6. In Compute Members click Set
  7. Select the Members tab
  8. Select Category NSX Segments
  9. Select Desktop-Net
  10. Click APPLY
  11. Click Save
  12. Using Steps 1 - 11 create a 2nd Group as follows
    • Name: DNS Servers
    • Type tab: IP Addresses
      • Values:
        • 192.168.110.10
        • 8.8.8.8
        • 8.8.4.4

Task 3.2 - Create FQDN Context Profile & Firewall Policy

In this task, we will create Context profile with two user-defined FQDNs, and one built-in System-defined FQDN. Later, we will use this context profile in a firewall policy to restrict web access from the windows desktop to only these three FQDNs

  1. In the SDDC NSX Manager UI click Inventory tab
  2. Click on Profiles
  3. Select Attribute Types Tab
  4. Click FQDNs sub-tab
  5. Click ACTIONS --> Add FQDN
    • Domain: *.google.com
  6. Click SAVE
  7. Repeat task 3 - 6 using *.vmware.com
  1. Click the Profiles Tab
  2. Click ADD CONTEXT PROFILE
  3. Name: Allowed FQDNs
  4. In attributes click Set
  5. Click ADD ATTRIBUTE --> Domain(FQDN) Name
  6. Select the following domains
    • *.google.com
    • *.vmware.com
    • *.verisign.net
    • *.verisign.com
  7. Click ADD
  8. Click APPLY
  9. Click SAVE
  1. Go to Security tab
  2. Click Distributed Firewall
  3. Click Add Policy
  4. Name the Policy FQDN Whitelist
  5. Select the Policy and add 3 firewall rules
  6. Configure the 3 firewall rules as follows: (Ensure the rules show up in order matching the screenshot)
    1. RULE 1
      • Name: FQDN DNS
      • Source: Desktops
      • Destination: DNS Servers
      • Service:
        • DNS-TCP
        • DNS-UDP
      • Context Profile: DNS
      • Action: Allow
    2. RULE 2
      • Name: Allow limited Public access
      • Source Desktops
      • Destination: ANY
      • Service:
        • HTTP
        • HTTPS
      • Context Profile: Allowed FQDNs
      • Action: Allow
    3. RULE 3
      • Name: Block All HTTP Access
      • Source: Desktops
      • Destination: ANY
      • Service:
        • HTTP
        • HTTPS
      • Context Profile: None
      • Action: Drop
  7. Click PUBLISH

The rule order MUST match the order seen in the screenshot below

  1. In the currently opened browser tab for your SDDC vCenter, login if required (See the setting tab or your worksheet for vCenter credentials)
  2. In the inventory select the Win10-Desktop VM. (Power it on, if it is Powered-off)
  3. Click LAUNCH WEB CONSOLE
  4. Select the Win10-Desktop Browser tab
  5. Click Send Ctrl+Alt+Deletein the upper right-hand of the screen
  6. Log into the VM as:
    • student
    • VMware1!
  7. Launch a browser from the Desktop
  8. Try accessing www.google.com, and www.vmware.com
  9. Try accessing any other website

Task 3.3 - Cleanup

  1. Go back to the SDDC NSX Manager UI
  2. Under Security tab click Distributed Firewall
  3. Expand the FQDN Whitelist Policy
  4. Select the Block All HTTP Traffic rule
  5. To the far right of the rule move the slider to the left to disable it
  6. Click PUBLISH
Task 4 - Distributed IDS/IPS

VMware NSX Distributed IDS/IPS provides security operators with a software-based IDS/IPS solution that enables them to achieve regulatory compliance, create virtual zones and detect lateral movement of threats on east-west traffic.

With the rise of distributed applications and micro-services withing VMC on AWS, internal network traffic now dominates traditional north-south traffic. At the same time, the SDDC boundary has diffused with edge and cloud applications as well as with end-user devices. Modern-day attackers noticed these changes and learned to move laterally, aggressively, from their initial point of attack. As a result, inspecting internal east-west (server-to-server) traffic with an advanced threat detection capability is increasingly critical to securing workloads and enterprise data in VMC on AWS.

Task 4.1 - Deploy Infection Monkey virtual Appliance

Infection Monkey is an open source breach and attack simulation (BAS) platform that allows organizations to discover security gaps and fix them. You can Simply infect a random machine with the Infection Monkey and automatically discover your security risks. Test for different scenarios - credential theft, compromised machines and other security flaws. We start by deploying Infection Monkey and we'll later use it to exploit a vulnerability.

  1. From the VDI desktop open your SDDC vCenter browser tab and log in
  2. Click vSphere Client  to access Shortcuts --> Content Library 
  3. Select the VMC Content Library
  4. Select the Templates -> OVF & OVA Templates tab
  5. Right-click on MonkeyIsland
  6. Click New VM from This Template
  1. Provide the following values in the New Virtual Machine from Content Library Wizard
    • Virtual Machine name:  MonkeyIsland01
    • Location:   Workloads
    • Compute resource:  Cluster-1--> Compute-ResourcePool
    • Storage:  WorkloadDatastore
    • Network: Desktop-Net
  1. Monitor the deployment progress, once the MonkeyIsland  appliance is deployed,
    Click the Hosts & Cluster view.
  2. Expand the Cluster --> Compute-ResourcePool
  3. Right-click MonkeyIsland01 choose Power --> Power-on
  1. Once powered-on, record the IP Address of the MonkeyIsland appliance

You will use this IP later to invoke the exploit.

Task 4.2 - Enable and Configure Distributed IDS/IPS

  1. In the VMware Cloud on AWS portal click the OPEN NSX MANAGER button
  2. Click ACCESS VIA THE INTERNET to connect to NSX Manager UI
  3. Click on Security tab
  4. Click IDS/IPS
  5. Click the Settings tab of IDS/IPS
  6. Click the IDS/IPS Sub-tab
  7. Click the Auto Update slider
  1. Click the Shared sub-tab
  2. Check Cluster-1
  3. Move the slider Enable to enable Distributed IDS/IPS for Cluster-1
  4. Click YES to confirm you want to Enable D-IDS/IPS

Now, we'll create an IDS/IPS profile to use with an IDS/IPS rule

  1. Click the Profiles tab under Distributed IDS/IPS
  2. Click ADD PROFILE
  3. Name: Desktop Profile
  4. Leave the default settings for every other option
  5. Click SAVE
  1. Click the Distributed Rules tab under Distributed IDS/IPS
  2. Click ADD POLICY
  3. Name the Policy Desktop IDPS Policy
  4. Select the Policy
  5. Click ADD RULE
  6. Set the Rule as follows
    • Name: Desktop Exploit Detection
    • Source: SDDC-Workloads
    • Destination: Any
    • Services: Any
    • Security Profiles: Desktop Profile
    • Mode: Detect Only
  7. Click the Gear at the far right of the rule
  8. Move the slider to Enable Logging
  9. Click APPLY
  10. Click PUBLISH
  11. Click the Events Tab, we'll return here later to review discovered exploits

Task 4.3 - Execute workload Exploits

We will now use Infection Monkey to inject some exploits.

  1. From the chrome browser tab for the SDDC vCenter, login if required
  2. Select the MonkeyIsland01 VM
  3. Confirm the VM is powered-on and take note of the IP address if you hadn't done so previously
  4. Select Win10-Desktop
  5. Take note of the IP Address
  6. Click LAUNCH WEB CONSOLE
  1. In the browser tab for Win10-Desktop Console, login as:
    • student
    • VMware1!
  2. Launch Chrome from the Win10-Desktop
  3. In the address bar type, https://<MonkeyIsland_IPAddress>:5000
  4. login as:
    • monkeyuser
    • VMware1!
  5. Click Configure Monkey
  1. Click the Network tab
  2. Change the Scan Target list IP address to the <IP address of your Win10-Desktop VM>
  3. Click Submit (scroll down)
  4. Click Submit
  1. In the left pane, click Run Monkey
  2. Click From Island
  3. Click Infection Map, to watch Infection Monkey run its exploits.
  4. Click the VDI browser tab for the SDDC NSX Manager UI
  5. From Security tab in IDS/IPS on the Events tab click refresh to review the Intrusion attempts
  6. Expand one or more of the events to review the details.

The exploits take up to 10 mins to run

None of the exploits will be successful against the Desktop VM as it isn't running those services, but the attempts will still be picked up by distributed IDS/IPS.

When setting up IDS/IPS policies you first want to use the “Detect only” action to identify all possible exploits and the possible vulnerable systems. Once you have a good handle on what could be exploited you can then move to tune the Policies and rules and as part of that start to use the “Detect and Prevent” action. In essence, starting with “Detect and Prevent” action could have unwanted and even disastrous effects on a production environment without 1st exploring the necessary due diligence.

Many customers tend to implement only a handful of "Detect and Prevent" rules and typically they would be tied to their critical systems for known exploits, while having Detect only rule for most signatures. This allows them to better identify and address false-positive that could impede production activities.

Task 4.4 - Review Intrusion Event In Log Insight

vRealize log Insight Cloud is a platform capable of bringing log data from your entire environment together, no matter where it resides, and extracting meaning from it. The solution brings order to the chaos of millions of unstructured data points, turning raw log data into actionable insights that can help you address both security and operational issues.

Log Insight Cloud contains built-in knowledge and native support for VMware SDDC technologies—from VMware vSphere, NSX-T firewall, VMware Cloud on AWS logs, AWS Services, to 3rd party technologies such as Docker, MS SQL, Apache and many more.

Log Insight Cloud can also capture and Analyze VMC on AWS Intrusion events detected by the NSX advances security Add-on. All you need to do is enable Logging in your Distributed IDS/IPS rule(s). In Task 4.2 you enabled logging for your IDS/IPS rule, now we will review the log entry in Log Insight.

  1. In the SDDC Console, click Activity Logs
  2. Click the vRealize Log Insight Cloud Link
  1. In the Left Pane Click Dashboard
  2. Click NSX - IDPS Traffic to view the dashboard. This will display the top sources, destinations, signatures, etc...
  3. In the upper-right, click 6H to adjust the time scale. The Dashboard defaults to the past 5 minutes.
  1. Repeat the steps above to view the NSX - IDPS - Overview Dashboard

Conclusion

The NSX Advanced Distributed Security for VMware Cloud on AWS workloads ensure workloads are secure and compliance goals are met. NSX Advanced Firewall for VMware Cloud on AWS customers provides layer 7 distributed security that scales linearly with VMs, with no blind spots during network traffic inspections. With The NSX Advanced Firewall enabled, you can make use of:

  • Distributed Firewall with Layer 7 Application ID - Deep Packet Inspection built into the hypervisor with built in profiles for common enterprise applications.
  • Distributed Firewall with Active Directory based User ID - Per user and session application access control with an Identity Firewall
  • Distributed Firewall with FQDN Filtering - Permit or deny communication to specific destinations in the Internet.
  • Distributed Firewall with Active Directory based User ID - Per user and session application access control with an Identity Firewall.

0 Comments

Add your comment

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Previous Article Lab 04 - On-Premises integration with VMC on AWS
Next Article Lab 06 - Inter-SDDC and Native VPC Connectivity

Message

VMware Cloud Expert: VMC on AWS (4-day) - VMCEXPERT1

Day 1 4
Day 2 3
Day 3 4
Day 4 3
Download Article PDF Print Article