VMware Cloud Expert

Lab 5 - Log Management

Updated

Introduction

vRealize Log Insight Cloud is a part of the VMware Cloud suite of services. Use this service to develop sophisticated analytics that aid in rapid troubleshooting of your SDDC or VMware Cloud on AWS environment..

As part of the VMware Cloud suite of services, vRealize Log Insight Cloud (formerly known as VMware Log Intelligence) provides a fully managed and integrated log analytics and troubleshooting service.

vRealize Log Insight Cloud includes VMware-authored SDDC (ESXi, VC, NSX, and VSAN) insight for troubleshooting, a flexible and comprehensive query facility that supports troubleshooting for novice and experienced administrators, built-in SDDC and custom alerting capability, flexible notification mechanisms, and centralized support for local or federated authentication.

Setting up Log Insight Cloud for non- Cloud services organization

Setting up Log Insight Cloud for VMware Cloud Subscribers

TASKS

Task 1 - Enabling Firewall logging and generating log entries
  1. From your VDI Desktop, open the browser and log into your VMC on AWS SDDC
    https://vmc.vmware.com/console/sddcs
    • Username: vmcexpert#-xx@vmware-hol.com
    • Password: VMware1!
  2. On your SDDC Tile Click View Details
  3. Click the Networking & Security Tab
  4. Click Distributed Firewall
  5. Expand the Default Layer3 Section Policy
  6. Select the Gear in the far right corner of the Default Layer3 Rule (Last Rule)
  7. Move the slider next to logging to enable logging
  8. Click Apply
  1. Click Publish
  2. Click Gateway Firewall
  3. Select the Gear in the far right corner of the ANY-ANY Rule (first Rule)
  4. Move the slider next to logging to enable logging
  5. Click Apply 
  6. Click Publish
  7. Click Open vCenter 
  8. Click Show Credentials
  9. Copy the Password and Click Open vCenter
  10. Log into vCenter as:
    • cloudadmin@vmc.local
    • {Paste in the copied Password}
  11. In vCenter, Click the frontend VM and record its IP address
  1. Select the Ubuntu-DT VM and click Open Console
  2. Enter the Password of VMware1! if prompted
  3. Launch the Firefox browser in the Ubunt-DT VM and type in {the address of your Frontend VM} for the Cats & Dogs Application
  4. Click the Gato & Cachorro buttons multiple times until the image that appears is a hedgehog.
  5. When this image (hedgehog) appears an error is generated and logged with log-insight and captured by vRealize Operations.  You are also performing this step to generate some firewall logs
  6. In a new Browser tab, go to VMware.com and one or more public websites
Task 2 - Log Insight Cloud Overview
  1. Click the stacked squares in the upper right-hand corner
  2. Right-Click VMware vRealize Log Insight Cloud
  3. Click Open link in new tab
  1. If Collapsed, Click the double arrows to expand the left pane
  2. Expand the Configuration Section
  3. Click Subscriptions
  4. Notice we are using the Free Subscription. This subscription comes with VMC on AWS and only allows you to view Audit logs and Firewall Logs.
    To view other types of logs (Application logs, non-SDDC logs, etc..) the subscription must be upgraded.
  1. Click Dashboards to view the available dashboards
  2. Select the Gateway Firewall Dashboard
  3. Review the Pie Graphs for Top Sources and Top Destinations
    You'll see an aggregated and processed view of all network traffic leaving the SDDC in the past 5 mins.
  4. In the Upper right corner note that you can increase the time scale to review date beyond the last 5 mins

All NSX related log events must first be enabled before those log messages will be sent to Log Insight. E.G. If you want to see DHCP, NAT or VPN related log messages then you must enable logging for those services as we did in task 1 for our firewall rule.

  1. Click Log Sources to see all of the endpoints Log Insight can natively pool and analyze logs from.
  2. Click Alert --> Alert Definitions to review he built-in alerts
  3. Select the Distributed Firewall Rule Created Alert definition
  4. Review the settings and notice there is currently no notification set when this alert is triggered.
  5. In the upper right hand, click the edit Icon
  6. Input Your Email address in the notify field of Trigger Condition 1 and click the +
  7. Move the slider to enable the alert
  8. Click Save
    Whenever a new firewall rule is created you will be notified.

Note: You can also create custom alerts.

  1. In the left pane, click Content Pack to review the available content packs for Log Insight Cloud
  2. Notice that not all content packs are enabled. Enabling a content pack allows Log Insight to begin processing log messages for the system
  3. If Disabled, Enable the content packs for:
    • Audit Events for VMware Cloud SDDC (v2)
    • General
  1. Expand Applications
  2. Enable Apache - HTTP Server and Nginx
  3. Expand Others
  4. Enable the Linux and Linux - Systemd Content packs
  1. Click on Dashboards, You'll now notice additional dashboards (Activity, Alerts, etc...)
Task 3 - Application Logs
  1. In the log Insight cloud interface expand Configuration
  2. Click Cloud Proxy
    If there are any existing inactive Proxies, click Delete and confirm the deletion to remove them
  3. Click ADD Proxy
  4. Click Existing
  5. Select the vRA Proxy you deployed earlier
  6. Click Add
  1. In the left pane under the Configuration section, click vRLI Agent
  2. Under Agent Configuration add configuration settings for
    • MongoDB
      • Directory: /var/log/mongodb
      • include files: *.log
    • syslog
      • Directory: /var/log
      • Include files: *.log
    • docker
      • Directory: /var/lib/docker/containers
      • Include files: *.log
  1. At the top of the page, in the Agents search field, click the drop-down and select Create New Group
  2. Name the Group Linux_XX (Linux_01) matching your student number
  3. Click OK
  1. Configure the Group as follows:
    • Filter
      • OS
      • Starts with
      • Ubuntu
  1. Under the Agent Configuration section add the following configuration settings:
    • Under General Click New, and add the general section logging
      • Set the logging level to verbose
    • Click Save
      • Under Common Click New, and add the section global
    • Click Save
  1. In the Agent Configuration section, Under Parsers, Let's create and define 4 parsers:
    1. Next to Parsers Click New, name the section syslog_appname_parser
      • Set the Parser to use/extend to CLF (default Common Log Format)
      • Set Format to:
%{appname}i[%{thread_id}i]

Click Save

  1. Next to Parsers Click New, name the section syslog_parser
    • Set the Parser to use/extend to CLF (default Common Log Format)
    • Decode Field click Add
      • set the field to appname
      • set the value to syslog_appname_parser
    • Set Format to:
%t %i %{appname}i: %M

Click Save

  1. Next to Parsers Click New, name the section auth_Parser_sles
    • Set the logging level to CLF (default Common Log Format)
    • Set Next Parser to syslog_parser
    • Set Format to:
%t %i %{appname}i[%{thread_id}i]: password changed - account=%{linux_user}i, uid=%{uid}i, %i

Click Save

  1. Net to Parsers Click New, name the section auth_Parser
    • Set the logging level to CLF (default Common Log Format)
    • Set Next Parser to syslog_parser
    • Set Format to:
%t %i %{appname}i[%{thread_id}i]: password changed - account=%{linux_user}i, uid=%{uid}i, %i

Click Save

  1. In the Agent Configuration section, Under File Logs, Let's create and define 4 log configurations:
    1. Next to File Log Click New, name the section auth
      • Set the Directory to: /var/log
      • Set Include files to: auth.log;auth.log.?
      • Tag Field click Add
        • set the field to vmc_cp
        • set the value to linux
      • Set Parse fields by to: auth_parser

Click Save

  1. Next to File Log Click New, name the section messages
    • Set the Directory to: /var/log
    • Set Include files to: messages;messages.?
    • Tag Field click Add
      • set the field to vmc_cp
      • set the value to linux
    • Set Parse fields by to: syslog_parser

Click Save

  1. Next to File Log Click New, name the section syslog
    • Set the Directory to: /var/log
    • Set Include files to: syslog;syslog.?
    • Tag Field click Add
      • set the field to vmc_cp
      • set the value to linux
    • Set Parse fields by to: syslog_parser

Click Save

  1. Next to File Log Click New, name the section docker
    • Set the Directory to: /var/log
    • Set Include files to: docker
    • Tag Field click Add
      • set the field to vmc_cp
      • set the value to linux
    • Set Parse fields by to: syslog_parser

Click Save

  1. At the top of the vRLI Agents page choose All Agents in the search field

You should now see the agents from your Cats & Dog application, there would also be more than 10 events recorded

Conclusion

0 Comments

Add your comment

E-Mail me when someone replies to this comment

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.