| We moved this page to our Documentation Portal. You can find the latest updates here. |
Question
Can I use log management tools for monitoring the vCD system in OnApp?
Environment
OnApp with vCloud Director integration
Answer
OnApp has some very detailed, verbose logging set by default, which allows you to easily troubleshoot errors or just monitor the status of your vCD system within OnApp. A side-effect of this is that the log files can get very large – and therefore, it can be a good idea to use a logging tool to assist and parse the logs in a structured way.
The instruction below describes how to integrate the OnApp logs into a free, hosted log management tool called Papertrail.
Get Papertrail:
The first step would be to sign up for an account at Papertrail and log in to your dashboard. Here you need to add your first system. The system you will be adding at this point will be your OnApp controller, so log into that via SSH and run the commands that it lists. Alternatively, there is a one-line command you can run at the top of this page that will automate much of the initial setup. It should look something like:
wget -qO - --header="X-Papertrail-Token: xhxhASnX5nucRPFrH9XZ" \ https://papertrailapp.com/destinations/3809456/setup.sh | sudo bash
Once this has run, it will automatically add your OnApp CP to the systems. However, you need to configure ‘remote_syslog’ to ensure you can gather all of the OnApp logs that are required.
Install and Configure remote_syslog:
1. Download the latest remote_syslog release:
https://github.com/papertrail/remote_syslog2/releases
2. Unpack the download:
tar xzf ./remote_syslog*.tar.gz
3. Open the remote_syslog directory:
cd remote_syslog
4. Copy the remote_syslog application to the /usr/local/bin directory:
sudo cp ./remote_syslog /usr/local/bin
5. Download the following config file and edit the Papertrail host and port to be the same as you use in your dashboard:
https://github.com/jimfreeman/onapp_vcd_papertrail/blob/master/log_files.yml
This file should be located at /etc/log_files.yml
6. Download the following file and place it into /etc/init.d/remote_syslog:
https://github.com/papertrail/remote_syslog2/blob/master/examples/remote_syslog.init.d
7. Run the following command to give it the correct permissions:
chmod +x /etc/init.d/remote_syslog
8. Run the following command to make it launch on startup:
update-rc.d remote_syslog defaults
9. Start the service:
service remove_syslog start
From now, if you go to the dashboard of the system in Papertrail, you should see your live logs being processed.
Saving Searches and Alerts:
Now all of the logs are being processed by Papertrail, then the next step is to configure searches and alert email address(es) based on the search results. Several examples are provided below.
Daily Resync: Each night at 1 am, OnApp runs a resync of the OnApp and vCD environment to ensure all assets inside of OnApp are up to date and the same as in vCloud Director. To retrieve the search criterion based on this, use the following search query:
program:production_vcloud.log AND "Daily resync"
Performance metric tracking: By default, OnApp runs an L2 stats transaction every 3 minutes (this is also customizable). If this transaction does not run correctly, there will be blank entries in performance metrics for virtual servers within your environment. The following search criterion will output all L2 stat collections to make sure they have run successfully. We recommend setting the frequency of this alert to one day.
program:production_vcloud_stats_runner.log AND "[L2]"
Failures during import or resync procedure: This search criterion will look for any failures during the import, such as a user not importing due to not having an email address assigned, or a user being from an external directory source. You can then configure this to be sent to you via an alert and resolve any failures that have happened.
program:production_vcloud.log AND “failure”