anacron run-parts generates invalid or malformed syslog messages

On RedHat and CentOS 6 and 7, anacron generates syslog messages that are mangled when they are forwarded by rsyslog. I found the cause and a solution in a comment by Tomas Heinrich on this old Fedora bug. Sadly, that bug was closed instead of getting fixed, probably because Fedora is oriented toward desktop users who are not generally forwarding syslog messages! Syslog uses the following default template for messages:

"< %PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"

The syslogtag field is truncated to 32 characters, but cronie-anacron includes a lot of data in its tag. The tag gets truncated to 32 characters, resulting in invalid syslog. If you try to send them to Logstash, you’ll get errors like this:

{"message":"<77>Nov 4 19:01:01 my-web-03 run-parts(/etc/cron.hourly)[1858 starting 0anacron","@version":"1","@timestamp":"2016-11-05T00:01:01.146Z","port":55456,"type":"syslog","tags":["_grokparsefailure"],"host_ip":"192.168.17.23"}

Continue reading

Failed to schedule Software Protection service for re-start: Error Code: 0x80041316

This post documents how to resolve the following error message, which may appear for no particular reason, and flood the Windows event log:

Failed to schedule Software Protection service for re-start at 2116-09-14T16:54:27Z. Error Code: 0x80041316.

The following steps will resolve the error on Windows Server 2012r2, but based on other links, similar steps should work for Windows 8, 8.1, or Server 2012.

Root Cause

The root cause, in my case, was a corruption in the XML files that control task scheduling for the Software Protection service. I am not sure what caused this error, but it appeared to start after a domain controller was not shut down cleanly.

Microsoft’s official documentation reports that another cause may be a mismatch between the permissions used to run the task and the permissions on the files that control the task. However, most people have reported that the problem is caused by corrupted XML files. Also, note that the Microsoft documentation is inaccurate when it states that the task must run as the NETWORK SERVICE account. If the task is triggered by an interactive user logon, it should run as account Interactive.

Continue reading

Making Windows work for Linux and OS X admins

If you are a Linux or OS X power user, then you’re used to having all the necessary tools built into your OS. When you log into a Windows system (What! No command line?) you may feel lost. These tools and shortcuts will help you be productive on Windows systems.

Windows Shortcuts

  • Alt-x is a magic shortcut key on Windows 8 and Server 2012. It pops up a little menu in the lower right corner of the screen which contains just the items that an admin needs. Try it!

Continue reading

Microsoft Baseline Security Analyzer Troubleshooting

Microsoft Baseline Security Analyzer (MBSA) is a useful tool for auditing the configuration and update status of Windows computers. Most of the time, its reports are useful and easy to understand. However, some of its responses are baffling, and some of its suggested solutions haven’t been updated since Server 2003. Here is my collection of odd MBSA reports, and how to resolve them.

Internet Explorer

MBSA Reports:

The use of Internet Explorer is not restricted for administrators on this server

Solution:
Enable IE Enhanced Security Configuration in the Server Manager
Continue reading

Troubleshooting SSSD, realm, kerberos, and SSH

SSSD (System Security Services Daemon) allows Linux systems (specifically, Red Hat, CentOS, and Fedora) to verify identity and authenticate against remote resources. If you have a CentOS or Red Hat enterprise system, and you need to authenticate against a domain controller such as FreeIPA or Active Directory, SSSD is the way to go. I use SSSD on CentOS 7 systems, but it is now available on CentOS 6 as well. A few years ago, adclient (an open-source project from Centrify) was your only option to make a CentOS 6 server authenticate against Active Directory. adclient seems to have reached end of life, so SSSD is definitely the path forward.

I won’t repeat the procedure for using Active Directory as an identity provider on a Red Hat 7 system. Instead, I want to provide a few troubleshooting tips, since limited information is available on SSSD and related tools.
Continue reading

LVM device-mapper: reload ioctl failed: Invalid argument

LVM2 (Logical Volume Management) is pretty amazing, but when something goes wrong, it’s not easy to troubleshoot. This is not the fault of the tools, but a reflection that LVM is relatively new in Linux, and not widely understood.

What I Tried to Do

I tried to increase the size of a logical volume with the lvextend command:

lvextend --extents 100%FREE /dev/VolumeGroup1/var

This form of the command is supposed to use all of the free space in the volume group.

What Failed

The command responded with an error message that contained this text:

device-mapper: reload ioctl failed: Invalid argument

Running lvdisplay showed the following status for the volume:

LV Status    suspended

Continue reading

Is it worthwhile to embed structured data in Web content?

Short answer: not really!

A few years ago, I was fascinated with the idea of microformats. The concept is to add structural data to common HTML tags so that certain types of data on a web page can be clearly identified. Adding structural information makes it easier to automate the parsing and indexing of web content. The original microformats included common data sets that contain well-defined data, such as contact information or a calendar event. In the intervening years, microformats were replaced by microformats2, with additional draft standards for more items such as recipes, resumes, etc.

I decided to use my resume as a “use case” for structured HTML data. My original plan was to use the h-resume draft specification from microformats2. I modified my plan when I realized that major search engines (Bing, Google, Yahoo! and Yandex) have decided to parse another semantic technology, microdata (microdata W3C spec). There is no point in creating semantic web pages if nobody is going to parse them, so I used the microdata specifications that are available at schema.org and placed them into an h-resume framework.  I used the event class to represent each position I held or educational achievement, because the event class is the only one with start and end dates. Continue reading

Monitoring with SNMP, Part 3: Automate active monitoring with Nagios

My last post showed how to monitor networked devices with SNMP. You could try to remember to manually check the status of things periodically, but that would be missing the point of computers. Instead, automate your monitoring with Nagios, a web-based monitoring tool for Linux that automates the process of actively querying devices and doing something with the information. Nagios is available as free open source software (Nagios Core), and the company offers additional non-free products with premium features. The open-source version is fine for getting started and setting up basic monitoring. Nagios does a lot more than just SNMP monitoring. I’ll refer you to the Nagios Core documentation to get Nagios up and running, and I’ll focus on how to set up Nagios to actively monitor devices with SNMP.
Continue reading

Monitoring with SNMP, Part 2: Command-line tools for active SNMP

In Part 1, I summarized the basic concepts of SNMP and defined the terms and acronyms used in this post. Now, I will show how to use SNMP to monitor actual devices. As an example, I will monitor an enterprise-grade uninterruptible power supply (UPS) and power distribution unit (PDUs) from Tripp-Lite. These devices have an SNMPWEBCARD installed to support communication over Ethernet.

Command-line tools for SNMP communication should be available for any Linux distribution (or any other UNIX-derived OS). Documentation for the basic SNMP tools is available online. The challenge with SNMP is figuring out what parameters are supported by a particular device. Most devices support a set of standard OIDs that return basic information such as device name, uptime, etc.
Continue reading

Monitoring with SNMP, Part 1: Fundamentals of SNMP

SNMP is a protocol for conveying information and controlling devices over a network. SNMP can be used in two ways:

  • Active: a device sends a command to set a parameter or request information for another device
  • Passive: a device sends an alert (called a trap) to another device, which is configured to receive traps and do something with the information.

The “payload” of an SNMP message is called an Object Identifier, or OID. An OID is an ordered list of non-negative numbers, such as:

1.3.6.1.2.1.1.3.0

The sequence is hierarchical, starting with the highest-level object and progressing to lower-level objects. The above sequence corresponds to:

iso(1) org(3) dod(6) internet(1) mgmt(2) mib-2(1) system(1) sysUpTime(3) 0

When this command is sent to a device, it will return the uptime of the device.

The translation between the numerical sequence and the human-readable form is stored in a text file called a Management Information Base, or MIB. The format of the MIB is defined in RFC 2578. Some MIB files are standard and contain object IDs that are recognized by almost all devices. Device manufacturers also provide custom MIB files in which they define specialized object IDs for a particular device. Unfortunately, some devices don’t have MIB files, and you will have to query the device to see what objects it supports and decipher what they mean.

In Part 2 of this series, I will use active SNMP to monitor infrastructure.