Tl;dr: Log4j is a mess, if you’re chasing down the applications, services and servers that use Java; consider the suggestions below to make zero day patching easier.
A high severity vulnerability (CVE-2021-44228) impacting multiple versions of the Apache Log4j 2 utility was disclosed publicly via the project’s GitHub on December 9, 2021. The vulnerability impacts Apache Log4j 2 versions 2.0 to 2.14.1.
Configuration Managers across the world earned their hard work last week as one of the most widespread 0-day exploits was announced. If your team had a mature configuration management practice, a well honed CMDB and some basic dependency mappings; you may already be done patching!
If not, you are probably putting out fires for a couple weeks and might be looking to develop tools to make this easier going forward.
How We Got Here
To say that application development is complicated would be an understatement. Simply building a login screen creates a ton of foundational questions. Should we use a third party tool? Open Source? Build it ourselves? What language and frameworks should we use?
The answers to these questions determine the tools, architecture and decisions you and your teams will make as the application matures and grows. Sometimes you will choose to build features and components, and sometimes you will choose to buy.
One of the foundational decisions the developers will make is how to manage logging. Application logs help developers identify bugs, troubleshoot performance and track application activity. Since application logging is almost completely hidden from customers, it makes sense that a lot of people use a third-party tool which is pre-built and easy to implement.
Finally, an open source logging platform is likely to capture a majority of this market, especially if it is free. Combine this fact with a widespread application platform like Java and you’ve got a high likelihood of widespread adoption.
How We Will Get Out
This exploit isn’t particularly difficult to patch. Some simple pattern matching fixes the problem. The real problem is when we are dependent on vendors to patch the vulnerability and they are either not able to, or they aren’t able to do it in a timely manner. So your services and applications that depend on software vendors who actively leverage Log4J will need to be identified, patched, retired or assessed to determine the next course of action.
How to Make it Easier Next Time
It may already be obvious, but the best way to patch up zero day exploits is to have a clear picture of your environment. A database of the applications, servers and services IT provides is just the start. Valuable though this database might be, the real value is realized when these items are related to one another. For example, mapping which servers have Java installed has now become critical information. A quick query and you can identify all the machines that are at risk.
Figure 1: A Configuration Management Database (CMDB) is one of easiest ways to begin tracking complex information about physical and virtual components.
If your teams and processes can connect the components that combine to deliver the service, then you now have a prioritized list of not only the servers and applications that need patching and inspection; you also have a way to prioritize that work (most critical services first).
Figure 2: A Service Component Map like this one from InvGate gives IT staff a quick and easy way to pinpoint problems, prioritize work and understand complexity.
What We Have Learned
Zero day exploits are going nowhere. As applications continue to get more complex and have more dependencies, accurately representing systems and services will become more important. The work to build the data may take a while, but it’s likely that your governance team(s) have some tricks up their sleeves.
Once your critical services are accurately documented and mapped, protect that data through change, release and project management processes. This way, when the next exploit is announced, you’ve got the data in place and you can trust it!
Figure 3: We could build a graphic to demonstrate how the CMDB data is protected by Change Management (which is a child to Release Management which is a child to Project Management).
There are tools to help this go faster. From agents that dial-home and report applications installed to network discovery tools; there are plenty of tools to help make this simpler. If you don’t have a tool to capture and document this information, try using InvGate Insight for free today!