SecureState Blog

Read SecureState's award winning blog.

A critical step in Defense Readiness

A critical step in Defense Readiness

As we progress in our discussions about the 2014 Top Attack Vectors, we come to systems that are unpatched.  Every System Administrator has dreaded the task of updating systems, the fear of executing a patch that is incompatible with an installed piece of software, the time it takes for everything to complete, weekends lost, and the list continues to grow.  However, we rarely think about what patching truly does.  Nor do we consider what the best approach to patching everything, from a single system to an entire data center, would be.  What remains true throughout the industry is that the action does not count, but rather the prep work is what makes for a successful patch deployment (or a dreaded weekend failure).

How do we build a patch management solution, get our weekends back, while still maintaining secure systems?

The following are key areas we need to take a look at in order to achieve these goals:

1. System Scope
2. Testing Environment
3. Change Management
4. Patch Deployment
5. Vulnerability Scanning

Fixing Unpatched Systems

 

We might think this process begins on the date that the patch is released, but this is far the truth.  Our work really begins well before the system is built and configured!

1. System Scope

Defining the system scope begins by sitting down with the system administrator, manager, director, and chief information officer to outline the purpose of the system types and roles; which needs to include a baseline configuration for each system in each role.

What will be the role of this system?

What applications run on each system?

What systems are mission critical, mission sensitive, business critical, general operations, and testing?

Once this system scope has been defined and documented, you can move onto the next step in the process.  The lack of a defined scope to cover these type of systems, roles these systems play and the category these systems reside in will result in a flawed patch management process.

2. Testing Environment

When designing and building the test environment, three key characteristics must be considered in order to be successful.

Is the environment flexible?

An environment which is flexible must have the ability to serve multiple roles.

How dynamic is the environment?

If an environment is dynamic then it serves to change the role of these test systems quickly.  If you have the flexibility to change the environment, but it takes days to build a test system with a specific roles, it fails to be dynamic.

Is the environment accurate to the production environment?

Most important, in my opinion, and critical to the success of any patch management solution is to build the environment to be an accurate representation of your production environment.  Lacking accuracy in the baseline and installed applications of a system being tested results in the inability to properly test patches and find issues before rolling these patches out onto the production environment.

Virtualization. Virtualize your test environment by cloning the production systems about to be patched using “Physical-2-Virtual” (P2V) technology, based on the defined scope.  Prior to testing the system, take a snapshot.  If the patch conflicts with a piece of software or setting, or totally destroys the system you can revert back to the snapshot and continue researching potential solutions.  Virtualization is the most vital element within a test environment, the cost verse reward is incredible when compared to the time and money saved if you have to recover a system after a failed weekend of patching.

3. Change Management

We now come to change management, often talked about but just as often forgotten.  When implementing change management it is key to have this process in place prior to the installation of patches, in the testing environment or production.  The most important aspect is to document the steps for the installation, but more importantly the back out plan in the event that something goes wrong.

4. Patch Deployment

All the hard work is done, you are moving on to actually installing the patch or application update.  Taking good notes during this stage is critical!  While all the testing has been completed, there is a slight chance for problems.  Prior to the actual rollout we must confirm the maintenance window and patching cycle, which will incorporate not only the time and day of the week you deploy patches, but also in what order the systems are patched, taking into account the defined system scope and security priority of the patch or update.

However, the process is not yet complete!

5. Vulnerability Scanning

Once the deployment of the patches or updates are completed we must monitor these systems for any operational issues, as well as verifying that the patches or updates haven’t created any vulnerabilities.  In order to accomplish this task a vulnerability scanner is used, such as Nessus.  You can not only use Nessus to verify the patch or update was installed successfully you can see if it actually fixed the issue it was supposed to resolve.

All these steps combined develops the methodology needed in order to maintain a successful patch management program.  While every organization is slightly different, understanding this methodology will not only save time and reduce fears of updating systems, it will dramatically reduce time and resources required.