Table of contents
This is a continuation of Part 1 you can find here:
Intro
In post PyPaloCleaner – Part 1 I mentioned two use cases: first was the address objects in security rules and the second was address objects and groups in security rules. I covered first use case and now it is time to take care of address groups.
This is a code we developed in part 1:
# Import Section
from panos.firewall import Firewall
from panos.objects import AddressObject, AddressGroup
from panos.policies import Rulebase, SecurityRule
import datetime
start = datetime.datetime.now()
# Login Details
HOSTNAME = '10.20.30.94'
USERNAME = 'bot'
PASSWORD = 'BotPassword123'
# Initiate firewall object
fw = Firewall(HOSTNAME, USERNAME, PASSWORD)
# Retrieve Data from the firewall
original_objects = AddressObject.refreshall(fw, add=False)
rulebase = fw.add(Rulebase())
security_rules = SecurityRule.refreshall(rulebase)
# Cleaning Section
unique_objects = original_objects.copy()
duplicates = []
duplicates_replacement = {}
for a in unique_objects:
for b in unique_objects:
if (a.value == b.value) and (a.name != b.name) and (a not in duplicates):
duplicates.append(b)
duplicates_replacement[b.name] = {"Value": b.value, "ReplaceWith": a.name}
fw.add(b)
unique_objects.remove(b)
# Fix for a problem in duplicates_replacement
for k, v in duplicates_replacement.items():
if a.value == v['Value'] and a.name != v['ReplaceWith']:
v['ReplaceWith'] = a.name
# Check rules for duplicates usage
modified_rules = []
for rule in security_rules:
for duplicate in duplicates:
if duplicate.name in rule.source:
print("Duplicate '{}' detected as a source in the rule '{}'".format(duplicate.name, rule.name))
if duplicates_replacement[duplicate.name]['ReplaceWith'] in rule.source:
rule.source.remove(duplicate.name)
else:
rule.source[rule.source.index(duplicate.name)] = duplicates_replacement[duplicate.name]['ReplaceWith']
if rule not in modified_rules:
modified_rules.append(rule)
if duplicate.name in rule.destination:
print("Duplicate '{}' detected as a destination in the rule '{}'".format(duplicate.name, rule.name))
if duplicates_replacement[duplicate.name]['ReplaceWith'] in rule.destination:
rule.destination.remove(duplicate.name)
else:
rule.destination[rule.destination.index(duplicate.name)] = duplicates_replacement[duplicate.name]['ReplaceWith']
if rule not in modified_rules:
modified_rules.append(rule)
# Apply Section
if len(modified_rules) > 0:
modified_rules[0].apply_similar()
print("*** Applied Changes to the Security Rules. ***")
if len(duplicates) > 0:
duplicates[0].delete_similar()
print("*** Removed duplicated address objects. ***")
print("### Cleaning took: {} ###".format(datetime.datetime.now() - start))
Our task for this part is to find duplicated object within the address groups.
Case Study Overview
In this use case we have 12 address objects, among which 4 are duplicates.
There are two address groups
And the groups are used in BeanstalkStartSystem security rule:
Development
First of all, we need to get address groups from the firewall and put them into a list. We will call it original_groups. Complete line should look like original_groups = AddressGroup.refreshall(fw, add=False). It can be put somewhere at the begining, before the code, that removes objects from the firewall. This means that the best place would be our # Retrieve Data from the firewall section.
# Retrieve Data from the firewall
original_objects = AddressObject.refreshall(fw, add=False)
rulebase = fw.add(Rulebase())
security_rules = SecurityRule.refreshall(rulebase)
original_groups = AddressGroup.refreshall(fw, add=False)
Let’s run part of our code in ipython:
# Import Section
from panos.firewall import Firewall
from panos.objects import AddressObject, AddressGroup
from panos.policies import Rulebase, SecurityRule
import datetime
start = datetime.datetime.now()
# Login Details
HOSTNAME = '10.20.30.94'
USERNAME = 'bot'
PASSWORD = 'BotPassword123'
# Initiate firewall object
fw = Firewall(HOSTNAME, USERNAME, PASSWORD)
# Retrieve Data from the firewall
original_objects = AddressObject.refreshall(fw, add=False)
rulebase = fw.add(Rulebase())
security_rules = SecurityRule.refreshall(rulebase)
original_groups = AddressGroup.refreshall(fw, add=False)
…and check what is inside of our original_groups list:
We can see original_groups is a list with two items, which are panos class objects. Let’s unpack Rose groups from the original_list and assign it to rose variable and see what we can do with it:
Oh! about() method is avaliable, let’s run it:
It returns some kind of dictionary with static values, dynamic values (depending on group type), description, tag and name. We need to focus on static value and name of the group.
A static_value of the rose group returns a list with names of address objects (type string). A name returns name of the rose group and it is a string type. So right now what we need to do is to iterate over original_groups and over our detected address duplicates in order see if duplicates are members of existing address groups. But first, run # Cleaning Section in ipython, in order to detect duplicates in address objects:
And now let’s run our new code:
We have a detection! Right now it is only printing out a label to the screen informing that duplicate.name was detected in og.name group. Now it is a good place to check, which object to use as a replacement for ‘Odysseus’ address. Run a query on duplicates_replacement dictionary:
A query returns information that ‘Clover’ address should be used as a replacement for ‘Odysseus’. Let’s embed our query into above for loop.
That is better. Our query works within a loop but it is still printing just a label. Remove a print() function entry. We will now actually update our group:
In this piece of code we iterate over two lists, then we check if string is in a list. If the comparison result is True, we replace a duplicate in static_value with a replacement object name from duplicates_replacement. Finnaly we run apply() method to push modifications to the firewall.
Below is a combined code from part 1 and part 2. I highlighted lines developed in this part (2).
# Import Section
from panos.firewall import Firewall
from panos.objects import AddressObject, AddressGroup
from panos.policies import Rulebase, SecurityRule
import datetime
start = datetime.datetime.now()
# Login Details
HOSTNAME = '10.20.30.94'
USERNAME = 'bot'
PASSWORD = 'BotPassword123'
# Initiate firewall object
fw = Firewall(HOSTNAME, USERNAME, PASSWORD)
# Retrieve Data from the firewall
original_objects = AddressObject.refreshall(fw, add=False)
rulebase = fw.add(Rulebase())
security_rules = SecurityRule.refreshall(rulebase)
original_groups = AddressGroup.refreshall(fw, add=False)
# Cleaning Section
unique_objects = original_objects.copy()
duplicates = []
duplicates_replacement = {}
# Find Duplicated Address Objects
for a in unique_objects:
for b in unique_objects:
if (a.value == b.value) and (a.name != b.name) and (a not in duplicates):
duplicates.append(b)
duplicates_replacement[b.name] = {"Value": b.value, "ReplaceWith": a.name}
fw.add(b)
unique_objects.remove(b)
# Fix for a problem in duplicates_replacement
for k, v in duplicates_replacement.items():
if a.value == v['Value'] and a.name != v['ReplaceWith']:
v['ReplaceWith'] = a.name
# Find Duplicated Address Objects in Address Groups
for og in original_groups:
for duplicate in duplicates:
if duplicate.name in og.static_value:
og.static_value[og.static_value.index(duplicate.name)] = duplicates_replacement[duplicate.name]['ReplaceWith']
og.apply()
# Check rules for duplicates usage
modified_rules = []
for rule in security_rules:
for duplicate in duplicates:
if duplicate.name in rule.source:
print("Duplicate '{}' detected as a source in the rule '{}'".format(duplicate.name, rule.name))
if duplicates_replacement[duplicate.name]['ReplaceWith'] in rule.source:
rule.source.remove(duplicate.name)
else:
rule.source[rule.source.index(duplicate.name)] = duplicates_replacement[duplicate.name]['ReplaceWith']
if rule not in modified_rules:
modified_rules.append(rule)
if duplicate.name in rule.destination:
print("Duplicate '{}' detected as a destination in the rule '{}'".format(duplicate.name, rule.name))
if duplicates_replacement[duplicate.name]['ReplaceWith'] in rule.destination:
rule.destination.remove(duplicate.name)
else:
rule.destination[rule.destination.index(duplicate.name)] = duplicates_replacement[duplicate.name]['ReplaceWith']
if rule not in modified_rules:
modified_rules.append(rule)
# Apply Section
if len(modified_rules) > 0:
modified_rules[0].apply_similar()
print("*** Applied Changes to the Security Rules. ***")
if len(duplicates) > 0:
duplicates[0].delete_similar()
print("*** Removed duplicated address objects. ***")
print("### Cleaning took: {} ###".format(datetime.datetime.now() - start))
Test
It is a time for script execution and validation of the results. We expect to have four duplicates removed from the configuration. Also in a Banana group a duplicate should be replaced with unique object.
Summary
As we can see in a Device Config Audit, script returned expected result.
In the next episode I will focus on detection of duplicated address groups and replacement of those duplicates (nested address groups).
If you are reading this sentence, you probably reached to the end of this post. Thank you for your attention! Leave a comment if you like 😉 Follow us on Twitter and Instagram.
Krzysztof.
Leave a Reply