Monday, July 23, 2018

The secrets of MTU - L2 MTU vs. L3 MTU - Where does the fragmentation happen?

"The Maximum Transmission Unit (MTU) is the largest possible frame size of a communications Protocol Data Unit (PDU) on an OSI Model Layer 2 data network." In today's network the standard MTU for Layer 3 IP packet is 1500 bytes. Meanwhile, the standard MTU for Layer 2 Ethernet frame is 1514 bytes ( 6 bytes source MAC + 6 bytes destination MAC + 2 bytes EtherType + 1500 bytes IP packet). For the Dot1Q trunk frame, extra 4 bytes for Dot1Q tag is added. So up to here, we understand that there are two types of MTUs. MTU for layer 2 frames and MTU for layer 3 packets.

In modern network, overlay encapsulation is more and more popular. Technology such as VxLAN requires switch fabric which has jumbo frame support. We may have a question, what happen if a switch receives a frame which has a larger MTU than it's local L2 MTU? The answer is that the switch will drop this over-size frame. Someone may ask why this frame cannot get fragmented? Actually the fragmentation is happening on the Layer 3. The DF bit is sitting in the IP header. So no fragment happens on the layer 2.


Click to Expand
So, every over-size L2 frame will be dropped. In order to avoid packet dropping, we should enable jumbo frame support in the switch fabric.

Ok, now we understand L2 MTU vs L3 MTU. Another interesting question comes up. When a L3 IP packet passing through a router, where the MTU will be checked and where the fragmentation will happen? The inbound interface or the outbound interface. In order to find out the answer of this question, let's do a simple Lab as the following topology.

Click to Expand
As shown in the above diagram, we configure G0/0 (inbound interface) on R2 as IP MTU 1300. All the other interfaces have standard IP MTU 1500. When we send a packet with IP Packet size 1500 from R1 to R3. We can see this packet is not been fragmented by R2. It arrives at R3 with the original packet size 1500.

Let's send another IP packet with size 1500 from R3 to R1. This time, we can see the packet has been fragmented into two by R2.

Click to Expand


by doing this lab, we understand that the IP MTU is checked by the outgoing interface of the router. If fragmentation is required, the packet is fragmented by the outgoing interface of the router. So if the DF bit is set, then the packet will be dropped on the outbound interface as well. 

Reference:








Sunday, July 22, 2018

Ansible REST API - Interacting with Cisco FirePower Management Center (FMC) - 05 - Request Access Token

This post belongs to my "Ansible REST API - Interacting with Cisco FMC" series. The following is the table of content of this series:
  1. Introduction and Ansible playbook download
  2. Script flow charts
  3. Introduction of REST API and Cisco FMC API Explorer
  4. Script prerequisites
  5. Request Access Token
  6. Get policy content, modify content and "PUT' in FMC - Part 1
  7. Get policy content, modify content and "PUT' in FMC - Part 2 
  8. Get deployable devices and deploy policy

In the last post, I described the prerequisites for our main task playbook. Now, we can start writing the main task yml file in "/etc/ansible/roles/FMC-enable-policyrule/tasks/main.yml".

First of all, we need to be authenticated by the FMC server. Different systems have different authentication methods for REST API calls. Some of them require user to provide username/password for every call. For FMC, it uses Token Based Authentication mechanism for API users. The following shows the message flows for obtaining and using the access token:


Click to Expand
The API user will initial a "POST" call to the Cisco FMC with username/password information (of course, this API call is encrypted by https). FMC will verify the login details and if the username/password is valid, it will reply the API user with the "x_auth_access_token" and status_code of "204" (regrading to the REST return code, please refer to my another post "REST API 101". 

In the following API calls, the API user can use this "x_auth_access_token" in the header field and no need to provide username/password again in the whole token life cycle.

location: /etc/ansible/roles/FMC-enable-policyrule/tasks/main.yml

---
  - name: POST
    uri:
      url: "https://{{FMC_IP}}/api/fmc_platform/v1/auth/generatetoken"
      method: POST
      user: "{{username}}"
      password: "{{password}}"
      validate_certs: no
      force_basic_auth: yes
      status_code: 204

    register: token


In this "POST" call, we specific the url, username, password and define "force_basic_auth" as "yes". By default, Ansible will use "200" as status return code if it's not specified. In our case, FMC will send "204" as return code so we need to specific it here as well.  

And the following shows the reply from the FMC server. And the highlighted line is the "x_auth_access_token" we need. 


 "token": {
        "accept_ranges": "bytes",
        "cache_control": "no-cache, no-store, must-revalidate, max-age=0",
        "changed": false,
        "connection": "close",
        "content_length": "0",
        "cookies": {},
        "date": "Thu, 21 Jun 2018 12:15:45 GMT",
        "domain_id": "111",
        "domain_uuid": "e276abec-e0f2-11e3-8169-6d9ed49b625f",
        "domains": "[{\"name\":\"Global\",\"uuid\":\"e276abec-e0f2-11e3-8169-6d9ed49b625f\"}, {\"name\":\"Global/Sydney\",\"uuid\":\"dd731f3e-8297-5b05-7ac3-000000000001\"}]",
        "failed": false,
        "global": "e276abec-e0f2-11e3-8169-6d9ed49b625f",
        "msg": "OK (0 bytes)",
        "redirected": false,
        "server": "Apache",
        "status": 204,
        "url": "https://192.168.1.205/api/fmc_platform/v1/auth/generatetoken",
        "user_uuid": "2bd20b46-708d-11e8-bc24-cd319511305b",
        "vary": "Accept-Charset,Accept-Encoding,Accept-Language,Accept",
        "x_auth_access_token": "a7103106-380d-4593-8184-044e8b2ea7fe",
        "x_auth_refresh_token": "042d27db-fa8e-435c-9250-3926f2841445",
        "x_frame_options": "SAMEORIGIN"
    }
}


Next, we need to set a new available ("acc_token") and assign this access token to it.

location: /etc/ansible/roles/FMC-enable-policyrule/tasks/main.yml


  - name: get token
    set_fact:
      acc_token: "{{token.x_auth_access_token}}"


In the next post, I will show you the most important part of this play book - how to modify the policy content of the existing access policy

Reference:

Sunday, July 15, 2018

Ansible REST API - Interacting with Cisco FirePower Management Center (FMC) - 04 - Script prerequisites

This post belongs to my "Ansible REST API - Interacting with Cisco FMC" series. The following is the table of content of this series:
  1. Introduction and Ansible playbook download
  2. Script flow charts
  3. Introduction of REST API and Cisco FMC API Explorer
  4. Script prerequisites
  5. Request Access Token
  6. Get policy content, modify content and "PUT' in FMC - Part 1
  7. Get policy content, modify content and "PUT' in FMC - Part 2 
  8. Get deployable devices and deploy policy

Before writing the main task script, we have to prepare couple of things:

1. Define the Ansible hosts file


We know that the Hosts file is the inventory file of Ansible. Here we define the Cisco FMC IP and it's hostname "Cisco_FMC".

location: /etc/ansible/hosts

[FMC]
Cisco_FMC ansible_host=192.168.1.205


2. Build the login detail files

In order to login to the FMC, we need to give the login details to the system. This is sensitive information. So we need to encrypted by the Ansible Vault.

First, we build the vault file under "/etc/ansible/group_vars/all/" by issue command "ansible-vault create vault" and the following is the file content:


location: /etc/ansible/group_vars/all/vault

---
  FMC_username: admin
  FMC_pass: cisco123


Please be noted that the above "vault" file is NOT only for this FMC script. It will be used for all of your Ansible scripts in this Linux host. In the next step, we will write a login detail file dedicated for our FMC playbook and reference the details from the "vault" file:

location: /etc/ansible/group_vars/FMC

---
  username: "{{FMC_username}}"
  password: "{{FMC_pass}}"


The file name "FMC" will be matching the "FMC" group in the hosts file. The values of {{FMC_username}} & {{FMC_pass}} will be obtained from the "vault" file. 

3. Build site yaml file

Here is the site YML file. It links the hosts "FMC" and the ansible role "FMC-enable-policyrule" together. 

location: /etc/ansible/FMC-enable-policyrule.yml

---
  - hosts: FMC
    gather_facts: no
    connection: local
    roles:
      - FMC-enable-policyrule


4. Initial the ansible role

Issue the command "ansible-galaxy init FMC-enable-policyrule" under "/etc/ansible/role" folder.

5. build the var file

location: /etc/ansible/roles/FMC-enable-policyrule/vars/main.yml

---
  FMC_IP: "192.168.1.205"  # define the FMC IP address

# here is the domain ID obtained from FMC's api explorer
  domain: "dd731f3e-8297-5b05-7ac3-000000000001"  

# here is the access policy name
  policy_name: "FTD-1-Access-Policy-SYD"

# here is the access policy rule name which needs to be enabled. 
  policy_rule_name: "URL Filter"

# here we define a variable which indicates whether the config has been modified
  config_has_been_changed: false


After finishing all the above steps, we are ready to start writing our main task playbook. In the next post, I will introduce the way to obtain the access token from FMC.


Sunday, July 8, 2018

Ansible REST API - Interacting with Cisco FirePower Management Center (FMC) - 03 Introduction of REST API and Cisco FMC API Explorer

This post belongs to my "Ansible REST API - Interacting with Cisco FMC" series. The following is the table of content of this series:
  1. Introduction and Ansible playbook download
  2. Script flow charts
  3. Introduction of REST API and Cisco FMC API Explorer
  4. Script prerequisites
  5. Request Access Token 
  6. Get policy content, modify content and "PUT' in FMC - Part 1
  7. Get policy content, modify content and "PUT' in FMC - Part 2 
  8. Get deployable devices and deploy policy

Cisco FirePower Management Center starts supporting REST API since version 6.1. But as many other early offerings of Cisco products, it takes time to become mature. At the latest release 6.2.3, the it become much more stable.

If you are not family with REST API, you can go through my post "REST API 101" and get a general idea about what REST API can do and what's the format of REST request and respond.

Cisco FirePower Management Center provides its own API Explorer. You will need a "admin" account to connect to the API interface. The REST API is enabled by default. If the REST API was disabled, you can login to the FMC console and go to "System -> Configuration -> REST API Preferences" and click the "Enable REST API":



Click to expand
In order to access the API Explorer of FMC, open your browser and input the URL as below:

https://{{FMC IP}}/api/api-explorer/

Click to expand
In the following, I will use an example to show how to use the api-explorer. And the photo below shows the steps of getting the device list and device details of the FMC:


Click to expand
First, we select the "Devices" from "API INFO" on the left hand side panel. Then the available API calls and methods will appear in the mid-panel. We can see for "device", we have two type of API calls: "copy config request" & "device records". In this example, we will focus on the "device records".

As we know, in FMC, devices can be registered in the Global domain or other sub-domains. In our example, we want to check the devices which registered in the "Global/Sydney" domain. So we need to select the "Global/Sydney" domain as below:


Click to expand

Once we select the domain, we can see the domain ID will appear in the URL link as below. In the later posts of this API series, we will use this domain ID quite often. So now we keep in mind how to get the domain ID here.
Click to expand
In the above screen-shot, we can see there are 4 types of request method for the "device records". They are DELETE (delete device object), PUT (modify device object), POST (add new device object) and GET (obtain existing device object info). Let's select the "GET" and we have the API CONSOLE shown as below:
Click to expand
In the above API console, we can see the URL, request method, query parameter and request/ response info. Please be note that when sending GET request to FMC, we should always use "expanded: true" in the query parameter as it will give you the detailed info in response. The following shows how the request parameter is appended to the URL:

{"url":"/api/fmc_config/v1/domain/dd731f3e-8297-5b05-7ac3-000000000001/devices/devicerecords?expanded=true"}


In the "Response Text" field, we can see the response body which contain the existing device info of the FMC under the "Global/Sydney" domain. The devices are in in the array "items[]". In each device's JSON, we have the key values such as "id", "links", "name", "description", "model", "accessPolicy " etc. The follow is an example of the response text. The lines highlighted in orange is device "FTD-2". The lines highlighted in light blue is device "FTD-1".


{
  "links": {
    "self": "https://192.168.1.205/api/fmc_config/v1/domain/dd731f3e-8297-5b05-7ac3-000000000001/devices/devicerecords?offset=0&limit=2&expanded=true"
  },
  "items": [
    {
      "id": "11a13c5e-7b9d-11e8-8701-be7fcc61faec",
      "type": "Device",
      "links": {
        "self": "https://192.168.1.205/api/fmc_config/v1/domain/dd731f3e-8297-5b05-7ac3-000000000001/devices/devicerecords/11a13c5e-7b9d-11e8-8701-be7fcc61faec"
      },
      "name": "FTD-2",
      "description": "NOT SUPPORTED",
      "model": "Cisco Firepower Threat Defense for VMWare",
      "modelId": "A",
      "modelNumber": "75",
      "modelType": "Sensor",
      "healthStatus": "green",
      "sw_version": "6.2.3",
      "healthPolicy": {
        "id": "e13c7d40-7083-11e8-97ce-02c515297e64",
        "type": "HealthPolicy",
        "name": "Initial_Health_Policy 2018-06-15 10:07:15"
      },
      "accessPolicy": {
        "name": "syd-test-access-policy",
        "id": "000c2911-ed30-0ed3-0000-030064771115",
        "type": "AccessPolicy"
      },
      "hostName": "192.168.1.207",
      "license_caps": [
        "THREAT",
        "MALWARE",
        "URLFilter"
      ],
      "keepLocalEvents": false,
      "prohibitPacketTransfer": false,
      "metadata": {
        "readOnly": {
          "state": false
        },
        "domain": {
          "name": "Sydney",
          "id": "dd731f3e-8297-5b05-7ac3-000000000001",
          "type": "domain"
        }
      }
    },
    {
      "id": "1b8e0ef8-7081-11e8-ab14-9aaa7606d632",
      "type": "Device",
      "links": {
        "self": "https://192.168.1.205/api/fmc_config/v1/domain/dd731f3e-8297-5b05-7ac3-000000000001/devices/devicerecords/1b8e0ef8-7081-11e8-ab14-9aaa7606d632"
      },
      "name": "FTD-1",
      "description": "NOT SUPPORTED",
      "model": "Cisco Firepower Threat Defense for VMWare",
      "modelId": "A",
      "modelNumber": "75",
      "modelType": "Sensor",
      "healthStatus": "red",
      "sw_version": "6.2.3",
      "healthPolicy": {
        "id": "e13c7d40-7083-11e8-97ce-02c515297e64",
        "type": "HealthPolicy",
        "name": "Initial_Health_Policy 2018-06-15 10:07:15"
      },
      "accessPolicy": {
        "name": "FTD-1-Access-Policy-SYD",
        "id": "000c2911-ed30-0ed3-0000-008589936316",
        "type": "AccessPolicy"
      },
      "hostName": "192.168.1.206",
      "license_caps": [
        "THREAT",
        "MALWARE",
        "URLFilter"
      ],
      "keepLocalEvents": false,
      "prohibitPacketTransfer": false,
      "metadata": {
        "readOnly": {
          "state": false
        },
        "domain": {
          "name": "Sydney",
          "id": "dd731f3e-8297-5b05-7ac3-000000000001",
          "type": "domain"
        }
      }
    }
  ],
  "paging": {
    "offset": 0,
    "limit": 2,
    "count": 2,
    "pages": 1
  }
}


In FMC, every object has its own ID. So getting the correct item ID is very important in the REST API operations. In the above example, we can get the Domain ID, Device ID, Access Policy ID etc.

Although our goal is to use Ansible to interact with the FMC, it's very important to understand how to use the FMC API Explorer. As I mentioned before, different vendors have different API data structure. API Explorer is the guide line of Cisco FMC REST API. From the API explorer, we can learn about the API request/ response format and we can write our Ansible scripts based on these instructions.

In the following post, I will take about the prerequisites of this Ansible Playbook.


Reference:

REST API 101

“A RESTful API -- also referred to as a RESTful web service -- is based on representational state transfer (REST) technology, an architectural style and approach to communications often used in web services development.”

Users can make HTTP - GET, PUT, POST, PATCH & DELETE requests to a device which support REST API and interact with the device. So before sending the REST requests to the device, we will need to have the instruction from the device vendor and understand the data structure/ format in the request and response. The data body exchange from local and remote system are in JSON format.

  • REST Request methods
First of all, let's talk about the standard Request methods. There are 5 major types of request method in REST: GET, PUT, PATCH, POST & DELETE.

GET – retrieves values for specified object

Most of time, GET request is the first call make from user to the device to obtain the specific information. For example, if you want to know interface details of a firewall, you can make a GET call to the firewall which support REST calls. Then obtain the interface name, duplex, speed, description, IP/mask etc.

The following example is a GET call. We can see there are 3 key components in the GET call. Basically they are the key components in all types of REST calls.

Request method: GET
Request URI:
Host:

Click to expand
PUT – Update the value for an existing object in the remote system

As explained above, PUT will update the object values. If the object does not exist in the remove system, an error code “404” will be returned.


PATCH – Applies a partial update for the remote object

A new REST user would be confused by PUT and PATCH because both of them can update the object values. However, if you want to use PUT, you will need to provide the full data structure of the object in the PUT call. And if you want to use PATCH, you only need to provide the data you want to update.

For example, if you want to update the IP address of the device and you want to use the PUT call, then you will have to provide not only the new IP address, but also all other settings in the interface data structure such as description, duplex, speed etc.

But if you want to use the PATCH call, then you will only need to provide the update IP address information. The updated information will be provided in the “body” field of the PATCH call.


POST – creates a new object with the provide values

POST call is used to create new object in the system. The body of the POST call will contain the values of this new object.

DELETE – remove the specified object

DELETE call is very self-explanatory. It can delete the specified object


  •  REST return codes
Once we make a REST request call to the server, we will expect a return message from the server. This return message will contain a code. Now let’s describe the meaning of these return codes.

Code
Description
20X
Two-hundred series code are successful return codes
200 OK
Request successful (standard return code)
201 Created
Request completed
202 Accepted
Request accepted, but processing not completed.
204 No Content
Request successful, but no content is returned
40X
Four-hundred series code caused by a client-side error
400 Bad Request
Invalid query parameters such as unrecognized parameters or invalid values
404 Not Found
The provided URI does not patch any existing resource
405 Method not Allowed
The request is not allowed on the specified object
5XX
Five-hundred series code indicates a server-side error

  •      Tools to make REST Calls
In the above post, we talked about the REST request and response. But how can we make these REST calls. There are many tools can help us to do that. The most famous one is POSTMAN application.

POSTMAN is a handy tool which allow us to make REST call by define every field in the call such as URI, Host, Body, method, authenticate type etc.



Click to expand
Postman can be downloaded from the following link:

https://www.getpostman.com/

If you don’t want to install the application but still want to play with the REST calls, you can install the REST Client Firefox add-on. It’s a lightweight method to enable you to consume the RESTful web services.

https://addons.mozilla.org/en-US/firefox/addon/restclient/



Click to expand
However, if you want to do programmable REST control to the device, a programmable language is required. Python or Ansible are your best friends then. 

In conclusion, REST is a very powerful tool to interact with remote system in the network. In the networking world, it’s also become more and more important. Although the data structure in JSON are vendor specified, the message formats are the same. So if we understand how to request and manipulate the JSON data in REST, we can easily control any of the systems no matter which vendors they are from.

In this blog, I have a post series which describe how to useAnsbile to interact with Cisco Fire Power Management Center with REST.


Reference:

https://searchmicroservices.techtarget.com/definition/RESTful-API



NSX Load Balancer "Application Rules" Examples:

Load Balancing is one of the features provided by the NSX Edge Services Gateway (ESG). It can provide L7 Load Balancing by utilizing the HA...