)
I’ve heard from many OpenStack operators over the last few years who are looking for a way to get rid of the mandatory implementation of IPv4 network address translation (NAT) with OpenStack. Historically, the way to achieve this was to deploy provider networks with VLANs, attach instances directly to these provider networks and just route out through the data center without any need/requirement of NAT until the traffic reached the Internet Edge.
In the Mitaka release of OpenStack, Address Scopes and Subnet Pools were introduced. One method of disabling IPv4 NAT is to match address scopes on the external and internal sides of the Neutron router (in a future post, I’ll talk about disabling SNAT as another option). This is done by creating an address scope, creating subnet pools that use that address scope and then creating networks/subnets from those subnet pools. If the external and internal (tenant) subnets use the same ‘scope’ then NAT is disabled in the Neutron L3 component. More can be found here: OpenStack Docs: Address scopes
A quick test reveals some issues with this as a mainstream method of removing IPv4 NAT from a cloud deployment:
- Address scope and subnet pool use are mandatory
- You cannot manually assign a subnet allocation pool range OR a custom gateway (“Bad subnets request: allocation_pools allowed only for specific subnet requests..”). This is a major issue when you need to define a custom upstream (provider) gateway address. Note: I resolved this by editing the subnet by hand (before creating any routers) and changing the allocation range and changing the default gateway IP address.
- You have to sub-allocate prefixes/CIDR ranges out of the subnet pool which is typically something the cloud/network operator would do upstream at the DC Agg layer, routed access or Border Leaf
Test scenario:
- All-in-One RDO VM (Newton) on VMware Fusion
- Eth1<>br-ex
- Public Network: 192.168.81.0/24
- One Address Scope
- Two Subnet Pools out of the 192.168.81.0/24 range, divided as /26s
Create an Address Scope:
[root@c7-n-aio ~]# neutron address-scope-create --shared test-no-nat 4
Created a new address_scope:
+------------+--------------------------------------+
| Field | Value |
+------------+--------------------------------------+
| id | 6f055310-6225-403c-a14d-acd723757869 |
| ip_version | 4 |
| name | test-no-nat |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| shared | True |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
+------------+--------------------------------------+
Create a new subnet pool using the new address scope:
[root@c7-n-aio ~]# neutron subnetpool-create --address-scope test-no-nat --shared --pool-prefix 192.168.81.0/24 --default-prefixlen 26 nonat-pool-ip4
Created a new subnetpool:
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| address_scope_id | 6f055310-6225-403c-a14d-acd723757869 |
| created_at | 2016-11-04T19:59:10Z |
| default_prefixlen | 26 |
| default_quota | |
| description | |
| id | 964ae834-9391-4aa2-bc7b-b1c197d4d4c5 |
| ip_version | 4 |
| is_default | False |
| max_prefixlen | 32 |
| min_prefixlen | 8 |
| name | nonat-pool-ip4 |
| prefixes | 192.168.81.0/24 |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| revision_number | 1 |
| shared | True |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated_at | 2016-11-04T19:59:10Z |
+-------------------+--------------------------------------+
Create a new network for the Public/External side:
[root@c7-n-aio ~]# neutron net-create Public-Net --provider:network_type vxlan --shared --router:external
Created a new network:
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2016-11-04T19:59:21Z |
| description | |
| id | 4d68a85a-eac9-44a3-84b7-eec3b731d97a |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False |
| mtu | 1450 |
| name | Public-Net |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 71 |
| revision_number | 2 |
| router:external | True |
| shared | True |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated_at | 2016-11-04T19:59:21Z |
+---------------------------+--------------------------------------+
Create a new subnet for the public/external network and reference the new subnet pool:
#Note: this is where you cannot dictate the ‘gateway_ip’ value manually; if you attempt to, you either get an error (see the output in my list of issues in the beginning of the doc) or it ignores it (not consistent).
[root@c7-n-aio ~]# neutron subnet-create Public-Net --name Public-Subnet --subnetpool nonat-pool-ip4 --dns-nameserver 8.8.8.8
Created a new subnet:
+-------------------+---------------------------------------------------+
| Field | Value |
+-------------------+---------------------------------------------------+
| allocation_pools | {"start": "192.168.81.2", "end": "192.168.81.62"} |
| cidr | 192.168.81.0/26 |
| created_at | 2016-11-04T20:00:44Z |
| description | |
| dns_nameservers | 8.8.8.8 |
| enable_dhcp | True |
| gateway_ip | 192.168.81.1 |
| host_routes | |
| id | 30c3f341-ad77-4f27-ae55-d469d8004266 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | Public-Subnet |
| network_id | 4d68a85a-eac9-44a3-84b7-eec3b731d97a |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| revision_number | 2 |
| service_types | |
| subnetpool_id | 964ae834-9391-4aa2-bc7b-b1c197d4d4c5 |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated_at | 2016-11-04T20:00:44Z |
+-------------------+---------------------------------------------------+
Create a Tenant network:
[root@c7-n-aio ~]# neutron net-create nonat-net-1 Created a new network: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | | | created_at | 2016-11-04T20:02:24Z | | description | | | id | e5ae8309-beb8-4bcd-a9ee-87be39004eff | | ipv4_address_scope | | | ipv6_address_scope | | | mtu | 1450 | | name | nonat-net-1 | | project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc | | provider:network_type | vxlan | | provider:physical_network | | | provider:segmentation_id | 32 | | revision_number | 2 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tags | | | tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc | | updated_at | 2016-11-04T20:02:24Z | +---------------------------+--------------------------------------+
Create a new subnet for the new Tenant network and use the new subnet pool:
[root@c7-n-aio ~]# neutron subnet-create --name nonat-subnet-1 --subnetpool nonat-pool-ip4 nonat-net-1
Created a new subnet:
+-------------------+-----------------------------------------------------+
| Field | Value |
+-------------------+-----------------------------------------------------+
| allocation_pools | {"start": "192.168.81.66", "end": "192.168.81.126"} |
| cidr | 192.168.81.64/26 |
| created_at | 2016-11-04T20:02:43Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 192.168.81.65 |
| host_routes | |
| id | f24de73f-8ff7-48df-a5e9-c2d5d43bb154 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | nonat-subnet-1 |
| network_id | e5ae8309-beb8-4bcd-a9ee-87be39004eff |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| revision_number | 2 |
| service_types | |
| subnetpool_id | 964ae834-9391-4aa2-bc7b-b1c197d4d4c5 |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated_at | 2016-11-04T20:02:43Z |
+-------------------+-----------------------------------------------------+
Create a new router:
[root@c7-n-aio ~]# neutron router-create no-nat-router
Created a new router:
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2016-11-04T20:02:54Z |
| description | |
| distributed | False |
| external_gateway_info | |
| flavor_id | |
| ha | True |
| id | 31a09399-fd79-4d0f-8354-f700393a8138 |
| name | no-nat-router |
| project_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| revision_number | 2 |
| routes | |
| status | ACTIVE |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated_at | 2016-11-04T20:02:54Z |
+-------------------------+--------------------------------------+
Link the router interfaces and gateway to the new networks:
[root@c7-n-aio ~]# neutron router-interface-add no-nat-router nonat-subnet-1
Added interface b27f6296-9353-4b4a-b96e-55c88ba97177 to router no-nat-router.
[root@c7-n-aio ~]# neutron router-gateway-set no-nat-router Public-Net
Set gateway for router no-nat-router
Boot an instance and attach it to the new “nonat-net-1” network which is using subnet pool “nonet-pool-ip4”:
[root@c7-n-aio ~]# nova boot --flavor m1.tiny --image cirros --key-name aio-key --security-groups default --nic net-name=nonat-net-1 nonat-vm
+--------------------------------------+-----------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | nonat-vm |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-rwlgkfoi |
| OS-EXT-SRV-ATTR:root_device_name | - |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | 5efCnNeo7P5a |
| config_drive | |
| created | 2016-11-04T20:06:50Z |
| description | - |
| flavor | m1.tiny (1) |
| hostId | |
| host_status | |
| id | 6f788d79-6b4b-4d58-a0be-84856de5207e |
| image | cirros (403de225-2186-4fb0-8b19-020c57300fb7) |
| key_name | aio-key |
| locked | False |
| metadata | {} |
| name | nonat-vm |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tags | [] |
| tenant_id | 67ddc0b4822f4b67b3e4a449dbbe65bc |
| updated | 2016-11-04T20:06:50Z |
| user_id | cab9fe684b0a4ddc9ba0f57acafeb2bb |
+--------------------------------------+-----------------------------------------------+
Grab reference information for the instance and the router:
[root@c7-n-aio ~]# nova list
+--------------------------------------+----------+--------+------------+-------------+---------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+---------------------------+
| 6f788d79-6b4b-4d58-a0be-84856de5207e | nonat-vm | ACTIVE | - | Running | nonat-net-1=192.168.81.74 |
+--------------------------------------+----------+--------+------------+-------------+---------------------------+
[root@c7-n-aio ~]# neutron router-list
+--------------------------------------+----------------+-----------------------------------------------+-------------+------+
| id | name | external_gateway_info | distributed | ha |
+--------------------------------------+----------------+-----------------------------------------------+-------------+------+
| 31a09399-fd79-4d0f-8354-f700393a8138 | no-nat-router | {"network_id": "4d68a85a- | False | True |
| | | eac9-44a3-84b7-eec3b731d97a", "enable_snat": | | |
| | | true, "external_fixed_ips": [{"subnet_id": | | |
| | | "30c3f341-ad77-4f27-ae55-d469d8004266", | | |
| | | "ip_address": "192.168.81.12"}]} | | |
| da5fc542-9028-4315-b66f-13a11d8ef87d | Private-Router | null | False | True |
+--------------------------------------+----------------+-----------------------------------------------+-------------+------+
From an outside (external to OpenStack) host, ping the IPv4 address of the router interface (it works):
# ping 192.168.81.12
PING 192.168.81.12 (192.168.81.12): 56 data bytes
64 bytes from 192.168.81.12: icmp_seq=0 ttl=64 time=1.523 ms
64 bytes from 192.168.81.12: icmp_seq=1 ttl=64 time=0.232 msFrom an outside (external to OpenStack host), ping the IPv4 address of the instance (it doesn’t work):
# ping 192.168.81.74
PING 192.168.81.74 (192.168.81.74): 56 data bytes
Request timeout for icmp_seq 0
Add a route on the network gear or the host pointing to the Tenant subnet via the Neutron router IP (in this example, it is my Mac, which is hosting the VMware Fusion VMs this is running on):
# sudo route -n add 192.168.81.64/26 192.168.81.12
add net 192.168.81.64: gateway 192.168.81.12
Ping the instance from external host. It works without a floating IP and no NAT:
# ping 192.168.81.74
PING 192.168.81.74 (192.168.81.74): 56 data bytes
64 bytes from 192.168.81.74: icmp_seq=0 ttl=63 time=1.101 ms
64 bytes from 192.168.81.74: icmp_seq=1 ttl=63 time=1.058 msThis post first appeared on the Shannon McFarland’s blog. Superuser is always interested in community content, email: [email protected].
Cover Photo // CC BY NC
Superuser Magazine is the Open Infrastructure Foundation's official online publication. It covers open infrastructure ecosystem news, case studies, event recaps, product updates and announcements, project releases, and more.
Latest posts by Superuser (see all)
- Demystifying Confidential Containers with a Live Kata Containers Demo - July 13, 2023
- OpenInfra Summit Vancouver Recap: 50 things You Need to Know - June 16, 2023
- Congratulations to the 2023 Superuser Awards Winner: Bloomberg - June 13, 2023
Alain Swanson
Emma L. Foley
Shannon McFarland
