Counting bytes and measuring latencies

AWS -VPC Networking for beginners

I have worked very little on networking. I have seen L2/L3 hardware but never got an opportunity to configure, deploy or automate them. As we experience the cloudification of traditional IT, some of the core IT services are already mature enough for production uses, or large scale deployments. Storage (s3 , openstack-swift) , block storage (EBS) or compute (ec2, openstack nova) are few of them. What always excited me is to understand how much network automation is possible in public or private clouds.  It is important for broader cloud adoption. 

I know there are significant innovations happening in that space also. OpenFlow might be the openstack equivalent in that space. Together they give you openfabrics. A cloud that accommodates your networking devices along with SAN and rack servers.


I have done some preliminary work on AWS-VPC. And i am listing down my learnings from it. I always read about these concepts, but never understood what they meant.  So remember in mind if you are on aws - vpc and you do not have prior experience with networking this might be helpful for you:



 A Virtual private cloud is a sub-cloud inside AWS public cloud. Sub-cloud means it is inside an isolated logical network. Other servers cant see instances that are inside a vpc. It is like a vlan an aws infrastructure inside a vlan.


In non-vpc aws cloud, the normal one all servers get a public ip. This is used to access the instance from outside. But if you run `ip addr show` you'll see it has a private ip. The instance itself is not aware of its public ip (its probably NAT-ed with the public ip in their switch). The information about private ip is also important as communications using private and public ip's incur different cost schemes (public ip based communications incur some cost). But the point is every single instance is visible, their names are and ips are reused. These things have significant impact on how you design and implement your network security as well. Security groups, which selectively allow individual types of ingress or incoming traffic, becomes more important.


A VPC is denoted by a subnet mask. For example, when one says vpc-x is , that means any instances inside this vpc will have an ip 10.123.X.Y where X and Y can be anything between 2 to 254. A vpc can have following global components:

  • An DHCP option set (server that assigns dynamic ips )
  • Internet gateway (will come to this shortly)
  • One or more subnets
  • One or more routing tables
  • One or more network ACLs

 Subnets: A subnet is a sub network inside a vpc. Example of subnets inside vpc (10.123.X.Y) can be 10.123.1.A/24. Which mean any instance that belongs to this subnet will have an ip 10.123.1.A where A can be anything between 2-254. These are also known as CIDR notations. An instance always belongs to a subnet. You can not have an instance inside a vpc that does not belong to any subnets. While spawning instances inside AWS vpc one must specify which subnet the instance should belong to.


 Routing tables: network traffic of any instance inside a subnet is dictated by a routing table. An example routing table can be:

CIDR ---   target --- local  -  igw  (internet gateway)

which mean any traffic destined for 10.123.X.Y ip (where X and Y can be anything from 2 to 254) will be sent directly. For the rest traffic will be directed to igw. 


Now, it is important to understand that a subnet is always attached to one and only one routing table. So, if we spawn an instance inside a subnet that has the above mentioned routing table attached to it still the instance will not be accessible from outside vpc. Because it does not have a public ip. One can attach an elastic ip (which is reusable public ip) to this instance and then access it. The instance in turn can access the internet. Remember for an instance to be directly available from the internet it has to have an elastic ip as well as it must be within a subnet that has a routing table where non local traffic is routed via an internet gateway. So, elastic ip, and igw in routing table are two criterias for an instance to be available directly from internet. Subnets which has such routing tables attached to them also known as public subnet (non-local traffic routed to internet gateway), as any instance with an elastic ip can from this subnet can be publicly available.

 On the other hand you can specify a NAT (a gateway) instance as target for non-local traffic inside a routing table. And keep the NAT box in a public subnet with an elastic ip attached to it. Now any subnet that has this type of routing table attached become private subnet. Because they can not be exposed publicly. Even if you assign elastic ip it wont be publicly available (recall, for an instance to be publicly available you need both, an elastic ip as well as a routing table that directs non-local traffic to internet gateway). Example of private subnet


CIDR --- target --- local  -  i-abcdef  (instance ip of the NAT box)


Network ACLs or network access control lists.  Apart from routing tables each subnet also assigned an network ACL. Network ACLs specify which type of traffic is allowed inside the subnet. By default it might havefollowing rules:

rule number --- port --- protocol --- source        -- action

   100 ---- ALL ---   ALL    --- 0.0.0/0 -- allow

This means all traffic is allowed within this network. You can think of Network ACLs are subnet wide security groups. They are effective while isolating subnets from each other, reduction of collision domains etc.


Entities such as RDS , ELB can be provisioned within VPC as well. Same rule applies for them as other ec2 instances. If they belong to public subnet they can be accessed from internet, otherwise not. 

In a typical web application example, you will be spawning the ELB and a NAT box inside the public subnet and your db servers (or RDS instances) and web servers in the private subnet. Since you have a NAT gateway (and a routing table attached to the private subnet that routes traffic via this NAT gateway) , instances from private subnets can access internet, but the reverse is not possible. If you do not want the instances fro private subnet to access internet , then you can remove the NAT box from the private subnet's routing table. Since all this can be done dynamically, via the web browser based console, command line  tools or aws webservices api , you can temporarily allow the instances from private subnet to access internet (like while provisioning,) and then revoke it later (before joining the elb )

I'll be writing another post how you can setup cross availability zone , highly available services using AWS VPC from an network standpoint, this will serve the foundation of that post.