network

Accessing docker containers from your mac

The most prevalent solution for running docker on the mac is using boot2docker or maybe a CoreOS vagrant VM. In either case the docker0 network is not accessible in the mac since the it is created as a host only network on Virtual Box, usually with the CIDR of 172.17.0.0/16

A simple way to get connectivity to the host only network created for docker is to add a route for the docker's network CIDR with the gateway of the VM's IP

  1. Find the IP of your VM, if you are using docker machine then: $: docker-machine ls
  2. Find the CIDR for you docker private network
    • $: docker-machine ssh yourVmName
    • $: ifconfig docker0 | grep "inet addr" if the netmask is 255.255.0.0 that means it is a /16 network so if the gateway is 172.17.42.1 its CIDR would be 172.17.0.0/16
  3. Create a route for docker0 network: $: sudo route -n add 172.17.0.0/16 192.168.99.100

To confirm that the route was added successfuly you can check the route table entries with netstat, you should see something like this:

netstat -nr  
172.17             192.168.99.100     UGSc            1        4 vboxnet  

Configuring docker0 private network CIDR range on boot2docker VM

I came across an issue with the docker0 virtual interface configuration while trying to use dnsdock

Docker0 virtual interace

Docker creates a bridged virtual interface named docker0 in the host machine. The range is selected randomly from the available CIDRs defined in the RFC 1918 However, in most cases the range of 172.17.42.1/16 is selected. One key point to keep in mind is that docker first make sure that the subnet range doesn't create a conflict with an interface already configured in the host.

dnsdock issue

The issue I was facing was that I was setting the primary DNS server of my mac to 172.17.42.1 which was IP bound to the dnsdock container:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9d32abb2169f tonistiigi/dnsdock "/go/bin/dnsdock -nam" 35 hours ago Up 34 hours 172.17.42.1:53->53/udp dnsdock

In the latest version of the boot2docker VM two interfaces are created by default

  • eth0 which is NATed to the host giving internet access connectivity
  • eth1 which is the host only network used for the docker containers.

The DNS server settings are also copied to the VM, you can check them on /etc/resolv.conf

So since I had the primary DNS set on my mac to 172.17.42.1 when the boot2docker VM was booted it would set the primary DNS of the VM to 127.17.42.1 as well.
Now when the docker daemon starts it would see that the subnet 172.17.0.0/16 was already being used it would select the next range available which was 172.18.0.0/16
So it was a chicken and the egg problem since to get the latest DNS settings I had to restart the VM, but by restarting the VM docker would select a different range for its private network which would force my to re-updated the DNS settings on the mac and repeat the cycle endlessly.

Configuring docker0 network

To solve the issue I instructed the docker daemon to use a static CIDR for its private network in accordance to the official docs

Update file /var/lib/boot2docker/profile
Add --bip="172.17.0.0/16" to EXTRA_ARGS variable

With that option every time the VM is restarted the docker0 CIDR will still be the same even though the host has the primary DNS server pointing to the docker0 gateway

Diving into Metasploit - Configuring local environment

This semester I have a great excuse to learn the Metasploit framework since it is a required topic for the course on Penetration Testing I’m taking at Seneca.

I want to document the steps of being introduced to metasploit from a software developer’s point of view.
I've never used metasploit before and the goal by the end of the semester if to be fairly fluent with the framework.

To get started I want to cover the environment installation.

1. Choosing virtualization tool

My dev machine is a mac, I’m running Mavericks.
There are a few options to virtualize an OS on a mac.
You could use Paralles, VMWare or VirtualBox. There is also the possibility of running containers but that’s the topic of another post.
So between the main three virtualization tools, hands down VirtualBox is the best  if you plan to run linux OS. It comes with pointer integration and drag and drop out of the box while Paralles and VMWare don’t. Also we can’t forget the fact that VirtualBox is free which makes even easier to get started with.

VirtualBox website

2. Planning network architecture

Once I had the tools in place to virtualize my environment it was time to plan out the network configuration.
I’m sticking with a very basic setup:
network: 10.10.0.0/24
static pool: 10.10.0.1-100
dhcp pool: 10.10.0.101-254
domain: dpi902.shogun
hosts: {osName}{number}

To create a network on VirtualBox is very simple, only a few steps required:
Screen Shot 2014-01-28 at 9.39.31 PM

Screen Shot 2014-01-28 at 9.39.27 PM

Screen Shot 2014-01-28 at 9.39.19 PM

To get more information on the network types supported by VirtualBox check out their manual:https://www.virtualbox.org/manual/ch06.html

3. Configure Interfaces

With the host-only network created, the next step is to configure the network interfaces of the VMs you’ll be using. I’m starting with Kali and Metasploitable-2

I like to set up as the eth0 the host-only network I’ll be configuring the static IPs.
eth2 I leave for the bridge interface where I’ll get internet connection whenever needed.
Screen Shot 2014-01-28 at 9.44.14 PM

Screen Shot 2014-01-28 at 9.44.08 PM
Since Kali and Metasploitable are debian base we can set static ips the same way we do it on ubuntu:

vim /etc/network/interfaces

 
 $: vim /etc/network/interfaces

 auto eth0  
 iface eth0 inet static  
 address 10.10.0.22  
 gateway 10.10.0.1  
 brodcast 10.10.0.255  
 netmask 255.255.255.0

auto eth1  
 iface eth1 inet dhcp

post-up route add default gw 10.0.0.1 metric 2  
 pre-down route del default gw 10.0.0.1  

A couple of things to note:

  1. By simply adding a virtual interface to VirtualBox doesn’t mean that it will be brought up by default by the network service, it needs to be brought up manually or configure in the interfaces file.
  2. I guess since I’m bridging eth1 the default gateway being used is from eth0, which doesn’t have internet connection. To circumvent the problem I just set the default gateway manually when the network service gets started. One issue I foresee with this is when I use a network with a segment different than 10.0.0.0. I’ll need to do some more readings on this topic but I’m thinking of configuring the gateway dynamically or setting the bridge interface on eth0. We’ll see.

So that’s pretty much it.
An environment to play around with metasploit

TODO:
Use the virtualbox api in conjunction with puppet to orchestrate the deployment/config of VMs in a test environment.

Installing OpenStack, Quantum problems

During the following weeks we plan to expand more on the subject of setting up an OpenStack cloud using Quantum.
For now we have been experimenting with different Quantum functionality and settings.
At first Quantum might look like a black box, not due to its complexity, but because it deals with several different plugins and protocols that if a person is not very familiar with them it becomes hard to understand why Quantum is there in the first place.

In a nutshell Quantum has the role to provide an interface to configure the network of multiple VMs in a cluster.

In the last few years the lines between a system, network and virtualization admin have become really blury.
The classical unix admin is pretty much non existent now a days since most services are offered in the cloud in virtualized environments.
And since everything seems to be migrating over to the cloud some network principles that were applied into physical networks in the past some times don’t translate very well to virtualized networks.

Later we’ll have some posts explaining what technologies and techniques underlie the network configuration of a cloud, in our case focusing specifically on OpenStack and Quantum.

With that being said, below are a few errors that came up during the configuration of Quantum:

1. ERROR [quantum.agent.dhcp_agent] Unable to sync network state.

This is error is most likely caused due a misconfiguration of the rabbitmq server.
A few ways to debug the issue is to:
Check if the file /etc/quantum/quantum.conf in the controller node(where the quantum server is installed) has the proper rabbit credentials

By default rabbitmq runs on port 5672, so run:

[sourcecode]
netstat -an | grep 5672
[/sourcecode]

and check if the rabbitmq server is up an running

On the network node(where the quantum agents are installed) also check if the /etc/quantum/quantum.conf have the proper rabbit credentials:

If you are running a multihost setup make sure the rabbit_host var points to the ip where the rabbit server is located.

Just to be safe check if you have a connection on the management networking by pinging all the hosts in the cluster and restart both the quantum and rabbitmq server as well the quantum agents.

2. ERROR [quantum.agent.l3agent] Error running l3nat daemon_loop

This error requires a very simple fix, however, it was very difficult to find information about the problem online.
Luckily, I found one thread on the mailing list of the fedora project explaining in more details the problem.

This is error is due to the fact that keystone authentication is not working.
A quick explanation – the l3 agent makes use of the quantum http client to interface with the quantum service.
This requires keystone authentication. If this fails then the l3 agent will not be able to communicate with the service.

To debug this problem check if the quantum server is up and running.
By default the server runs on port 9696

[sourcecode]
root@folsom-controller:/home/senecacd# netstat -an | grep 9696
tcp 0 0 0.0.0.0:9696 0.0.0.0:* LISTEN
tcp 0 0 192.168.0.11:9696 192.168.0.12:40887 ESTABLISHED
[/sourcecode]

If nothing shows up is because the quantum server is down, try restarting the service to see if the problems goes away:

[sourcecode]
quantum-server restart
[/sourcecode]

You can also try to ping the quantum server from the network node(in a multihost scenario):

[sourcecode]
root@folsom-network:/home/senecacd# nmap -p 9696 192.168.0.11

Starting Nmap 5.21 ( http://nmap.org ) at 2013-01-28 08:07 PST
Nmap scan report for folsom-controller (192.168.0.11)
Host is up (0.00038s latency).
PORT STATE SERVICE
9696/tcp open unknown
MAC Address: 00:0C:29:0C:F0:8C (VMware)

Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds
[/sourcecode]

3.ERROR [quantum.agent.l3agent] Error running l3nat daemon_loop – rootwrap error

I didn’t come across this bug, but I found a few people running into this issue.
Kieran already wrote a good blog post explaining the problem and how to fix it

You can check the bug discussion here

4. Bad floating ip request: Cannot create floating IP and bind it to Port , since that port is owned by a different tenant.

This is just a problem of mixed credentials.
Kieran documented the solution for the issue here

There is also a post on the OpenStack wiki talking about the problem.

Conclusion

This should help fixing the problems that might arise with a Quantum installation.
If anybody knows about any other issues with Quantum or has any suggestions about the problems listed above please let us know!

Also check the official guide for other common errors and fixes