[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt-users] Networking issues with lxc containers in AWS EC2

On 04/01/2016 07:04 PM, Peter Steele wrote:
On 04/01/2016 02:07 PM, Laine Stump wrote:
On 03/31/2016 06:43 PM, Peter Steele wrote:
I've created an EC2 AMI for AWS that essentially represents a CentOS 7 "hypervisor" image. I deploy instances of these in AWS and create an number of libvirt based lxc containers on each of these instances. The containers run fine within a single host and have no problem communicating with themselves as well as with their host, and vice versa. However, containers hosted in one EC2 instance cannot communicate with containers hosted in another EC2 instance.

We've tried various tweaks with our Amazon VPC but have been unable to find a way to solve this networking issue. If I use something like VMware or KVM and create VMs using this same hypervisor image, the containers running under these VMs can communicate with with each other, even across different hosts.

What is the <interface> config of your nested containers? Do they each get a public IP address?
Yes, they all have public IPs on the same subnet. When deployed in a VM environment on premises, the containers have no problems. Amazon clearly does something with the packets though and the containers can't talk to each other.

You say they can talk among containers on the same host, and with their own host (I guess you mean the virtual machine that is hosting the containers), but not to containers on another host. Can the containers communicate outside of the host at all? If not, perhaps the problem is iptables rules for the bridge device the containers are using - try running this command:

  sysctl net.bridge.bridge-nf-call-iptables

If that returns:

  net.bridge.bridge-nf-call-iptables = 1

then run this command and see if the containers can now communicate with the outside:

  sysctl -w net.bridge.bridge-nf-call-iptables=0

My real question is has anyone tried deploying EC2 images that host containers and have figured out how to successfully communicate between containers on different hosts?

No experience with EC2, sorry.

I think we'll need to go to Amazon themselves to resolve this issue. There is very little information out there about how to get lxc containers to work properly in EC2.

Well, if they've allowed your virtual machine to acquire multiple IP addresses, then it would make sense that they would allow them to actually use those IP addresses. I'm actually more inclined to think that the packets simply aren't getting out of the virtual machine (or the responses aren't getting back in).

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]