FTP active mode in AWS

FTP is one of the oldest Internet protocols. Unfortunately, it was designed for environments where clients and servers interact with each other with a minimum of restriction. Therefore, FTP protocol doesn't work well in scenarios including NAT and/or Firewalls.

As you may know, AWS instances from OS perspective live in a network with private IPs. When an instance (EC2 classic or VPC) needs to communicate with another VPC, region or outside AWS network, some special devices handle the NAT translation between the private and the public IP. This AWS mechanism provides you maximum flexibility, allowing you to move a public IP from one instance to another easily, for example.

FTP protocol was designed without this concepts in mind. So, in complex network including NAT and/or Firewalls, the protocol doesn't work well. An example is FTP active mode. Imagine next scenario:

• One instance (FTP client) in Frankfurt region
• Private IP: 172.31.16.100
• Public IP: 52.28.244.154
• Another instance (FTP server) in Tokyo region
• Private IP: 172.31.14.185
• Public IP: 52.69.174.237
• Security groups allow any traffic between both instances

 If we try to communicate using a standard FTP client such as ncftp with active mode, you will experience next issue:

1. FTP client will be able to connect
2. FTP client will be able to authenticate
3. FTP client will fail when trying to list files

The issue is well covered in next public documentation from ncftp team. Please, take a look on previous information (specially the 'Why PORT Poses Problems for Routing Devices' section).

Inside the instance, FTP client can only see private IPs. So, when the client try to connect with the remote FTP server, it will send private IP information. If we perform a packet capture in FTP server side during previous test, we will see how the FTP protocol information inside the packet includes 'PORT' request using the private IP of the FTP client instead of the public IP (check the packet highlighted in blue):

Because of this, FTP data connection from server will fail (basically: FTP server won't be able to find a route back to source instance).

To work around this, I recommend you:

1. Use FTP passive mode instead of active. The passive mode works better under NAT/Firewalls. Most of the FTP services include parameters to specify the public IP address to handle this kind of scenarios.
2. Use another method to send files. FTP is an old protocol and has many limitations about functionality and security. Probably, move to a modern transfer protocol (like SSH) could be a good alternative.
3. Use a FTP client with options to specify public IP. FileZilla ftp client has a specific parameter to specify the public IP when using active mode.
4. Patch FTP to specify Public IP instead of Private IP. With this method, both instances will be able to establish and complete the communication. 

If for whatever reason you need to use FTP active mode in AWS and the 3rd option is not possible, I suggest you to patch the FTP client to workaround the issue. For example, you can obtain the public IP of an instance from metadata information. 

I created a patch for the latest stable version of ncftp client to replace the private IP in 'PORT' request with the public IP (if available through metadata) when the FTP Server (destination) is not a private IP. The result: the FTP server is able to establish the FTP data connection and complete the request, even in active mode. Example output of the same test with the customized FTP client:

Please, note in the previous tcpdump output the highlighted packet in blue how the 'PORT' directive includes the public IP of the instance instead of the private IP.

If you want to use this customized ftp client, you can download the patched source-code version from the next link.

Also, if you want to patch from the vanilla version, download the latest stable version (v3.2.5) of ncftp from here and patch next files:

• configure
• libncftp/ftp.c

 The customized ncftp client requires development curl libraries (libcurl-devel package, in Amazon Linux) in order to compile correctly.

Finally, here you can see patch details:

configure

libncftp/ftp.c