Proxy

Introduction

All outbound HTTP or HTTPS traffic in a VPC gets sent through the proxy by default.

Though machines inside the VPC need to access resources on the internet, best practices dictate that they can't have unrestricted access.

The proxy provides a single point to manage what resources on the internet can or cannot be accessed.

Our proxy servers run Squid3, an open source web caching proxy, on port 3128.

Whitelisting Domains (allow_list.txt)

When provisioning your proxy we will whitelist a default set of domains that your VPC's instances will require access to.

If you need to whitelist additional domains to your proxy, you can do so by SSH'ing into the proxy and editing /etc/squid3/allow_list.txt

After updating the proxy's allow_list.txt, you'll have to run sudo service squid3 restart before the changes take effect.

There are plans to improve the experience of this in the future using the CLI.

Proxy Environment Variables and HTTP Libraries

If an application is making HTTP requests that time out, it’s because it’s probably not going through the proxy.

If your proxy instance IP address is 10.38.18.22, you'll have the following environment variables set on all your instances.

HTTP_PROXY=http://10.38.18.22:3128
HTTPS_PROXY=http://10.38.18.22:3128
http_proxy=http://10.38.18.22:3128
https_proxy=http://10.38.18.22:3128
NO_PROXY=localhost,127.0.0.1,169.254.169.254,s3.amazonaws.com,internal-myclient-ElbElast-1983RX2W81RAU-1031583335.us-east-1.elb.amazonaws.com
no_proxy=localhost,127.0.0.1,169.254.169.254,s3.amazonaws.com,internal-myclient-ElbElast-1983RX2W81RAU-1031583335.us-east-1.elb.amazonaws.com

These environment variables are the standard way of telling software running on your instance that requests should go through a proxy.

The no_proxy NO_PROXY environment variables are a comma delimited list of domains/hosts that you don't want to send through the proxy.

By default requests to localhost, S3, the Elasticsearch ELB domain, and the EC2 metadata service are all excluded from going through the proxy.

Debugging tip: If you're looking to confirm/deny whether HTTP requests are going through the proxy, you can SSH into the proxy and run tail -f /var/log/squid3/access.log.

If your application has requests that are timing out, it's best to make sure you're using an HTTP library that supports the environment variables:

It's worth noting that Net::HTTP in Ruby's STDLIB does have proxy support, though you'll want to carefully read the documentation as not all request methods support reading from ENV.

Sending Email

All Workarea applications are setup to go send email through the proxy, which acts as an SMTP relay.

Debugging tip: If you're having trouble sending email from your application, SSH into your VPC's proxy and run sudo tail -f /var/log/mail.log.