Soon after we updated to Rails 4.2.1 from 4.2.0 deployed - the site stopped responding and we saw this error in the passenger log:
phusion_passenger/preloader_shared_helpers.rb:69:in `fork': Cannot allocate memory - fork
That was also the last time emails were successfully sent from Elastic Beanstalk. Now SMPT emails are silently failing with no clues in the logs.
We were unsure if the memory issue was with this AWS instance or this update. Later decided it was the instance.
To get the site up again, we tried a few things like cloning the EB environment, in the end we got the site back up on the clone running on a previous code base release, then on the current release ( after we granted the new EC2 security group to the RDS security group ). We swapped the CNAME and site is back working. In retrospect we are not sure why autoscaling didn’t kill this unhealthy instance and spin up a new one. Maybe we didn’t wait long enough for the rule to trigger? Odd.
So back to the email issue. We are not a heavy user of emails but this month may have been heavier than usual, about 500 sent. Also the act of creating a new EB environment meant a new IP was assigned.
The fix was to submit a request to AWS to have the email limit removed About 12 hours later AWS approved the request, and after a further 12 hours emails were being sent again. AWS recommended “assigning a reverse DNS record to the Elastic IP address you use to send email”. But with Elastic Beanstalk this is not best practice, the alternative was to start using SES as explained here.
We haven’t moved to using SES yet, but doesn’t look too difficult to do with Rails : Getting started quickly with Amazon Email Sending Service (SES) and Ruby on Rails