To scrape a javascript-heavy web page you will need a web browser to load the dynamic page into. To automate the browser, you will need something like Puppeteer or Selenium. We opted for selenium-webdriver with Google-chrome running in headless mode. It was useful to install the chrome dev package locally for developing and testing. However running this way in production didn’t work, the Rails App server will grind to a halt after some hours when it eventually runs out of memory. ... Read more

How we successfully used Kamal to leave AWS Beanstalk and deploy anywhere. We were new to containers, so this was part of the learning curve as well. In Part 1 We simply wanted to replicate the example by DHH and use Kamal to deploy to a single server on Hetzner. Includes some errors we overcame. In Part 2 We added the pieces in preparation for deploying the product. The Gotchas are some errors we saw and how to fix them. ... Read more

Up till now we had been developing Rails 6 applications on linux using VMWare on a windows laptop. But now with the general availability of WSL 2, we can retire the old laptop and do everything together in one place. Given that WSL 2 does not currently support GUI applications, we cant expect to replicate the Pepermint distro that was so nice to use. Instead we are taking the opportunity to retool to use VS codeOur toolset uncludes “Git”, “Rails”, “PostgreSQL”, and deployment to “AWS Beanstalk” ... Read more

New Opportunity: Linux on Windows via WSL 2With the now general availability of WSL 2 on Windows 10, it has now become possible to run Linux desktop on a Windows PC without the need of a Virtual Machine. WSL 2 is the 2nd iteration of the Windows Subsystem for Linux, this time with a Linux kernel. We can already utilize WSL 1 with the availability of Bash shell which is vastly superior to the command window (Cmd). ... Read more

To be ready for Rails 6 upgrade we needed to install Ruby 2.5 or above, so we went for ruby 2.6.1 Up till now we had been running Rails 5.2.2 on Ruby 2.4.1. So here are the steps we took:- update ruby version in /.ruby-version file Intall Ruby 2.6 $ sudo apt-get update $ rvm get stable --autolibs=enable gpg: Signature made Sun 30 Dec 2018 10:44:46 GMT using RSA key ID 39499BDB gpg: Can't check signature: No public key Warning, RVM 1. ... Read more

On Jun 29 Nate Berkopec (@nateberkopec) tweeted this: “Observation: when scaling on AWS/AWS-based VPS services, newer, lower-volume Rails applications tend to be memory-bottlenecked, older, high-volume Rails applications tend to be cpu-bottlenecked.” This is really useful to know. We recently saw our tiny new Rails app running on an AWS t2.micro server getting into a “WARN” state after running a sucker punch job. It was using over 90% of its RAM, and stayed that way until we restarted the server. ... Read more

Here are two quick wins for security that will help you sleep better at night: If you are running Rails 5.2 or higher, chances are you are already using the new Content-Security-Policy feature for Cross-Site-Scripting (XSS) protection, it just needs configuring. The Rack-attack gem for throttling spammy clients, and preventing brute-force login attacks. Both of these are quick to get started with, but will benefit from monitoring and fine tuning. ... Read more

How we receive email for our domain using email forwarding on Amazon SES, with minimum effort. Our web-app sends emails no problem, but we also want to receive emails into the same accounts. For example our web-app sends an automatic email from info@mySite.com, but if it receives an email sent to info@mySite.com, we want to handle that too. The simplest way is to implement email forwarding. (The hard way would be to build an email client inside our web-app, then decide how and when to notify whoever needs to handle it). ... Read more

How we aim to meet the requirements for GDPR compliance on a website with minimal use of personal data, using minimum effort. *Disclaimer - we are not lawyers, so don’t take any of this as legal advice! What is GDPR?: GDPR is the EU “General Data Protection Regulation”. It is more detailed and specific than the UK Data Protection Act 1998. It will be enforced from 25 May 2018. Compliance for simple cases like ours appears to be mainly about implementing effective privacy notice(s) and reveiwing how you ask for, manage and record consent. ... Read more

How we started to use Rails 5.2 Encrypted Secrets for securing Amazon S3 key pairs for use with Active Storage. Here is a useful starting point: Rails Encrypted Credentials on Rails 5.2 Our development and production envionment is Linux, and we precompile our assets before deployment. Part 1: Encrypt your secrets With Rails 5.2 a sort of ‘password safe’ is introduced, whereby you commit your encrypted secret credentials to source control, but not your master password. ... Read more