Skip to main content

reveille and caelumvox.com are live!

As part of a series of projects I'm putting together in an online portfolio, I created reveille (reveille.caelumvox.com), a website that shows articles of local websites inserted by an AWS Step Function job whose lambdas scrape the website and load it into a MariaDB instance. Some details:

  • The Step Function Lambdas are written in Python,
  • The backend API is written in the Express Node Framework,
  • The frontend app is written in the Angular Node Framework using bootstrap for frontend styling and placement for desktop and mobile browsing.

To keep costs low, the frontend, backend, and database are all hosted on one EC2 instance. The frontend and backend are hosted by the same nginx container with a Let's Encrypt certificate.

I also created a home page at caelumvox.com as a starting place for visitors, but it still needs a bit of work. The site is hosted on an AWS Cloudfront distribution. HTTPS only!

Comments

Popular posts from this blog

Software Design Principles - SOLID

The SOLID software design principles weren't called SOLID while I was in grad school, but the concepts were there in my Object Oriented Design course. They're worth mentioning here, primarily because I think once you start coding and become dangerous, it's one of the best ways to stay organized once you incorporate it into your daily coding routines, and it even changes your way of thinking for the better: https://en.wikipedia.org/wiki/SOLID

Matlab and MySQL

I had a lot of data in a MySQL database that I wanted to analyze. I had a copy of Matlab, so I figured the best way to look at this all would be to plot this data and use some GUI elements to go through various combinations. After some Googling, I found this database connector that seemed to do the trick. I downloaded the files, configured mex to use MSVC 2008, built the connector, then I was able to successfully connect over the network! I ran into two problems, though: The connector does not support fetching columns of type TIMESTAMP, and With the magnitude of data (about 180k rows), access times were really slow. I was able to solve problem #1 by changing my columns to DATETIME, which was supported. I'm still trying to figure out problem #2. It may come down to importing all the data directly into Matlab.

AWS Development On A Budget

I was interested in hosting a small portfolio of applications in AWS, but I wanted to keep costs down.  I was willing to maybe host about $30-$40 a month of servers just to showcase some of the applications I was putting together.  Here's what I came up with. The basic capabilities here were: serve up about 6-10 applications that would require some level of web application hosting, database, file storage for larger items, and possibly queuing or email schedule jobs to run periodically to either scrape websites for new data or access APIs regularly These were the services in mind, all out of US-East-1 (Virginia): EC2 instances are pretty cheap.  You can run a t3a.nano for about $3.38 a month, or a t3a.micro for about double that for $6.77.  The more things can run on one of these instances, the better. Lambdas also don't really add up to much if they are scheduled jobs that do not consume large amounts of memory or computing.   The cost of a Lambda run as part of an API gateway