Introduction

calendar is a helper to track your goals.

Source Code

The source code from which this book is generated can be found on Github.

Get started

This page describes how to start the calendar on local machine for development purposes.

calendar-backend

Prerequisites

Before starting the backend, you will need to have the rust toolchain installed.

Follow the instructions from official site.

Start the backend

Clone the git repo

git clone https://github.com/calendar-team/calendar-backend.git

cd into the directory

cd calendar-backend

Start the app

cargo run

calendar-frontend

Prerequisites

Before starting the frontend, you will need to have Node.js and npm installed.

Follow the instruction from official site.

Start the frontend

Clone the git repo

git clone https://github.com/calendar-team/calendar-frontend.git

cd into the directory

cd calendar-frontend

Install locally all the dependencies from the project

npm install

Start the app

npm run dev

Architecture overview

Currently, the calendar project has 3 components:

flowchart TD
  subgraph browser["fab:fa-internet-explorer Browser"]
    direction TB
    calendar-frontend-client-side("Client side of calendar-frontend")
  end

  subgraph vercel["fab:fa-server Vercel"]
    calendar-frontend-server-side("Server side of calendar-frontend")
  end

  subgraph ec2["fab:fa-server EC2"]
    direction TB
    calendar-backend --> sqlite
  end

  client("fal:fa-user Client")
  sqlite("fab:fa-database SQLite")
  calendar-backend(calendar-backend)

  client -- "<a href='https://calendar.aguzovatii.com' target='_blank'>https://calendar.aguzovatii.com</a>" --> browser
  browser --> vercel --> ec2
  browser --> ec2

calendar-fronted

Deployment

The calendar-fronted is deployed on Vercel. The deployment is done automatically every time a new commit is made on the main branch.

Source Code

The calendar-fronted source code can be found on Github.

calendar-backend

Currently it also contains an in-memory sqlite database.

Deployment

The calendar-backend is deployed on EC2.

The deployment is done automatically every time a new commit is made on master branch.

Source Code

The calendar-backend source code can be found on Github.

Deploy calendar-backend to EC2

Currently, the calendar-backend is deployed to EC2.

This guide will describe how to deploy on a new EC2 machine and how to configure the deployment pipeline.

  1. First we need to launch a new EC2 instance new_ec2_instance_for_backend.png

    1. Choose Amazon Linux as OS and a 64 bit architecture new_ec2_instance_for_backend_os.png
    2. Instance type: t2.nano new_ec2_instance_for_backend_type.png This is the cheapest and smallest instance type.
    3. Choose calendar-backend as the key pair new_ec2_instance_for_backend_key_pair.png Or we can create a new key pair, just make sure to update the github secret (details will follow in a step below).
    4. Use calendar-backend security group new_ec2_instance_for_backend_sg.png Or we can create a new security group, we will need the following ports to be added under the inbound rules:
      • 22 - SSH port used by our github-actions workflow to deploy new version of calendar-backend
      • 80 - used by certbot to renew the TLS certificate (details will follow in a step below)
      • 8080 - this is the port that calendar-backend uses
    5. Allocate a 30 GB gp3 disk new_ec2_instance_for_backend_disk.png
    6. Create S3 bucket for DB backup
      • Set calendar-db-backup as the name and use eu-central-1 as the region: new_s3_bucket_for_db_backup.png
      • Keep ACLs disabled as object ownership option: new_s3_bucket_for_db_backup_oo.png
      • Block public access: new_s3_bucket_for_db_backup_pa.png
      • Enable versioning: new_s3_bucket_for_db_backup_v.png
      • Use default encryption: new_s3_bucket_for_db_backup_enc.png
    7. Create a new IAM Policy for S3 put access:
      • Choose S3 as the service and PutObject as access level: new_iam_policy_for_s3_upload.png
      • Add a new ARN for calendar-db-backup bucket and database.db3 file: new_iam_policy_for_s3_upload_arn.png
      • Use calendar_db_backup as the pollicy name: new_iam_policy_for_s3_upload_name.png
    8. Create a new IAM role
      • Choose AWS service as the trusted entity type: new_iam_role_for_s3_upload.png
      • Use EC2 as the use case: new_iam_role_for_s3_upload_uc.png
      • Choose the previously created IAM Policy under permissions policies: new_iam_role_for_s3_upload_policy.png
      • Use calendar_db_backup as the name: new_iam_role_for_s3_upload_name.png
    9. Attach role to EC2 instance:
      • Choose the EC2 instance, then click on Actions, then click on Modify IAM role under Security attach_iam_role_to_ec2.png
      • Choose the previously created IAM role: attach_iam_role_to_ec2_new_role.png
    10. Configure cron on EC2
      • SSH into the EC2 machine
      • Install crond because it's not present by default:
        • install cronie 
          sudo dnf install cronie
        • enable crond 
          sudo systemctl enable crond
        • check that crond is running as expected 
          sudo systemctl status crond
          The result should look like: 
          ● crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; preset: enabled)
Active: active (running) since Tue 2024-04-16 06:59:54 UTC; 4min 57s ago
Main PID: 1858743 (crond)
Tasks: 2 (limit: 510)
...
      • install sqlite 
        sudo yum install sqlite-devel
      • create a new file db_backup.sh in home directory (/home/ec2-user): 
        cd /home/ec2-user
sqlite3 database.db3 ".backup 'database_backup.db3'"
aws s3 cp database.db3 s3://calendar-db-backup/database.db3
rm database_backup.db3
      • give exec permissions to the previous file: 
        chmod +x db_backup.sh
      • run: 
        echo "0 0,6,12,18 * * * root /home/ec2-user/db_backup.sh" | sudo tee -a /etc/crontab > /dev/null
        This will upload the database to S3 bucket every day at 00:00, 06:00, 12:00, 18:00 UTC time

  2. Configure the TLS certificate by following the tutorial from here.

    Note that you will have to ssh into the EC2 machine for setting the certificate. To do this we will need:

    • the private key from the key pair that we chose on instance creation from above
    • the public DNS of our EC2 instance. This can be found in AWS console on instance details
    • run the following command from a terminal: 
      ssh -i PATH_TO_SSH_KEY ec2-user@EC2_PUBLIC_DNS

    Also make sure that the calendar-backend is pointing to the correct certificate.

  3. Configure the deployment pipeline. For this we will have to update a few github secrets:

  4. Finally we can trigger Deploy to EC2 workflow to deploy calendar-backend to the newly created EC2 instance

You can find a sequence diagram explaining the process of deploying to EC2 here.

ADRs

Use Markdown Any Decision Records

Context and Problem Statement

We want to record any decisions made in this project independent whether decisions concern the architecture ("architectural decision record"), the code, or other fields. Which format and structure should these records follow?

Considered Options

Decision Outcome

Chosen option: "MADR 3.0.0", because

  • Implicit assumptions should be made explicit. Design documentation is important to enable people understanding the decisions later on. See also A rational design process: How and why to fake it.
  • MADR allows for structured capturing of any decision.
  • The MADR format is lean and fits our development style.
  • The MADR structure is comprehensible and facilitates usage & maintenance.
  • The MADR project is vivid.
status: accepted
date: 2023-09-03

calendar-backend CI/CD pipeline

Context and Problem Statement

We want to fully automate the process of deployment of calendar-backend.

On every commit on master branch, a new version of the application should be deployed in production.

On every commit on any branch, the application is compiled and all the tests are executed. No merge is allowed if at least one test fails.

Considered Options

  • Jenkins
  • Github Actions with AWS CodeDeploy
  • Github Actions with SSH and rsync

Decision Outcome

Chosen option: Github Actions with SSH and rsync

This option was chosen because it is the simplest one, so it allowed us to achieve a functional CI/CD pipeline faster.

Confirmation

Implementation of the ADR was confirmed through a PR review in #3

Pros and Cons of the Options

Jenkins

Deploy a self-managed Jenkins instance, connect to Github, and react to events from Github.

On every commit on any branch, build the app, then run all the tests.

On commit on master branch, deploy the new version on EC2.

  • Good, because this is the most configurable option. Jenkins allows to configure the Pipeline however we need
  • Good, becauses there is no limit in number of builds, we can run as often as we need
  • Bad, because we need to host the Jenkins instance somewhere, and this may imply additional costs
  • Bad, because we need to maintain the instance up to date and healthy all the time, and this may imply additional effort in the future
  • Bad, because it will slow us down, we need to configure and run the Jenkins instance before starting to configure the CI/CD pipeline itself

Github Actions with AWS CodeDeploy

See the details in the article: link

  • Good, because we don't manage any services, Github Actions can be used out of the box
  • Good, because it decouples from any particular EC2 instance, so EC2 instances can be killed at any moment without disruption
  • Bad, because we need to configure additional AWS services, which makes us more coupled with AWS
  • Bad, because we need to configure multiple AWS services, which will slow us down
  • Bad, because we need to configure multiple AWS services, which may imply additional costs

Github Actions with SSH and rsync

Explained through a sequence diagram:

sequenceDiagram
    autonumber
    actor D as Developer
    participant G as Github
    D->>G: merge PR to `master`
    participant GA as Github Actions
    G->>GA: trigger `Deploy to EC2` workflow
    activate GA
    participant EC2
    GA->>EC2: rsync
    Note right of GA: upload the new binary to EC2
    GA->>EC2: kill the current instance of `calendar-backend`
    GA->>EC2: start the newly uploaded binary
    GA-->>G: mark commit as successful
    deactivate GA

The implementation can be found in the git repo: link

  • Good, because we don't manage any services, Github Actions can be used out of the box
  • Good, because we don't couple to AWS too much
  • Good, because it is a simple solution which can be implemented fast
  • Bad, because it is fragile, every time the EC2 instance is restarted and changes the IP address, the CI/CD pipeline breaks, because we are connecting directly to the instance