The Tech Chorus DevOps Platform

In the last decade, the way people develop and deploy software has evolved drastically. There is an amalgamation of various sub-disciplines such as

• IT engineering - primarily deals with hardware and networking infrastructure
• System administration - responsible for managing servers in the rack, data center or at a colocation service
• Cloud infrastructure engineering - exclusively manage the infrastructure in the public cloud
• Platform engineering - build and maintain a platform to deploy software applications. Shields the users from the complixity.

All of these sub-disciplene can be called DevOps engineering.

In a previous series of blog posts, we discussed the Kubernetes objects typically used to run a web application. We covered, Namespace, Pod, ConfigMap, Secret, Service, Deployment, ServiceAccount, Ingress, PDB, HPA, PV, PVC, Job and CronJob.

In this post, we will discuss some fundamental building blocks for the Kubernetes cluster administration: RBAC objects.

In most situations, Kubernetes API server is started with the flag --authorization-mode=RBAC which enables RBAC in the cluster.

Introduction

We have the host machine with the OS Ubuntu 22.04. On this PC or server, we will create two virtual machine guests:

• myubuntu2204test01 having static IP of 192.168.122.146
• myubuntu2204test02 having static IP of 192.168.122.147

The guest VMs will use the default network created by libvirt. The gateway IP for the default network is 192.168.122.1.

We will achieve automation using libvirt, qemu and cloud-init. To go through the article and exercise, you should have a rudimentary understanding of Linux system administration and networking.

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

Purpose Of Having Naming Conventions For An Ansible Project

• Consistency: Adopting a naming convention standardizes naming across the project and organization. This makes it easier for developers to switch between projects. Typically, an organization with an infrastructure team will have several Ansible projects and source code repositories. A developer working on one such Ansible project can seamlessly switch to another given a standard naming convention.
• Error Reduction: With improper naming there can be pitfalls. For instance, using hyphens(-) in variable names can cause parsing errors since Ansible might interpret them as the minus operator. Standardizing naming for variables can help avoid collisions.
• Readability: With a standard naming convention, it helps developers to find variables, files and tasks quickly.

Naming Conventions For Ansible Roles And Playbooks

• Role name : use lowercase letters and hyphens to separate words: For example, web-server or database-backup.
• Task file name: user lowercase letters and underscores to separate words. When statements such as include are used, it is convenient to have file names without hyphens. For example: install_web_ubuntu.yml
• YAML file extension: yml instead yaml. To be consistent and succinct.
• Task name: start with a verb: Use an action verb at the beginning of the task name to indicate what the role does. For example, Install Nginx or Configure firewall. Start with a capital letter. No need to end with a period for a few words of task description.
• Variable name: lowercase letters and underscores to separate words. The variable must start with the role name. For example, if the role name is nginx, the variable name should be nginx_default_hostname.

Content Guide

• Always include a README file for the role. Describe in detail how to use the role. Create a table to show the role variables and their defaults.
• Include automated tests using Molecule.

Ansible Prerequisites

Before jumping on to learning Ansible, have a firm grounding in Linux system administration and shell scripting. You can use Ansible for a lot of automation projects. The primary target audience for this blog post are DevOps engineers, IT infrastructure engineers and system administrators who create and manage IT infrastructure to run workloads. A good understanding of YAML is required before starting to write Ansible playbooks. A background in at least one programming language helps. Python programing is not a requirement per se. But Python programming familiarity helps put together some automated testing.

Architecture 1: Kubernetes Control Plane Without HA

Create three Virtual Machine guests on your laptop or workstation.

• VM 01 - Kubernetes Control Plane
• VM 02, VM 03 - Kubernetes Worker nodes

Architecture 2: Kubernetes Control Plane With HA

Create five Virtual Machine guests on your laptop or workstation.

• VM 01, VM 02, VM 03 - Kubernetes Control Plane with HA
• VM 04, VM 05 - Kubernetes Worker nodes

Use the Kubeadm tool to create the cluster.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

In the Kubernetes Objects Required For A Typical Web Application post we talked about few Kubernetes objects that a web application developer should get accustomed to. In this post, we will extend the series and talk about more objects that can help web developers scale their applications.

As we delve deeper into Kubernetes topics, the demarcation of roles and skill sets start to reveal. In larger organizations, a dedicated team of infrastructure engineers design and make choices of network topology, IAC tooling and orchestration of the Kubernetes clusters and CI/CD pipelines. Typically, such DevOps engineers manage scaling and storage by installing the required controllers and CSI drivers. Application developers whose applications are deployed on the cluster maybe able to tune certain parameters of certain objects to manage the scaling needs of their applications. Depending on the situation, the developers maybe able to request and use storage volumes with certain restrictions. Regardless of the organization’s team structure, you should be able to learn about these Kubernetes concepts and objects and play with them locally on your laptop using Minikube.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

NFS Server

To run your own NFS (Network File System) server, you’ll need to first install Linux on your server. I’d go with Debian, Ubuntu or CentOS distributions. Once you’ve done that, you can install the NFS server software, which is typically included with most Linux distributions. After the NFS server software is installed, you’ll need to configure it to specify which directories on the server should be shared with NFS clients, and what type of access they should have. This typically involves editing the server’s configuration file, which is typically located at /etc/exports. Once the server is configured, you can start the NFS server and begin sharing files with NFS clients. To access the shared files, the NFS clients will need to mount the NFS share using the mount command.

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

There are many tools and software programs that can be used for monitoring and performance analysis on Linux systems. Some popular options include:

• top - This is a command-line utility that shows real-time information about the processes running on a Linux system, such as their CPU and memory usage.
• htop - This is a more advanced version of top that provides a more user-friendly interface and additional features, such as the ability to sort processes by different metrics and to kill processes.
• sar - This is a command-line utility that collects and displays performance metrics for a Linux system over time. It can be used to analyze CPU, memory, I/O, and network usage, as well as other metrics.
• iostat - This is a command-line utility that shows real-time information about I/O performance on a Linux system. It can be used to monitor the performance of disks and other storage devices.
• vmstat - This is a command-line utility that shows real-time information about various system resources, such as memory, CPU, and I/O. It can be used to monitor the overall health of a Linux system.
• netstat - This is a command-line utility that shows information about network connections on a Linux system. It can be used to monitor the status of network connections and to diagnose networking issues.

There are also many modern monitoring tools and software programs available for Linux, such as Prometheus and Zabbix. These tools typically offer more advanced features and capabilities than the built-in Linux utilities, such as the ability to collect and store metrics over time, and to generate alerts when certain conditions are met.

To run your own email server using Linux and other open source software, you’ll need to first choose a Linux distribution and install it on your server. I’d go with either Ubuntu, Debian or Rocky Linux. Once you’ve done that, you can choose an email server software that is compatible with Linux, such as Postfix or Exim. After installing and configuring the email server software, you’ll need to set up DNS records and configure authentication and encryption to ensure that your email server is secure. Finally, you’ll need to test your email server to make sure it is working properly and can send and receive messages.

Definition

A load balancer is a type of software or hardware that distributes incoming traffic across multiple servers or resources. This allows the load balancer to distribute the workload evenly, improving the performance and availability of the application.

The Load Balancer Lab

To run your own load balancer using open source software, you will need to:

• Install and configure the load balancer software on a server. Some popular open source options include HAProxy, Nginx, and Envoy.
• Configure the load balancer to distribute incoming traffic to the appropriate servers or resources. This typically involves setting up virtual servers and defining rules for routing traffic.
• Test the load balancer to ensure that it is working correctly and distributing traffic as expected.
• Monitor the load balancer and the underlying servers to ensure that the system is performing well and handling traffic effectively.
• Continually tune and optimize the load balancer configuration to improve performance and ensure that the application is always available and responsive.

Your web applications need a solution to store and retrieve its data. A relational database is often used in web applications. MySQL, MariaDB and PostgreSQL are some popular relational databases. There’s also SQLite. Many applications can use any of these relational databases by the virtue of the database layer abstraction. If you are writing your own web application, pick any one database and install it on your web server.

Install The Database

• Install the package.
• Enable and start the database systemd unit.
• Initialize the database server.
• Create databases.
• Create database users and set their passwords.

Connect The Web Application To The Database Server

I see a lot of people are trying to break into software engineering and DevOps. There are many inspiring stories out there. People from other industries and verticals have switched to IT and are having a successful career. You can do it too.

Let us look at the resources available to you.

Online Vs. Offline Training And Learning Resources

Online

All you need to break into DevOps is:

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

From an application developer and Kubernetes user’s point of view, you have to have a working knowledge of Kubernetes. The post outlines the most important Kubernetes objects required to deploy a typical web application. Let us assume that the web application uses the two-tier architecture. We also assume that the cluster is created and administered by an infrastructure or DevOps engineer and the necessary access is provided to the developer to deploy their web application onto the Kubernetes cluster. The Kubernetes operations are performed from the web application developer’s perspective.

Introduction

If you have chosen the DevOps engineering path for your career, Linux system administration skills are a must. Companies run their workloads on Linux on-premise as well as in the cloud. Many developers write software on their Linux laptops and workstations. In this post, I will lay out a plan to master Linux system administration skills. The journey begins with you becoming a Linux desktop user. Gradually, you start using your laptop as a Linux server. Become more productive and create your own labs using virtualization. You will be able to simulate a lot of Linux infrastructure scenarios on your laptop.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

As soon as you are ready to deploy services and applications on the Internet, the first thing you need is a registered domain and a DNS server. Often people just use the DNS service provided by the domain registrar or the hosting or cloud provider. Using the managed service is not a requirement. You can host your own DNS server and manage the DNS for your domains. There are many open source software using which you can build your own DNS servers. Bind is the traditional and probably the most popular DNS solution out there. There are other options such as Knot, PowerDNS and Unbound.

Introduction

The Linux Kernel parameters are settings that can be configured to control the behavior of the Linux kernel. They are typically used to fine-tune system performance or to enable/disable certain features.

Some examples of kernel parameters include:

Memory-related parameters: These parameters control how the kernel manages system memory, including how much memory is allocated to user processes and how aggressively the kernel caches data.

Processor-related parameters: These parameters control how the kernel interacts with the system’s processors, including how it schedules processes and how it handles interrupts.

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

The build and deploy engineer is a role in IT/infrastructure/devops teams of organizations that is responsible for deploying a set of applications using CI/CD pipelines. Usually, they use the pipelines, tools and processes created by DevOps engineers, consultants or practitioners. They tweak things with tools like Ansible, Jenkins, Git, Maven, etc. here and there a bit.

Are you one such build and deploy engineer looking to transition your career into DevOps engineering? Here’s a path you could follow:

The Tech Chorus blog by Sudheer Satyanarayana started way back in 2008. The blog has used various hosting technologies and software over the years.

Initially, the blog site used Drupal content management system. LAMP was a popular technology stack those days. Drupal was one of my favorite CMSes back then. The Drupal site was hosted on a cPanel server.

As life progressed, there was not enough time to patch Drupal and its plugins. Upgrading between major versions of Drupal also took considerable amount of time and effort. I moved on to a static site generator. I found and loved Lektor. With Lektor, I managed the content on my workstation and generated the static site. I used to deploy the generated static site to an AWS EC2 instance with Nginx.

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

Install the Ansible role gavika.aws_cloudwatchagent via Galaxy

ansible-galaxy install gavika.aws_cloudwatchagent

Create The Playbook File - cw-play.yml :

---
- hosts: all
  become: true
  vars:
  roles:
    - role: gavika.aws_cloudwatchagent

Prepare the AWS CloudWatch Agent configuration

In your variables file, use aws_cloudwatch_agent_config

agent:
    metrics_collection_interval: 60
    run_as_user: "cwagent"
    metrics:
      namespace: "Gavika"
      append_dimensions:
        InstanceId: "${aws:InstanceId}"
      metrics_collected:
        disk:
          measurement:
            - used_percent
          metrics_collection_interval: 60
          resources:
            - "*"
        mem:
          measurement:
            - mem_used_percent
          metrics_collection_interval: 60

In this example, I am using the namespace, Gavika. Feel free to change it. We collect the cpu, disk, diskio, mem and swap metrics. The agent will send these metrics once in 360 seconds.

Install PostgreSQL Server

Fedora and CentOS:

sudo dnf install postgresql-server

Ubuntu 18.04:

sudo apt install postgresql

New Server Initialization

On CentOS 7/Fedora 30:

sudo postgresql-setup initdb

Upgrading From An Older Version

sudo postgresql-setup --upgrade

Administering The Database Server

Managing The postgresql Daemon

Starting PostgreSQL server

sudo systemctl start postgresql

Checking PostgreSQL Server Status:

sudo systemctl status postgresql

Enabling PostgreSQL Server Systemd Unit/Enabling PostgreSQL Server On Boot:

sudo systemctl enable postgresql

Allowing Password Based Login From localhost

Edit /var/lib/pgsql/data/pg_hba.conf as privileged user(root) and add this line:

The Tech Chorus DevOps Platform

In the last decade, the way people develop and deploy software has evolved drastically. There is an amalgamation of various sub-disciplines such as

• IT engineering - primarily deals with hardware and networking infrastructure
• System administration - responsible for managing servers in the rack, data center or at a colocation service
• Cloud infrastructure engineering - exclusively manage the infrastructure in the public cloud
• Platform engineering - build and maintain a platform to deploy software applications. Shields the users from the complixity.

All of these sub-disciplene can be called DevOps engineering.

In a previous series of blog posts, we discussed the Kubernetes objects typically used to run a web application. We covered, Namespace, Pod, ConfigMap, Secret, Service, Deployment, ServiceAccount, Ingress, PDB, HPA, PV, PVC, Job and CronJob.

In this post, we will discuss some fundamental building blocks for the Kubernetes cluster administration: RBAC objects.

In most situations, Kubernetes API server is started with the flag --authorization-mode=RBAC which enables RBAC in the cluster.

Purpose Of Having Naming Conventions For An Ansible Project

• Consistency: Adopting a naming convention standardizes naming across the project and organization. This makes it easier for developers to switch between projects. Typically, an organization with an infrastructure team will have several Ansible projects and source code repositories. A developer working on one such Ansible project can seamlessly switch to another given a standard naming convention.
• Error Reduction: With improper naming there can be pitfalls. For instance, using hyphens(-) in variable names can cause parsing errors since Ansible might interpret them as the minus operator. Standardizing naming for variables can help avoid collisions.
• Readability: With a standard naming convention, it helps developers to find variables, files and tasks quickly.

Naming Conventions For Ansible Roles And Playbooks

• Role name : use lowercase letters and hyphens to separate words: For example, web-server or database-backup.
• Task file name: user lowercase letters and underscores to separate words. When statements such as include are used, it is convenient to have file names without hyphens. For example: install_web_ubuntu.yml
• YAML file extension: yml instead yaml. To be consistent and succinct.
• Task name: start with a verb: Use an action verb at the beginning of the task name to indicate what the role does. For example, Install Nginx or Configure firewall. Start with a capital letter. No need to end with a period for a few words of task description.
• Variable name: lowercase letters and underscores to separate words. The variable must start with the role name. For example, if the role name is nginx, the variable name should be nginx_default_hostname.

Content Guide

• Always include a README file for the role. Describe in detail how to use the role. Create a table to show the role variables and their defaults.
• Include automated tests using Molecule.

Architecture 1: Kubernetes Control Plane Without HA

Create three Virtual Machine guests on your laptop or workstation.

• VM 01 - Kubernetes Control Plane
• VM 02, VM 03 - Kubernetes Worker nodes

Architecture 2: Kubernetes Control Plane With HA

Create five Virtual Machine guests on your laptop or workstation.

• VM 01, VM 02, VM 03 - Kubernetes Control Plane with HA
• VM 04, VM 05 - Kubernetes Worker nodes

Use the Kubeadm tool to create the cluster.

In the Kubernetes Objects Required For A Typical Web Application post we talked about few Kubernetes objects that a web application developer should get accustomed to. In this post, we will extend the series and talk about more objects that can help web developers scale their applications.

As we delve deeper into Kubernetes topics, the demarcation of roles and skill sets start to reveal. In larger organizations, a dedicated team of infrastructure engineers design and make choices of network topology, IAC tooling and orchestration of the Kubernetes clusters and CI/CD pipelines. Typically, such DevOps engineers manage scaling and storage by installing the required controllers and CSI drivers. Application developers whose applications are deployed on the cluster maybe able to tune certain parameters of certain objects to manage the scaling needs of their applications. Depending on the situation, the developers maybe able to request and use storage volumes with certain restrictions. Regardless of the organization’s team structure, you should be able to learn about these Kubernetes concepts and objects and play with them locally on your laptop using Minikube.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

Introduction

If you have chosen the DevOps engineering path for your career, Linux system administration skills are a must. Companies run their workloads on Linux on-premise as well as in the cloud. Many developers write software on their Linux laptops and workstations. In this post, I will lay out a plan to master Linux system administration skills. The journey begins with you becoming a Linux desktop user. Gradually, you start using your laptop as a Linux server. Become more productive and create your own labs using virtualization. You will be able to simulate a lot of Linux infrastructure scenarios on your laptop.

The Tech Chorus DevOps Platform

In the last decade, the way people develop and deploy software has evolved drastically. There is an amalgamation of various sub-disciplines such as

• IT engineering - primarily deals with hardware and networking infrastructure
• System administration - responsible for managing servers in the rack, data center or at a colocation service
• Cloud infrastructure engineering - exclusively manage the infrastructure in the public cloud
• Platform engineering - build and maintain a platform to deploy software applications. Shields the users from the complixity.

All of these sub-disciplene can be called DevOps engineering.

In a previous series of blog posts, we discussed the Kubernetes objects typically used to run a web application. We covered, Namespace, Pod, ConfigMap, Secret, Service, Deployment, ServiceAccount, Ingress, PDB, HPA, PV, PVC, Job and CronJob.

In this post, we will discuss some fundamental building blocks for the Kubernetes cluster administration: RBAC objects.

In most situations, Kubernetes API server is started with the flag --authorization-mode=RBAC which enables RBAC in the cluster.

This is a guide to write a Kubernetes controller. We will kick off by inspecting the Kubernetes API from inside a pod within the cluster. Minikube suffices for this exercise. But you can conduct the exercise to any Kubernetes cluster.

The controller watches events related to Kubernetes pods using the Kubernetes API. When there is a new event, the controller logs the event’s type and the name of the affected pod. This controller can be extended to perform other actions when pod events occur, such as scaling the number of replicas for a deployment, sending notifications, or triggering a custom script or program.

Architecture 1: Kubernetes Control Plane Without HA

Create three Virtual Machine guests on your laptop or workstation.

• VM 01 - Kubernetes Control Plane
• VM 02, VM 03 - Kubernetes Worker nodes

Architecture 2: Kubernetes Control Plane With HA

Create five Virtual Machine guests on your laptop or workstation.

• VM 01, VM 02, VM 03 - Kubernetes Control Plane with HA
• VM 04, VM 05 - Kubernetes Worker nodes

Use the Kubeadm tool to create the cluster.

In the Kubernetes Objects Required For A Typical Web Application post we talked about few Kubernetes objects that a web application developer should get accustomed to. In this post, we will extend the series and talk about more objects that can help web developers scale their applications.

As we delve deeper into Kubernetes topics, the demarcation of roles and skill sets start to reveal. In larger organizations, a dedicated team of infrastructure engineers design and make choices of network topology, IAC tooling and orchestration of the Kubernetes clusters and CI/CD pipelines. Typically, such DevOps engineers manage scaling and storage by installing the required controllers and CSI drivers. Application developers whose applications are deployed on the cluster maybe able to tune certain parameters of certain objects to manage the scaling needs of their applications. Depending on the situation, the developers maybe able to request and use storage volumes with certain restrictions. Regardless of the organization’s team structure, you should be able to learn about these Kubernetes concepts and objects and play with them locally on your laptop using Minikube.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

From an application developer and Kubernetes user’s point of view, you have to have a working knowledge of Kubernetes. The post outlines the most important Kubernetes objects required to deploy a typical web application. Let us assume that the web application uses the two-tier architecture. We also assume that the cluster is created and administered by an infrastructure or DevOps engineer and the necessary access is provided to the developer to deploy their web application onto the Kubernetes cluster. The Kubernetes operations are performed from the web application developer’s perspective.

The Tech Chorus DevOps Platform

In the last decade, the way people develop and deploy software has evolved drastically. There is an amalgamation of various sub-disciplines such as

• IT engineering - primarily deals with hardware and networking infrastructure
• System administration - responsible for managing servers in the rack, data center or at a colocation service
• Cloud infrastructure engineering - exclusively manage the infrastructure in the public cloud
• Platform engineering - build and maintain a platform to deploy software applications. Shields the users from the complixity.

All of these sub-disciplene can be called DevOps engineering.

Data Structures And Algorithms

• Data Structures: A data structure is a technique of storing data in a computer so that it can be accessed and modified efficiently.
• Algorithms: An algorithm is a step-by-step instruction to perform a task.

Why Should I Learn Data Structures And Algorithms?

• Enhance your problem solving skills.
• Write efficient code that is performant and scalable.
• Specialized fields: machine learning, data science, artificial intelligence and other engineering fields deal with complex data and require efficient processing. To delve into such fascinating fields of engineering, having a firm grounding in data structures and algorithms is beneficial and in many cases required.

Can I Develop Applications Without Studying Data Structures And Algorithms?

Yes, sure. A lot of applications just store some data in a datastore, typically a relational database and have some procedures to show this data in a user interface, most commonly the web browser. The crux of such applications can be defined using the acronym CRUD. CRUD stands for Create, Read, Update and Delete. Often you are doing one of these CRUD functions in the context of persistent storage of your application:

Purpose Of Having Naming Conventions For An Ansible Project

• Consistency: Adopting a naming convention standardizes naming across the project and organization. This makes it easier for developers to switch between projects. Typically, an organization with an infrastructure team will have several Ansible projects and source code repositories. A developer working on one such Ansible project can seamlessly switch to another given a standard naming convention.
• Error Reduction: With improper naming there can be pitfalls. For instance, using hyphens(-) in variable names can cause parsing errors since Ansible might interpret them as the minus operator. Standardizing naming for variables can help avoid collisions.
• Readability: With a standard naming convention, it helps developers to find variables, files and tasks quickly.

Naming Conventions For Ansible Roles And Playbooks

• Role name : use lowercase letters and hyphens to separate words: For example, web-server or database-backup.
• Task file name: user lowercase letters and underscores to separate words. When statements such as include are used, it is convenient to have file names without hyphens. For example: install_web_ubuntu.yml
• YAML file extension: yml instead yaml. To be consistent and succinct.
• Task name: start with a verb: Use an action verb at the beginning of the task name to indicate what the role does. For example, Install Nginx or Configure firewall. Start with a capital letter. No need to end with a period for a few words of task description.
• Variable name: lowercase letters and underscores to separate words. The variable must start with the role name. For example, if the role name is nginx, the variable name should be nginx_default_hostname.

Content Guide

• Always include a README file for the role. Describe in detail how to use the role. Create a table to show the role variables and their defaults.
• Include automated tests using Molecule.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

In the Kubernetes Objects Required For A Typical Web Application post we talked about few Kubernetes objects that a web application developer should get accustomed to. In this post, we will extend the series and talk about more objects that can help web developers scale their applications.

As we delve deeper into Kubernetes topics, the demarcation of roles and skill sets start to reveal. In larger organizations, a dedicated team of infrastructure engineers design and make choices of network topology, IAC tooling and orchestration of the Kubernetes clusters and CI/CD pipelines. Typically, such DevOps engineers manage scaling and storage by installing the required controllers and CSI drivers. Application developers whose applications are deployed on the cluster maybe able to tune certain parameters of certain objects to manage the scaling needs of their applications. Depending on the situation, the developers maybe able to request and use storage volumes with certain restrictions. Regardless of the organization’s team structure, you should be able to learn about these Kubernetes concepts and objects and play with them locally on your laptop using Minikube.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

From an application developer and Kubernetes user’s point of view, you have to have a working knowledge of Kubernetes. The post outlines the most important Kubernetes objects required to deploy a typical web application. Let us assume that the web application uses the two-tier architecture. We also assume that the cluster is created and administered by an infrastructure or DevOps engineer and the necessary access is provided to the developer to deploy their web application onto the Kubernetes cluster. The Kubernetes operations are performed from the web application developer’s perspective.

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

In this blog post, I will explain how I implemented a graph algorithm to solve the database migration problem in an application pluggable module system.

Prerequisites:

• Working knowledge of Python
• Working knowledge of Graph Theory. Familiar with the terms: Edge, vertex, path, sink, source, digraph, path graph, etc.

Gavika Web Framework has a pluggable module system. The modules can be developed independently. They can be installed, upgraded and removed from the main application. Gavika Web Framework is written using Python, Flask, SQLAlchemy and a bunch of other related technologies and libraries.

You access Python dictionary keys using the syntax:

my_dicy[my_key]

For example:

>>> my_dict = {'food': 'idly'}
>>> my_dict['food']
'idly'

Sometimes, you might want to access the dictionary keys using:

my_dict.my_key

syntax. If you do this is what happens:

>>> my_dict.food
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'food'

How can you solve this? Easy.

 pip install attrdict

How do you use the newly installed package?

>>> from attrdict import AttrDict
>>> my_dict = AttrDict({'food': 'idly'})
>>> my_dict.food
'idly'

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Euler’s Totient Function.

Euler’s Totient Function, denoted as ϕ(n), is a function that counts the number of positive integers less than or equal to n that are relatively prime to n. Two numbers are relatively prime (or coprime) if their greatest common divisor (GCD) is 1.

Prerequisites

• Familarity with the Inclusion-Exclusion principle
• Familarity with the modular arithmetic

Definition

For any positive integer n, the Euler’s Totient Function ϕ(n) is defined as:

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Chinese Remainder Theorem.

Prereseqisites

• Modular arithmetic, especially modular inverses
• The Extended Euclidean Algorithm

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) provides a solution to a system of simultaneous congruences with different moduli. Here’s a step-by-step explanation:

Problem Setup

Suppose you have a system of congruences like this:

x ≡ a₁ (mod m₁)

Introduction

In this series of articles about number theory and cryptography, we have discussed

• The Euclidean algorithm to compute the GCD for two integers a and b
• The Bezout’s Identity: ax + bc = GCD(a,b)

This article will introduce the reader to the Extended Euclidean algorithm to compute the coefficients of the Bezout’s Identity.

Bézout’s identity states that for any two integers a and b, their greatest common divisor (GCD) can be expressed as:

Introduction

In this series of articles on number theory, today we talk about the Bézout’s Identity.

Let a and b be integers with greatest common divisor d. Then there exist integers x and y such that ax + by = d. The integers of the form az + bt are exactly the multiples of d.

The integers x and y are called Bézout coefficients for (a, b); they are not unique

This is a long-form post about the Euclidean algorithm to compute the greatest common divisors of two integers. The article starts from the fundamentals and explains why it works better than the naive algorithm. The author also explains the computational complexity and the mathematics of the algorithm.

Prereseqisites

• Fundamental arithmetic and an understanding of the greatest common divisor(GCD).
• Introductory concepts in algebra and logarithms, including mathematical induction.
• Overview of modular arithmetic.
• Familiarity with the Fibonacci series. An excellent video tutorial of the Fibonacci series on Youtube.
• An introduction to the big O notation.
• Basics Python programming skills.
• Curiosity.
• Patience to read long articles. It is okay to read it in more than one sitting. Taking notes will be benefitial for the learner. Re-reading the article a few times is encouraged.

Introduction

What Is GCD?

GCD, the greatest common divisor is the largest number that divides the given two integers.

In this blog post, I will explain how I implemented a graph algorithm to solve the database migration problem in an application pluggable module system.

Prerequisites:

• Working knowledge of Python
• Working knowledge of Graph Theory. Familiar with the terms: Edge, vertex, path, sink, source, digraph, path graph, etc.

Gavika Web Framework has a pluggable module system. The modules can be developed independently. They can be installed, upgraded and removed from the main application. Gavika Web Framework is written using Python, Flask, SQLAlchemy and a bunch of other related technologies and libraries.

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Euler’s Totient Function.

Euler’s Totient Function, denoted as ϕ(n), is a function that counts the number of positive integers less than or equal to n that are relatively prime to n. Two numbers are relatively prime (or coprime) if their greatest common divisor (GCD) is 1.

Prerequisites

• Familarity with the Inclusion-Exclusion principle
• Familarity with the modular arithmetic

Definition

For any positive integer n, the Euler’s Totient Function ϕ(n) is defined as:

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Chinese Remainder Theorem.

Prereseqisites

• Modular arithmetic, especially modular inverses
• The Extended Euclidean Algorithm

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) provides a solution to a system of simultaneous congruences with different moduli. Here’s a step-by-step explanation:

Problem Setup

Suppose you have a system of congruences like this:

x ≡ a₁ (mod m₁)

Introduction

In this series of articles about number theory and cryptography, we have discussed

• The Euclidean algorithm to compute the GCD for two integers a and b
• The Bezout’s Identity: ax + bc = GCD(a,b)

This article will introduce the reader to the Extended Euclidean algorithm to compute the coefficients of the Bezout’s Identity.

Bézout’s identity states that for any two integers a and b, their greatest common divisor (GCD) can be expressed as:

Introduction

In this series of articles on number theory, today we talk about the Bézout’s Identity.

Let a and b be integers with greatest common divisor d. Then there exist integers x and y such that ax + by = d. The integers of the form az + bt are exactly the multiples of d.

The integers x and y are called Bézout coefficients for (a, b); they are not unique

This is a long-form post about the Euclidean algorithm to compute the greatest common divisors of two integers. The article starts from the fundamentals and explains why it works better than the naive algorithm. The author also explains the computational complexity and the mathematics of the algorithm.

Prereseqisites

• Fundamental arithmetic and an understanding of the greatest common divisor(GCD).
• Introductory concepts in algebra and logarithms, including mathematical induction.
• Overview of modular arithmetic.
• Familiarity with the Fibonacci series. An excellent video tutorial of the Fibonacci series on Youtube.
• An introduction to the big O notation.
• Basics Python programming skills.
• Curiosity.
• Patience to read long articles. It is okay to read it in more than one sitting. Taking notes will be benefitial for the learner. Re-reading the article a few times is encouraged.

Introduction

What Is GCD?

GCD, the greatest common divisor is the largest number that divides the given two integers.

Data Structures And Algorithms

• Data Structures: A data structure is a technique of storing data in a computer so that it can be accessed and modified efficiently.
• Algorithms: An algorithm is a step-by-step instruction to perform a task.

Why Should I Learn Data Structures And Algorithms?

• Enhance your problem solving skills.
• Write efficient code that is performant and scalable.
• Specialized fields: machine learning, data science, artificial intelligence and other engineering fields deal with complex data and require efficient processing. To delve into such fascinating fields of engineering, having a firm grounding in data structures and algorithms is beneficial and in many cases required.

Can I Develop Applications Without Studying Data Structures And Algorithms?

Yes, sure. A lot of applications just store some data in a datastore, typically a relational database and have some procedures to show this data in a user interface, most commonly the web browser. The crux of such applications can be defined using the acronym CRUD. CRUD stands for Create, Read, Update and Delete. Often you are doing one of these CRUD functions in the context of persistent storage of your application:

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Euler’s Totient Function.

Euler’s Totient Function, denoted as ϕ(n), is a function that counts the number of positive integers less than or equal to n that are relatively prime to n. Two numbers are relatively prime (or coprime) if their greatest common divisor (GCD) is 1.

Prerequisites

• Familarity with the Inclusion-Exclusion principle
• Familarity with the modular arithmetic

Definition

For any positive integer n, the Euler’s Totient Function ϕ(n) is defined as:

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Chinese Remainder Theorem.

Prereseqisites

• Modular arithmetic, especially modular inverses
• The Extended Euclidean Algorithm

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) provides a solution to a system of simultaneous congruences with different moduli. Here’s a step-by-step explanation:

Problem Setup

Suppose you have a system of congruences like this:

x ≡ a₁ (mod m₁)

Introduction

In this series of articles about number theory and cryptography, we have discussed

• The Euclidean algorithm to compute the GCD for two integers a and b
• The Bezout’s Identity: ax + bc = GCD(a,b)

This article will introduce the reader to the Extended Euclidean algorithm to compute the coefficients of the Bezout’s Identity.

Bézout’s identity states that for any two integers a and b, their greatest common divisor (GCD) can be expressed as:

Introduction

In this series of articles on number theory, today we talk about the Bézout’s Identity.

Let a and b be integers with greatest common divisor d. Then there exist integers x and y such that ax + by = d. The integers of the form az + bt are exactly the multiples of d.

The integers x and y are called Bézout coefficients for (a, b); they are not unique

This is a long-form post about the Euclidean algorithm to compute the greatest common divisors of two integers. The article starts from the fundamentals and explains why it works better than the naive algorithm. The author also explains the computational complexity and the mathematics of the algorithm.

Prereseqisites

• Fundamental arithmetic and an understanding of the greatest common divisor(GCD).
• Introductory concepts in algebra and logarithms, including mathematical induction.
• Overview of modular arithmetic.
• Familiarity with the Fibonacci series. An excellent video tutorial of the Fibonacci series on Youtube.
• An introduction to the big O notation.
• Basics Python programming skills.
• Curiosity.
• Patience to read long articles. It is okay to read it in more than one sitting. Taking notes will be benefitial for the learner. Re-reading the article a few times is encouraged.

Introduction

What Is GCD?

GCD, the greatest common divisor is the largest number that divides the given two integers.

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Euler’s Totient Function.

Euler’s Totient Function, denoted as ϕ(n), is a function that counts the number of positive integers less than or equal to n that are relatively prime to n. Two numbers are relatively prime (or coprime) if their greatest common divisor (GCD) is 1.

Prerequisites

• Familarity with the Inclusion-Exclusion principle
• Familarity with the modular arithmetic

Definition

For any positive integer n, the Euler’s Totient Function ϕ(n) is defined as:

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Chinese Remainder Theorem.

Prereseqisites

• Modular arithmetic, especially modular inverses
• The Extended Euclidean Algorithm

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) provides a solution to a system of simultaneous congruences with different moduli. Here’s a step-by-step explanation:

Problem Setup

Suppose you have a system of congruences like this:

x ≡ a₁ (mod m₁)

Introduction

In this series of articles about number theory and cryptography, we have discussed

• The Euclidean algorithm to compute the GCD for two integers a and b
• The Bezout’s Identity: ax + bc = GCD(a,b)

This article will introduce the reader to the Extended Euclidean algorithm to compute the coefficients of the Bezout’s Identity.

Bézout’s identity states that for any two integers a and b, their greatest common divisor (GCD) can be expressed as:

Introduction

In this series of articles on number theory, today we talk about the Bézout’s Identity.

Let a and b be integers with greatest common divisor d. Then there exist integers x and y such that ax + by = d. The integers of the form az + bt are exactly the multiples of d.

The integers x and y are called Bézout coefficients for (a, b); they are not unique

This is a long-form post about the Euclidean algorithm to compute the greatest common divisors of two integers. The article starts from the fundamentals and explains why it works better than the naive algorithm. The author also explains the computational complexity and the mathematics of the algorithm.

Prereseqisites

• Fundamental arithmetic and an understanding of the greatest common divisor(GCD).
• Introductory concepts in algebra and logarithms, including mathematical induction.
• Overview of modular arithmetic.
• Familiarity with the Fibonacci series. An excellent video tutorial of the Fibonacci series on Youtube.
• An introduction to the big O notation.
• Basics Python programming skills.
• Curiosity.
• Patience to read long articles. It is okay to read it in more than one sitting. Taking notes will be benefitial for the learner. Re-reading the article a few times is encouraged.

Introduction

What Is GCD?

GCD, the greatest common divisor is the largest number that divides the given two integers.

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Euler’s Totient Function.

Euler’s Totient Function, denoted as ϕ(n), is a function that counts the number of positive integers less than or equal to n that are relatively prime to n. Two numbers are relatively prime (or coprime) if their greatest common divisor (GCD) is 1.

Prerequisites

• Familarity with the Inclusion-Exclusion principle
• Familarity with the modular arithmetic

Definition

For any positive integer n, the Euler’s Totient Function ϕ(n) is defined as:

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the Chinese Remainder Theorem.

Prereseqisites

• Modular arithmetic, especially modular inverses
• The Extended Euclidean Algorithm

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) provides a solution to a system of simultaneous congruences with different moduli. Here’s a step-by-step explanation:

Problem Setup

Suppose you have a system of congruences like this:

x ≡ a₁ (mod m₁)

Introduction

In this series of articles about number theory and cryptography, we have discussed

• The Euclidean algorithm to compute the GCD for two integers a and b
• The Bezout’s Identity: ax + bc = GCD(a,b)

This article will introduce the reader to the Extended Euclidean algorithm to compute the coefficients of the Bezout’s Identity.

Bézout’s identity states that for any two integers a and b, their greatest common divisor (GCD) can be expressed as:

Introduction

In this series of articles on number theory, today we talk about the Bézout’s Identity.

Let a and b be integers with greatest common divisor d. Then there exist integers x and y such that ax + by = d. The integers of the form az + bt are exactly the multiples of d.

The integers x and y are called Bézout coefficients for (a, b); they are not unique

This is a long-form post about the Euclidean algorithm to compute the greatest common divisors of two integers. The article starts from the fundamentals and explains why it works better than the naive algorithm. The author also explains the computational complexity and the mathematics of the algorithm.

Prereseqisites

• Fundamental arithmetic and an understanding of the greatest common divisor(GCD).
• Introductory concepts in algebra and logarithms, including mathematical induction.
• Overview of modular arithmetic.
• Familiarity with the Fibonacci series. An excellent video tutorial of the Fibonacci series on Youtube.
• An introduction to the big O notation.
• Basics Python programming skills.
• Curiosity.
• Patience to read long articles. It is okay to read it in more than one sitting. Taking notes will be benefitial for the learner. Re-reading the article a few times is encouraged.

Introduction

What Is GCD?

GCD, the greatest common divisor is the largest number that divides the given two integers.

Introduction

In our last post, we wrote some Python functions in preparation to implement the RSA algorithm. In this post, we will implement the RSA algorithm.

Program On The Receiver’s Side

Create the public and private keys. Share the public key with the sender.

import random
from sympy import isprime

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

def modinv(a, m):
    m0, x0, x1 = m, 0, 1
    if m == 1:
        return 0
    while a > 1:
        q = a // m
        m, a = a % m, m
        x0, x1 = x1 - q * x0, x0
    if x1 < 0:
        x1 += m0
    return x1

def generate_prime_candidate(length):
    p = random.getrandbits(length)
    # Apply a mask to set MSB and LSB to 1 to ensure a proper length prime
    p |= (1 << length - 1) | 1
    return p

def generate_large_prime(length=1024):
    p = 4
    while not isprime(p):
        p = generate_prime_candidate(length)
    return p

def generate_keypair(keysize):
    p = generate_large_prime(keysize)
    q = generate_large_prime(keysize)
    n = p * q
    phi = (p - 1) * (q - 1)

    # Choose e
    e = random.randrange(2, phi)
    g = gcd(e, phi)
    while g != 1:
        e = random.randrange(2, phi)
        g = gcd(e, phi)

    # Generate d
    d = modinv(e, phi)

    return ((n, e), (n, d))

def decrypt(private_key, ciphertext):
    n, d = private_key
    # Decrypt each character
    plain = [chr((char ** d) % n) for char in ciphertext]
    return ''.join(plain)

# RSA Key Generation
keysize = 8  # Small size for demonstration; use 1024 or 2048 for real applications
public_key, private_key = generate_keypair(keysize)

print(f"Public Key: {public_key}")
print(f"Private Key: {private_key}")

Execute the program on the receiver’s computer. Take note of the public and private key. Share the public key with the sender.

Prerequisites

• Familiarity with binary arithmetic. Know what are MSB and LSB.
• Bitwise operations. Logical shift.
• Primality Testing

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

Before writing the RSA program, let’s prepare some useful functions.

GCD

Here’s a simple recurisve function to computed the GCD of two integers.

def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a

We have discussed the non-recursive implementation of the Euclidean algorithm at length in this series.

Introduction

Continuing with the series of posts on number theory and cryptograpy, let us take a look at the RSA algorithm.

RSA is an algorithm to perform encryption and decryption of data.

The Basic Idea

RSA uses two keys to encrypt and decrypt messages:

• Public Key: Anyone can see this key. It’s used to encrypt the message.
• Private Key: Only you should know this key. It’s used to decrypt the message.

When someone wants to send you a secret message, they use your public key to encrypt the message. Once it’s encrypted, only your private key can decrypt it.

The Traditional Way Of Learning

In the olden days, you had to go to school or college to educate yourself. In the classroom, listen to the lectures, make notes and enjoy some discussions with the lecturer and classmates. At home, you would refer to textbooks and complete the work suggested in the curriculum.

Other than college, you had and have the option of private tuition. There are many brick and mortar training academies that supplement the colleges and universities.

Data Structures And Algorithms

• Data Structures: A data structure is a technique of storing data in a computer so that it can be accessed and modified efficiently.
• Algorithms: An algorithm is a step-by-step instruction to perform a task.

Why Should I Learn Data Structures And Algorithms?

• Enhance your problem solving skills.
• Write efficient code that is performant and scalable.
• Specialized fields: machine learning, data science, artificial intelligence and other engineering fields deal with complex data and require efficient processing. To delve into such fascinating fields of engineering, having a firm grounding in data structures and algorithms is beneficial and in many cases required.

Can I Develop Applications Without Studying Data Structures And Algorithms?

Yes, sure. A lot of applications just store some data in a datastore, typically a relational database and have some procedures to show this data in a user interface, most commonly the web browser. The crux of such applications can be defined using the acronym CRUD. CRUD stands for Create, Read, Update and Delete. Often you are doing one of these CRUD functions in the context of persistent storage of your application:

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

I see a lot of people are trying to break into software engineering and DevOps. There are many inspiring stories out there. People from other industries and verticals have switched to IT and are having a successful career. You can do it too.

Let us look at the resources available to you.

Online Vs. Offline Training And Learning Resources

Online

All you need to break into DevOps is:

Introduction

If you have chosen the DevOps engineering path for your career, Linux system administration skills are a must. Companies run their workloads on Linux on-premise as well as in the cloud. Many developers write software on their Linux laptops and workstations. In this post, I will lay out a plan to master Linux system administration skills. The journey begins with you becoming a Linux desktop user. Gradually, you start using your laptop as a Linux server. Become more productive and create your own labs using virtualization. You will be able to simulate a lot of Linux infrastructure scenarios on your laptop.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

The build and deploy engineer is a role in IT/infrastructure/devops teams of organizations that is responsible for deploying a set of applications using CI/CD pipelines. Usually, they use the pipelines, tools and processes created by DevOps engineers, consultants or practitioners. They tweak things with tools like Ansible, Jenkins, Git, Maven, etc. here and there a bit.

Are you one such build and deploy engineer looking to transition your career into DevOps engineering? Here’s a path you could follow:

AWS Certified Solutions Architect - Associate is one of the most sought-after certifications in the IT industry.

Here’s a few tips for those seeking this certification.

Background Knowledge And Experience

AWS recommends “at least one year of hands-on experience designing available, cost-efficient, fault-tolerant, and scalable and distributed systems on AWS.” In other words, if you recently started your career in IT, there are a few things you have to do before you start preparing for this certification.

The Traditional Way Of Learning

In the olden days, you had to go to school or college to educate yourself. In the classroom, listen to the lectures, make notes and enjoy some discussions with the lecturer and classmates. At home, you would refer to textbooks and complete the work suggested in the curriculum.

Other than college, you had and have the option of private tuition. There are many brick and mortar training academies that supplement the colleges and universities.

The Traditional Way Of Learning

In the olden days, you had to go to school or college to educate yourself. In the classroom, listen to the lectures, make notes and enjoy some discussions with the lecturer and classmates. At home, you would refer to textbooks and complete the work suggested in the curriculum.

Other than college, you had and have the option of private tuition. There are many brick and mortar training academies that supplement the colleges and universities.

When you begin your career in IT, you will be required to learn Git sooner or later. Git is a key technology used in everyday software engineering and DevOps. It has become persuasive and quintessential. There is no need to hard sell Git to anyone at this point.

There is a lot to learn about Git. The good news is that you can start with a subset of Git. There is a no need to become a Git master on your day one of software engineering or DevOps engineering. You can follow a tutorial and take home a Git workflow on day one. In about three days of studying and practising Git, you will be ready to use Git in software engineering and DevOps workflows.

I see a lot of people are trying to break into software engineering and DevOps. There are many inspiring stories out there. People from other industries and verticals have switched to IT and are having a successful career. You can do it too.

Let us look at the resources available to you.

Online Vs. Offline Training And Learning Resources

Online

All you need to break into DevOps is:

Data Structures And Algorithms

• Data Structures: A data structure is a technique of storing data in a computer so that it can be accessed and modified efficiently.
• Algorithms: An algorithm is a step-by-step instruction to perform a task.

Why Should I Learn Data Structures And Algorithms?

• Enhance your problem solving skills.
• Write efficient code that is performant and scalable.
• Specialized fields: machine learning, data science, artificial intelligence and other engineering fields deal with complex data and require efficient processing. To delve into such fascinating fields of engineering, having a firm grounding in data structures and algorithms is beneficial and in many cases required.

Can I Develop Applications Without Studying Data Structures And Algorithms?

Yes, sure. A lot of applications just store some data in a datastore, typically a relational database and have some procedures to show this data in a user interface, most commonly the web browser. The crux of such applications can be defined using the acronym CRUD. CRUD stands for Create, Read, Update and Delete. Often you are doing one of these CRUD functions in the context of persistent storage of your application:

Introduction

This is an anchor post about machine learning and artificial intelligence.

Observations On AI

• Revolution in Text Generation: Generative AI is making significant strides in text production, promising a surge in the quality and quantity of text-based content. This encompasses everything from blog posts and articles to comprehensive reference books, transforming how information is created and disseminated.

• Advancements In AI-Generated Imagery: The evolving capabilities of AI in image generation are noteworthy. These advancements are set to enrich various fields with innovative and creative visual content, offering new dimensions in digital art, marketing, and visual storytelling.

Introduction

This is an anchor post about machine learning and artificial intelligence.

Observations On AI

• Revolution in Text Generation: Generative AI is making significant strides in text production, promising a surge in the quality and quantity of text-based content. This encompasses everything from blog posts and articles to comprehensive reference books, transforming how information is created and disseminated.

• Advancements In AI-Generated Imagery: The evolving capabilities of AI in image generation are noteworthy. These advancements are set to enrich various fields with innovative and creative visual content, offering new dimensions in digital art, marketing, and visual storytelling.

In a previous series of blog posts, we discussed the Kubernetes objects typically used to run a web application. We covered, Namespace, Pod, ConfigMap, Secret, Service, Deployment, ServiceAccount, Ingress, PDB, HPA, PV, PVC, Job and CronJob.

In this post, we will discuss some fundamental building blocks for the Kubernetes cluster administration: RBAC objects.

In most situations, Kubernetes API server is started with the flag --authorization-mode=RBAC which enables RBAC in the cluster.

Introduction

We have the host machine with the OS Ubuntu 22.04. On this PC or server, we will create two virtual machine guests:

• myubuntu2204test01 having static IP of 192.168.122.146
• myubuntu2204test02 having static IP of 192.168.122.147

The guest VMs will use the default network created by libvirt. The gateway IP for the default network is 192.168.122.1.

We will achieve automation using libvirt, qemu and cloud-init. To go through the article and exercise, you should have a rudimentary understanding of Linux system administration and networking.

So, you have an SSH key pair. The public key has been added on to the server. On the client, ie your laptop/desktop you have your private key. For some reason, your private key has not been loaded into memory.

All you have to do is start the ssh-agent and then load the key into memory via ssh-add.

Step 1: Start the agent

eval $(ssh-agent)

Step 2: load the key into memory

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

Ansible Prerequisites

Before jumping on to learning Ansible, have a firm grounding in Linux system administration and shell scripting. You can use Ansible for a lot of automation projects. The primary target audience for this blog post are DevOps engineers, IT infrastructure engineers and system administrators who create and manage IT infrastructure to run workloads. A good understanding of YAML is required before starting to write Ansible playbooks. A background in at least one programming language helps. Python programing is not a requirement per se. But Python programming familiarity helps put together some automated testing.

Architecture 1: Kubernetes Control Plane Without HA

Create three Virtual Machine guests on your laptop or workstation.

• VM 01 - Kubernetes Control Plane
• VM 02, VM 03 - Kubernetes Worker nodes

Architecture 2: Kubernetes Control Plane With HA

Create five Virtual Machine guests on your laptop or workstation.

• VM 01, VM 02, VM 03 - Kubernetes Control Plane with HA
• VM 04, VM 05 - Kubernetes Worker nodes

Use the Kubeadm tool to create the cluster.

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

Introduction

If you have chosen the DevOps engineering path for your career, Linux system administration skills are a must. Companies run their workloads on Linux on-premise as well as in the cloud. Many developers write software on their Linux laptops and workstations. In this post, I will lay out a plan to master Linux system administration skills. The journey begins with you becoming a Linux desktop user. Gradually, you start using your laptop as a Linux server. Become more productive and create your own labs using virtualization. You will be able to simulate a lot of Linux infrastructure scenarios on your laptop.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

Introduction

The Linux Kernel parameters are settings that can be configured to control the behavior of the Linux kernel. They are typically used to fine-tune system performance or to enable/disable certain features.

Some examples of kernel parameters include:

Memory-related parameters: These parameters control how the kernel manages system memory, including how much memory is allocated to user processes and how aggressively the kernel caches data.

Processor-related parameters: These parameters control how the kernel interacts with the system’s processors, including how it schedules processes and how it handles interrupts.

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

The Tech Chorus blog by Sudheer Satyanarayana started way back in 2008. The blog has used various hosting technologies and software over the years.

Initially, the blog site used Drupal content management system. LAMP was a popular technology stack those days. Drupal was one of my favorite CMSes back then. The Drupal site was hosted on a cPanel server.

As life progressed, there was not enough time to patch Drupal and its plugins. Upgrading between major versions of Drupal also took considerable amount of time and effort. I moved on to a static site generator. I found and loved Lektor. With Lektor, I managed the content on my workstation and generated the static site. I used to deploy the generated static site to an AWS EC2 instance with Nginx.

What is SSH Key Management?

The blog post is intended towards someone new to SSH key management. Do you have just one SSH key pair? Do you have multiple key pairs? What are some pros and cons of having single or multiple key pairs? The post answers such questions about SSH key usage in practice.

The Premise

So, someone walked up to you or in a video call asked for your SSH public key?

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

Install PostgreSQL Server

Fedora and CentOS:

sudo dnf install postgresql-server

Ubuntu 18.04:

sudo apt install postgresql

New Server Initialization

On CentOS 7/Fedora 30:

sudo postgresql-setup initdb

Upgrading From An Older Version

sudo postgresql-setup --upgrade

Administering The Database Server

Managing The postgresql Daemon

Starting PostgreSQL server

sudo systemctl start postgresql

Checking PostgreSQL Server Status:

sudo systemctl status postgresql

Enabling PostgreSQL Server Systemd Unit/Enabling PostgreSQL Server On Boot:

sudo systemctl enable postgresql

Allowing Password Based Login From localhost

Edit /var/lib/pgsql/data/pg_hba.conf as privileged user(root) and add this line:

Introduction

We have the host machine with the OS Ubuntu 22.04. On this PC or server, we will create two virtual machine guests:

• myubuntu2204test01 having static IP of 192.168.122.146
• myubuntu2204test02 having static IP of 192.168.122.147

The guest VMs will use the default network created by libvirt. The gateway IP for the default network is 192.168.122.1.

We will achieve automation using libvirt, qemu and cloud-init. To go through the article and exercise, you should have a rudimentary understanding of Linux system administration and networking.

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

Introduction

We have the host machine with the OS Ubuntu 22.04. On this PC or server, we will create two virtual machine guests:

• myubuntu2204test01 having static IP of 192.168.122.146
• myubuntu2204test02 having static IP of 192.168.122.147

The guest VMs will use the default network created by libvirt. The gateway IP for the default network is 192.168.122.1.

We will achieve automation using libvirt, qemu and cloud-init. To go through the article and exercise, you should have a rudimentary understanding of Linux system administration and networking.

So, you have an SSH key pair. The public key has been added on to the server. On the client, ie your laptop/desktop you have your private key. For some reason, your private key has not been loaded into memory.

All you have to do is start the ssh-agent and then load the key into memory via ssh-add.

Step 1: Start the agent

eval $(ssh-agent)

Step 2: load the key into memory

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

Ansible Prerequisites

Before jumping on to learning Ansible, have a firm grounding in Linux system administration and shell scripting. You can use Ansible for a lot of automation projects. The primary target audience for this blog post are DevOps engineers, IT infrastructure engineers and system administrators who create and manage IT infrastructure to run workloads. A good understanding of YAML is required before starting to write Ansible playbooks. A background in at least one programming language helps. Python programing is not a requirement per se. But Python programming familiarity helps put together some automated testing.

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

NFS Server

To run your own NFS (Network File System) server, you’ll need to first install Linux on your server. I’d go with Debian, Ubuntu or CentOS distributions. Once you’ve done that, you can install the NFS server software, which is typically included with most Linux distributions. After the NFS server software is installed, you’ll need to configure it to specify which directories on the server should be shared with NFS clients, and what type of access they should have. This typically involves editing the server’s configuration file, which is typically located at /etc/exports. Once the server is configured, you can start the NFS server and begin sharing files with NFS clients. To access the shared files, the NFS clients will need to mount the NFS share using the mount command.

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

There are many tools and software programs that can be used for monitoring and performance analysis on Linux systems. Some popular options include:

• top - This is a command-line utility that shows real-time information about the processes running on a Linux system, such as their CPU and memory usage.
• htop - This is a more advanced version of top that provides a more user-friendly interface and additional features, such as the ability to sort processes by different metrics and to kill processes.
• sar - This is a command-line utility that collects and displays performance metrics for a Linux system over time. It can be used to analyze CPU, memory, I/O, and network usage, as well as other metrics.
• iostat - This is a command-line utility that shows real-time information about I/O performance on a Linux system. It can be used to monitor the performance of disks and other storage devices.
• vmstat - This is a command-line utility that shows real-time information about various system resources, such as memory, CPU, and I/O. It can be used to monitor the overall health of a Linux system.
• netstat - This is a command-line utility that shows information about network connections on a Linux system. It can be used to monitor the status of network connections and to diagnose networking issues.

There are also many modern monitoring tools and software programs available for Linux, such as Prometheus and Zabbix. These tools typically offer more advanced features and capabilities than the built-in Linux utilities, such as the ability to collect and store metrics over time, and to generate alerts when certain conditions are met.

Introduction

If you have chosen the DevOps engineering path for your career, Linux system administration skills are a must. Companies run their workloads on Linux on-premise as well as in the cloud. Many developers write software on their Linux laptops and workstations. In this post, I will lay out a plan to master Linux system administration skills. The journey begins with you becoming a Linux desktop user. Gradually, you start using your laptop as a Linux server. Become more productive and create your own labs using virtualization. You will be able to simulate a lot of Linux infrastructure scenarios on your laptop.

Introduction

The Linux Kernel parameters are settings that can be configured to control the behavior of the Linux kernel. They are typically used to fine-tune system performance or to enable/disable certain features.

Some examples of kernel parameters include:

Memory-related parameters: These parameters control how the kernel manages system memory, including how much memory is allocated to user processes and how aggressively the kernel caches data.

Processor-related parameters: These parameters control how the kernel interacts with the system’s processors, including how it schedules processes and how it handles interrupts.

Using Git with SSH is a common practice among software developers. The convenience of not having to remember passwords is a huge productivity boost in software development workflows.

In a previous blog post, I wrote about managing SSH keys.

Having many SSH keys can cause few issues:

• ssh-agent doesn’t work well with too many keys.
• Can’t always map SSH keys to servers in SSH client configuration. Both personal and company projects Git can be hosted on the same server or third-party service such as Github.
• Some Git hosting services do not allow you to use the same SSH key in more than one user profile. You are forced to have a unique SSH keypair per user profile.
• The SSH server might refuse to allow too many authentication attempts.

Git allows you to set the environment variable GIT_SSH_COMMAND. If you set this environment variable, Git uses the specified command for push and pull operations.

What is SSH Key Management?

The blog post is intended towards someone new to SSH key management. Do you have just one SSH key pair? Do you have multiple key pairs? What are some pros and cons of having single or multiple key pairs? The post answers such questions about SSH key usage in practice.

The Premise

So, someone walked up to you or in a video call asked for your SSH public key?

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY). Instead of writing such tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github.
Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in the future.

These are some guidelines for the Ansible role projects that Gavika follows:

Ansible comes with a vault feature. It is meant to be used in the context of configuration management. But you can also use it as a standalone simple password vault for your personal or organization’s use.

Initial setup of password vault:

• Create or clone a Git or another SCM repository

git init 

• Create the password vault

ansible-vault create myvault.secret

Type the new master password and confirm, ansible-vault will open your text editor. Type your secrets in the editor and save and quit. To open your vault for viewing or editing in the future, you will need your vault password.

In this tutorial, I will provide step-by-step process to implement MySQL replication. We will create one master and one slave. We will use two CentOS 6 servers - one for master and the other for slave. This following steps have been tested on two virtual machines.

Our master server will have IP address 192.168.122.10. Our slave server will have IP address 192.168.122.12.

You might want to run SELinux in permissive mode.

Moving around

You can use the arrow keys on keyboard to move around in the command line. Bash also provides convenient keyboard shortcuts to navigate effectively. Try them out and see for yourself.

To become a Bash pro user you have to get yourself familiar with the keyboard shortcuts. Once you do, you’ll find yourself productive.

Moving around words using ESC+f and ESC+b are my favourites in this list. Jumping to first and last lines of the history list is also useful.

Introduction

We have the host machine with the OS Ubuntu 22.04. On this PC or server, we will create two virtual machine guests:

• myubuntu2204test01 having static IP of 192.168.122.146
• myubuntu2204test02 having static IP of 192.168.122.147

The guest VMs will use the default network created by libvirt. The gateway IP for the default network is 192.168.122.1.

We will achieve automation using libvirt, qemu and cloud-init. To go through the article and exercise, you should have a rudimentary understanding of Linux system administration and networking.

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

This is a guide to write a Kubernetes controller. We will kick off by inspecting the Kubernetes API from inside a pod within the cluster. Minikube suffices for this exercise. But you can conduct the exercise to any Kubernetes cluster.

The controller watches events related to Kubernetes pods using the Kubernetes API. When there is a new event, the controller logs the event’s type and the name of the affected pod. This controller can be extended to perform other actions when pod events occur, such as scaling the number of replicas for a deployment, sending notifications, or triggering a custom script or program.

This is a guide to write a Kubernetes controller. We will kick off by inspecting the Kubernetes API from inside a pod within the cluster. Minikube suffices for this exercise. But you can conduct the exercise to any Kubernetes cluster.

The controller watches events related to Kubernetes pods using the Kubernetes API. When there is a new event, the controller logs the event’s type and the name of the affected pod. This controller can be extended to perform other actions when pod events occur, such as scaling the number of replicas for a deployment, sending notifications, or triggering a custom script or program.

This is a guide to write a Kubernetes controller. We will kick off by inspecting the Kubernetes API from inside a pod within the cluster. Minikube suffices for this exercise. But you can conduct the exercise to any Kubernetes cluster.

The controller watches events related to Kubernetes pods using the Kubernetes API. When there is a new event, the controller logs the event’s type and the name of the affected pod. This controller can be extended to perform other actions when pod events occur, such as scaling the number of replicas for a deployment, sending notifications, or triggering a custom script or program.

This is a guide to write a Kubernetes controller. We will kick off by inspecting the Kubernetes API from inside a pod within the cluster. Minikube suffices for this exercise. But you can conduct the exercise to any Kubernetes cluster.

The controller watches events related to Kubernetes pods using the Kubernetes API. When there is a new event, the controller logs the event’s type and the name of the affected pod. This controller can be extended to perform other actions when pod events occur, such as scaling the number of replicas for a deployment, sending notifications, or triggering a custom script or program.

In this blog post, I will explain how I implemented a graph algorithm to solve the database migration problem in an application pluggable module system.

Prerequisites:

• Working knowledge of Python
• Working knowledge of Graph Theory. Familiar with the terms: Edge, vertex, path, sink, source, digraph, path graph, etc.

Gavika Web Framework has a pluggable module system. The modules can be developed independently. They can be installed, upgraded and removed from the main application. Gavika Web Framework is written using Python, Flask, SQLAlchemy and a bunch of other related technologies and libraries.

You access Python dictionary keys using the syntax:

my_dicy[my_key]

For example:

>>> my_dict = {'food': 'idly'}
>>> my_dict['food']
'idly'

Sometimes, you might want to access the dictionary keys using:

my_dict.my_key

syntax. If you do this is what happens:

>>> my_dict.food
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'food'

How can you solve this? Easy.

 pip install attrdict

How do you use the newly installed package?

>>> from attrdict import AttrDict
>>> my_dict = AttrDict({'food': 'idly'})
>>> my_dict.food
'idly'

So, you have an SSH key pair. The public key has been added on to the server. On the client, ie your laptop/desktop you have your private key. For some reason, your private key has not been loaded into memory.

All you have to do is start the ssh-agent and then load the key into memory via ssh-add.

Step 1: Start the agent

eval $(ssh-agent)

Step 2: load the key into memory

What is SSH Key Management?

The blog post is intended towards someone new to SSH key management. Do you have just one SSH key pair? Do you have multiple key pairs? What are some pros and cons of having single or multiple key pairs? The post answers such questions about SSH key usage in practice.

The Premise

So, someone walked up to you or in a video call asked for your SSH public key?

So, you have an SSH key pair. The public key has been added on to the server. On the client, ie your laptop/desktop you have your private key. For some reason, your private key has not been loaded into memory.

All you have to do is start the ssh-agent and then load the key into memory via ssh-add.

Step 1: Start the agent

eval $(ssh-agent)

Step 2: load the key into memory

Using Git with SSH is a common practice among software developers. The convenience of not having to remember passwords is a huge productivity boost in software development workflows.

In a previous blog post, I wrote about managing SSH keys.

Having many SSH keys can cause few issues:

• ssh-agent doesn’t work well with too many keys.
• Can’t always map SSH keys to servers in SSH client configuration. Both personal and company projects Git can be hosted on the same server or third-party service such as Github.
• Some Git hosting services do not allow you to use the same SSH key in more than one user profile. You are forced to have a unique SSH keypair per user profile.
• The SSH server might refuse to allow too many authentication attempts.

Git allows you to set the environment variable GIT_SSH_COMMAND. If you set this environment variable, Git uses the specified command for push and pull operations.

What is SSH Key Management?

The blog post is intended towards someone new to SSH key management. Do you have just one SSH key pair? Do you have multiple key pairs? What are some pros and cons of having single or multiple key pairs? The post answers such questions about SSH key usage in practice.

The Premise

So, someone walked up to you or in a video call asked for your SSH public key?

Using Linux KVM, Expose Virtual Guests On The LAN

Audience: The blog post is for beginner to intermediary Linux system administrators.

• You should have a thorough understanding of the shell commands and
• be comfortable on the command line
• be able to install and configure packages, etc.
• be able to start and stop services using systemd
• be familiar with Linux configuration files
• be able to set Linux kernel parameters using sysctl
• be able to enable and disable Kernel modules
• be comfortable installing and using guest VMs using libvirt

You should have a rudimentary understanding of networking concepts and tooling such as

KVM (Kernel-based Virtual Machine), libvirt, and QEMU are three components that work together to provide a virtualization solution on Linux systems.

KVM is a virtualization infrastructure built into the Linux kernel that allows it to run multiple virtual machines (VMs) on a single physical host. It provides the underlying virtualization technology, such as hardware acceleration for virtualized CPU and memory, and device emulation.

Purpose Of Having Naming Conventions For An Ansible Project

• Consistency: Adopting a naming convention standardizes naming across the project and organization. This makes it easier for developers to switch between projects. Typically, an organization with an infrastructure team will have several Ansible projects and source code repositories. A developer working on one such Ansible project can seamlessly switch to another given a standard naming convention.
• Error Reduction: With improper naming there can be pitfalls. For instance, using hyphens(-) in variable names can cause parsing errors since Ansible might interpret them as the minus operator. Standardizing naming for variables can help avoid collisions.
• Readability: With a standard naming convention, it helps developers to find variables, files and tasks quickly.

Naming Conventions For Ansible Roles And Playbooks

• Role name : use lowercase letters and hyphens to separate words: For example, web-server or database-backup.
• Task file name: user lowercase letters and underscores to separate words. When statements such as include are used, it is convenient to have file names without hyphens. For example: install_web_ubuntu.yml
• YAML file extension: yml instead yaml. To be consistent and succinct.
• Task name: start with a verb: Use an action verb at the beginning of the task name to indicate what the role does. For example, Install Nginx or Configure firewall. Start with a capital letter. No need to end with a period for a few words of task description.
• Variable name: lowercase letters and underscores to separate words. The variable must start with the role name. For example, if the role name is nginx, the variable name should be nginx_default_hostname.

Content Guide

• Always include a README file for the role. Describe in detail how to use the role. Create a table to show the role variables and their defaults.
• Include automated tests using Molecule.

Ansible Prerequisites

Before jumping on to learning Ansible, have a firm grounding in Linux system administration and shell scripting. You can use Ansible for a lot of automation projects. The primary target audience for this blog post are DevOps engineers, IT infrastructure engineers and system administrators who create and manage IT infrastructure to run workloads. A good understanding of YAML is required before starting to write Ansible playbooks. A background in at least one programming language helps. Python programing is not a requirement per se. But Python programming familiarity helps put together some automated testing.

Introduction

The Linux Kernel parameters are settings that can be configured to control the behavior of the Linux kernel. They are typically used to fine-tune system performance or to enable/disable certain features.

Some examples of kernel parameters include:

Memory-related parameters: These parameters control how the kernel manages system memory, including how much memory is allocated to user processes and how aggressively the kernel caches data.

Processor-related parameters: These parameters control how the kernel interacts with the system’s processors, including how it schedules processes and how it handles interrupts.

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

This is a blog post in IAC with Terraform series.

IAC stands for Infrastructure As Code. Modern IT infrastructure can be orchestrated using programmatic methods. Terraform is(was?) a popular open source software used to orchestrate infrastructure in the cloud and elsewhere too.

Terraform has the concept of modules. With modules, you can code abstract infrastructure. For example, if you are creating a pattern of infrastructure over and over again, you could abstract the pattern into a Terraform module. Let’s take the example of a web application. It consists of:

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY). Instead of writing such tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Short answer: use ipify

ipify provides a simple public address API.

Using the tool, you can determine your public IP address programmatically. If you are using the shell:

curl 'https://api.ipify.org'

Using it in a shell script:

my_ip=$(curl 'https://api.ipify.org' -s)
echo $my_ip

Using the Ansible ipify module:

- hosts: localhost
  vars:
  tasks:
    - name: Get my public IP
      ipify_facts:
        timeout: 20
      delegate_to: localhost
      register: public_ip
    - name: output
      debug: msg="{{ ipify_public_ip }}"

Sample output of Ansible playbook execution:

ansible-playbook ipify.yml 
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Get my public IP] *******************************************************************************************************************************************************************************************
ok: [localhost -> localhost]

TASK [output] *****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "49.206.13.205"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0   

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github.
Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in the future.

These are some guidelines for the Ansible role projects that Gavika follows:

Install the Ansible role gavika.aws_cloudwatchagent via Galaxy

ansible-galaxy install gavika.aws_cloudwatchagent

Create The Playbook File - cw-play.yml :

---
- hosts: all
  become: true
  vars:
  roles:
    - role: gavika.aws_cloudwatchagent

Prepare the AWS CloudWatch Agent configuration

In your variables file, use aws_cloudwatch_agent_config

agent:
    metrics_collection_interval: 60
    run_as_user: "cwagent"
    metrics:
      namespace: "Gavika"
      append_dimensions:
        InstanceId: "${aws:InstanceId}"
      metrics_collected:
        disk:
          measurement:
            - used_percent
          metrics_collection_interval: 60
          resources:
            - "*"
        mem:
          measurement:
            - mem_used_percent
          metrics_collection_interval: 60

In this example, I am using the namespace, Gavika. Feel free to change it. We collect the cpu, disk, diskio, mem and swap metrics. The agent will send these metrics once in 360 seconds.

Ansible comes with a vault feature. It is meant to be used in the context of configuration management. But you can also use it as a standalone simple password vault for your personal or organization’s use.

Initial setup of password vault:

• Create or clone a Git or another SCM repository

git init 

• Create the password vault

ansible-vault create myvault.secret

Type the new master password and confirm, ansible-vault will open your text editor. Type your secrets in the editor and save and quit. To open your vault for viewing or editing in the future, you will need your vault password.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

NFS Server

To run your own NFS (Network File System) server, you’ll need to first install Linux on your server. I’d go with Debian, Ubuntu or CentOS distributions. Once you’ve done that, you can install the NFS server software, which is typically included with most Linux distributions. After the NFS server software is installed, you’ll need to configure it to specify which directories on the server should be shared with NFS clients, and what type of access they should have. This typically involves editing the server’s configuration file, which is typically located at /etc/exports. Once the server is configured, you can start the NFS server and begin sharing files with NFS clients. To access the shared files, the NFS clients will need to mount the NFS share using the mount command.

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

There are many tools and software programs that can be used for monitoring and performance analysis on Linux systems. Some popular options include:

• top - This is a command-line utility that shows real-time information about the processes running on a Linux system, such as their CPU and memory usage.
• htop - This is a more advanced version of top that provides a more user-friendly interface and additional features, such as the ability to sort processes by different metrics and to kill processes.
• sar - This is a command-line utility that collects and displays performance metrics for a Linux system over time. It can be used to analyze CPU, memory, I/O, and network usage, as well as other metrics.
• iostat - This is a command-line utility that shows real-time information about I/O performance on a Linux system. It can be used to monitor the performance of disks and other storage devices.
• vmstat - This is a command-line utility that shows real-time information about various system resources, such as memory, CPU, and I/O. It can be used to monitor the overall health of a Linux system.
• netstat - This is a command-line utility that shows information about network connections on a Linux system. It can be used to monitor the status of network connections and to diagnose networking issues.

There are also many modern monitoring tools and software programs available for Linux, such as Prometheus and Zabbix. These tools typically offer more advanced features and capabilities than the built-in Linux utilities, such as the ability to collect and store metrics over time, and to generate alerts when certain conditions are met.

To run your own email server using Linux and other open source software, you’ll need to first choose a Linux distribution and install it on your server. I’d go with either Ubuntu, Debian or Rocky Linux. Once you’ve done that, you can choose an email server software that is compatible with Linux, such as Postfix or Exim. After installing and configuring the email server software, you’ll need to set up DNS records and configure authentication and encryption to ensure that your email server is secure. Finally, you’ll need to test your email server to make sure it is working properly and can send and receive messages.

Definition

A load balancer is a type of software or hardware that distributes incoming traffic across multiple servers or resources. This allows the load balancer to distribute the workload evenly, improving the performance and availability of the application.

The Load Balancer Lab

To run your own load balancer using open source software, you will need to:

• Install and configure the load balancer software on a server. Some popular open source options include HAProxy, Nginx, and Envoy.
• Configure the load balancer to distribute incoming traffic to the appropriate servers or resources. This typically involves setting up virtual servers and defining rules for routing traffic.
• Test the load balancer to ensure that it is working correctly and distributing traffic as expected.
• Monitor the load balancer and the underlying servers to ensure that the system is performing well and handling traffic effectively.
• Continually tune and optimize the load balancer configuration to improve performance and ensure that the application is always available and responsive.

Your web applications need a solution to store and retrieve its data. A relational database is often used in web applications. MySQL, MariaDB and PostgreSQL are some popular relational databases. There’s also SQLite. Many applications can use any of these relational databases by the virtue of the database layer abstraction. If you are writing your own web application, pick any one database and install it on your web server.

Install The Database

• Install the package.
• Enable and start the database systemd unit.
• Initialize the database server.
• Create databases.
• Create database users and set their passwords.

Connect The Web Application To The Database Server

As soon as you are ready to deploy services and applications on the Internet, the first thing you need is a registered domain and a DNS server. Often people just use the DNS service provided by the domain registrar or the hosting or cloud provider. Using the managed service is not a requirement. You can host your own DNS server and manage the DNS for your domains. There are many open source software using which you can build your own DNS servers. Bind is the traditional and probably the most popular DNS solution out there. There are other options such as Knot, PowerDNS and Unbound.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

Architecture 1: Kubernetes Control Plane Without HA

Create three Virtual Machine guests on your laptop or workstation.

• VM 01 - Kubernetes Control Plane
• VM 02, VM 03 - Kubernetes Worker nodes

Architecture 2: Kubernetes Control Plane With HA

Create five Virtual Machine guests on your laptop or workstation.

• VM 01, VM 02, VM 03 - Kubernetes Control Plane with HA
• VM 04, VM 05 - Kubernetes Worker nodes

Use the Kubeadm tool to create the cluster.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

NFS Server

To run your own NFS (Network File System) server, you’ll need to first install Linux on your server. I’d go with Debian, Ubuntu or CentOS distributions. Once you’ve done that, you can install the NFS server software, which is typically included with most Linux distributions. After the NFS server software is installed, you’ll need to configure it to specify which directories on the server should be shared with NFS clients, and what type of access they should have. This typically involves editing the server’s configuration file, which is typically located at /etc/exports. Once the server is configured, you can start the NFS server and begin sharing files with NFS clients. To access the shared files, the NFS clients will need to mount the NFS share using the mount command.

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

There are many tools and software programs that can be used for monitoring and performance analysis on Linux systems. Some popular options include:

• top - This is a command-line utility that shows real-time information about the processes running on a Linux system, such as their CPU and memory usage.
• htop - This is a more advanced version of top that provides a more user-friendly interface and additional features, such as the ability to sort processes by different metrics and to kill processes.
• sar - This is a command-line utility that collects and displays performance metrics for a Linux system over time. It can be used to analyze CPU, memory, I/O, and network usage, as well as other metrics.
• iostat - This is a command-line utility that shows real-time information about I/O performance on a Linux system. It can be used to monitor the performance of disks and other storage devices.
• vmstat - This is a command-line utility that shows real-time information about various system resources, such as memory, CPU, and I/O. It can be used to monitor the overall health of a Linux system.
• netstat - This is a command-line utility that shows information about network connections on a Linux system. It can be used to monitor the status of network connections and to diagnose networking issues.

There are also many modern monitoring tools and software programs available for Linux, such as Prometheus and Zabbix. These tools typically offer more advanced features and capabilities than the built-in Linux utilities, such as the ability to collect and store metrics over time, and to generate alerts when certain conditions are met.

To run your own email server using Linux and other open source software, you’ll need to first choose a Linux distribution and install it on your server. I’d go with either Ubuntu, Debian or Rocky Linux. Once you’ve done that, you can choose an email server software that is compatible with Linux, such as Postfix or Exim. After installing and configuring the email server software, you’ll need to set up DNS records and configure authentication and encryption to ensure that your email server is secure. Finally, you’ll need to test your email server to make sure it is working properly and can send and receive messages.

Definition

A load balancer is a type of software or hardware that distributes incoming traffic across multiple servers or resources. This allows the load balancer to distribute the workload evenly, improving the performance and availability of the application.

The Load Balancer Lab

To run your own load balancer using open source software, you will need to:

• Install and configure the load balancer software on a server. Some popular open source options include HAProxy, Nginx, and Envoy.
• Configure the load balancer to distribute incoming traffic to the appropriate servers or resources. This typically involves setting up virtual servers and defining rules for routing traffic.
• Test the load balancer to ensure that it is working correctly and distributing traffic as expected.
• Monitor the load balancer and the underlying servers to ensure that the system is performing well and handling traffic effectively.
• Continually tune and optimize the load balancer configuration to improve performance and ensure that the application is always available and responsive.

Your web applications need a solution to store and retrieve its data. A relational database is often used in web applications. MySQL, MariaDB and PostgreSQL are some popular relational databases. There’s also SQLite. Many applications can use any of these relational databases by the virtue of the database layer abstraction. If you are writing your own web application, pick any one database and install it on your web server.

Install The Database

• Install the package.
• Enable and start the database systemd unit.
• Initialize the database server.
• Create databases.
• Create database users and set their passwords.

Connect The Web Application To The Database Server

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

As soon as you are ready to deploy services and applications on the Internet, the first thing you need is a registered domain and a DNS server. Often people just use the DNS service provided by the domain registrar or the hosting or cloud provider. Using the managed service is not a requirement. You can host your own DNS server and manage the DNS for your domains. There are many open source software using which you can build your own DNS servers. Bind is the traditional and probably the most popular DNS solution out there. There are other options such as Knot, PowerDNS and Unbound.

A reference application is a sample application that is used as a guide for developers to learn and implement best practices for software development, testing, deployment, and operations within a specific technology stack or framework. It can serve as a blueprint for building and maintaining similar applications and can be used to demonstrate the use of tools, processes, and techniques for achieving high levels of automation, scalability, and reliability.

If you are learning how to deploy applications to Kubernetes of Virtual Machines in cloud, create your own tiny application.

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

When you begin your career in IT, you will be required to learn Git sooner or later. Git is a key technology used in everyday software engineering and DevOps. It has become persuasive and quintessential. There is no need to hard sell Git to anyone at this point.

There is a lot to learn about Git. The good news is that you can start with a subset of Git. There is a no need to become a Git master on your day one of software engineering or DevOps engineering. You can follow a tutorial and take home a Git workflow on day one. In about three days of studying and practising Git, you will be ready to use Git in software engineering and DevOps workflows.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

Using Git with SSH is a common practice among software developers. The convenience of not having to remember passwords is a huge productivity boost in software development workflows.

In a previous blog post, I wrote about managing SSH keys.

Having many SSH keys can cause few issues:

• ssh-agent doesn’t work well with too many keys.
• Can’t always map SSH keys to servers in SSH client configuration. Both personal and company projects Git can be hosted on the same server or third-party service such as Github.
• Some Git hosting services do not allow you to use the same SSH key in more than one user profile. You are forced to have a unique SSH keypair per user profile.
• The SSH server might refuse to allow too many authentication attempts.

Git allows you to set the environment variable GIT_SSH_COMMAND. If you set this environment variable, Git uses the specified command for push and pull operations.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

Ansible comes with a vault feature. It is meant to be used in the context of configuration management. But you can also use it as a standalone simple password vault for your personal or organization’s use.

Initial setup of password vault:

• Create or clone a Git or another SCM repository

git init 

• Create the password vault

ansible-vault create myvault.secret

Type the new master password and confirm, ansible-vault will open your text editor. Type your secrets in the editor and save and quit. To open your vault for viewing or editing in the future, you will need your vault password.

In a previous post, we discussed using age to manage secrets in Git repositories. In this post, let’s improve our secrets management workflow in Git repositories using SOPS.

sops is an editor of encrypted files that supports popular configuration formats such as YAML and various encryption techniques such as age.

Read the blog post about age to install the package and creating the key file.

This time, we will use sops to perform encryption and decryption operations instead of the age command.

Are you storing secrets such as database credentials, API keys, etc. unencrypted in Git repositories? Stop.

To protect your secrets, do not store them anywhere unencrypted. Especially in Git repositories. Ideally, your organization must have some vault solution where secrets can be stored and securely shared with people on a need-to-know basis. In many small organizations, having such a central secrets management solution is still a luxury. The need to store such secret information in Git repositories is obvious. There are a few ways in which you can encrypt secrets. We discussed using Ansible Vault in one of the previous blog posts.

When you begin your career in IT, you will be required to learn Git sooner or later. Git is a key technology used in everyday software engineering and DevOps. It has become persuasive and quintessential. There is no need to hard sell Git to anyone at this point.

There is a lot to learn about Git. The good news is that you can start with a subset of Git. There is a no need to become a Git master on your day one of software engineering or DevOps engineering. You can follow a tutorial and take home a Git workflow on day one. In about three days of studying and practising Git, you will be ready to use Git in software engineering and DevOps workflows.

When you begin your career in IT, you will be required to learn Git sooner or later. Git is a key technology used in everyday software engineering and DevOps. It has become persuasive and quintessential. There is no need to hard sell Git to anyone at this point.

There is a lot to learn about Git. The good news is that you can start with a subset of Git. There is a no need to become a Git master on your day one of software engineering or DevOps engineering. You can follow a tutorial and take home a Git workflow on day one. In about three days of studying and practising Git, you will be ready to use Git in software engineering and DevOps workflows.

When you begin your career in IT, you will be required to learn Git sooner or later. Git is a key technology used in everyday software engineering and DevOps. It has become persuasive and quintessential. There is no need to hard sell Git to anyone at this point.

There is a lot to learn about Git. The good news is that you can start with a subset of Git. There is a no need to become a Git master on your day one of software engineering or DevOps engineering. You can follow a tutorial and take home a Git workflow on day one. In about three days of studying and practising Git, you will be ready to use Git in software engineering and DevOps workflows.

NFS Server

To run your own NFS (Network File System) server, you’ll need to first install Linux on your server. I’d go with Debian, Ubuntu or CentOS distributions. Once you’ve done that, you can install the NFS server software, which is typically included with most Linux distributions. After the NFS server software is installed, you’ll need to configure it to specify which directories on the server should be shared with NFS clients, and what type of access they should have. This typically involves editing the server’s configuration file, which is typically located at /etc/exports. Once the server is configured, you can start the NFS server and begin sharing files with NFS clients. To access the shared files, the NFS clients will need to mount the NFS share using the mount command.

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

To run your own email server using Linux and other open source software, you’ll need to first choose a Linux distribution and install it on your server. I’d go with either Ubuntu, Debian or Rocky Linux. Once you’ve done that, you can choose an email server software that is compatible with Linux, such as Postfix or Exim. After installing and configuring the email server software, you’ll need to set up DNS records and configure authentication and encryption to ensure that your email server is secure. Finally, you’ll need to test your email server to make sure it is working properly and can send and receive messages.

Definition

A load balancer is a type of software or hardware that distributes incoming traffic across multiple servers or resources. This allows the load balancer to distribute the workload evenly, improving the performance and availability of the application.

The Load Balancer Lab

To run your own load balancer using open source software, you will need to:

• Install and configure the load balancer software on a server. Some popular open source options include HAProxy, Nginx, and Envoy.
• Configure the load balancer to distribute incoming traffic to the appropriate servers or resources. This typically involves setting up virtual servers and defining rules for routing traffic.
• Test the load balancer to ensure that it is working correctly and distributing traffic as expected.
• Monitor the load balancer and the underlying servers to ensure that the system is performing well and handling traffic effectively.
• Continually tune and optimize the load balancer configuration to improve performance and ensure that the application is always available and responsive.

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

As soon as you are ready to deploy services and applications on the Internet, the first thing you need is a registered domain and a DNS server. Often people just use the DNS service provided by the domain registrar or the hosting or cloud provider. Using the managed service is not a requirement. You can host your own DNS server and manage the DNS for your domains. There are many open source software using which you can build your own DNS servers. Bind is the traditional and probably the most popular DNS solution out there. There are other options such as Knot, PowerDNS and Unbound.

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

Short answer: use ipify

ipify provides a simple public address API.

Using the tool, you can determine your public IP address programmatically. If you are using the shell:

curl 'https://api.ipify.org'

Using it in a shell script:

my_ip=$(curl 'https://api.ipify.org' -s)
echo $my_ip

Using the Ansible ipify module:

- hosts: localhost
  vars:
  tasks:
    - name: Get my public IP
      ipify_facts:
        timeout: 20
      delegate_to: localhost
      register: public_ip
    - name: output
      debug: msg="{{ ipify_public_ip }}"

Sample output of Ansible playbook execution:

ansible-playbook ipify.yml 
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Get my public IP] *******************************************************************************************************************************************************************************************
ok: [localhost -> localhost]

TASK [output] *****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "49.206.13.205"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0   

There are many applications and technologies enabled by VPN. We will focus on one of them in this post: client VPN.

Connecting To A Private Network

If the server you are trying to access is on a private network, you can use a VPN to connect to the network as if you were physically present on the same network. This can be useful if you need to access resources or devices that are only available on the private network.

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

There are many tools and software programs that can be used for monitoring and performance analysis on Linux systems. Some popular options include:

• top - This is a command-line utility that shows real-time information about the processes running on a Linux system, such as their CPU and memory usage.
• htop - This is a more advanced version of top that provides a more user-friendly interface and additional features, such as the ability to sort processes by different metrics and to kill processes.
• sar - This is a command-line utility that collects and displays performance metrics for a Linux system over time. It can be used to analyze CPU, memory, I/O, and network usage, as well as other metrics.
• iostat - This is a command-line utility that shows real-time information about I/O performance on a Linux system. It can be used to monitor the performance of disks and other storage devices.
• vmstat - This is a command-line utility that shows real-time information about various system resources, such as memory, CPU, and I/O. It can be used to monitor the overall health of a Linux system.
• netstat - This is a command-line utility that shows information about network connections on a Linux system. It can be used to monitor the status of network connections and to diagnose networking issues.

There are also many modern monitoring tools and software programs available for Linux, such as Prometheus and Zabbix. These tools typically offer more advanced features and capabilities than the built-in Linux utilities, such as the ability to collect and store metrics over time, and to generate alerts when certain conditions are met.

To run your own email server using Linux and other open source software, you’ll need to first choose a Linux distribution and install it on your server. I’d go with either Ubuntu, Debian or Rocky Linux. Once you’ve done that, you can choose an email server software that is compatible with Linux, such as Postfix or Exim. After installing and configuring the email server software, you’ll need to set up DNS records and configure authentication and encryption to ensure that your email server is secure. Finally, you’ll need to test your email server to make sure it is working properly and can send and receive messages.

Definition

A load balancer is a type of software or hardware that distributes incoming traffic across multiple servers or resources. This allows the load balancer to distribute the workload evenly, improving the performance and availability of the application.

The Load Balancer Lab

To run your own load balancer using open source software, you will need to:

• Install and configure the load balancer software on a server. Some popular open source options include HAProxy, Nginx, and Envoy.
• Configure the load balancer to distribute incoming traffic to the appropriate servers or resources. This typically involves setting up virtual servers and defining rules for routing traffic.
• Test the load balancer to ensure that it is working correctly and distributing traffic as expected.
• Monitor the load balancer and the underlying servers to ensure that the system is performing well and handling traffic effectively.
• Continually tune and optimize the load balancer configuration to improve performance and ensure that the application is always available and responsive.

Your web applications need a solution to store and retrieve its data. A relational database is often used in web applications. MySQL, MariaDB and PostgreSQL are some popular relational databases. There’s also SQLite. Many applications can use any of these relational databases by the virtue of the database layer abstraction. If you are writing your own web application, pick any one database and install it on your web server.

Install The Database

• Install the package.
• Enable and start the database systemd unit.
• Initialize the database server.
• Create databases.
• Create database users and set their passwords.

Connect The Web Application To The Database Server

Install PostgreSQL Server

Fedora and CentOS:

sudo dnf install postgresql-server

Ubuntu 18.04:

sudo apt install postgresql

New Server Initialization

On CentOS 7/Fedora 30:

sudo postgresql-setup initdb

Upgrading From An Older Version

sudo postgresql-setup --upgrade

Administering The Database Server

Managing The postgresql Daemon

Starting PostgreSQL server

sudo systemctl start postgresql

Checking PostgreSQL Server Status:

sudo systemctl status postgresql

Enabling PostgreSQL Server Systemd Unit/Enabling PostgreSQL Server On Boot:

sudo systemctl enable postgresql

Allowing Password Based Login From localhost

Edit /var/lib/pgsql/data/pg_hba.conf as privileged user(root) and add this line:

In this tutorial, I will provide step-by-step process to implement MySQL replication. We will create one master and one slave. We will use two CentOS 6 servers - one for master and the other for slave. This following steps have been tested on two virtual machines.

Our master server will have IP address 192.168.122.10. Our slave server will have IP address 192.168.122.12.

You might want to run SELinux in permissive mode.

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

Once upon a time, Apache was the de-facto web server solution. Later, Nginx became popular. If you are getting started with DevOps and Linux system administration, I would recommend you to start with Nginx.

Start With A Static Website

What is a static website? A website made from HTML and CSS. And maybe some JavaScript, images, videos, fonts, etc. The key takeaway is that there is no server side application involved. Install the Nginx web server on your Linux VM. Configure it to serve a static website. You will need a static website as a per-requisite. Create a static website by assembling some HTML, CSS, JavaScript and images. Optionally, add some fonts and videos. Access the website from your web browser by typing the IP address of the web server in the address bar. Take it to the next level by pointing the DNS A record of your domain to the VM. For our purposes, a fake domain or a local unregistered domain is sufficient. Manipulating /etc/hosts is also fine. Enjoy viewing the website from the browser.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

The Tech Chorus blog by Sudheer Satyanarayana started way back in 2008. The blog has used various hosting technologies and software over the years.

Initially, the blog site used Drupal content management system. LAMP was a popular technology stack those days. Drupal was one of my favorite CMSes back then. The Drupal site was hosted on a cPanel server.

As life progressed, there was not enough time to patch Drupal and its plugins. Upgrading between major versions of Drupal also took considerable amount of time and effort. I moved on to a static site generator. I found and loved Lektor. With Lektor, I managed the content on my workstation and generated the static site. I used to deploy the generated static site to an AWS EC2 instance with Nginx.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

The Tech Chorus blog by Sudheer Satyanarayana started way back in 2008. The blog has used various hosting technologies and software over the years.

Initially, the blog site used Drupal content management system. LAMP was a popular technology stack those days. Drupal was one of my favorite CMSes back then. The Drupal site was hosted on a cPanel server.

As life progressed, there was not enough time to patch Drupal and its plugins. Upgrading between major versions of Drupal also took considerable amount of time and effort. I moved on to a static site generator. I found and loved Lektor. With Lektor, I managed the content on my workstation and generated the static site. I used to deploy the generated static site to an AWS EC2 instance with Nginx.

1. Learn Linux. Install Linux on your laptop. Get familiar with Linux commands. Learn virtualization. Unlock the path to learn more Linux.
2. Programming. Acquiring some programming skills using a general purpose programming language like Python goes a long way. Learning some web development is required in most DevOps engineering contexts. Although you don’t have to be an expert in web development, you should have a clear understanding of workloads consisting of web applications and microservices. Learning some HTML, CSS and JavaSript is required for web development. At this point, you have to learn at least one database system. I recommend PostgreSQL to get started with relational database system. Git is also an essential tool to manage source code.
3. Automation is the cornerstone of DevOps engineering. Learn the basics of automation with Ansible and shell scripting.
4. CI/CD. The purpose of DevOps is to create a culture and practice where developers can ship their applications to customers quickly, safely and continuously. The CI/CD pipelines often constitute the backbone of the DevOps practice. Jenkins is a popular tool to create CI/CD pipelines. Learn the basics of Jenkins and create pipelines to automatically test and deploy applications.
5. Observability and monitoring. Start with Linux commands such as top, free, du and progress towards Prometheus. The typical Prometheus stack includes Alert Manager and Grafana. Then explore the world of traces with tools such as Jaeger and OpenTelemetry.
6. Kubernetes. The quintessential container orchestration platform.
7. Cloud engineering. Start with one of the popular clouds such as AWS, Azure or GCP.
8. IAC. Take automation to the next level in the cloud. Use Terraform to orchestrate resources in the cloud.

Tech Chorus References

• Learning Linux For Devops
• Preparing For A DevOps Engineer Job With A Personal Project
• Three Day Plan To Learn Git

Learning Resources

DevOps

• Coursera Course: Google IT Automation with Python Professional Certificate
  • Crash Course On Python
  • Using Python to Interact with the Operating System
  • Introduction to Git and GitHub
  • Troubleshooting and Debugging Techniques
  • Configuration Management and the Cloud
  • Automating Real-World Tasks with Python

Python

• Free interactive Python tutorial: LearnPython.org

The blog post attempts to answer some questions like:

• How to become a DevOps engineer?
• How to prepare for a DevOps engineer interview?
• How to get a DevOps engineer job?

In a previous blog post, I wrote about the path an aspiring DevOps engineer could follow.

In this blog post, I will lay out a concrete plan using which you can prepare yourself for a DevOps engineer job. If you follow the steps carefully, you will be armed with practical DevOps knowledge, and you will be able to apply for DevOps engineer jobs confidently.

As soon as you are ready to deploy services and applications on the Internet, the first thing you need is a registered domain and a DNS server. Often people just use the DNS service provided by the domain registrar or the hosting or cloud provider. Using the managed service is not a requirement. You can host your own DNS server and manage the DNS for your domains. There are many open source software using which you can build your own DNS servers. Bind is the traditional and probably the most popular DNS solution out there. There are other options such as Knot, PowerDNS and Unbound.

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

The List Of Web Application Ideas

• An e-commerce platform where people can buy and sell products online.
• A platform for booking and managing doctor’s appointments.
• A social networking site for connecting with friends and family.
• A task and project management tool for teams and organizations.
• A recipe and meal planning app for people who want to eat healthy.
• A travel planning and booking app that helps people plan their trips and find the best deals on flights and accommodations.
• An online education platform that offers courses and tutorials on a variety of subjects.
• A fitness and workout tracker that helps users set goals, track their progress, and stay motivated.
• A budgeting and personal finance app that helps people manage their money and save for the future.
• A to-do list and productivity app that helps users organize their tasks and get things done.

Some Common Features Of These Applications

• User accounts and authentication, so users can create an account and log in to the app.
• A user-friendly interface that is easy to navigate and use.
• Search and filtering tools, so users can find what they are looking for quickly and easily.
• Support for different devices and browsers, so users can access the app from any device with an internet connection.
• Social sharing and collaboration features, so users can share content and work together on projects.
• Security measures to protect user data and keep it safe from unauthorized access.
• Analytics and reporting tools to help users track their progress and make data-driven decisions.
• Integration with other tools and platforms, such as payment processors or email marketing services.
• Support for multiple languages and localization, so users can use the app in their native language.
• Responsive design that adjusts to different screen sizes and resolutions, so the app looks good on any device.

Programming Languages, Tools And Technologies

• HTML, CSS, and JavaScript for creating the user interface and front-end of the web application.
• Server-side languages such as Python, Java, or Go for handling the back-end logic and data processing.
• Databases such as MySQL, PostgreSQL for storing and managing data.
• Web frameworks such as Django or Express.js for building the application and handling common web development tasks.
• Libraries and tools such as ReactJS or Angular for creating interactive and dynamic user interfaces.
• Git for source control.
• Cloud services such as Amazon Web Services or Google Cloud Platform for hosting the application and supporting its infrastructure.
• Payment processing APIs and integrations for enabling transactions and handling payments.
• Email marketing and messaging APIs for sending notifications and updates to users.
• Analytics and tracking tools for gathering data and generating reports.
• Security tools and practices for protecting user data and ensuring the security of the application

References

• DevOps Lab: Create Your Own Reference Application

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

If you are wondering how to automate the installation and configuration of Jenkins using code, this post is for you.

Jenkins is a popular open source tool to build CI/CD pipelines.

Ansible is a popular open source tool to automate a lot of things in IT, including CI/CD and infrastructure orchestration.

Ansible can be used to deploy applications in the cloud. Ansible is a nice tool to execute steps such as:

This is a blog post in IAC with Terraform series.

IAC stands for Infrastructure As Code. Modern IT infrastructure can be orchestrated using programmatic methods. Terraform is(was?) a popular open source software used to orchestrate infrastructure in the cloud and elsewhere too.

Terraform has the concept of modules. With modules, you can code abstract infrastructure. For example, if you are creating a pattern of infrastructure over and over again, you could abstract the pattern into a Terraform module. Let’s take the example of a web application. It consists of:

This is a blog post in IAC with Terraform series.

IAC stands for Infrastructure As Code. Modern IT infrastructure can be orchestrated using programmatic methods. Terraform is(was?) a popular open source software used to orchestrate infrastructure in the cloud and elsewhere too.

Terraform has the concept of modules. With modules, you can code abstract infrastructure. For example, if you are creating a pattern of infrastructure over and over again, you could abstract the pattern into a Terraform module. Let’s take the example of a web application. It consists of:

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY). Instead of writing such tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github.
Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in the future.

These are some guidelines for the Ansible role projects that Gavika follows:

Last Saturday, the Bangalore PHP User Group conducted a meeting. The venue was same as the last time, Microsoft office, Bangalore! The topic of the meeting was Framework Shootout. The frameworks represented were:

• Zend Framework
• CakePHP
• Symfony
• WordPress

I was glad to get an opportunity to represent the Zend Framework. The slides I presented with Ganesh H S can be downloaded or viewed online at SlideShare.

I liked all the presentations. Personally, I believe Zend Framework and Symfony are the two PHP5 frameworks you would want to seriously consider using in your projects. The strengths and weaknesses of each framework varies. In a previous post we discussed the reasons to use Zend Framework. Sjoerd de Jong has offered to conduct training sessions on Symfony for free.

Today, the Bangalore PHP User Group had a meeting. The meetup.com site reports that sixty eight people attended the meeting. The venue was at Microsoft. The increasing participation of Microsoft in PHP conferences and meetings have taken many by surprise. Microsoft were kind enough to offer free Pizza for all the attendees.

They have given me a copy of Windows 7 Release Candidate which expires on June 1, 2010. If time permits, I will sure try to install it as a virtual guest.

This is a blog post in IAC with Terraform series.

IAC stands for Infrastructure As Code. Modern IT infrastructure can be orchestrated using programmatic methods. Terraform is(was?) a popular open source software used to orchestrate infrastructure in the cloud and elsewhere too.

Terraform has the concept of modules. With modules, you can code abstract infrastructure. For example, if you are creating a pattern of infrastructure over and over again, you could abstract the pattern into a Terraform module. Let’s take the example of a web application. It consists of:

This is a blog post in IAC with Terraform series.

IAC stands for Infrastructure As Code. Modern IT infrastructure can be orchestrated using programmatic methods. Terraform is(was?) a popular open source software used to orchestrate infrastructure in the cloud and elsewhere too.

Terraform has the concept of modules. With modules, you can code abstract infrastructure. For example, if you are creating a pattern of infrastructure over and over again, you could abstract the pattern into a Terraform module. Let’s take the example of a web application. It consists of:

The Tech Chorus blog by Sudheer Satyanarayana started way back in 2008. The blog has used various hosting technologies and software over the years.

Initially, the blog site used Drupal content management system. LAMP was a popular technology stack those days. Drupal was one of my favorite CMSes back then. The Drupal site was hosted on a cPanel server.

As life progressed, there was not enough time to patch Drupal and its plugins. Upgrading between major versions of Drupal also took considerable amount of time and effort. I moved on to a static site generator. I found and loved Lektor. With Lektor, I managed the content on my workstation and generated the static site. I used to deploy the generated static site to an AWS EC2 instance with Nginx.

In this blog post, I will explain how I implemented a graph algorithm to solve the database migration problem in an application pluggable module system.

Prerequisites:

• Working knowledge of Python
• Working knowledge of Graph Theory. Familiar with the terms: Edge, vertex, path, sink, source, digraph, path graph, etc.

Gavika Web Framework has a pluggable module system. The modules can be developed independently. They can be installed, upgraded and removed from the main application. Gavika Web Framework is written using Python, Flask, SQLAlchemy and a bunch of other related technologies and libraries.

AWS Certified Solutions Architect - Associate is one of the most sought-after certifications in the IT industry.

Here’s a few tips for those seeking this certification.

Background Knowledge And Experience

AWS recommends “at least one year of hands-on experience designing available, cost-efficient, fault-tolerant, and scalable and distributed systems on AWS.” In other words, if you recently started your career in IT, there are a few things you have to do before you start preparing for this certification.

Install the Ansible role gavika.aws_cloudwatchagent via Galaxy

ansible-galaxy install gavika.aws_cloudwatchagent

Create The Playbook File - cw-play.yml :

---
- hosts: all
  become: true
  vars:
  roles:
    - role: gavika.aws_cloudwatchagent

Prepare the AWS CloudWatch Agent configuration

In your variables file, use aws_cloudwatch_agent_config

agent:
    metrics_collection_interval: 60
    run_as_user: "cwagent"
    metrics:
      namespace: "Gavika"
      append_dimensions:
        InstanceId: "${aws:InstanceId}"
      metrics_collected:
        disk:
          measurement:
            - used_percent
          metrics_collection_interval: 60
          resources:
            - "*"
        mem:
          measurement:
            - mem_used_percent
          metrics_collection_interval: 60

In this example, I am using the namespace, Gavika. Feel free to change it. We collect the cpu, disk, diskio, mem and swap metrics. The agent will send these metrics once in 360 seconds.

AWS Certified Solutions Architect - Associate is one of the most sought-after certifications in the IT industry.

Here’s a few tips for those seeking this certification.

Background Knowledge And Experience

AWS recommends “at least one year of hands-on experience designing available, cost-efficient, fault-tolerant, and scalable and distributed systems on AWS.” In other words, if you recently started your career in IT, there are a few things you have to do before you start preparing for this certification.

Introduction

The article explains how to run your own OpenVPN server. We will create a Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY). Instead of writing such tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github.
Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in the future.

These are some guidelines for the Ansible role projects that Gavika follows:

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY). Instead of writing such tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github.
Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in the future.

These are some guidelines for the Ansible role projects that Gavika follows:

Last Saturday, the Bangalore PHP User Group conducted a meeting. The venue was same as the last time, Microsoft office, Bangalore! The topic of the meeting was Framework Shootout. The frameworks represented were:

• Zend Framework
• CakePHP
• Symfony
• WordPress

I was glad to get an opportunity to represent the Zend Framework. The slides I presented with Ganesh H S can be downloaded or viewed online at SlideShare.

I liked all the presentations. Personally, I believe Zend Framework and Symfony are the two PHP5 frameworks you would want to seriously consider using in your projects. The strengths and weaknesses of each framework varies. In a previous post we discussed the reasons to use Zend Framework. Sjoerd de Jong has offered to conduct training sessions on Symfony for free.

Today, the Bangalore PHP User Group had a meeting. The meetup.com site reports that sixty eight people attended the meeting. The venue was at Microsoft. The increasing participation of Microsoft in PHP conferences and meetings have taken many by surprise. Microsoft were kind enough to offer free Pizza for all the attendees.

They have given me a copy of Windows 7 Release Candidate which expires on June 1, 2010. If time permits, I will sure try to install it as a virtual guest.

Short answer: use ipify

ipify provides a simple public address API.

Using the tool, you can determine your public IP address programmatically. If you are using the shell:

curl 'https://api.ipify.org'

Using it in a shell script:

my_ip=$(curl 'https://api.ipify.org' -s)
echo $my_ip

Using the Ansible ipify module:

- hosts: localhost
  vars:
  tasks:
    - name: Get my public IP
      ipify_facts:
        timeout: 20
      delegate_to: localhost
      register: public_ip
    - name: output
      debug: msg="{{ ipify_public_ip }}"

Sample output of Ansible playbook execution:

ansible-playbook ipify.yml 
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Get my public IP] *******************************************************************************************************************************************************************************************
ok: [localhost -> localhost]

TASK [output] *****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "49.206.13.205"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0   

Moving around

You can use the arrow keys on keyboard to move around in the command line. Bash also provides convenient keyboard shortcuts to navigate effectively. Try them out and see for yourself.

To become a Bash pro user you have to get yourself familiar with the keyboard shortcuts. Once you do, you’ll find yourself productive.

Moving around words using ESC+f and ESC+b are my favourites in this list. Jumping to first and last lines of the history list is also useful.

Install the Ansible role gavika.aws_cloudwatchagent via Galaxy

ansible-galaxy install gavika.aws_cloudwatchagent

Create The Playbook File - cw-play.yml :

---
- hosts: all
  become: true
  vars:
  roles:
    - role: gavika.aws_cloudwatchagent

Prepare the AWS CloudWatch Agent configuration

In your variables file, use aws_cloudwatch_agent_config

agent:
    metrics_collection_interval: 60
    run_as_user: "cwagent"
    metrics:
      namespace: "Gavika"
      append_dimensions:
        InstanceId: "${aws:InstanceId}"
      metrics_collected:
        disk:
          measurement:
            - used_percent
          metrics_collection_interval: 60
          resources:
            - "*"
        mem:
          measurement:
            - mem_used_percent
          metrics_collection_interval: 60

In this example, I am using the namespace, Gavika. Feel free to change it. We collect the cpu, disk, diskio, mem and swap metrics. The agent will send these metrics once in 360 seconds.

Ansible comes with a vault feature. It is meant to be used in the context of configuration management. But you can also use it as a standalone simple password vault for your personal or organization’s use.

Initial setup of password vault:

• Create or clone a Git or another SCM repository

git init 

• Create the password vault

ansible-vault create myvault.secret

Type the new master password and confirm, ansible-vault will open your text editor. Type your secrets in the editor and save and quit. To open your vault for viewing or editing in the future, you will need your vault password.

Install PostgreSQL Server

Fedora and CentOS:

sudo dnf install postgresql-server

Ubuntu 18.04:

sudo apt install postgresql

New Server Initialization

On CentOS 7/Fedora 30:

sudo postgresql-setup initdb

Upgrading From An Older Version

sudo postgresql-setup --upgrade

Administering The Database Server

Managing The postgresql Daemon

Starting PostgreSQL server

sudo systemctl start postgresql

Checking PostgreSQL Server Status:

sudo systemctl status postgresql

Enabling PostgreSQL Server Systemd Unit/Enabling PostgreSQL Server On Boot:

sudo systemctl enable postgresql

Allowing Password Based Login From localhost

Edit /var/lib/pgsql/data/pg_hba.conf as privileged user(root) and add this line:

Install PostgreSQL Server

Fedora and CentOS:

sudo dnf install postgresql-server

Ubuntu 18.04:

sudo apt install postgresql

New Server Initialization

On CentOS 7/Fedora 30:

sudo postgresql-setup initdb

Upgrading From An Older Version

sudo postgresql-setup --upgrade

Administering The Database Server

Managing The postgresql Daemon

Starting PostgreSQL server

sudo systemctl start postgresql

Checking PostgreSQL Server Status:

sudo systemctl status postgresql

Enabling PostgreSQL Server Systemd Unit/Enabling PostgreSQL Server On Boot:

sudo systemctl enable postgresql

Allowing Password Based Login From localhost

Edit /var/lib/pgsql/data/pg_hba.conf as privileged user(root) and add this line:

You access Python dictionary keys using the syntax:

my_dicy[my_key]

For example:

>>> my_dict = {'food': 'idly'}
>>> my_dict['food']
'idly'

Sometimes, you might want to access the dictionary keys using:

my_dict.my_key

syntax. If you do this is what happens:

>>> my_dict.food
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'food'

How can you solve this? Easy.

 pip install attrdict

How do you use the newly installed package?

>>> from attrdict import AttrDict
>>> my_dict = AttrDict({'food': 'idly'})
>>> my_dict.food
'idly'

In this tutorial, I will provide step-by-step process to implement MySQL replication. We will create one master and one slave. We will use two CentOS 6 servers - one for master and the other for slave. This following steps have been tested on two virtual machines.

Our master server will have IP address 192.168.122.10. Our slave server will have IP address 192.168.122.12.

You might want to run SELinux in permissive mode.

In this tutorial, I will provide step-by-step process to implement MySQL replication. We will create one master and one slave. We will use two CentOS 6 servers - one for master and the other for slave. This following steps have been tested on two virtual machines.

Our master server will have IP address 192.168.122.10. Our slave server will have IP address 192.168.122.12.

You might want to run SELinux in permissive mode.

In this tutorial, I will provide step-by-step process to implement MySQL replication. We will create one master and one slave. We will use two CentOS 6 servers - one for master and the other for slave. This following steps have been tested on two virtual machines.

Our master server will have IP address 192.168.122.10. Our slave server will have IP address 192.168.122.12.

You might want to run SELinux in permissive mode.

If you have to insert a comment on several lines do you do manually insert the comment character in every line? Stop. Vim is a good editor and has a nice feature to accomplish this quickly. Here are the steps:

1. Enter visual blocking mode by pressing CTRL V (CTRL key and the lowercase v).
2. Make your selection using motion keys(jklm, etc.).
3. Press I(uppercase I) to enter block insert mode.
4. Press #, the comment character.
5. Press Esc key. The comment character # will be inserted on each line the visual block selection.

Read more about visual blocking mode using the vim help topic visual-block. At the command line(:), type help visual-block.

If you have to insert a comment on several lines do you do manually insert the comment character in every line? Stop. Vim is a good editor and has a nice feature to accomplish this quickly. Here are the steps:

1. Enter visual blocking mode by pressing CTRL V (CTRL key and the lowercase v).
2. Make your selection using motion keys(jklm, etc.).
3. Press I(uppercase I) to enter block insert mode.
4. Press #, the comment character.
5. Press Esc key. The comment character # will be inserted on each line the visual block selection.

Read more about visual blocking mode using the vim help topic visual-block. At the command line(:), type help visual-block.

If you have to insert a comment on several lines do you do manually insert the comment character in every line? Stop. Vim is a good editor and has a nice feature to accomplish this quickly. Here are the steps:

1. Enter visual blocking mode by pressing CTRL V (CTRL key and the lowercase v).
2. Make your selection using motion keys(jklm, etc.).
3. Press I(uppercase I) to enter block insert mode.
4. Press #, the comment character.
5. Press Esc key. The comment character # will be inserted on each line the visual block selection.

Read more about visual blocking mode using the vim help topic visual-block. At the command line(:), type help visual-block.

Moving around

You can use the arrow keys on keyboard to move around in the command line. Bash also provides convenient keyboard shortcuts to navigate effectively. Try them out and see for yourself.

To become a Bash pro user you have to get yourself familiar with the keyboard shortcuts. Once you do, you’ll find yourself productive.

Moving around words using ESC+f and ESC+b are my favourites in this list. Jumping to first and last lines of the history list is also useful.

Moving around

You can use the arrow keys on keyboard to move around in the command line. Bash also provides convenient keyboard shortcuts to navigate effectively. Try them out and see for yourself.

To become a Bash pro user you have to get yourself familiar with the keyboard shortcuts. Once you do, you’ll find yourself productive.

Moving around words using ESC+f and ESC+b are my favourites in this list. Jumping to first and last lines of the history list is also useful.

Last Saturday, the Bangalore PHP User Group conducted a meeting. The venue was same as the last time, Microsoft office, Bangalore! The topic of the meeting was Framework Shootout. The frameworks represented were:

• Zend Framework
• CakePHP
• Symfony
• WordPress

I was glad to get an opportunity to represent the Zend Framework. The slides I presented with Ganesh H S can be downloaded or viewed online at SlideShare.

I liked all the presentations. Personally, I believe Zend Framework and Symfony are the two PHP5 frameworks you would want to seriously consider using in your projects. The strengths and weaknesses of each framework varies. In a previous post we discussed the reasons to use Zend Framework. Sjoerd de Jong has offered to conduct training sessions on Symfony for free.

Today, the Bangalore PHP User Group had a meeting. The meetup.com site reports that sixty eight people attended the meeting. The venue was at Microsoft. The increasing participation of Microsoft in PHP conferences and meetings have taken many by surprise. Microsoft were kind enough to offer free Pizza for all the attendees.

They have given me a copy of Windows 7 Release Candidate which expires on June 1, 2010. If time permits, I will sure try to install it as a virtual guest.

Last Saturday, the Bangalore PHP User Group conducted a meeting. The venue was same as the last time, Microsoft office, Bangalore! The topic of the meeting was Framework Shootout. The frameworks represented were:

• Zend Framework
• CakePHP
• Symfony
• WordPress

I was glad to get an opportunity to represent the Zend Framework. The slides I presented with Ganesh H S can be downloaded or viewed online at SlideShare.