Capstone Project: Week 2, Proposal
This week is proposal week, and I have deciced on a project! I am kind of late but I have not been slacking. I have been researching as much as I can and working with dozens of different tools so that I could get a good grasp on the project before proposing it. While I am not confident in anything yet, I have decided that this project is doable and I will enjoy it!
Below is my project proposal as well as the associated files. Thank you for reading!
DevOps CI/CD Automation
Objective
The overarching goal of this project is to use Development & Operations (DevOps) methodologies to automate an IT infrastructure from code commits to deployment to the cloud. The main objective of this project is to create continuous integration and continuous delivery/deployment (CI/CD) pipeline to automate the deployment of a web application to a cloud server. DevOps automation using CI/CD sets out to solve a multitude of problems including issues, bugs, and delays with integration, slow deployment times, slow manual testing, terrible bug finding efficiency, ineffective communication between team members, improper code practices, among other problems (Longbottom, 1).
Background and Rationale
Every minute that is spent on a product that is not adding value to the customer is a waste and should be eliminated. For example, if it takes 5 days of coding and 1 day of deployment to release an update or feature to the customer, but it takes 20 days due to delays, integration issues, manual testing, and miscommunication, among other things, then your business process is inefficient (Microsoft, 4). These issues have largely stemmed from the waterfall methodology of application development, where each stage of the project requires the previous stage to be completed before work can begin.
Many businesses have been turning to the DevOps methodology to increase efficiency with their business process. Efficiency is key when it comes to DevOps, and the way we do that is by using automation. The key to DevOps is the CI/CD pipeline which automates every step of the way from code commits to deployment to servers. Through continuous integration, one can modify the source code, such as a git commit from a team member to a git repository, and the CI tools will automatically validate those changes by building the application and running tests on the application to make sure everything is functioning correctly (Redhat, 1). If not, errors are swiftly detected, and the pipeline will fail, potentially triggering a new pipeline for errors. This triggered pipeline usually involves creating a new work item or a GitHub issue item, resulting in quick conflict resolution. The final step involved with continuous integration is the creation of an artifact which can be a multitude of things such as a package of files or a docker image.
Continuous delivery is the next phase of CI/CD and it is typically triggered by the creation of the artifact produced in continuous integration. This artifact is what you want to deliver to some environment, whether it be a testing environment, staging environment, or most commonly, the production environment with the option to hold deployment. Continuous deployment is continuous delivery to production except there is no hold, so any change that passes through the CI/CD pipeline with no errors will be deployed to the customer (Chandrasekara & Herath, 1). Finally, DevOps does not end at just continuous deployment. There is some risk when it comes to automatically deploying changes to software and there needs to be some way to alert the team that something went wrong. Thus, an integral part to any CI/CD system is to install and configure some sort of monitoring system for your server or application. As an example, if you see a sudden spike in memory usage soon after deployment then you can quickly respond to the issue with an idea of where the issue originated.
When using CI/CD, a company can have as many steps as it wants. The company could skip the build stage entirely if it is not needed, or, they may decide not to utilize continuous deployment and skip that part of the chain completely. Additionally, a company may utilize various tools to perform multiple different tasks within each stage. There is no defined way to implement CI/CD within an IT infrastructure. What matters is that it works, and your team is comfortable with the tools that they use. The main issue for CI/CD comes from finding the right tools for the job when there are hundreds of tools out there and every company is using a different “stack” (stackshare, 1).
Proposed Solution
The solution to bugs and delays with integration, slow deployment times, slow testing, improper code practices, and all the other problems that stem from an improper development methodology is a DevOps infrastructure featuring a robust and inclusive continuous integration and continuous deployment pipeline. This project will focus on the CI/CD pipeline aspect of DevOps which will remediate many of the proposed problems. The CI/CD pipeline will implement various tools and technologies to accomplish the task of automating the deployment of a web application to a cloud server. Figure 3 shows the current plans for the CI/CD pipeline.
The diagram illustrates the two pathways that I have elected to pursue further research and pursue for my CI/CD pipeline. All the tools in Figure 3 are subject to change for reasons such as unexpected pricing issues, incompatibility issues, and insufficient capabilities. However, after working with and studying dozens of technologies, I believe either one will be suited for the task. I will now briefly explain the tools and processes that are currently planned to be used for this project.
First, the developer will utilize the Visual Studio Code app to work on an application. The developer may also include Ansible playbooks which are YAML files that can be deployed to servers for configuration management. The developer may also make use of Docker to containerize or do manual testing on the code. When done, the developer will commit the changes to the repository which could be either GitHub or GitLab.
Once the change is committed, the continuous integration tool, whether it be Azure DevOps or GitLab CI, will recognize the changes and build the application and perform automatic scans, linting, security testing, etc. The CI tool will also send out a webhook to alert communication software such as Slack or GitHub to alert for new commits or other notifications. Once done, an artifact will be created, and it will be passed on to the continuous deployment tool which is also apart of Azure DevOps or GitLab. The artifact may then be stored in its respective container registry. It may then have some more tests run before delivering to the cloud server.
The cloud server will be hosted on either an Amazon AWS EC2 instance, or Microsoft Azure Virtual Machine instance. There will be separate server that will manage the monitoring and security of the application as well as a server to host Ansible. With that, we should have a fully functional CI/CD pipeline that meets the goals of this project. The hard goals for this project, or requirements that are necessary to fulfill said solution, are creating a CI/CD pipeline that takes code from a repo, performs some tests on it, and automatically deploys it to a server while notifying a team service such as Slack for commits. Also, we will require an encompassing monitoring service for the application such as Data Dog, Grafana, or the Elk stack.
The soft goals for this project, or goals that are proposed but not critical to the functionality of the project, are the creation and usage of Ansible with Ansible Playbooks to configure the server, implementing a security service using Data Dog, Elk, or some other service besides just one with monitoring capabilities, as well as an in-depth usage of docker containers such as creating a container out of the application and deploying that to the server instead of plain code.
The optional goals of this project are goals that will only be done if time permits, as in if I finish the above tasks before the deadline. These goals are not requirements for the completion of the project. Stretch goals that may be included are additional security testing which practices the methodology of DevSecOps (NotSoSecure, 1), Infrastructure as Code using Terraform, and container orchestration using Kubernetes or Docker Swarm. Again, these goals are out of scope of the project but would be beneficial to add if time allows.
Schedule and References
The Gantt Chart outlying the schedule for this project as well as the reference page for the proposal are attached below:
Concept-base illustration of creative idea with light bulb image Illustration by Iconscout Store on Iconscout. Developer Illustration by Manypixels Gallery