GitLab vs. GitHub: DevSecOps Pipeline

by Romano Roth and Patrick Steger

This video series will show you how to build up an enterprise-ready DevSecOps Pipeline with GitLab and GitHub and compare the two platforms.

​Topic	​GitLab	​GitHub
​Introduction	Provides integration points to include security tools. Offers GitLab maintained integrations for GitLab recommended default security tools.	Provides integration points to include security tools. There are no default GitHub recommended/maintained security tools. There are security tools provided by the community for open source and commercial tools.
​Creating a simple project	Can create new empty repository Offers to start with GitLab provided default project/repo-structure (e.g. C#, SpringBoot, PHP, etc). They are easy to find. Can import from other repositories like any Git repository (URL), GitHub, Jira, Bitbucket, etc. .gitlab-ci.yml is the entry point defining the whole pipeline. Option to include additional files (their content is included automatically at runtime). Pipeline is broken down into stages and jobs, where stages are executed after each other and all jobs of a stage run in parallel (dependencies can be configured) This is a very neat approach but can become tricky to understand for complex pipelines. So it was easier to understand for us at the beginning, but we feel its slightly harder to maintain and understand for projects with many pipelines / complex scenarios) Pipeline runs are found in menu CI/CD -> pipelines. Very easy to create a simple first pipeline because GitLab provides defaults for most everything that is required to start Nice UI for navigating the repositories/projects Artifacts can simply be declared and are then available in the pipeline results view for download, this is GitLab provided and well documented. Standard artifacts like test-results can be made visible in the GitLab UI using default GitLab functionality. Integreation into the GitLab UI is professional out-of-the-box. Different pipelines (ways to execute the pipeline) must be configured with rules (e.g. based on what branch was commited). This may become complicated when basically wanting different pipelines to run for different conditions (because pipelines view does not cleanly separates them and offers just views for different branches and tags). It may get a bit messy for complex scenarios. Nice graphical view to visualize stages and jobs as well as their dependencies (but not the conditions)	Can create new empty repository Can create a new repository based on a template project. Template projects are hard to find when there are none in your organization. (Searching on GitHub with tag called #github-templates or #template-repository produces some...). Can import from existing Repositories. Including GitLab. But does not work when using Social-Login (i.e. Login with Account not provided by GitLab). Pipelines are called Workflows. We can have a dedicated file for each workflow we want. Within the worflow file we define what triggers the workflow/pipeline. It is possible to "call" sub-workflows (files) from a workflow file. Pipelines/Workflows are cleanly separated in the UI and make it easy to find the right workflow/pipeline. Pipeline/Workflow runs are found cleanly listed by Workflow in tab "Actions" in the UI. Workflows use actions to implements jobs/steps. Such Actions are available over a Marketplace and not always easy to be found. This is a flexible way to design pipelines. It took us longer to understand how everything works, but we feel that it is easier to maintain and understand projects with many pipelines/complex scenarios) Quite complex to get the first simple pipeline running because not much is automatically provided by GitHub Nice UI for navigating the repositories/projects. Artifacts are made available using an Action (for example using upload-artifact Action found in the Marketplace) Standard artifacts like test-results require other actions from the marketplace to make them available in the GitHub UI. The UI quality and integration into the GitHub UI varies. Different pipelines are simply configured in extra workflow files. This becomes natural after some time and supports complex scenarios but requires clear guidance within the project team. Nice graphical view to visualize the workflow/pipeline run for executed runs.
​​​Software Composition Analysis (SCA)	GitLab provided open source scanner available, documented and easily found Simply add a GitLab provided template (or copy the file to the project) The default tool has adequate quality and findings and is enterprise ready Findings are well integrated in GitLab UI (Pipeline view, Security&Compliance -> Vulnerability report) Vulnerabilities are also available as a download (json). Commercial tools and custom tools can also be integrated but require quite a bit of work. Integrating their findings into GitLab as a central platform can proof difficult. GitLab uses a GitLab specific format to integrate vulnerabilities in the GitLab UI. No built-in scheduled re-scans, but can be defined explicitly. (See later video). Special feature: Comparable simple editor of files in the project.	No GitHub provided default tool Find one in the Marketplace. We choose CRDA of RedHat based on Snyk (which proofed to be not enterprise ready because it was instable regarding availability and it limits the number of scans).There are not to many open source options available that integrate well into GitHub. Using commercial tools usually leads to situations where the findings are only found within the commercial tool selected. Marketplace tools (Actions) are provided by third parties. Be aware of the risk and consider what providers you trust. Consider also building your own tool. GitHub has also some built-in functionality (called Dependabot, enabled in the project settings) that can be enabled in settings and allows regular, scheduled scans of unchanged code. That is great. The GitHub format to display vulnerabilities in the GitHub UI is based on standards. This proofs to be better for custom integration of tools because some tools provide these standard formats. Dependabot can alert of new findings and offer pull requests for known fixes. But it is not executed on commit and cannot block merges and similar. Dependabot dependencies are visibale in the GitHub UI. Dependabot findings can be found in tab Security -> Dependabot. We did not get any findings here when the video was made although we should have. Special Feature: Fully integraded development environment (similar to Visual Studio Code). This proofed to be very powerful.
​License Compliance	License compliance is provided as a managed GitLab tool (based on License-Finder open source project) that is easy and straightforward to add. Yet at time of recording the tool had a bug we required to fix (marking a file as executable) All used licenses of the project are visible in a default GitLab UI. Can configure acceptable licenses as part of the platform (project settings; we can define a license policy). Support of basic workflows to verify and accept new licenses (i.e. include a security team member as acceptor/reviewer) Failed (not marked as acceptable) licenses are reported.	Requires community provided (Marketplace) or custom Action to implement this. It's meaningful to define an extra sub-workflow (pipeline) for this. Our session on this topic led to improvements for Action license-finder-scan (GitHub support helped us to create this, it is now in the Marketplace available). Note that the license finder image we use in the video had a bug requiring us to mark a file as executable in the image (i.e. same as with GitLab, as both use the same open source tool/image). Can define what licenses are configurable, get error/failed pipeline when other licenses are found. Failed (not marked as acceptable) licenses are - in our case - reported as "customized Test Results". Results are visible in the pipeline run only (not in the scan results view of the platform). There is no easy way to see "all licenses used by the project" (this is only in the log-output of the pipeline run).
​​Static Application Security Testing (SAST)	GitLab provided standard tool based on semgrep and other open source tools (e.g. spotbugs) available. Easy to integrate with a few lines of configuration. GitLab added extra rules to the open source tools, so that it finds way more security issues than the plain open source semgrep tool. Overall the findings look good enough. Findings of the tools are integrated in the GitLab user interface (vulnerability view)	We added the SAST tools in a distinct sub-workflow that is triggered from our main workflow/pipeline. GitHub provides a standard tool for static analysis (CodeQL). We added in addition the open source tools semgrep and spotbugs, some of them required us to change the report format into a format GitHub understands. The combined results of all the three tools still miss important findings. Custom work (i.e. defining custom rules) or extra licenses are required to improve. Finding appropriate SAST tools (Actions) in the Marketplace is not straight forward. Many tools in the Marketplace require licenses for these tools. Findings of the tools (they need to support sarif) are integrated in the GitHub user interface (security tab).
​Container Scanning	Standard tool provided by GitLab (based on Trivy and Grype). Result is Integrated in the GitLab UI in two places: In the pipeline run, in the Vulnerability Report.. Findings are reasonably good. Easy to add. But needs a container image built first. Creating a container image and making it available in the default container registry requires a bit more effort but is documented in GitLab documentation. Findings will not break the pipeline; they are just reported...which is often what companies do. But we cannot configure GitLab to break a pipeline when vulnerabilities are found.	Create two sub-workflows/pipelines: One to create a container image. One to scan the docker image. We selected Trivy as container scanner, there is no GitHub standard, so you have to find it in the Marketplace. Results are integrated into the GitHub UI in Security -> Code Scanning (when they provide a sarif report, as our job does). Duplicates with SCA may happen. Findings are reasonably good (with our selected tool). Creating a container and making it available in the container registry is reasonably documented in GitHub but not trivial. As always it requires Actions found in the Marketplace. Findings will not break the pipeline; they are just reported...which is often what companies do. Making the pipeline break will require some custom work.
​​Secret Detection	GitLab provides a standard tool based on GitLeaks. Integration is easy. Secrets found are limited; a good SCA tool is required as complement. GitLeaks focus is on security tokens (e.g. AWS access tokens), not regular keys or passwords. Findings are reported in the GitLab UI; both in the pipeline and the Vulnerability Report. Btw: GitLab has no integrated secure way to store secrets. They support HashiCorp Vault, but the Vault must be hosted somewhere else and managed by you.	GitHub has built-in functionality for secrets. Integration is easy. Finds secrets in the repository – so reports secrets that are disclosed and must be replaced. At the time when we recorded the video “push protection” of secrets did not work (i.e. do not allow to push commits with secrets). Secrets found are limited, a good SCA tool is required as complement. The focus is on security tokens (e.g. AWS access tokens), not regular keys or passwords. You can define and test your own secret patterns. Btw: GitHub has a well-protected secrets area where secrets can be stored securely. AND they also support Azure KeyVault out-of-the-box which gives you a very secure alternative for storing and managing secrets. A huge plus point.
​​Dynamic Application Security Testing (DAST)	Provides default tooling. Based on OWASP ZAP. Scan capabilities are very limited out-of-the-box, needs considerable configuration Out-of-the-box findings are mostly irrelevant Having to create a container first that can be started to run DAST against Container can be started in GitLab internal runtime environment Results are nicely integrated in Pipeline run. Results can also be found in the vulnerability management report of the GitLab UI	No out-of-the-box solution. Need to find one in the marketplace. We choose OWASP ZAP based on the action zaproxy/action-full-scan Scan capabilities are limited out-of-the-box, but results better than in GitLab Having to create a container first that can be started to run DAST against. Container can be started in GitHub using an Ubuntu image that has docker installed and passing it our image we want to run and test there. Results are not integrated in the GitHub UI. And therefore also not integrated in the GitHub vulnerability management. Just a standalone report is made available. The Reason is missing sarif support (report format) in OWASP ZAP.
​​Vulnerability Management	GitLab offers functionality to manage vulnerabilities in the platform UI. You can add Vulnerabilities manually. With a lot of goodwill you may be able to have GitLab as your single vulnerability management tool. We are missing most the capability to add a description why we dismissed (false positive or accepted risk) a finding. SO dismissing a finding is possible but without reasoning. You can create an issue from a vulnerability -> “Developer Task” You can manually mark findings as resolved or accept the automatically detected resolving. While it is possible to integrate findings of additional tools they are required in a non-standard format, so hard to get.	GitHub offers limited functionality to manage vulnerabilities in the platform UI. Most critically the capability to add custom vulnerabilities/findings. This means, that you always need another tool to do vulnerability management anyway. You can create an issue from a vulnerability/finding-> “Developer Task” Dismissing findings as false positive, won’t fix (accepted risk), test code only is possible with a comment. Resolving is only automatically possible. No marking as fixed in the vulnerability management, You can integrate findings of additional tools -> when they deliver the report in sarif format. You cannot dismiss for a limited time. You cant change the severity of a finding.
​Merge Request / Pull Request	You can protect branches, so that merge requests are mandatory Lists security findings delta between the merged and the default branch So choose the default branch wisely Deltas and findings are available for all the security tools integrated in GitLab. Deltas/Findings are nicely presented and can be navigated to. You can have defined people eligible to approve if new vulnerabilities of a given criticality are found You can see also non-security stuff such as: code changes, review comments, test results, pipeline runs	You can enforce that pull requests are mandatory for a branch. You can present some results of the security tools, quite a bit of customization is required to get acceptable quality. (Branch protection status checks configuration as well as user interface definitions. And a trigger on the pipeline for the branches to execute the checks/tools.) New findings are then available - to some degree - for the security tools (checks). Resolved findings are not visible. Note that findings are not aggregated between tools, you might see a finding multiple times. You can have defined people eligible to approve the pull request, but this is not based on new findings/vulnerabilities. You can see also non-security stuff such as: code changes, review comments, test results.
​Schedule Pipeline
Our Recommendation
​Code	​Here you can find the code: https://gitlab.com/romano_roth/gitlabdevsecopspipeline	​Here you can find the code: https://github.com/romanoroth/GitHubDevSecOps

Summary