Swaggerfy

Swaggerfy operationalizes NightVision’s Swagger Extractor by generating Swagger docs from your entire GitHub organization, within minutes.

Example report: https://nvsecurity.github.io/swaggerfy/

Requesting ✨ Feedback ✨

This is a work in progress and experimental. We are seeking some customer feedback to see if this is useful to them in performing API Extraction across their code bases, and how we can build this into our product.

TL;DR

Setup

# Installation
brew install gh
brew install nvsecurity/taps/nightvision
# Update installation if it is already installed
brew update && brew upgrade nightvision
pip3 install https://downloads.nightvision.net/swaggerfy/latest/swaggerfy.tar.gz --user

# Login to NightVision and GitHub
nightvision login
gh login
gh auth refresh -s workflow,admin:public_key,gist,read:org,repo

Usage

Example 1: The vulnerable-apps GitHub organization

# 1a. Query GitHub to discover all repositories in your organization that build REST APIs
swaggerfy query-github --org vulnerable-apps --frameworks all --forks
# 1b. No forks
swaggerfy query-github --org vulnerable-apps --frameworks all

# 2a. Download the repositories that are eligible for Swagger extraction
swaggerfy clone --owner vulnerable-apps
# 2b. If the repository exists locally, pull it anyway for updates
swaggerfy clone --owner vulnerable-apps --force-pull

# 3. Analyze the GitHub repositories and generate Swagger docs for all the GitHub repos, using the NightVision CLI
swaggerfy analyze --owner vulnerable-apps -o analysis.json

# 4. Generate an HTML report of the work that was performed and the Swagger docs for all the repositories
swaggerfy report -a analysis.json -o report.html

The above commands, when executed, would generate an HTML report file that provides an API Inventory, with Swagger docs that were generated using NightVision’s Swagger Extraction capability. See the example report here: https://nvsecurity.github.io/swaggerfy/

Example 2: The spring-projects GitHub organization

swaggerfy query-github --org spring-projects --frameworks all
swaggerfy clone --owner spring-projects
swaggerfy analyze --owner spring-projects -o analysis.json
swaggerfy report -a analysis.json -o report.html

How does this work?

query-github command

The query-github command performs queries against the GitHub Code Search API to look for dependency manager files that contain patterns that indicate the usage of different frameworks. For example, a Python Flask application will always have flask mentioned inside one of three files: requirements.txt, Pipfile, or pyproject.toml. While there are some other variations, this covers 99% of cases where Flask can be installed.

For example, here is the GitHub CLI command equivalent of the API call that we make to discover potential Java Spring APIs:

gh search code 'flask fork:true' --owner=vulnerable-apps --filename requirements.txt

This logic is carefully written to fly below the very low rate limits for the GitHub Code Search API.

We save this data locally in the ~/.swaggerfy folder, which is leveraged in later steps.

clone command

The clone command performs a git clone using multiprocessing for parallelized, rapid downloads of the code bases.

analyze command

The analyze command is a wrapper around the NightVision CLI’s swagger-extract command. It performs the swagger-extract command in parallel on the cloned GitHub repositories and generates Swagger files from static analysis. More information on NightVision’s Swagger Extraction feature can be found here. It outputs a JSON analysis file.

report command

The report command takes the JSON analysis file from the previous step and generates an HTML report file. The HTML report file is local-only, and none of the information is ever uploaded. If you want some support, feel free to share the file with us, but that is not mandatory to experiment with this tool.

FAQ

How is my code protected?

• The CLI runs on your machine - it never runs on NightVision servers.
• When you run swaggerfy clone, the code is cloned to your computer, and it is never uploaded.
• When you run swaggerfy analyze, the API specs are not uploaded to NightVision servers.
• When you run swaggerfy report, it generates a single, large, HTML report file. No data is uploaded to NightVision.

We will likely turn this experimental tool into an official part of our NightVision product. At that point, we will evaluate whether it makes sense to provide this as a GitHub app instead of having a fully offline solution like this.

Why do I have to install the GitHub CLI?

Swaggery queries the GitHub API for information. The easiest way to ensure that you can run this locally with little hassle is to install the GitHub CLI and log in. We felt this was easier than giving instructions on setting up a GitHub Personal Access Token.