# evsecrets

The purpose of the **evsecrets** npm library is to detect **secrets**
in your codebase, **before** you push your code to GitHub, where the 
secret values are defined in your **environment variables**.
**evsecrets** is a command-line interface (CLI) program.

Version 1.1.0 of this library added support for optional **.env** files,
such as used by the Python **python-dotenv** library and the Node.js
**dotenv** library.

Per the Twelve Factor App best practices, configuration should be stored in 
environment variables; see https://12factor.net/config.
Docker and containerized environments also commonly use environment variables.

---

## Global Installation

evsecrets can be installed globally on your system, using -g, as follows:

```
$ npm install -g evsecrets
```

Note: When installed globally, the envsecrets library can be used to scan
the codebase of **any** project on your system (i.e. - **Python, Java, C#, Node.js, etc.**).
Thus, this is the recommended installation approach.

## Configuration

See the **Configuration: .evsecrets.json** section below.

## CLI subcommands

These five subcommands are implemented:

| Subcommand | Function                                                                                |
| ---------- | --------------------------------------------------------------------------------------- |
| version    | Display the version of the evsecrets library (i.e. - '1.0.0')                           |
| init       | Create a .evsecrets.json file in the current directory.  Edit it as necessary           |
| secrets    | Display the pattern-matched environment variables and their values (i.e. - the secrets) |
| files      | Display the filtered list of files that will be scanned per your .evsecrets.json file   |
| scan       | Scan the filtered files list in your codebase for the identified secrets                |

The **scan** subcommand is the primary function. It will identify the files to be scanned,
then scan them for your secrets.  The other subcommands (i.e. - version, secrets, files)
are informational.

## Use 

The **npx** program within Node.js can be used to execute this library,
and the four subcommands, as follows:

```
$ npx -- evsecrets version
$ npx -- evsecrets init
$ npx -- evsecrets secrets
$ npx -- evsecrets files
$ npx -- evsecrets scan
```

You can also specify a specific version number with npx.

```
$ npx -- evsecrets@1.1.0 scan
```

---

### Project Installation 

Alternatively, you can install the library locally in your Node.js project.
See the **console_app** directory in this repo for a demonstration of this.

```
$ npm install evsecrets
```

Add a script alias command in package.json:

```
    "scripts": {
        ...
        "secrets": "./node_modules/evsecrets/dist/index.js"
    },
```

Then, execute the alias command:

```
$ npm run secrets version
$ npm run secrets init
$ npm run secrets secrets
$ npm run secrets files
$ npm run secrets scan
```

### Optional CLI flag arguments 

The **--verbose** command-line arg can be used to produce additional output
for your understanding of exactly which files are included and excluded
in the scanning process.

The **--tmp-file-outputs** command-line arg can be used to write json
files to the **tmp** directory within the current directory.
This json files contain a list all files found, and the files that are
included for scanning.

For example:

```
$ npm run scan -- --tmp-file-outputs

> evsecrets@1.1.0 scan
> node ./dist/index.js scan --tmp-file-outputs

file written: tmp/evsecrets-walkFs.json
file written: tmp/evsecrets-filteredFilenamesList.json

--- 1
WARNING: Secret found at line 3 of file /Users/cjoakim/github/evsecrets-js/.env
content: AZURE_COSMOSDB_EMULATOR_URI=https://localhost:8081/
--- 2
WARNING: Secret found at line 4 of file /Users/cjoakim/github/evsecrets-js/.env
content: KAGGLE_KEY=dd64Wup8RwYrNCReZQPB
--- 3
WARNING: Secret found at line 6 of file /Users/cjoakim/github/evsecrets-js/.env
content: SOME_DOUBLE_QUOTED_API_KEY="Tdvs4352oeSe6o6ULU7Umb3pZQ6u3RqDQ"
--- 4
WARNING: Secret found at line 7 of file /Users/cjoakim/github/evsecrets-js/.env
content: SOME_SINGLE_QUOTED_API_KEY= "Tdvs4352oeSe6o6ULU7Umb3pZQ6u3RqSQ"
--- 5
WARNING: Secret found at line 123 of file /Users/cjoakim/github/evsecrets-js/README.md
content: content: # Secret value => C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==

...
```

Verbose scanning, with the **--verbose** flag, gives you additional output,
such as explicitly listing each file along with a boolean indicating if it
was included in the scanning process.

```
$ npm run scan -- --verbose


includeThisFile: /Users/cjoakim/github/evsecrets-js/tmp/evsecrets-filteredFilenamesList.json --> false
includeThisFile: /Users/cjoakim/github/evsecrets-js/tmp/evsecrets-walkFs.json --> false
...
```

## Configuration: .evsecrets.json

In the root directory of your project, optionally create a file named
**.evsecrets.json** that looks like the following JSON.

This file, with default values, can be created for you when you execute
the 'init' subcommand as described above.

Edit the values within **env_var_patterns**, **exclude_file_patterns** and
**exclude_file_suffixes** per your needs.  The values shown below are
the **defaults** implemented in the program.

These determine what environment variables to obtain your secrets from,
the files to be excluded from scanning, and the filetypes to be excluded
from scanning, respectively.

```
{
    "env_var_patterns": [
        "_KEY",
        "_URI",
        "_URL",
        "CONN_STR",
        "CONNECTION_STR",
        "CONNECTION_STRING"
    ],
    "exclude_file_patterns": [
        "__MACOSX/",
        "__pycache__/",
        ".code-workspace",
        ".git/",
        ".git/",
        ".gradle/",
        ".idea/",
        ".vscode/",
        "bin/",
        "build/",
        "htmlcov/",
        "man/",
        "node_modules/",
        "obj/",
        "opt/",
        "tmp/",
        "venv/"
    ],
    "exclude_file_suffixes": [
        ".acc",
        ".avi",
        ".bmp",
        ".class",
        ".dll",
        ".doc",
        ".docx",
        ".DS_Store",
        ".exe",
        ".gif",
        ".jar",
        ".jpeg",
        ".jpg",
        ".mov",
        ".mp3",
        ".mp4",
        ".pdf",
        ".png",
        ".ppt",
        ".pptx",
        ".pyc",
        ".so",
        ".tar",
        ".tgz",
        ".tiff",
        ".wav",
        ".xls",
        ".xlsx",
        ".vscode",
        ".zip"
    ]
}
```

## Example

Assuming the above **.evsecrets.json** configuration file, 
and the following environment variable in your system:

```
SOME_SECRET_KEY=C2y6yDjf5
```

Then the value 'C2y6yDjf5' will be identified if it exists in your codebase
during a **scan**.

### .env files

The evsecrets library will read, parse, and use the values in your optional
**.env** file.  Some programming language ecosystems support this alternative
way to define environment variables, such as the Python **python-dotenv** library
and the Node.js **dotenv** library.

The GitHub repo for the evsecrets library contains the an example .env file,
named **example_dot_env**, which is shown below.

Your secrets can thus be defined in either the actual environment variables
and/or your .env file.  If your environment and .env file contain different
values for a given environment variable name, then BOTH values will be scanned.

As shown in this example, several formats of quoted and unquoted values are supported.

```
AZURE_COSMOSDB_EMULATOR_URI=https://localhost:8081/
KAGGLE_KEY=dd64Wup8RwYrNCReZQPB
KAGGLE_USERNAME=   Miles   
SOME_DOUBLE_QUOTED_API_KEY="Tdvs4352oeSe6o6ULU7Umb3pZQ6u3RqDQ"
SOME_SINGLE_QUOTED_API_KEY= "Tdvs4352oeSe6o6ULU7Umb3pZQ6u3RqSQ" 
```

---

## Version History

| Version |    Date    | Changes                                                             |
| ------- | ---------- | ------------------------------------------------------------------- |
|  1.1.0  | 2025/05/02 | Added support for optional .env files                               |
|  1.0.0  | 2025/04/28 | Dotfile '.evsecrets.json' replaces 'evsecrets.json'                 |
|  0.9.0  | 2025/04/15 | Logging each file with the 'files' command instead of a json array  |
|  0.8.0  | 2025/04/15 | Initialization handling for absent config file                      |
|  0.7.0  | 2025/04/15 | Updated GitHub URL, updated installation.  Added init subcommand    |
|  0.6.0  | 2025/04/14 | Ported to the @nodelib/fs.walk library, made codebase synchronous   |
|  0.5.0  | 2025/04/13 | Added 'secrets' subcommand, removed 'patterns'                      |
|  0.4.0  | 2025/04/13 | npx usage and -g installation                                       |
|  0.3.0  | 2025/04/13 | Simplified bin command, added version CLI function                  |
|  0.2.0  | 2025/04/13 | Sample console_app                                                  |
|  0.1.0  | 2025/04/13 | Initial release                                                     |

## Common Errors

### Permission denied

```
./node_modules/evsecrets/dist/index.js: Permission denied
```

In this case, on Linux and macOS, make the file executable with this command:

```
$ chmod 744 ./node_modules/evsecrets/dist/index.js
```