Allstar

[!IMPORTANT] The OpenSSF-hosted Allstar GitHub App has been retired. Allstar, the OpenSSF Scorecard subproject, itself continues to be maintained — you must now run it yourself, either as a GitHub Action or as a service daemon.

See ossf/allstar#881 for more details.

If your organization was relying on the hosted app, see Migrating off the hosted app.

Overview

• What Is Allstar?

What's new with Allstar

• whats-new.md

Disabling Unwanted Issues

• Help! I'm getting issues created by Allstar and I don't want them!

Getting Started

• Background
• Org-Level Options
• Installation Options
  • Create your GitHub App
  • Create your .allstar control repository
  • Running Allstar as a GitHub Action
  • Running Allstar as a service daemon
• Migrating off the hosted app

Policies and Actions

• Actions
• Policies

Advanced

• Configuration Definitions
• Example Configurations
• Run Your Own Instance of Allstar

Contribute

• Contributing

Overview

What is Allstar?

Allstar is a GitHub App that continuously monitors GitHub organizations or repositories for adherence to security best practices. If Allstar detects a security policy violation, it creates an issue to alert the repository or organization owner. For some security policies, Allstar can also automatically change the project setting that caused the violation, reverting it to the expected state.

Allstar’s goal is to give you finely tuned control over the files and settings that affect the security of your projects. You can choose which security policies to monitor at both the organization and repository level, and how to handle policy violations. You can also develop or contribute new policies.

Allstar is developed as a part of the OpenSSF Scorecard project.

What's new with Allstar

Disabling Unwanted Issues

If you're getting unwanted issues created by Allstar, follow these directions to opt out.

Getting Started

Background

Allstar is highly configurable. There are three main levels of controls:

• Org level: Organization administrators can choose to enable Allstar on:
  • all repositories in the org;
  • most repositories, except some that are opted out;
  • just a few repositories that are opted in.

These configurations are done in the organization's .allstar repository.

• Repo level: Repository maintainers in an organization that uses Allstar can choose to opt their repository in or out of organization-level enforcements. Note: these repo-level controls are only functional when "repo override" is allowed in the org-level settings. These configurations are done in the repository's .allstar directory.

• Policy level: Administrators or maintainers can choose which policies are enabled on specific repos and which actions Allstar takes when a policy is violated. These configurations are done in a policy yaml file in either the organization's .allstar repository (admins), or the repository's .allstar directory (maintainers).

Org-Level Options

Before installing Allstar at the org level, you should decide approximately how many repositories you want Allstar to run on. This will help you choose between the Opt-In and Opt-Out strategies.

• The Opt In strategy allows you to manually add the repositories you'd like Allstar to run on. If you do not specify any repositories, Allstar will not run despite being installed. Choose the Opt In strategy if you want to enforce policies on only a small number of your total repositories, or want to try out Allstar on a single repository before enabling it on more. Since the v4.3 release, globs are supported to easily add multiple repositories with a similar name.

• The Opt Out strategy (recommended) enables Allstar on all repositories and allows you to manually select the repositories to opt out of Allstar enforcements. You can also choose to opt out all public repos, or all private repos. Choose this option if you want to run Allstar on all repositories in an organization, or want to opt out only a small number of repositories or specific type (i.e., public vs. private) of repository. Since the v4.3 release, globs are supported to easily add multiple repositories with a similar name.

Installation Options

Allstar acts on your organization as a GitHub App: you create the app, and you run the process that authenticates as it. Setup is therefore two steps that are common to every deployment — create the app and create the control repository — and then a choice of how to run it:

The Action is the lower-overhead of the two and is where most organizations should start; you can move to a daemon later without changing any policy configuration.

Create your GitHub App

An App is a user-like identity with a set of permissions in your organization. Allstar needs read access to most settings and file contents in order to detect compliance, and write access to issues and checks in order to file issues and support the block action.

Follow Operator instructions - Create a GitHub App, and record the App ID and private key. Both run modes need them.

Create your .allstar control repository

Allstar reads its configuration from a repository named .allstar in your organization.

The fastest way to create one is from the sample:

1. Open the sample repository and click the "Use this template" button
2. In the field for Repository Name, type .allstar
3. Click "Create repository from template"

This enables all current Allstar policies on all repositories using the Opt Out strategy, with the issue action. You can change any of it later.

For granular control from the start — choosing the Opt In or Opt Out strategy and writing individual policy files yourself — follow the manual installation directions instead.

Running Allstar as a GitHub Action

This option runs Allstar as a scheduled job using GitHub Actions, so there is no infrastructure to operate beyond GitHub itself.

Follow the GitHub Actions installation directions to set up a recurring Action in your .allstar repository, harden it, and monitor its results.

Running Allstar as a service daemon

This option runs Allstar as a persistent process, which detects and resolves violations continuously rather than on a schedule.

See Operator instructions for running the process, managing secrets, sizing, and the available environment variables.

Migrating off the hosted app

If your organization used the OpenSSF-hosted app, your configuration carries over as-is. The .allstar control repository, allstar.yaml, and every policy file keep working unchanged; what you are replacing is only the process that reads them.

To migrate:

1. Create your own GitHub App and install it on your organization with the same repository access the hosted app had.
2. Keep your existing .allstar repository exactly as it is.
3. Run Allstar as an Action or as a daemon.
4. Uninstall allstar-app from your organization, if it still appears under Settings -> GitHub Apps.

Issues previously filed by the hosted app remain in your repositories. Your own instance identifies its issues by the same allstar label (or your configured issueLabel), so it will adopt and close them as violations are resolved, rather than filing duplicates.

Policies and Actions

Actions

Each policy can be configured with an action that Allstar will take when it detects a repository to be out of compliance.

• log: This is the default action, and actually takes place for all actions. All policy run results and details are logged. Logs are currently only visible to the app operator, plans to expose these are under discussion.
• issue: This action creates a GitHub issue. Only one issue is created per policy, and the text describes the details of the policy violation. If the issue is already open, it is pinged with a comment every 24 hours without updates (not currently user configurable). If the policy result changes, a new comment will be left on the issue and linked in the issue body. Once the violation is addressed, the issue will be automatically closed by Allstar within 5-10 minutes.
• fix: This action is policy specific. The policy will make the changes to the GitHub settings to correct the policy violation. Not all policies will be able to support this (see below).

Proposed, but not yet implemented actions. Definitions will be added in the future.

• block: Allstar can set a GitHub Status Check and block any PR in the repository from being merged if the check fails.
• email: Allstar would send an email to the repository administrator(s).
• rpc: Allstar would send an rpc to some organization-specific system.

Action configuration

Two settings are available to configure the issue action:

• issueLabel is available at the organization and repository level. Setting it will override the default allstar label used by Allstar to identify its issues.

• issueRepo is available at the organization level. Setting it will force all issues created in the organization to be created in the repository specified.

Policies

Similar to the Allstar app enable configuration, all policies are enabled and configured with a yaml file in either the organization's .allstar repository, or the repository's .allstar directory. As with the app, policies are opt-in by default, also the default log action won't produce visible results. A simple way to enable all policies is to create a yaml file for each policy with the contents:

optConfig:
  optOutStrategy: true
action: issue

The details of how the fix action works for each policy is detailed below. If omitted below, the fix action is not applicable.

Branch Protection

This policy's config file is named branch_protection.yaml, and the config definitions are here.

The branch protection policy checks that GitHub's branch protection settings are setup correctly according to the specified configuration. The issue text will describe which setting is incorrect. See GitHub's documentation for correcting settings.

The fix action will change the branch protection settings to be in compliance with the specified policy configuration.

Binary Artifacts

This policy's config file is named binary_artifacts.yaml, and the config definitions are here.

This policy incorporates the check from scorecard. Remove the binary artifact from the repository to achieve compliance. As the scorecard results can be verbose, you may need to run scorecard itself to see all the detailed information.

CODEOWNERS

This policy's config file is named codeowners.yaml, and the config definitions are here.

This policy checks for the presence of a CODEOWNERS file on your repositories.

Outside Collaborators

This policy's config file is named outside.yaml, and the config definitions are here.

This policy checks if any Outside Collaborators have either administrator(default) or push(optional) access to the repository. Only organization members should have this access, as otherwise untrusted members can change admin level settings and commit malicious code.

SECURITY.md

This policy's config file is named security.yaml, and the config definitions are here.

This policy checks that the repository has a security policy file in SECURITY.md and that it is not empty. The created issue will have a link to the GitHub tab that helps you commit a security policy to your repository.

Dangerous Workflow

This policy's config file is named dangerous_workflow.yaml, and the config definitions are here.

This policy will run against all branches, see rationale here.

This policy checks the GitHub Actions workflow configuration files (.github/workflows), for any patterns that match known dangerous behavior. See the OpenSSF Scorecard documentation for more information on this check.

Generic Scorecard Check

This policy's config file is named scorecard.yaml, and the config definitions are here.

This policy runs any scorecard check listed in the checks configuration. All checks run must have a score equal or above the threshold setting. Please see the OpenSSF Scorecard documentation for more information on each check.

SARIF Upload

The Scorecard policy can optionally upload results as SARIF to each repository's Security > Code Scanning tab. This gives organization administrators visibility into Scorecard findings alongside other security tools (CodeQL, Dependabot, etc.) without requiring per-repository workflow setup.

To enable SARIF upload, add the upload field to your scorecard.yaml:

optConfig:
  optOutStrategy: true
action: issue
checks:
  - Binary-Artifacts
  - Signed-Releases
threshold: 8
upload:
  sarif: true

Requirements:

• The Allstar GitHub App must have the Code scanning alerts repository permission set to Read & write (API scope: security_events). This is not among the permissions Allstar otherwise needs, so add it to your app before enabling SARIF upload.
• SARIF upload is non-blocking: if the upload fails (e.g., due to missing permissions), the policy check continues normally.
• Change detection compares the repository HEAD commit SHA and skips the scan and upload when the repo has not been pushed to since the last upload.

SARIF upload works with both ways of running Allstar: as a service daemon or as a GitHub Action.

GitHub Actions

This policy's config file is named actions.yaml, and the config definitions are here.

This policy checks the GitHub Actions workflow configuration files (.github/workflows) (and workflow runs in some cases) in each repo to ensure they are in line with rules (eg. require, deny) defined in the organization-level config for the policy.

Repository Administrators

This policy's config file is named admin.yaml, and the config definitions are here.

This policy checks that by default all repositories must have a user or group assigned as an Administrator. It allows you to optionally configure if users are allowed to be administrators (as opposed to teams).

Future Policies

• Ensure dependabot is enabled.
• Check that dependencies are pinned/frozen.

Example Config Repository

See this repo as an example of Allstar config being used. As the organization administrator, consider a README.md with some information on how Allstar is being used in your organization.

Advanced

Configuration Definitions

• Organization level enable configuration
• Repository Override enable configuration

Secondary Org-Level configuration location

By default, org-level configuration files, such as the allstar.yaml file above, are expected to be in a .allstar repository. If this repository does not exist, then the .github repository allstar directory is used as a secondary location. To clarify, for allstar.yaml:

This is also true for the org-level configuration files for the individual policies, as described below.

Repo policy configurations in the Org Repo

Allstar will also look for repo-level policy configurations in the organization's .allstar repository, under the directory with the same name as the repository. This configuration is used regardless of whether "repo override" is disabled.

For example, Allstar will lookup the policy configuration for a given repo myapp in the following order:

Org-level Base and Merge Configuration Location

For org-level Allstar and policy configuration files, you may specify the field baseConfig to specify another repository that contains base Allstar configuration. This is best explained with an example.

Suppose you have multiple GitHub organizations, but want to maintain a single Allstar configuration. Your main organization is "acme", and the repository acme/.allstar contains allstar.yaml:

optConfig:
  optOutStrategy: true
issueLabel: allstar-acme
issueFooter: Issue created by Acme security team.

You also have a satellite GitHub organization named "acme-sat". You want to re-use the main config, but apply some changes on top by disabling Allstar on certain repositories. The repository acme-sat/.allstar contains allstar.yaml:

baseConfig: acme/.allstar
optConfig:
  optOutRepos:
  - acmesat-one
  - acmesat-two

This will use all the config from acme/.allstar as the base config, but then apply any changes in the current file on top of the base configuration. The method this is applied is described as a JSON Merge Patch. The baseConfig must be a GitHub <org>/<repository>.

Contributing

See CONTRIBUTING.md