How to Vet Open Source Software for Production

Somewhere in your production stack right now, there is a piece of code you have never read, written by a person you have never met, updated by a maintainer who may have burned out two years ago. That is not a criticism. It is just how modern software works. The average Node.js application pulls in over 1,000 transitive dependencies, and a typical enterprise codebase is roughly 70 to 90 percent open source by volume. You are not building software so much as assembling it.

Here is the uncomfortable stat: Sonatype's State of the Software Supply Chain report has repeatedly found that the majority of open source components with known vulnerabilities are downloaded even when a patched version exists. In other words, teams keep pulling in broken libraries not because they cannot fix them, but because nobody looked. The supply chain attacks that made headlines, from event-stream to the XZ Utils backdoor, all shared one root cause: trust granted without verification.

This article walks through exactly how to vet open source software before it touches production. You will get a repeatable checklist, a worked scoring example with real numbers, a comparison of the tools that automate this work, and a step-by-step process you can run on any dependency in about 20 minutes. No hand-waving, no "just be careful." Actual signals you can measure.

Key Takeaways

Health beats popularity. A high GitHub star count tells you almost nothing about whether a project is maintained, funded, or safe.

Score, don't guess. Use a weighted rubric across maintenance, security, license, and dependency risk so decisions are consistent across your team.

Read the last 90 days. Recent commit activity, open-vs-closed issue ratios, and time-to-patch on CVEs predict future reliability better than lifetime totals.

Audit the transitive tree, not just the top-level package. Most real risk hides three or four levels deep.

Pin, lock, and monitor. Vetting is not a one-time event. Lock versions and set up alerts so a clean package staying clean is verified, not assumed.

Commercial alternatives are a valid answer. When a critical component has no maintainer, a supported product with an SLA is often cheaper than the eventual incident.

What Does It Mean to Vet Open Source Software?

To vet open source software is to evaluate a component across four dimensions before you depend on it: its maintenance health, its security posture, its legal license terms, and its dependency footprint. Vetting answers one question: if this thing breaks or gets compromised, how badly does it hurt, and how likely is that?

The mistake most teams make is treating "it's on GitHub with 40,000 stars" as the whole evaluation. Stars measure popularity at some point in the past. They do not measure whether the maintainer still responds to security reports, whether the license is compatible with your product, or whether the package quietly pulls in 200 sub-dependencies you have never audited.

Good vetting is boring and systematic. It produces a score, a decision, and a paper trail. That is exactly what you want when a security auditor or a customer asks why you trusted a given library.

The Four Pillars

Maintenance: Is someone actively fixing bugs and merging patches?
Security: Are there known vulnerabilities, and how fast does the project respond to new ones?
Licensing: Can you legally use it the way you intend, including redistribution?
Dependencies: What does it drag in behind it, and are those clean too?

The Signals That Actually Predict Risk

Not all signals carry equal weight. After reviewing hundreds of dependencies over the years, these are the ones I trust, roughly in order of predictive value.

1. Recent maintenance activity

Look at commits in the last 90 days, not the last five years. A project with 8,000 lifetime commits but nothing since 2022 is functionally abandoned. Check the commit graph on the repo and the date of the latest release.

2. Issue and PR responsiveness

Open the Issues tab and sort by newest. Are maintainers replying within days or weeks, or are there 400 open issues with no response since last year? A healthy ratio is when closed issues significantly outnumber open ones and recent issues get triaged.

3. Security response history

Search the repo for a SECURITY.md file and check whether past CVEs were patched quickly. A project that shipped a fix within 7 days of a disclosed vulnerability is showing you exactly how it will behave next time.

4. Bus factor

If one person authored 95 percent of commits, the project has a bus factor of one. That single maintainer getting bored, burning out, or being targeted is a real supply chain risk. The XZ backdoor happened because a lone, overworked maintainer accepted "help" from a malicious contributor.

5. License clarity

MIT, Apache 2.0, and BSD are permissive and safe for commercial use. GPL and AGPL carry copyleft obligations that can force you to open source your own code. "No license" means all rights reserved, which is often riskier than a restrictive license.

6. Dependency depth

Run npm ls --all or pip show and count the transitive tree. A package that pulls in 3 clean dependencies is a very different bet from one that pulls in 180.

A Worked Example: Scoring Two Candidate Libraries

Say you need a date-parsing library for a Node.js service that handles payment scheduling. You have narrowed it down to two candidates. Instead of picking by gut feel, you score each on a 0 to 5 scale across six weighted criteria.

Here is the rubric applied to a hypothetical Library A (a mature, widely used package) versus Library B (a newer, trendier one with more GitHub stars this month).

Criterion	Weight	Library A	Library B
Recent maintenance (last 90 days)	25%	5	4
Security response history	25%	5	2
License compatibility (MIT/Apache)	15%	5	5
Dependency footprint	15%	4	2
Bus factor / contributor spread	10%	4	1
Documentation quality	10%	4	5

Now do the weighted math:

Library A: (5×.25) + (5×.25) + (5×.15) + (4×.15) + (4×.10) + (4×.10) = 1.25 + 1.25 + 0.75 + 0.60 + 0.40 + 0.40 = 4.65 / 5
Library B: (4×.25) + (2×.25) + (5×.15) + (2×.15) + (1×.10) + (5×.10) = 1.00 + 0.50 + 0.75 + 0.30 + 0.10 + 0.50 = 3.15 / 5

Library B has prettier docs and more buzz, but its weak security history, heavy dependency tree, and single-maintainer risk drag it below a threshold most teams would set at 4.0 for production use. The rubric caught what the star count hid. That is the entire point of scoring: it makes the invisible tradeoffs visible.

A Step-by-Step Vetting Walkthrough

Here is the exact process I run on any new dependency. Budget about 20 minutes per package for a first pass. You can do this without any paid tooling.

Find the canonical repository. Do not trust the package registry page alone. Click through to the actual source repo and confirm the org and author match the project you think you are installing. Typosquatting (e.g. crossenv vs cross-env) is a common attack.
Check the last release date and commit graph. Anything with no release in 18+ months goes on a watch list. No commits in 12+ months and it needs a very good reason to enter production.
Read the issues, newest first. Look for unresolved security concerns, complaints about broken builds, or maintainers who have gone silent. Note the open-to-closed ratio.
Run a vulnerability scan. Use npm audit, pip-audit, or an OSV lookup. For a deeper check, osv-scanner queries the Open Source Vulnerabilities database directly against your lockfile.
Inspect the dependency tree. Run npm ls --all or deptree equivalents. Anything surprising in the transitive list gets its own quick pass. This is where event-stream-style attacks live.
Verify the license. Confirm the license file exists, matches what the registry claims, and is compatible with your distribution model. If you ship a commercial product, flag any GPL/AGPL immediately.
Check the OpenSSF Scorecard. The scorecard tool from the Open Source Security Foundation automatically grades a repo on branch protection, signed releases, dependency update automation, and more. A score above 7 is a good sign.
Look for signs of a supply chain identity problem. Newly transferred ownership, a maintainer email domain that changed recently, or a package that suddenly gained a new "helpful" co-maintainer all deserve scrutiny.
Pin the version and record your decision. Lock the exact version in your lockfile, write a one-line note in an ADR or a spreadsheet with the score and date, and move on.

That last step matters more than it looks. A vetting process nobody records is a process you will repeat from scratch every quarter. If you manage many small internal scripts and snippets, keeping those decisions somewhere searchable, even a self-hosted paste tool like LionPaste, saves real time when an audit lands on your desk.

Tools That Automate the Grunt Work

You cannot manually vet 1,000 transitive dependencies. Automation handles the volume; your judgment handles the edge cases. Here is how the main options compare.

Cover image: Software value feedback loop by jakuza, licensed under BY-SA 2.0 via Openverse.