Every few years a piece of default software quietly wears out its welcome. Your password manager gets acquired and starts nagging you to upgrade. Your note app moves to a subscription. Your image editor phones home more than it edits. So you go looking for an open-source replacement, and within an hour you have four candidates, three GitHub repos, and zero idea whether any of them will steal your data or crash on save.
Here is the uncomfortable fact: "open source" tells you the code could be audited, not that anyone actually did. A 2023 review of package registries found that a meaningful slice of popular repositories had no security review in the previous 12 months, and typosquatted malicious packages regularly slip into npm and PyPI for days before takedown. Open code is a permission, not a guarantee.
This guide walks through exactly how to vet open source apps before you trust them with your files, your logins, or your customers. You will get a repeatable checklist, a worked scoring example, a comparison table of vetting signals, and a step-by-step audit you can run in about 30 minutes per app.
Key Takeaways
- Health beats hype. Commit recency, contributor count, and open-issue response time predict long-term safety better than star counts.
- Read the permissions, not the pitch. Match what the app requests (network, filesystem, telemetry) against what it actually needs to do its job.
- Verify the download chain. Confirm checksums and signatures so the binary you install matches the source you reviewed.
- Score, don't vibe. Use a simple weighted rubric so you compare candidates on the same criteria instead of gut feel.
- Plan the exit first. Confirm you can export your data in an open format before you migrate into anything.
What "vetting an open source app" actually means
Vetting is the process of checking that an application is safe, maintained, and honest about what it does before you rely on it. It is not the same as reading every line of source code. Almost nobody does that, and you don't have the time.
Instead, vetting is a layered risk assessment. You look at signals that correlate with trustworthiness: who maintains the project, how active it is, what permissions it demands, how cleanly it handles your data, and whether the thing you download matches the thing you reviewed.
Think of it like hiring. You don't interrogate a candidate's entire life. You check references, look for red flags, and confirm they can do the specific job. Open source vetting works the same way, and the higher the stakes, the deeper you dig. A throwaway CLI tool deserves five minutes. A password manager or a plugin running on your production site deserves an afternoon.
The 7 signals that separate safe apps from risky ones
When I evaluate an unfamiliar project, I run through the same seven signals in order. Each one is cheap to check and eliminates a chunk of candidates fast.
1. Maintenance activity
- Last commit date. Anything untouched for 12+ months is a liability, especially if it handles network requests.
- Release cadence. Look for tagged releases, not just a moving
mainbranch. Regular releases signal discipline. - Issue response time. Open a few issues and sort by recent. Do maintainers reply within days, or do bug reports sit for months?
2. Contributor breadth
A single-maintainer project is one burnout or one acquisition away from abandonment. Projects with 5+ regular contributors survive maintainer turnover. Check the "Contributors" tab and see whether commits come from multiple people or one heroic solo dev.
3. Security posture
- Is there a
SECURITY.mdwith a disclosure process? - Are there past CVEs, and were they patched quickly?
- Does the repo run automated scans (Dependabot, CodeQL, Snyk) in CI?
A project that has had vulnerabilities and fixed them fast is often safer than one that claims to have none. If you're evaluating plugins specifically, our guide on how to detect a vulnerable WordPress plugin before you install it goes deeper on CVE lookups.
4. Permission footprint
This is where most people stop paying attention, and it's the most important signal. A Markdown editor that requests your full filesystem and outbound network access is asking for more than it needs. Match requested permissions against the job.
5. Dependency hygiene
A clean app with 400 transitive dependencies inherits the risk of all 400. Check the dependency tree size and whether packages come from trustworthy sources. Our walkthrough on how to verify a package manager's repository isn't serving malware covers the supply-chain angle in detail.
6. Documentation and license
Real docs mean real users. A permissive, clearly stated license (MIT, Apache 2.0, GPL) means you know your obligations. A missing or custom "source available but not really open" license is a warning sign.
7. Download integrity
The binary you install should match the source you reviewed. That means published checksums (SHA-256) and, ideally, signed releases. If none exist, you are trusting a stranger's build server.
A worked example: scoring three note-taking apps
Abstract advice is easy to nod along to and hard to apply. So let's run a real scenario. Say your team of 8 uses a proprietary note app that just moved to a $10-per-user monthly plan. That's $960 a year. You want to replace it with an open-source option, and you've narrowed it to three candidates. We'll call them NoteA, NoteB, and NoteC.
I score each candidate 0–5 on six criteria, then weight the two that matter most for a tool holding company notes: maintenance and permissions.
| Criteria (weight) | NoteA | NoteB | NoteC |
|---|---|---|---|
| Maintenance activity (×2) | 5 | 3 | 1 |
| Contributor breadth (×1) | 4 | 2 | 1 |
| Permission footprint (×2) | 5 | 4 | 2 |
| Security posture (×1) | 4 | 3 | 0 |
| Docs & license (×1) | 5 | 4 | 3 |
| Export/exit options (×1) | 5 | 3 | 4 |
| Weighted total (max 40) | 38 | 26 | 13 |
The math does the arguing for you. NoteA scores 38 of a possible 40. NoteC looks tempting because it has a slick UI, but a maintenance score of 1 (last commit 14 months ago) and a security score of 0 (no SECURITY.md, one unpatched CVE) knock it out. That's a project one exploit away from becoming your incident.
The whole exercise took about 90 minutes for three apps. Compared to a botched migration or a data breach, that's the cheapest insurance you'll ever buy.
Signal-by-signal: what to check and where
Here's how the vetting signals map to concrete checks and the red flags that should stop you.
| Signal | Where to look | Green flag | Red flag |
|---|---|---|---|
| Maintenance | Repo commit history, releases | Commit in last 60 days | No activity for 12+ months |
| Contributors | Contributors tab | 5+ regular committers | Single maintainer, no backups |
| Security | SECURITY.md, CVE databases, CI | Fast past patches, active scans | Ignored reports, no disclosure path |
| Permissions | Manifest, install prompts, network logs | Requests match function | Broad access with no explanation |
| Integrity | Release page, signing keys | Published SHA-256 + signatures | Bare binary, no checksum |
A 30-minute vetting walkthrough you can follow now
Here's the exact process I run before letting a new app replace a default one. It works for desktop utilities, browser extensions, WordPress plugins, and CLI tools alike.
- Read the README and the license first (3 min). Confirm the project does what you think, and that the license permits your use. If the license is "custom" or missing, stop here.
- Check the pulse (5 min). Open the commit history, the releases page, and the issues tab. Note the last commit date, the number of open vs. closed issues, and how recently a maintainer replied to anything. Abandoned projects reveal themselves in minutes.
- Scan the security tab (4 min). Look for
SECURITY.md, past advisories, and whether Dependabot or CodeQL runs in CI. Search the project name plus "CVE" and "vulnerability" in a search engine. - Inspect the permission footprint (5 min). For a browser extension, read the requested permissions in the manifest. For a desktop app, note what filesystem and network access it wants. Our guide on how to audit browser extensions for ransomware before installing shows exactly what to look for.
- Watch its network behavior (5 min). Install it in a sandbox or VM, run a network monitor, and use the app normally. A local note editor calling out to an analytics endpoint on every keystroke is a finding. If your concern is credential exposure specifically, cross-reference our piece on how to spot an AI browser that leaks your login credentials.
- Verify the download (3 min). Compare the SHA-256 of your download against the published checksum. On Windows:
certutil -hashfile app.exe SH
