Other than reputation analysis by Snyk, I honestly don’t know how to objectively analyze one repo/package versus another.
Obviously I can make some basic heuristics, but I can’t reasonably evaluate all of the components of trust for every library, package, container, framework, repo even at a regular interval, let alone fast enough to just maintain patch levels (nevermind being reasonably productive).
I actually considered first steps in making a business out of this idea, but I’m convinced that every developer overestimates their ability to identify untrustworthy repos/packages and companies aren’t willing to pay the actual cost (with either subscription dollars or in the friction it would add to reject almost all 3rd party code because it doesn’t meet high standards of quality and security in a transparent and verifiable way).
>Obviously I can make some basic heuristics, but I can’t reasonably evaluate all of the components of trust for every library, package, container, framework, repo even at a regular interval, let alone fast enough to just maintain patch levels (nevermind being reasonably productive).
Related: have you tried throwing a file (or a hash) at VirusTotal lately? If it's executable, they'll run it in a sandbox and give you a forensic report of everything it touched and did.
I'm so suspicious of software I can't at least review the source code of that originates outside of trusted channels that I probably wouldn't run anything meeting that description that I couldn't compile myself if it weren't for that (and similar) tools.
Eg. How do I know with any certainty that a library passes all OWASP best practices?
Or is well documented?
Or is maintained?
Or is responsive to security reports?
[… and dozens of other similar properties]
Obviously I can make some basic heuristics, but I can’t reasonably evaluate all of the components of trust for every library, package, container, framework, repo even at a regular interval, let alone fast enough to just maintain patch levels (nevermind being reasonably productive).
I actually considered first steps in making a business out of this idea, but I’m convinced that every developer overestimates their ability to identify untrustworthy repos/packages and companies aren’t willing to pay the actual cost (with either subscription dollars or in the friction it would add to reject almost all 3rd party code because it doesn’t meet high standards of quality and security in a transparent and verifiable way).