We all know how common terminal one liners have became as a installation method on GNU/Linux and what are the issues with it but let's recap quickly.
You go to a pager of some project and it tells you to do curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs/ | sh or curl -fsSL https://deno.land/install.sh | sh. The only way to verify that this command will not delete all your files or install malware is to manually review the entire script.
So... why not create a secure script repository? On a central website you would create an account for a project and submit a script. On the other side we would provide a binary client that will download and execute the script (we can call it grunt from get and run it). So as a user you would run for example grunt rustup and it would get and execute the script created by rustup project. I imagine it shouldn't be that difficult to add a tiny package to the major distros.
I believe this would be a fairly simple project that would solve all the security issues typical terminal one liners have.
On the website for uploading scripts we could introduce:
- multi user approval flow for script updates
- 2FA
- static checks of the scripts
- reporting system for compromised scripts
verified projectstatus
On the client side we could:
- provide info about this script's security (how many people reviewed it, when was it last updated, is the project verified)
- provide info about downloads (how many time was this script downloaded since the last update)
- do additional checks (maybe the project could provide MD5 of the script on their servers and grunt could verify it?)
So it would look something like this:
# grunt rustp
Downloading rustp.sh from https://getandrun.it/...
Last updated 30 days ago.
Downloads since last update: 5
Verified project: No
Reviewed by 1 user
Execute script [y/N]
Clearly something is wrong...
# grunt rustup
Downloading rustup.sh from https://getandrun.it/...
Last updated 60 days ago.
Downloads since last update: 5342
Verified project: Yes
Reviewed by 3 users
Comparing MD5 checksum with https://rustup.rs/grunt_md5... Passed
Execute script [y/N]
That's better!
Right? So why don't we have something like this? Or we do and it simply didn't get enough traction?
========
So just to address some of the comments. No, it's not a package manager. Package managers are complex tools that handle versioning, dependencies, updates, uninstalls and so on. Package mangers are also distro specific. A lot of devs decide not to use package managers and use bash scripts that are distro agnostic and don't rely on external maintainers and packagers. It would be ideal if everyone used secure package managers but the reality is they don't. This solution is a compromise that offers devs full control of software distribution while introducing decent security.
=======
Someone suggested brew. How do you install brew according to https://brew.sh/ ?
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
See the problem?
I mentioned an alternative to the what these websites do, using a package manager to install these instead of their bash scripts.
Both of the bash scripts you mentioned as an example are being used to install software. If you have examples of bash scripts that do things other than install software, then it's worth discussing how to handle those.
However, the reason why bash is so popular for usecases like configuration scripts or an Arch install script though, is because no other software besides wget/curl and bash is required to get it. Having to get an extra tool on the Arch install iso just to run an install script in bash, or to run a script that installs tools on a fresh, clean install, somewhat defeats the point of the script being written in bash imo.
Bash is inherently insecure. I consider security not just issues with malice, but also footguns like the steam issues mentioned above. Centralizing all the bash scripts to a "repo" doesn't fix the issues with arbitrary bash scripts.
And if you are concerned about malice, then the bash scripts almost always download a binary that does further arbitrary code execution and cannot be audited. What's the difference between a bash script from the developers website and a binary from the developers website?
I would say package manager generally has the ability to track version and uninstall packages. That's why I don't consider bash scripts a package manager.
I said that the tool would have to be installed by default on the main distros. I would be a single binary and a man page. I don't think it would be very difficult to get it included.
As I explained in another comment I'm not worried about malice of the developer but of a hacker. Bash scripts are not signed in any way, are not verified by anything client side, are not versioned. There's 0 security. Even tarballs are usually provided with MD5 checksum that you can verify client side. With bash there's nothing. Central repository would provide a lot. Yes, there could still be bugs in bash but at least you know you're downloading reviewed, approved, statically checked script that was previously downloaded by 10k people.
It is very difficult. The core problem here is the philosophy of the distros will cause them to avoid this tool for various reasons. Minimalist distros, like Arch, will avoid this by default because they are minimal. On the other hand, Debian really dislikes users not using packages to install things, for a variety of reasons that could be their own post, but the short version is that they also won't package this tool. A gentoo developer explains some of this, but also why staticly compiled (single binary) setups are disliked by distro packages as well.
It's a very long post, but to paraphrase a common opinion from it: Developers are often bad at actually installing software and cannot really be trusted to manage their own installer, and the dependencies of the software they create. For example, here is a pastebin of me running cargo-audit on Deno. Just in that pastebin, there are two CVE's, one is 5.9, and also an unmaintained package. Except, one of the CVE's has a patch available. But, in the Cargo.lock:
They have "vendored" and "pinned" the package, meaning that it is essentially stuck on an insecure version. Although I'm sure that this version will be updated shortly, what sometimes happens is that a non-backwards compatible update that includes a security fix is released, and lazy developers, instead of updating their software, will pin the insecure version.
In a distro's package manager, the distro would step up to patch vulnerabilities like that one, or do security maintenance for unsupported packages. Although Debian's extremely slow movement is frustrating, they are a particularly excellent example of this because they maintain packages in such a way that all their packages are backwards compatible for the duration of their lifecycle in the stable release, meaning that a developer making a package for Debian would have no need to pin the version, but they would still get security updates for the libraries they are using for 6 years.
Deno is an extremely popular package, and thankfully it has very few issues, but I have seen much worse than this, and it's because of issues like these that I am generally opposed to developers being package maintainers, and I think that should be left up to distro maintainers or package maintainers.
MD5 hashes are not enough. Modern packaging systems, like Debian's or Arch's have developers sign the packages to ensure that it was the real developer (or at least someone on the real developers computer...) who uploaded the package. Even with MD5 hashes, there is no such verification.
The other step needed is reproducible builds: If multiple people build a package, they will have the same output. I can verify the XZ tarball and see that the MD5 hash matches, but it's meaningless when that tarball has a backdoor in it, because they added something when they compiled it on their own machine (real story btw, also the xz backdoor didn't make it into Debian stable because of Debian's slow release policy and the fact that they essentially maintain and build forks of their own packages).
If the rust binary is not being built reproducibly, then it's meaningless to verify the MD5 hash.
You're talking about difficulty packaging something not adding something to a distro. Standard installations of Ubuntu, Debian, Fedora, Mint and so on include hundreds of packages. Yes, minimal Arch docker image may be a different story but I think the main distros would accept a tiny package that solves security issues. Of course I don't know that but I think it would be worth trying. No one is trying to substitute package manager. It's just a way to make bash installers more secure.
I didn't say that MD5 is enough. I'm saying that it's something tarball providers do to offer a slimmer of security. Because it's well understood that downloading unsigned archive from a webpage is not a secure method of distribution. Bash scripts offer even less security. What I'm describing is a secure method of distributing such scripts.
bash installers from the developers, and vendored/pinned dependencies in general will never be secure for the reasons I mentioned above. Developers are bad at security. Developers should not be installing software on people's machines.
Well, they are doing it. And they are not using package managers because clearly they don't meet their needs. So we can either give up and keep doing it with 0 security or add some security to it. I don't understand why you treat it as all or nothing problem. It's clearly not. We can't have the perfect solution but we can improve on the current one.
There are clear alternatives to using developer install scripts to install software though: package managers
Developers incorrectly believe that they need to vendor dependencies or control the way software is installed, which package managers of distros do not offer them. So they don't mention the way that their software (deno, rust) is packaged in nixpkgs, and instead mention the install script. Actually Deno mentions nixpkgs, and Rust mentions apt on their less immediately visible docs, but the first recommendation is to use the install script.
The core problem mentioned here is one of packager control vs developer control. With an install script that downloads a binary (usually vendored) the developer has control over things like: the version of the software, how it is installed, and what libraries it uses. They like this for a variety of reasons, but it often comes to the detriment of user security for the reasons I have mentioned above. Please, please read the blog post about static linking or look into my cargo audit. Developers are not security experts and should not be allowed to install software, even though they want to and continue to do this.
One the other hand, with package maintainers, they value the security of users more than things like getting a new version out. With package maintainers however, they take control over how packages are installed, often using older versions to dodge new security vulnerabilities, at the cost of keeping the same set of non-security related bugs, and sometimes the developers whine about this, like when the Bottles devs tried to get unofficial versions of bottles taken down. Bottles even intentionally broke non-flatpak builds.
But I don't care about developer control. I don't care about the newest version. I don't care about the latest features. I don't care about the non-security bugs not getting ironed out until the next stable release. Developers care about these things.
But I care only about the security of the users. And that means stable release. That means package managers. That means developers not installing software.
I get all that. What I don't get is what are you proposing.
For me it's obvious that developers will keep providing bash installers and users will keep running them. Everyone knows it's bad. I see comments about how bad piping scripts from curl to bash is daily yet it's still being done everywhere. What I described is a good compromise: dev can keep using bash but users get some security. Your solution is to keep complaining about it without doing anything to improve security.
Or course proper packages are better. Simply knowing this is not fixing anything.
I don't want "devs to keep using bash". My security problems are with the developer distributions of these softwares themselves, rather than bash. Even if developers offered a rust binary as an installer (or a setup.exe), I would still be miffed and disappointed with them for doing things like vendoring CVE's into their software!
Simply having this discussion brings attention to the issue, and to alternatives for getting packages onto the users machine, thereby increasing their security. There's a reason why it's a hot topic whenever it's brought up.