OK. I was going to make this post as part of a deeper thread about installer problems, but I feel the need to post this where it will get more visibility:
This script is a huge security risk.
It takes arbitrary commands issued to it via the API and runs those on your system, then uploads the stdout from those commands back to their servers, where it's stored for an indeterminate amount of time with indeterminate security. (Their privacy policy makes no mention of the storage, usage, or dissemination of data collected via this tool)
There is literally no limit to what commands to be run, and you can't easily audit which commands it will run prior to them being executed. There's even a built-in privilege escalation path, via the upgrade script.
You might as well give them a login with passwordless sudo to any system you install this to.
Even without privilege escalation a feel like this is a big security risk.
I can't imagine having dozens of servers bound to a web service, with a user interface accessible from anywhere with a simple login and password. and from that interface you can add any number of arbitrary command.
I've created the account, copy the command, pasted in a terminal but I just couldn't hit enter... and that is in my home computer. I downloaded the link and the target manually, just checkout the content.
This isn't about the installer. It's about allowing arbitrary commands to be executed as specified by the API.
Specifically, lib/authentication.js receives a new command, passes it to addActivity in lib/execute_services.js, which schedules it to `exec` periodically (the period is defined by the API response) via ExecuteCommand.executeCommand.
As for the user that the sealion-agent is run as, it's started as root, then setuid'ed to the sealion user in index.js, so any update can undo the setuid and allow full root access.
Basically, by installing sealion-agent, I have given you, all of your current employees and of your future employees (both in CA and India) full access to the server.
Is this actually something people are willing to do? I.e. pipe from the Internet directly into a root shell? It seems especially crazy since none of the tools (ps, top, uptime, iostat, vmstat) require special privileges.
It is a script that you can audit if you don't pipe it straight away into bash (with sudo!).
I don't know Sealion and they can be amazing and all, so don't get this comment wrong.
If they want to support Linux systems properly, instead of using a script like that, I would recommend them to use packages and add support per distribution. Each distribution has different tools to add and manage services, and there are different ways to boot the system services. It's not just adding symlinks anymore because upstart and systemd, etc.
Also I'd love to cleanly upgrade/uninstall/etc their agent, so add a repo and I'll install your agent with more confidence.
How do I know I don't like how they install their agent? Because I didn't pipe the script directly into bash.
Curl has an option to explicitly disable cert validation:
-k, --insecure
(SSL) This option explicitly allows curl to perform "insecure" SSL connections and transfers. All SSL connections are attempted to be made secure
by using the CA certificate bundle installed by default. This makes all connections considered "insecure" fail unless -k, --insecure is used.
This is way better than downloading a random executable and running it. (I assume you mean running it as root.) With this, you can grab the install script and see exactly what it uses root for. It only takes a couple minutes. You could even modify the install script to comply to whatever installation scheme you wanted, and probably even make a version that runs without root privs if you're willing to do a few things manually. This mechanism is WAY better than a binary executable installer.
As for the product, it looks really good to me. No mention of encryption of stored data in the privacy statement, which might be a concern for some.
Also it wouldn't hurt to make checksums available for the installer and downloaded tar, for more paranoid folks like me.
Yes we wanted to give you the flexibility to see what we are installing on your servers. We understand the importance of checksums and will be realizing checksums for our agents in future releases.
Let's look at the established alternatives: packages. Packages are checked for corruption and validity.
This isn't so much different from downloading a random executable and running it. Hell, it's worse than that (since the download doesn't get stored for future audits)! But who said executing random executables from the internet is good in the first place?
Unfortunate. It's a bad habit to get into. Random forum goer pastes something in, you're used to installing via this method, you'll execute it without thought. Boom. No longer your system.
The command you will be executing is to download and install the SeaLion agent.
SeaLion requires you to use sudo to install it as a service so that it will automatically restart whenever system reboots. All the commands executed by the SeaLion agent are run in the context of Sealion user which is a normal system user.
How can we be sure that the package that's being installed is the one that you intended to be installed?
How can I be sure that the install will be identical across all of my servers?
How can I pin a version that I've vetted for security holes?
How can I distribute this to servers that don't have arbitrary internet access?
How do I configure selinux/apparmor to work with your tool?
How can I uninstall this cleanly?
Curling into a elevated shell is a terrible way to do installs. Packages are cleaner, more secure, and more capable of being trusted. You're making the assumption that your s3 buckets will never be hacked, that your own webservers can never be hacked, and that experienced sysadmins are going to ignore your tool because of the amateur install method.
> How can we be sure that the package that's being installed is the one that you intended to be installed?
We will give "checksums" with our agents in our upcoming releases.
> How can I be sure that the install will be identical across all of my servers?
Whenever you install SeaLion Agent, the most latest binary is installed. The agents in other servers are also updated, hence making them identical always.
> How can I pin a version that I've vetted for security holes?
SeaLion agents are released after rigorous testing, to ensure we do not leave any security holes and features, currently SeaLion Agent is designed for auto update.
> How can I distribute this to servers that don't have arbitrary internet access?
SeaLion agent requires Internet access to send data to its server. Hence internet access is mandatory.
> How do I configure selinux/apparmor to work with your tool?
SeaLion agent works fine with SELinux in enforcing mode without any specific configuration.
> How can I uninstall this cleanly?
sudo /usr/local/sealion-agent/uninstall.sh
> We will give "checksums" with our agents in our upcoming releases.
Checksums are useless if your install script is compromised, as well as if the server hosting the checksums is compromised. Signing everything while using an installer that verifies signatures (like yum, rpm and apt) is more secure, and just relies on you not losing control of your private key, instead of entire servers.
> Whenever you install SeaLion Agent, the most latest binary is installed. The agents in other servers are also updated, hence making them identical always.
Assuming your upgrade location is not compromised. Does your auto-updater check checksums or signatures, or does it rely on your servers always serving up the correct files? What if you push an auto-update that breaks servers?
> SeaLion agents are released after rigorous testing, to ensure we do not leave any security holes and features
No offense, but when I'm responsible for what happens on multiple servers I would like to ensure that myself, by checking the tcp payloads, endpoints connected to, files opened, etc. I'm certainly not alone in this.
> SeaLion agent requires Internet access to send data to its server. Hence internet access is mandatory.
Sending data can be accomplished individual firewall rules to whitelisted IPs. s3 is too broad of a target to allow unlimited access to.
> SeaLion agent works fine with SELinux in enforcing mode without any specific configuration.
If the only requirement is that it be started at boot, there are non-root ways to ensure that. Modern cron implementations have an "at reboot" option (@reboot in vixiecron and cronie). Alternatively, a cronjob that runs every 5 minutes or so and checks if the service is running and starts it if its not is doable.
Yeah, with sudo is especially bad. In the old days (20 years ago), I remember some things (ircII maybe) offered by an easy installer command line like:
telnet host.domain.com 12345 | sh
And then for a very long time that sort of thing fell out of favor because people got appropriately cautious about security. But I've seen it a lot more lately, starting with Homebrew: http://brew.sh/
The "trusted by 30,000 companies" at the bottom of this product intro page doesn't refer to this product does it?
If not then it seems a bit sleazy to me. It's great that their Yog products are widely used but they've over-stepped the line IMO. This seems more like wilful deception than a reminder that the company's other wares are widely used.
Was wondering the same thing. How come a "Show HN" project is already trusted by all those many companies and big-name brands?? This didn't feel quite right.
It looks great. But I don't run any servers at the moment - I'm just an annoying stickler for the truth in marketing.
Thanks for your response. FWIW I think it just needs a line like "our related Yog line of products are trusted [I'll let you have that one!] by 30,000 companies". Assuming that you can verify 30k different companies are actively using them. It's actually got a double edge too as it also tells people that you have other products in this space that will transform their daily drudge of server maintenance in to a veritable nirvana ;0)>
This is really good - +1 for the shell based installer and time sorting, and a level of polish that infrastructure tools often lack.
I'd add (in addition to what you have now) your own procfs tools to extend some of the knowledge the older command line tools.
- For a C app you care about argv[0] that top batch mode displays. But for some shell, python or perl or ruby you care about the actual script name. And for Java you care about the .jar (especially in enterprise envs). Even in your example, it's just some anonymous java process. There could be 12 'java's - I want to know whether it's our app or some Enterprise Crapware agent.
- Likewise load average - for a multicore system a load average of 10 could be quite underutilized - if I remember correctly, the run queue is actually (number of logical cores) long.
There's also some things you audience might not be familiar with, but would find valuable - tools like this can be a good way to show off some of this stuff.
- Do you have ethtool in there? Particularly patch disconnect events.
- You can also ask the switch what ports the host is connected to with a two liner LLDP/CDP thing.
These kinds of things are GOLD to people that know they exist, but as you know, Unix tools aren't very discoverable - your app could help with that.
- Not sure if this is relevant, but if you do monitor network stuff, try and steer users away from ifconfig, it's not maintained and will simply skip certain interfaces (eg, trading arbitrage desks used vlans on top of virtual IPs on top of bonds, these are completely invisible to ifconfig. There are problably other cases I haven't run into yet).
Are you guys on Twitter? I'm also working in similar space (used to work at Red Hat and IBM as a Linux specialist, now I write node apps all day). Not competing, but doing similar stuff.
Thanks for your great feedback. We wanted to build a framework which can be easily extendable by adding custom commands. This is just the beginning, we will soon roll out more tools which will help integrate tightly with your server and applications.
We plan to integrate monitoring and many other related features after analyzing how the users are using our system. Regarding monitoring network layer, you can always write your own bash which records network activity. Interesting thoughts, we have added them in our pipeline.
We are @SeaLionTeam on twitter, we would love further inputs from you
I get what you mean re: letting users add their own tools, but I think there's some utility from providing/suggesting things people don't already know.
Eg: a cable gets pulled:
- syslog, which is well known, might show a NIC event.
- ethtool and openlldp will tell you your were connected to row 4 port 26 of MySwitch and now you're not. Which is super useful when you need a NOC person to go investigate.
Good input, yeah after seeing the demo I've also noticed that the anomaly detection monitoring cannot be solved by simply pointing to a process.
I'd suggest aggregating the data of vmstat,ps,lsof,nc etc. into a simplified and abstract summary by incorparating a structure based AI model, like HTM.
You could build rules and action lists and anomaly type lists, then let the AI categorize anomalies by pattern and structure detection. Heck you could go much further, ie. building an NLP query interface.. Don't get me wrong, I don't want to smoke your brain, it's just what I would do. Keep working on it!
+1 there's a lot of stuff you can find by correlating between the tools.
Since many of these tools are just /procfs readers and you can make something tighter by hitting their data sources directly, rather than scraping existing command line apps (and the source for these apps is generally quite readable).
"/usr/local/sealion-agent/etc/sealion: 127: ./bin/sealion-node: not found"
Might want to fix that.
$ curl -s https://agent.sealion.com | sudo bash /dev/stdin -o 7260187a-df41-455d-9999-9a8207257de2
Downloading agent...
############################################################
############ 100.0%
Created 'sealion' user successfully
Extracting files to /usr/local/sealion-agent...
Files extracted successfully
Installed SeaLion as a service
Starting agent...
/usr/local/sealion-agent/etc/sealion: 127: ./bin/sealion- node: not found
Sealion service started successfully
Installation successful. Please continue on https://sealion.com
$ curl -s https://agent.sealion.com | sudo bash /dev/stdin -o etc
Downloading agent...
######################################################################## 100.0%
Created 'sealion' user successfully
Extracting files to /usr/local/sealion-agent...
Files extracted successfully
Installed SeaLion as a service
Starting agent...
./bin/sealion-node: 1: Syntax error: word unexpected (expecting ")")
Sealion service started successfully
Installation successful. Please continue on https://sealion.com
$ ./bin/sealion-node: 1: ELF: not found
Just a wild guess but maybe the installer wants the "real" bash and you have dash set up (I think ubuntu and debian do that nowadays).
That or you're missing an executable or some command returns an error that's poorly handled (since we all know proper error handling is borderline impossible in shell scripts...).
That's why I don't get the point of bash scripts: if you want something truly portable you might as well use /bin/sh and if you're willing to compromise a bit of portability for more advanced features you might as well switch to perl or python (perl is probably as likely to be installed on any un*x machine as bash).
I tried it out on two of my API servers and it seems to work flawlessly. I'm impressed with the beautiful style of your website.
Honestly I'm not willing to pay $19 a month for more than two servers though. Maybe if it was around $5 a month per server it would be worth it. As is the price is high enough that I could take a few days of work and create and run my own logging system for all my servers and while it wouldn't have that of an pretty interface it wouldn't be costing me $400 a month.
I got the impression that I can achieve similar (or better) results with logstash and its interval exec input [1] as well as the usual logs, events and perf counters, etc. And it has nice Kibana interface [2] on top of elasticsearch.
Seems like a thing I could build in a couple of nights.. but I won't, so good to have someone who will DEDICATE themselves just for that. There are tons of stuff that can be added.
Unfortunately pricetag is too high. As this is most useful when you have multiple servers then it quickly rises to hundreds of dollars per month. So I would stick with Nagios/etc.
And yes, our target audience is users having multiple servers. You can use nagios and some other tools for logging data from few servers. Handling and debugging of logs spread across several servers is a pain in the neck.
Is there any non-commercial vs. commercial distinction? I.E. do I pay the same price for my EC2 instance running tmux+weechat as my multinational enterprise software employer would pay for a production server?
I really wish this were able to be run internally on my own server without handing these guys the keys to my castle. I'd do it in a heartbeat. Unfortunately that's not the case.
Apart from HN, we were doing great on reddit.com/r/sysadmin/. Crazy traffic. Unfortunately, the site couldn't handle it. Ngnix was going out of TCP handles. Arrrrrrgh, we should have done a better job in load testing. Fixed it by increasing filehandles. Give it a spin now!
This script is a huge security risk.
It takes arbitrary commands issued to it via the API and runs those on your system, then uploads the stdout from those commands back to their servers, where it's stored for an indeterminate amount of time with indeterminate security. (Their privacy policy makes no mention of the storage, usage, or dissemination of data collected via this tool)
There is literally no limit to what commands to be run, and you can't easily audit which commands it will run prior to them being executed. There's even a built-in privilege escalation path, via the upgrade script.
You might as well give them a login with passwordless sudo to any system you install this to.