The Filesystem Hierarchy Standard [0] has been around for ~25 years but it still took quite a long time before most Linux distros decided to adhere to it (for the most part; some still do "non-standard" things at times).
Now that we're to that point, please stop screwing it up and coming up wih your own locations for application binaries, data, etc.
To be clear, the "/data" directory -- under which Moloch's pre-built packages apparently install to, according to the README [1] -- is not part of the FHS.
If the use of /data is the reason why you're not using this software, I'm pretty sure you wouldn't find any use of it if it stored data someplace in /var either.
Personally, I think the FHS is stupid marketing wankery that barely made sense in 1995 when the "preciousness" of the root partition was relevant, and has in the annals of time, caused more problems than it ever could have solved. Hey, what's the path to python? Is it:
This is perhaps the only question I need a "standard" to solve, and I've given up[†]. I don't need to know mail is "sometimes" in /var and sometimes in /var/local. I don't need to know that some of my configuration files are in /etc and others are in /usr/local/etc whilst still others are in /lib someplace. Where are my libraries? maybe /lib, /usr/lib /usr/local/lib /usr/lib64 /usr/local/lib64 and who knows where else. Fuck it, I'll just use find /.
I saw this great quote recently -- I know it was about something else, but I still wish Dan and the other FSSTND/FHS people could have seen and understood it all those years ago:
I suppose people blindly following suggestions on the Internet will eventually learn a hard lesson -- don't.
Because that is what the FHS was: A bunch of suggestions on a mailing list, with a smattering of armchair philosophy lacking any real analysis. But I mean, that's just my opinion. What if I'm wrong? ¯\_(ツ)_/¯
[†]: The answer is whatever /usr/bin/env python says.
If you look at the source code of most open source software, the devs typically install by default to arbitrary directories. They either don't know about, or care about, operating system standards. When Linux distributions package that software, they choose to change how that software works to align with standards. Usually their own standards, but those can often align with distro-independent standards too. But as a packager, you expect that you'll be modifying paths and making patches.
So really it doesn't matter where a random dev decides to install their software to by default, since either A) it's proprietary and it's non-standard anyway, or B) it's up to us to package it the way we want it.
Not everyone is using a package manager for every install, and even then it's not a magic bullet. I'm a Mac user and I've no idea why devs insist on installing to places like /usr/local or putting configuration in a dot file in my home directory when there's a standard place to put those (~/Library/Application Support), among other well thought out directory structuring[1].
Further to this, the moment you want to install somewhere else or have a dependency elsewhere (e.g. /opt/anything) builds break because of hard coded paths and poor assumptions. To top it all, I've no idea why a package manager like Homebrew then contributes to all of this by acting like no of this matters because a Mac is just like Linux, apparently.
If they don't like Apple's way then why not XDG[2]? Reinventing the wheel badly is more than just hubris, it breaks things and it's annoying.
This post 'gonna cost me, but what you write is endogenous to the FHS making little sense to most people who don't have a unix background.
"Where ARE my programs / settings that I installed, I always have to google it!"
is interrelated with someone else asking
"Where should I PUT this program / setting / data? I guess I'll chose one of those three or so locations where it may fit, or several"
For example: Why isn't there a data directory to be found? Where does the data go?
Why not replace nondescript and unhelpful names like "opt" and "usr" (which is btw. not where the user data is!)?
/usr is absolutely where user data was. That's what it stands for "users".
First when / ran out of space, new programs were put on the other disk /usr in /usr/bin so everyone could assume programs were in /bin except ones that were newly installed so they would be in /usr/bin
When /usr ran out of space, a new disk was added as /home and things that were easier to move (user directories) were moved first leaving things that everyone's script was depending on (#!/usr/bin...) where they were.
This was done out of necessity, not out of good taste.
In the original Unix implementations, /usr was where the home directories of the users were placed (that is to say, /usr/someone was then the directory now known as /home/someone).
It still had /usr/bin and /usr/sbin in it and scripts, muscle memory and general inertia to contend with. Yes of course it should have gone away, perl should have been at v6 and python at v3 years ago.
I'm off to saddle up my piggie squadron for a flypast.
This is cool, thanks for sharing! I'll be honest, I didn't think AOL did anything...interesting...but this is interesting to me! What are some open-source alternatives / analogues to this product?
Sorta like https://www.wireshark.org/. But Moloch is a very active project, used by many, and used internally at Verizon Media. Aol is part of Verizon Media (which brought AOL and Yahoo together). Open source is very active here. ;-)
Oh woah cool! I love the Verizon dashboard, looks very polished for an internal tool! I have a friend who did some network security work for Comcast, interesting stuff it is.
You can see https://www.verizonmedia.com/our-brands to see the collection of online brands in the family. You might use lots of these brands today without really noticing. There's a lot of internet content you get via https://www.verizondigitalmedia.com/ which is also part of the same company. Aol is still a thing, people do use it. Many people use lots of these brands as part of their internet experience.
You might even be looking for a job as an information security professional. You can join "The Paranoids" team (now that's a good name, don't you think!) by checking out some of their jobs. https://www.verizonmedia.com/careers/search.html?q=paranoids
Digital Programmatic Ad Buying Platforms for Brands by Verizon, formerly Oath Ads Platforms formerly BrightRoll, ONE by AOL (formerly Millennial Media) and Yahoo Gemini, which themselves have other ad tech acquisitions burred in them (Gravity, Adap.tv, Convertro among others come to mind) is the fifth largest digital marketing provider behind Google, Facebook, Amazon, and Microsoft.
Verizon Media (Engadget, Huffpost, TechCrunch, Yahoo Finance, News Sports etc) are the other big component of the company.
Seems like there would be at least some overlap with Metron[1] and/or Spot[2].
Of the two, I'm more familiar with Metron (I actually did a small amount of work on it back before it was an Apache project). The core "thing" of Metron was always a large-scale, high-speed packet capture mechanism that would allow you to apply real-time streaming analytics / ML to packet streams, as well as supporting indexing the packets with ElasticSearch for post-hoc retrieval / analysis.
Spot seems to employ some similar ideas, but I haven't dug into it as deeply.
Yep! As far as I'm aware, a number of military services (I think the USAF) have been public about using Moloch on their networks. It was money well spent by DARPA.
Fun fact, Moloch was initially created with funding from DARPA's famous Cyber Fast Track program! It's great to see that Moloch is still going strong since ~2013.
Not sure if you are joking, but Moloch was never part of DARPA's short lived Cyber Fast Track program. :) We do welcome contributions from everyone, and lots of different folks use Moloch. If interested join us over at https://molo.ch
That's odd. Are you one of the original authors? The CFT project list had Moloch on it. I'll try and dig it up, it's probably floating around my Google Drive. You may want to speak with Eoin Miller, as I believe he was the point of contact for the project in the document I'm thinking of.
Fuckkkkk I think I found the source of my confusion. I am wrong, you are right.
I DID find documents about Moloch floating around my Google Drive from ~2013-ish. I believe I invited your co-author Eion to present at a conference I was running, THREADS, in 2014 and that he was not able to make it. The focus the _year prior_ was exclusively on DARPA CFT. I combined those two events in my head and thought your project got some seed funding from DARPA too. I'm sorry!
I seem to be mistaken :-x. Moloch was never provided with early funding from CFT. I confused a few interactions I had with their project's original authors in 2013. It _feels_ like something CFT would have funded, but it was started on its own. See more here: https://news.ycombinator.com/item?id=22951925
In large part you're not looking at TLS application-data with this stuff; you're monitoring internal networks and all the protocols they run, in part so you can retroactively see if exploits, once revealed, have been run. For that kind of stuff you often care a lot more about, say, SMB dissection than you do about what stupid websites people are looking at.
The longstanding existence of tools like these --- and there are "better" ones that aren't open source, and have been for decades --- is one reason that "vulnerability equities processes" don't make sense; if the DoD uses an exploit against a foreign target, it can't just reveal it a few months later without compromising sources and methods.
(That doesn't mean you should care about that problem; I'm just reporting).
There should be a competition for most useful software with most offensive name. Then we can make all the software stacks of top companies likely to result in jail time when discussed in public. That would be funny.
I read the instructions but was not clear. Is this just a pcap viewer or something more? The way I read it, it appears to be a running daemon which listens on all port which saves the pcap file, which then exposes API's for accessing such data.
If you had an application http server running, is traffic sent to Moloch first, and forwarded to the http server like a proxy?
When running something like this on a large scale to capture all traffic going across a network, you'd typically use a "network packet broker" (cf. Google) that sends a copy of all traffic to the machine(s) running this software.
Your hypothetical application server would not even be aware that this was taking place.
Seriously, AOL named this after the god of child sacrifice, you might want to put a new marketeer on this project, I won't be touching anything with a name like that.
When this project was first published, the name of the company that published it was AOL. Before that, the company was called America Online. Subsequently it was named Oath, and now it's Verizon Media.
Names change. The common theme: names are not easy, sometimes they are beloved brands, sometimes they fall out of favor. Sometimes they were just bad ideas from the start, but happened anyway.
This gives rise to an interesting challenge for open source projects when you have an open source project in a github org, and the name of your company changes (or your company gets acquired), should you move the project? The problem is real since you don't want to lose your community, but you don't want to be stuck in the past.
This gets posted once in a while and I always, without fail, think of Ginsburg.
Moloch whose mind is pure machinery! Moloch whose blood is running money! Moloch whose fingers are ten armies! Moloch whose breast is a cannibal dynamo! Moloch whose ear is a smoking tomb!
Not the greatest choice of name: Moloch[a] (also Molech, Mollok, Milcom, or Malcam) is the biblical name of a Canaanite god associated with child sacrifice, through fire or war.
I think this is a bit distracting. It's an ancient name. Probably a better paragraph to fixate on would be:
> Moloch has been used figuratively in English literature from John Milton's Paradise Lost (1667) to Allen Ginsberg's "Howl" (1955), to refer to a person or thing demanding or requiring a very costly sacrifice.
I mean a project called Zeus would hardly founder on it's namesakes fetish for turning into a bull and assaulting human women.
Interesting, newer knew the word is of biblical origin. In German the word Moloch is in somewhat common use to refer to an abstract merciless, all-consuming or all devouring power.
E.g big cities might get referred to as molochs the way they eat into nature around it or suck people into its anonymity
I was thinking the same. In German Moloch is mostly used for groteskly big/growing cities. But thinking about it, the "eating into nature"-theme aligns well with the biblical "eating into kids"-theme.
For me at least, because of the city-context, Moloch matches: The moloch could stand for the amount of activity in a system that is humanly insurmountable to get a better-than-superficial understanding of.
I didn't have that strong of an association of Moloch with big cities as it felt like when I looked it up for the above answer, so I googled a bit of its usage on news sites and it looks like in Germany it is almost exclusively used for big cities, whereas in Switzerland, cities is the most common usage but it is applied way more liberally.
Within a minute or so I saw it used to refer to: The EU, a government bicycle program, a 2km long tunnel in Zürich, iTunes, the Tour de France, Goldman Sachs and fashion.
Reminds me of when someone saw that the NSA's public key (http://www.cypherspace.org/adam/hacks/lotus-nsa-key.html) to be included in Lotus Notes had an organizational name of "MiniTruth", and a common name of "Big Brother".
Moloch isn't the good guy in Howl.
Moloch! Solitude! Filth! Ugliness! Ashcans and unobtainable dollars! Children screaming under the stairways! Boys sobbing in armies! Old men weeping in the parks!
Moloch! Moloch! Nightmare of Moloch! Moloch the loveless! Mental Moloch! Moloch the heavy judger of men!
Moloch the incomprehensible prison! Moloch the crossbone soulless jailhouse and Congress of sorrows! Moloch whose buildings are judgment! Moloch the vast stone of war! Moloch the stunned governments!
which is the same referent, and begs the same question: "What sphinx of cement and aluminum bashed open their skulls and ate up their brains and imagination?"
It's an inside joke to the Deep State that actively needs captures of network data, AOL... is a Verizon company after all. In these discordian times, even Eris would blush.
Funny you mentioned the deep state. I remember being perturbed that the email leak from the 2016 race had references to child-sacrifice moloch. Strange times
Naming open source projects is a challenging task. The team hears this all the time, but hey, open source is about code you can use, a community you can join to make the code better, and the pride that people around the world want to use your code. If you are on a blue-team, you'll want to look at this project. If you are making lists of names that distract attention this goes on it.
If someone names their CLI JDahmer which then requires me to use it jdahmer <command> <options>, it is going to cringe me out.
That said, I totally respect the project's decision to name it so. I'm sorry if it comes off as an outrage or an unnecessary rant. I understand it takes away the energy which is otherwise better spent on a constructive discussion of this excellent project, which is also surprisingly well-maintained. But, I hope you realise why the name is controversial, especially since everyone that has to use the project has to get used to seeing/reading/using/typing Moloch everywhere from the docs, to the CLI, to the FAQ pages, and what-not.
May be the project can explain their rationale in a separate webpage (if they want to) and that might help?
The amount of professional-grade hand-wringing virtue-signaling in this comment section makes me feel ill.
Why get bent out of shape over the name of a software project? It's virtually a meaningless factor in day-to-day life.
What about hearing phrases "Sacrificing a Chicken to Moloch," the "spirit cooking" culture and related symbolism rampant in elite political circles? Shouldn't we be more interested in that?
Lots of low-hanging fruit to pick, I guess. Someday I might unlock the "downvote" ability on this platform. Until then my opinions don't carry weight here. Also, uhh who decides the "threshold" for downvoting? Hint: nobody knows. [0]
This platform has become more of an echo chamber than a host of rational discussion based on merit. I suppose that's a problem with growth.
There's a lot of conversation around "Moloch" as a name and the subsequent emotional responses... but not a lot of discussion about the tech at-hand. And it's a repost.
Edit: while this is high-ranking comment, I'd point out that if I had the ability I would have just downvoted the comments I didn't like. Take that for whatever it's worth.
Moloch is a serious open source project, run by serious people who care about network security. They published their code under an open source license, showing off just how confident they are that it is solid. You can use this project to inspect packets on your network, you can learn how they built it and become a valuable network security engineer with a job somewhere finding people trying to hack in to your site. You can propose modifications to make this even better (and if your code is good enough, it will get accepted and used by security teams around the world). Or you could focus on the name they chose and the name of the company at the time this was published.
I'd seize the opportunity to talk tech and focus on network security. Infosec jobs pay better than brand marketing jobs.
The fact we have access to tools like this (and many, many others) is so damn motivating. I am working through some structured course material in preparation for net-sec baseline `$vendor-$certification` exams, and there are so many complexities and rabbit holes that it's easy to be overwhelmed. Some nights are harder than others to see forest for the trees when it feels I'm trying to understand how to pick up a single pine cone to study.
Then I see projects like this, or Jaeger/OpenZipkin, or Chaos Monkey, and I'm simply inspired. Much like listening to a killer record.
I don't spend time feeling sorry for myself because an ancient, somewhat esoteric proper noun was used to describe the project.
Unless a bunch of people deleted their comments, at the time you posted, the comments actually criticizing the name had about as many words as the first half of your comment, with replies mostly pushing back instead of agreeing.
None of the critical comments were particularly strident ('bent out of shape').
Not to mention the OP participated in the debate about the name themselves. This being the top rated comment is a self-fulfilling prophecy, it’s getting in the way of a discussion about the tech!
Oh but now I’ve contributed to it as well. Uh oh...
> Why get bent out of shape over the name of a software project? It's virtually a meaningless factor in day-to-day life.
A lot of people do care about what things (living or not) are named and even how, sometimes and attach a lot of emotional value to it for a good reason.
- They strategically distinguish the product from its competitors by conveying its unique positioning.
- They hold appeal for the product’s target audience.
- They imply or evoke a salient brand attribute, quality or benefit.
- They allow companies to bond with their customers to create loyalty.
- They have a symbolic association that fortifies the image of a company or a product to the consumers.
- They help motivate customers to buy the product.
I'm sorry but claiming that an uncomfortable discussion on a project's name is not the response one wants to see is exactly what turns this into an echo chamber? To be noted how discussion on a product's technical merits isn't mutually exclusive to discussion on its naming. Both could easily be had, and both have their places, because I don't particularly think either is off-topic. In fact, a previous news.yc thread on this product brought up a similar line of discussion. I guess one could argue the discussion is counter-productive in a technical forum, but I don't think one should avoid it just because... unless the community rules explicitly state so. Technical people aren't robots, at least not yet.
You have good points. Clearly the name of a thing matters.
I'm arguing that the naming of a thing matters less than the efficacy of the thing itself. Doubly so in a hacker-oriented forum.
As for the echo-chamber, that's a classic human fallacy which I think is magnified by the centralization of content aggregation into heavily personalized "newsfeed" style design choices perpetuated by major players.
The epidemic of "outrage culture," or "offense culture" is something comedians and free thinkers have been discussing for awhile, and I think is underpinning of my personal frustrations.
-----
"This idea of "I'm offended". I've got news for you - I'm offended by a lot of things too. Where do I send my list? Life is offensive. Get in touch with your outer adult, and grow up, and move on."
-----
Post, comment, discuss as y'all like, right? That's the beauty of the format. I'm simply one of ∞.
The folks in the community here are brilliant, and the reasonable ones are off doing something meaningful, whereas I'm here arguing for practice, basically.
If you have a strong, negative emotional response to the name of a project, I think letting the project author know can be helpful.
I’m a rational guy to the best of my ability, but FWIW I would avoid using this project solely because of its name (just the thought of that name evokes a sick feeling to me - that’s how strong the negative association is... I won’t explain why as this isn’t an appropriate venue for that).
We are human, and sometimes visceral responses can’t be ignored regardless of their irrationality.
I could invoke Godwin’s law to give a more universal example of words with negative associations, but I’ll refrain.
If they named it something like “Pinochet” or PIZZAGATE, I’m sure we’d be getting completely different responses from the same people who think naming a project after a child sacrifice idol is “funny” or “cool“.
In a professional environment, I’d keep that to myself unless someone asked me for feedback or I had a hand in the decision But I think once the point is brought up in an online forum, that’s an appropriate venue to give feedback like that.
Like it or not, people will have internal reactions to things—and I for one would prefer if others let me know if they found a project name I chose to be difficult in some way.
Why not name the product "ILikeToKill"? Is that professional? Because that's exactly what the name of this software is implying. It is disgusting, and I would avoid using this project over the name as well. Names mean something, this is an easily understood concept.
> What about hearing phrases "Sacrificing a Chicken to Moloch," the "spirit cooking" culture and related symbolism rampant in elite political circles? Shouldn't we be more interested in that?
We had a client that forbade us from using their real name even in internal communication. Formaly they where called "client 123" but informaly we called them "Voldemort". They where (really!) absolutely delighted when they found out
Now that we're to that point, please stop screwing it up and coming up wih your own locations for application binaries, data, etc.
To be clear, the "/data" directory -- under which Moloch's pre-built packages apparently install to, according to the README [1] -- is not part of the FHS.
---
[0]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
[1]: https://raw.githubusercontent.com/aol/moloch/master/release/...