Sounds like you would probably like the EdgeSwitch line of products better than Unifi. Cheaper, faster, and easier to manage if you are trying to do something specific.
I was thinking about trying EnGenius or even Cisco next, but I'd like to stay within the Unifi universe because of my investment in wifi access points and video cameras. I should definitely investigate the edgeswitch line! I sort of ignored them thinking that they were last generation, but I'm going to take a look.
If you're interested in spending a bunch of money buying stuff again, wifi 6e APs are trickling out from the usual vendors. If your local spectrum is congested, the new 6ghz band might be nice.
How does it handle merging log files that are in different time zones? I wrote a small script to merge log files from lots of different network appliances a long time ago, it was extremely useful for debugging problems that occurred across our distributed system. The most unfortunate part was that the appliances all logged in local time and didn't include their UTC offsets in the timestamps :(
Timezones are not automatically handled by lnav at the moment, which isn't great. Timestamps are parsed and treated as UTC internally, which usually works out fine. You can manually adjust all of the timestamps for a file using the ":adjust-log-time" command:
Thanks, I see. At least with `:adjust-log-time` you can make it work!
With my old log merging program, you had to supply a regex with groups for the different timezone components and optionally a UTC offset. That worked really well but was a pain to set up. Typically I was using it to look at the same format of files all the time though, so in practice it wasn't that bad.
I'm not really a C/C++ person but maybe I'll try and hack on lnav a bit and see if I can figure out how to add timezone support.
A project you may want to look into adding is Tern [0]. I've had a good time reading through the code over the past couple of weeks, and have found it to be at least not "bad" code, and pretty easy to understand.
Specifically how they are untarring each container layer and creating a chroot jail to run commands inside is fairly self-contained and interesting.
Sometimes you can't use production data in your staging environment, like when prevented by privacy laws. Often times developers who would not have access to production data would have permissions to view data in a staging environment...
Creating tools to produce realistic test data and also using fuzzing tools can be just as good and sometimes even better than using actual production data.
I'm having a hard time visualizing this - are we talking outside devs (consultants) or folks in a different jurisdiction who don't have rights under law, or just a company administrative policy? Agree that fake data can work well, but if you can clone the production DB, that seems a preferable and easier approach.
There's a variety of reasons this may happen that stem from either legal, certification, or company reasons.
Your production database may have medically-sensitive PII (or for something like SOC-2 compliance any PII at all) that cannot be shared any human (other than the original user) unless with prior approval.
Even for non-externally mandated reasons, companies may (and often do) wish to restrict access to production data by developers to minimize concerns around data exfiltration and snooping on user data by company employees.
that makes sense - I suppose using my model you could mask some of the data in a versioned graph or a collection contained in the database that can be surfaced up to other users who can then clone the collection that excludes PII. You could run the main collection and the PII free collection in the same data product. This might be an easier approach than creating fake data & fake schema.
> This might be an easier approach than creating fake data & fake schema.
I doubt this. This is for two reasons: the first is that the development database usually shares the same schema as the production database so that's not an issue.
The second is that fake data convincingly takes care of various issues surrounding de-anonymization of data using correlations among bits of data that ostensibly have had their PII-sensitive bits removed.
If protection of user data is a priority, there are far fewer headaches associated with creating entirely fake data to populate the same schema than trying to figure out post-hoc censoring of production data.
That's not to say there aren't valid use cases of the latter. You often will want to do post-hoc censoring/aggregation if you wish to track e.g. usage metrics. This is in fact often a component of ETLs. However, those are removed from everyday development tasks.
I am struggling to think of any time it would be appropriate to clone a production database and use it for staging. That almost inevitably translates into any developer who ever deploys database-related changes having access to the contents of the production database, or at least every part of it that might ever be affected by code changes they are making. There are numerous legal, regulatory and ethical issues with granting such broad access, unless you're small enough that the few people you obviously do need to have the ability to grant full DB access happen to include anyone who is likely to change code that talks to the database. And even then it still seems like bad practice to actually grant anyone that access when it's not strictly necessary!
Legally. I've worked in healthcare and any direct access to production is strictly forbidden. If you can't figure it out through your logs and monitoring, and you can't reproduce in earlier environments, then the problem isn't that you don't have prod access, the problem is that your instrumentation has a gap.
Personally, I think we rely on prod access as a crutch because it's easier to expect that than it is to build a sufficient infrastructure. Cloning a prod database or allowing ad-hoc r/w access is on my list of strictly forbidden operations.
For example, we, as a company have decided on some rules around PII (personally identifyable information) that means, amongst other things, that we don’t have any of it outside of production. We therefore cannot just copy data from production to staging/dev.
If you prefer a GUI tool, DeepGit[0] is a very nice tool that allows you to do some pretty amazing code archeology. I use this all the time for figuring out how legacy code evolved over time.
Also related - don't read random tutorials on "how to do X", go and find the official documentation for X and RTFM. In my experience this is the easiest and most direct way to learn any new technology.
> go and find the official documentation for X and RTFM.
Yes and no. The official documentation can be obtuse and bad for new learners. (I'm looking at you, gradle[1]). Ideally, the official documentation includes tutorials, but also three other forms: 1) how-to guides showing the steps for accomplishing a common goal. Think of them as extended FAQ answers. 2) simple reference docs. Your typical api-centered doc, or man page cli flag list. Answers the question of "what was the function/class/flag to do that?" Assumes you already know how. 3) explanations, the discursive articles about how and why. A lot of people can get by with never reading these, but once in a while they can clear up some problem you're having. The tutorials, which should exist, would be short guides for learning by doing. They aren't how-to guides, because they don't assume you know anything yet. They are often the best entry points for picking up something entirely new.
I’ve had the opposite experience. I’ve attempted to read a number of specs, and by the way I’m completely unable to parse them, you’d assume I was illiterate. On the other hand, I’ve found good, technical articles detailing how the same things worked and have found them much more digestible.
Sometimes the official docs are just right. Django is good, for example. I had to do small Ubuntu sysadmin task recently, the tutorials I found were incomprehensible. The official docs were crystal clear.
But we've all seen the reverse, as well, and a combination of SO, YouTube, Reddit, and random tutorials is the only way to figure it out. Especially when you encounter the "This describes version 3.1, and everything in 4.5 is completely different, good luck" official docs.
For me personally, the answer is a combination of the two. Having the docs open side-by-side with a good technical write-up usually helps me understand a lot faster and better than either resource does on its own.
If you're competent enough to judge the technical article's credibility, this can be a good supplement to the official docs. I see junior developers running into this problem more often because they don't know how to tell the difference between a quality writeup and a garbage tutorial written by somebody trying to pad their blog.
There are definitely cases where the official docs (or RFCs) are practically incomprehensible to a newcomer, but I usually will try and come back to them quickly after getting a foothold using a blog post or YouTube video.
I find that even with the best technical writeups, there are often things applicable to MY use case that may not have been relevant to the author. The official docs (or sometimes the source) is the best place to find out about those.
The F'ing M is often an afterthought, and sometimes you need to dig into the source code to understand how the thing works, or at least to find the right handles to grasp.
Not necessarily the right approach for beginners, but I've wasted a lot of time trying to find the thing in the documentation, when I should have just lifted the hood in the first place.
This! There is very little selective pressure reducing the quantity or visibility of bad intros/tutorials. But for successful projects, the community puts a lot of pressure on the improvement and accuracy of core documentation. Truly complementary tutorials often ended up getting linked from or incorporated into primary project docs/pages -- so it is safer to explore from there than a Google search.
And it's much, much rarer but equally as important corollary: UTFM (Use TFM). UTFM means: when you're designing a system, adhere your design so f**ing tightly to standard usage. RTFM/UTFM with standard tooling gets you so, so far to building high-quality software.
Edit: I actually just made up (as far as I know) UTFM, but I really wish it were a thing.
I recently found the need to teach myself Java. While the Java Docs and official Java tutorial seem to be decent, especially once you know what you’re doing and want to dig deeper, the “Baeldung articles” have been really useful as starting points as a survey of options available. Since this seemed like a pretty random website, I’ve been cautious and cross referencing everything I read. But I have to admit, it was pretty damn useful.
So unofficial tutorials do have their place in introducing design patterns or listing trade offs between different approaches. But I do agree the source of truth should still be the official docs.
This is nowhere near being viable for real world use. Far more concerning is how easy it is to bypass most physical security measures. Forgetting about how easy it is to pick open most locks, many doors aren't even installed properly in the first place. Check out this YouTube video [0] to see what I'm talking about.
