Not nearly as surprising as it is to me that now that most developers have forgotten perl, they're turning to awk as an inspiring example of a bygone era.

Seriously: perl basically replaced awk in the mid-90's. It absorbed all the great lessons and added two or three dozen innovations. But it had scary syntax, so everyone used python (which did not replace awk very effectively) and forgot perl. So now we're back at awk. And the scary syntax is all in the Rust world.

The Perl hate is so prevalent in most companies, that you can get into serious issues should you even write a one-liner in it. So here I am, reluctantly using awk/sed and asking too much of Bash/Python.

[0] https://www.manning.com/books/minimal-perl

I could do all that in perl, but it's more stuff I'd need to dig up every time I wanted to do something awk-like. People who code perl regularly already have it, for sure. It's a barrier for those of us who don't, and one that awk doesn't have.

Awk isn't perfect, for sure, but I've transitioned from writing simple text processing in perl to doing it in awk because awk provides the basic framework that I'd end up doing ad-hoc and non-idiomatically in perl every time.

Again, all this with that caveat that I'm not a perl programmer. I used it for a couple of semesters in college, which was enough to be dangerous, but not truly proficient.

`perl -p` wraps code in a sed-alike start and finish stanza.

I used to maintain and debug a relatively large and an extremely complex build engine in Perl for a living for several years, and there was no construct in there that could not have been easily written in AWK.

Could you expand a bit on that comment?

Consider which language is sold to people-off-the-street as "How to Program!" it's python. Most of these people do not enjoy the cognitive effort, detailed typing and symbolic work, algorithmic thinking, architectural thinking, linguistic thinking that is "programming". At best, the enjoy the end result, but not the process.

Python taught in this way at least, fools people into thinking that programming is about something far simpler and more toylike than it is. Many "hard" disciplines do this: sell children and lay people on toy experiments that momentarily captive but have no relationship to what a practitioner of that discipline does.

The older system of recruitment would be to polarized hard, ie., to throw people in at the deep-end to scare off all the people who would waste time/resources training. Who remained really wanted to do that thing (eg. physics, programming, ...).

Today we're doing something perhaps vaguely immoral: selling people on a career that has no relationship to the sales pitch.

Somebody playing football in a Sunday pub league is still playing football, and still loves playing football, even if they're not at the same standard of Lionel Messi.

For me at least, I feel the same about programming. I love understanding somebody's problem, and building a solution for them that solves that problem. I normally use PHP (and sometimes Python or Ruby or JavaScript) because they make it easier for me to focus on the problem, rather than language details. I can't always solve the problem because it's too difficult, and perhaps some of my solutions are not 'optimal'. But I feel hurt by the idea that because I don't have a strong understanding of how Python works at a really deep level, I'm not a real programmer.

I also think that's a great way to piss people off who are just getting into the industry, and may one day become great programmers - even Linus Torvalds was a junior once. I'd encourage them to keep going, keep learning, and keep helping people solve their problems (and getting paid good money for doing that).

The vast majority of people do not want to be programmers and would not enjoy programming. Delaying the moment the actually have to do something difficult with a programming language is not especially heathly.

How would you feel about a career in football sold to you on the basis of table pong? Keep playing the pong, and then one day, you're face is in the dirt and you drop out.

The self-esteem hack psychology of the 60s-90s equivocated encouraging people with lying to them, as-if the only way we can get programmers is by lying about what programming is about. This isnt encouraging anyone, it's lying to them.

You can’t run a business expecting every employee to be a rockstar. It just doesn’t scale. So you skill it down and put the high-skill people where they can have the most impact.

The complexity of the code is not part of the definition. Nor is your understanding of the hardware/software involved.

Unless you've experienced this sales pitch you're not going to connect with the point i'm making.

The whole point of programming is power and flexibility. Else there would be no need to move beyond logic gates.

The problem is tools sold for newbies tend to take away too much power to make it easy for people to start, but then keep them, right there, all life.

If you didn't enjoy reading, puzzling, and banging your head against the wall you would wash out after a couple of classes.

Like many of us, I started programming with BASIC, not C or assembly, but I turned out fine. I don’t think people need to or necessarily can absorb all the high level details straight off. To me python seems well organized enough that it makes a great language for beginners.

Python is perhaps my favorite language, and I use a dozen fairly regularly.

My point was about how python sometimes gets used, and how its design is especially facilitating to that use.

It's not necessarily a bad thing to be productive, instead of inventing new problems for yourself to keep having programming things to do.

(Note that none of this necessarily reflects my personal opinion; it is just an alternative way to read the grandparent.)

It's disgusting how much we all rely on intuition and gut feelings when evaluating large swathes of technology. The internet hates perl so people criticize it without even using it. People will use what everybody else is using without actually trying out the options. There's too much information and so we must go with what others have said, and it all becomes hearsay. Keeping up is more like wizardry than engineering. Go with what the crowd says because I can't possibly install all of those libraries, play with the examples, and give my own evaluation. Hey look a new js framework just came out...

What Perl did that was amazing was bring regular expressions "to the masses", and Perl-compatible regular expressions (pcre) are still the defacto standard that most subsequent libraries have used (more or less).

"The internet" is an abstraction and doesn't hate (or love) anything. That itself is the kind of gross generalization you are criticizing. And one can criticize a language and still have respect for it.

I can count the number of tool-agnostic development teams that I've met on one hand. Many more have claimed they are when they are not.

If you aren't aiming for the top 10% of jobs (vague quality metric that you can interpret as you wish), then you want to have above-average knowledge of _just_ Python, Go, React, Docker and Kubernetes.

The situation only changes when high profile current/ex-Googlers (or similar) start talking about a language/tool a lot. Then the mass hops on board that train too.

And I don't think "high-profile" people talking has much effect. Paul Graham talked up lisp for a while, and I was certainly interested (I like lisps), but... there are still precious few jobs that use Lisp. Most people learn technologies that they either need currently or are in use in jobs they know of and might conceivably get.

Paul Graham, while a thought-leader of sorts, isn't generally thought of as someone working at either the edge of tech or in large-scale systems. That's why nobody wants to chase the tech he's using vs what Google/Facebook/etc are.

There's something like necessary complexity you can't easily abstract away. I find Rust does a fine job at cleaning up syntax. I'm not a fan of snake_case. Other than that I can't think of anything that's more difficult than the underlying concept in Rust. And it's still close to C (braces and functions) and ML syntax (type after colon and variable name, let bindings) in many ways.

Especially compared with a similarly complex language like C++. Now that is scary syntax, if you're not used to it from 20 years of using C++ and developing Stockholm Syndrome.

My point above was that this decision is basically one of fashion and not technology. And the proof is how the general consensus about awk has evolved over 20 years as perl has declined.

C++ and Perl, meanwhile, both have tons and tons of syntax, such that they're 1. harder to grasp for people who haven't seen them before, and 2. harder to learn (especially by attempting to Google language features "by name.")

If there was a spectrum with Lisp [or Forth] on one end and APL on the other, Rust might be somewhere right-of-center... but it'd still be pretty far left of C++ and Perl.

Also, given the languages that occupy the ends of said spectrum, I think it should be clear that your position on said spectrum has no correspondence with "expressive power" :)

There's necessary complexity to express certain concepts, but C++ has accumulated a lot of unnecessary complexity over the years.

Rust, in terms of GC-less languages with mostly zero-cost abstractions is the simplest language that I've seen. And having a GCless language with memory safety is not just fashion. It's pretty much the greatest single advancement in language design since the GC itself.

Have you taken a serious look at a lisp? You might be pleasently surprised. Everything that can be said about lisp has probably already been said, but I'd argue that sexprs have a much higher expressive/complexity ratio than say the C++ grammar.

Somewhat relative xkcd[1]

[1] https://www.explainxkcd.com/wiki/index.php/224:_Lisp

Perl -pie is a very powerful idiom.

I would rather use a subset of Perl than awk.

Edit: a-ok, probably elegance is meant in the same sense that C is elegant by cramming everything in a single statement using pre-/post-increment operators and assignments-as-expressions

However, I've never believed it to be possible to have a language both as a shell language and a proper programming language for large-scale projects. I believe the two usecases are fundamentally antithetical, but I'd be happy to be proven wrong.

I'd say Powershell proves you right. Powershell has a great design, it has optional typing and access to a cornucopia of libraries via .NET.

Even so, they had to make some compromises because of the shell parts (functions return output, for example) which makes is quite finicky as a "proper" programming language.

On the shell side, the very nice nomenclature which makes it very readable and discoverable makes is annoying sometimes to use as a shell. That and the somewhat unwieldy launch of non-Powershell commands.

Someone who attempts to bridge the two has a ton of work to do, both in the research and in the implementation department. I guess Oil Shell (https://www.oilshell.org/) is the most realistic approach we have today. And it's probably still 1-2 years away from release and many more years from mass adoption (if that ever happens).

other nice features I'd like in awk is `tr` and `join`

That's part of why people avoid perl. It's very capable, but that wide scope is counter to the unix philosophy that prefers simple, focused utilities that can be combined in pipelines.

you could ask why have sub/gsub when there is sed... that's because you need that for specific field or string in addition to other processing.. similarly, having tr for specific string/field is useful..

I meant join as in perl's join - to construct a string out of array values with specified separator

Some examples:

* https://stackoverflow.com/questions/48920626/sort-rows-in-cs...

* https://stackoverflow.com/questions/45571828/execute-bash-co...

* https://stackoverflow.com/questions/48925359/sorting-groups-...

But now we live in a world where no one knows perl, and languages like python and javascript are clumsy and weird in this space. And that makes awk look clever and elegant.

All I'm saying is that in perl-world (which was a real place!), awk wasn't clever and elegant, it was stale and primitive. And that to me is more surprising than awk's cleverness.

And yet in 2018, I never think about Perl anymore, but use sed, awk, and grep daily.

Eventually the cycle will repeat again. The moment you will have non trivial text work to do, awk will have to give way to Perl.

The rise of Python was really use case for dealing with standard interfaces like DBs/XMLs/JSONs becoming common. Python hasn't actually replaced Perl in any meaningful way.

In the web space, Python is not huge, but it definitely supplanted the niche Perl used to have. No one I know writes web stuff in Perl anymore.

I entered the field at the tail end of the Perl era, so I've only toyed with it a long time ago.

Meanwhile, if you randomly mashed on your keyboard, it would output a perl script.

I jumped to Python as soon as I found out about it in the late 90's, because it was exactly what I was looking for in self-documenting structuring. It was great for creating parsers with state-machines. I was also the user of my scripts, and I didn't want to have to relearn what I coded a year or so earlier. Python let me pick that up, and to a lesser extent, awk. State machine programming was really self-documenting in Python.

"#awk doc shows you how to implement a rel-DB and a compiler. #perl doc talks 20p+ about nested data structures."

Who hasn't made some weird dictionary with its values being lists and forgotten to check for and write code to create it when it doesn't exist, and had to fix that later after a runtime crash? In perl that's literally covered by the one-line assignment you used to set the field you parsed.

This is why it's sad that everyone's forgotten perl.

    open (my $FILEHANDLE, '<', $file) or 
        die "Cannot open $file\n";
    while(<FILEHANDLE>) {
        chomp;
        #Do you stuff here
    }
    close(FILEHANDLE);

In fact the whole point of Perl was Larry wall reaching the upper limits of what one could do with awk, sed and other utilities and having to use them all over the place in C. Then realizing there was a use case for a whole new language.

    LINE:
    while (<>) {
	...		# your program goes here
    } continue {
	print or die "-p destination: $!\n";
    }

...

Deal with multiple files, or other advanced use cases.

I do exactly this every day with AWK. Solving exactly these kinds of problems features prominently in the AWK programming language book.

Perl is that plus more things.

And I would hereby like to remind you that every computing problem is fundamentally an input-output problem, and because of this intrinsic property, it is possible to reduce all problems in computing to input-processing-output.

Which is exactly the kind of problem AWK is designed to address.

And AWK doesn’t work with lines, it works on records, for which the fathers of the language cleverly chose the default of ‘\n’, which is reconfigurable.

Have you ever pondered the number of projects that, in addition to sh, make and common base utilities, require perl during compilation where awk could have sufficed?

As a single example, have you looked at compiling openssl without perl, using awk instead?

Whenever I see a perl prerequisite I question whether it is truly a requirement or whether other base utilities1 such as awk could replace it.

Assuming it could be removed, how much effort is one willing to expend in order to extinguish a perl dependency?

1. Some OS projects like OpenBSD make perl a base utility.

Yes, writing a build engine in AWK would be perfectly doable, but the right tool for that job is Make.

But that's not the end of it. The whole point of Perl is avoid a salad of C and shell utils. Also in many cases the moment you have to deal with >2 files at a time shell utilities begin to show their limits.

The resulting code is often far more unreadable than anything you will ever write in Perl.

This is of practical importance for portable scripts, since Debian uses mawk [1], RH ships gawk, and Mac OS uses nawk.

[1] mawk has recently received new commits by its original author after a hiatus of 20 years or so; see https://github.com/mikebrennan000

https://blog.steve.fi/if_line_noise_is_a_program__all_fuzzer...

Though sadly still unfixed:

https://bugs.debian.org/816277

Awk is stupidly confused whether it is a two-namespace or one-namespace language (think Lisp-2 vs. Lisp-1). For instance, this works:

  function x()
  {
    print "x called"
  }

  function foo(x)
  {
    x() # function x, not parameter x.
  }

That is simply not the case. x() unambiguously treats x as a function, and will fail if there is no such function, even if there is a variable x.

  $ awk 'function foo(x)
         {
           x()
         }

         BEGIN { foo(42) }'
  awk: cmd. line:2: fatal: function `x' not defined

Also not the case. The purely syntactic context distinguishes whether or not the identifier is being used as a variable or function. Awk sometimes uses it, sometimes not. This doesn't diagnose:

  function x()
  {
  }

  function foo(x)
  {
     x()   # function call allowed
     x = 3 # assignment allowed
  }       
 
  BEGIN { foo(42) }

  function foo(x)
  {
  }       
 
  BEGIN { foo = 3 }

  fatal: function `foo' called with space between name and `(',
  or used as a variable or an array

    function f(x, otherArgs) {
      r = otherArgs["x"]
      for (v in otherArgs)
        delete otherArgs[v]
      return r
    }

I have only used sed once to edit the passwd file on an early linix system when vi wasn't installed and awk never. Though having being trained in sed and awk helped with picking up Perl

FWIW I've used both Perl and Python professionally and Python rules.

Nokogiri is hands-down the best tool for dealing with XML that there is.

I have much the same experience with Python and anything significant developed in Python here ends up getting rewritten in Go.

Im not directly in the software industry though, just writing programs for data analysis in quality and safety programs. I'm sure once you can think directly in something like Go, it'd be faster to write that program first. But it decreases the cognitive load for me.

We have a bunch of Python scripts for operations work and it's rare that performance becomes a major concern (...except in/regarding Ansible). Development isn't my team's primary responsibility so this state of things is fine -- our SysEng team can grok Python pretty well, whereas with other languages I wouldn't say this is true.

Maybe Perl 6 fixes these things, but learning it is too far down on my to-do list, where it sits just below Ruby.

If I have a problem that can be solved by looping through the lines of a text file and applying some combination of regular expression matching and substitution, the split and join functions, simple arrays and hash tables, I reach for Perl 5.

The reason I personally left Perl was the extremely hostile community. When someone asked a simple question, not only would they get an overhaul and namecalling, but so would anyone that tried to help them.

Python seemed to do a much better job at onboarding new people. To me, that seems like the most important reason they won in the long run.

FWIW perl6 takes extra special effort to be nice.

I don't have links handy, but there were official blog posts and such about how they endeavored to fix the problem. Of course I'm not saying it was universal, nor trying to take away from the fact that there are a lot of great people in the Perl community (PerlMonks is a good example)

They did fix it, as far as I can tell anyway, but it was too little too late.

Did it? Not used Perl for a while but I never needed to worry about backward compatibility and versions with CPAN, while I find I do with Python (to be fair I am programming more complex stuff than I did in my Perl days).

This gave me flashbacks to installing some perl module and watching it download half of CPAN.

Or maybe, hiding out in PHP?

Steeling syntax, releasing versions bigger than 5, smoking regex benchmarks...

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

JavaScript is heading in the direction of Haskell, Scala, and (to a lesser extent) Rust, too [1].

[1]: https://medium.freecodecamp.org/here-are-three-upcoming-chan...

If you are doing it for immediate application in a job or wish to acquire a skill you think might be required in a job then the answer is probably Perl 5. Despite the noise about Perl, it is still used a lot in many industries for flow control and inter-tool format conversion as well as many other applications. Many, many people understand perl and use it for rapid development of automation tools. I can tell you for a fact that it is pervasive in the semiconductor industry and is practically a job requirement, there.

Remember that they are two different languages with some similarities. Perl6 has really good support for making big apps I would say as it has really good OO support out of the box in addition to making FP concepts easy too. Perl5 has atrocious support baked in, but they have the excellent libraries that everyone uses to give them world class support for OO. Still, it feels more natural in Perl6.

On another note, Groovy is another great and fast language that runs on the JVM and works very well in the scripting space with a lot of DSLs for GUI, XML, DB...cool stuff.

But the same versioning issue exists for Apache Groovy (2 or 3 ?) that exists for Perl (5 or 6 ?). The Groovy PMC seem to be handling the issue, though, by slowing down the development of Groovy 3 to a crawl so it won't ever ship.

Oh, and calling Groovy "fast" is a bit of a stretch. It hardly matters anyway because scripting for classes written in Java doesn't need "fast".

In many ways, p6 is the superior language. But as an AWK competitor specifically, p5 might be the better choice for performance reasons alone (while p6 has been improving, as far as regex performance goes, it just isn't there yet [1]). You might even be able to write tighter code in p5 (for one, p6 regexes return objects, not strings, so in cases where you actually want the latter, you'll have to throw in boilerplate).

[1] https://gist.github.com/cygx/9c94eefdf6300f726bc698655555d73...

    it always blows my mind that so few people really know how to use it at a decent level

One thing I wish there was better documentation on, though, is creating complete gawk scripts. With @include directives coupled with AWK_PATH, it's really convenient to build up a handful of utility functions that do frequent things and I've found, on several occasions, that I end up writing an .awk script instead of a bash/zsh script and it ends up being a much more straight-forward set of code.

[0] Well, specifically, the 'gawk' variant

Too many times I encountered 200+ line Python scripts that I could replace with an Awk one-liner and a cron job.

My lazy habit in my Ruby work eventually became to parse JSON/XML/whatever into usable text, shelling out to sed/awk and working with the results. This would not only save me LoC, but was less error-prone.

There are a few things I have done which felt more difficult than they should. However, the vast majority of dealing with structured data using awk really has been a pleasant surprise.

My favorite bit of Awk code that I ever wrote was this print formatting program for the HP LaserJet II:

https://github.com/geary/awk/blob/master/LJPII.AWK

It printed source code in a "2-up" format, with two pages printed side by side in landscape mode. It looked for the kind of separators that I and many people were fond of back in those days (the source code is a good example) and converted those to graphic boxes. And it tried to be smart about things like breaking pages before a comment box, not after.

I wish I still have a sample printout handy. For 300 lines of what I thought was fairly cleanly written Awk code, it made some nicely formatted printouts.

It's true that many programs might well just have a single action (or a single action plus a BEGIN and END), but I'm not sure that resorting to awk -f will give great offense.

if you're interested, you could give my tutorial(https://github.com/learnbyexample/Command-line-text-processi...) a try

it has over 300 examples..

if you are interested in text processing in general, do check the other chapters for grep/sed/perl/sort/paste/pr/etc

I learned a lot writing them up and marvel at the speed and functionality provided by these tools.. for many cases, it is lot easier to write a bash script combining these tools than to write a Perl/Python/Ruby script..

Extremely powerful programming language for big data processing and data driven programming, I often use it to generate shell scripts based on some arbitrary data. With no memory management and providng hash arrays, AWK is an absolute delight to program in. Using it with the functional programming paradigm makes it even more powerful. It blows my mind that Aho, Weinberger and Kernighan managed to design something so versatile and yet so small and fast. I’ve also been using it to replace Python scripts with a ratio of Python:AWK being anywhere from 3:1 to 10:1 as far as lines of code needed to get the same tasks done.

It does essentially what Javadoc does, except it uses Markdown to format text, which I find much more pleasing to the eyes when you read source code.

The benefit of doing it in Awk is that if you want to use it in your project, you can just distribute a single script with your source code and add a two lines to your Makefile. Because of the ubiquity of Awk, you never have to worry whether people building your library has the correct tools installed.

It doesn't have all the features that more sophisticated tools like Doxygen provides, but I'm going to keep on doing the documentation for my small hobby projects this way.

[1] https://github.com/wernsey/d.awk

As I recall, gawk in particular has some extensions that are explicitly a superset of the standard funcionality. Which is great if you're only targeting gawk and know about it. It's less great if you think you're only targeting gawk, and discover later on that there's a system with a different awk you need to support.

Your project looks neat, by the way. I'm looking forward to taking a closer look.

I've also been burned when I discovered that Raspbian ships with an older version of mawk that did something differently in the way it processed regexen that caused my script to break.

Oracle databases and other databases exported data in fixed width files and I had to download from several Nix systems to import into one general Nix system using Oracle and then a DOS based Clipper 5 system and an Access 2.0 Windows system and they all had to get the same results.

If not for Awk I could not filter the files from the Nix systems.

[0]https://nixos.org/

http://www.nongnu.org/txr/txr-manpage.html#N-000264BC

> "Unlike Awk, the awk macro is a robust, self-contained language feature which can be used anywhere where a TXR Lisp expression is called for, cleanly nests with itself and can produce a return value when done. By contrast, a function in the Awk language, or an action body, cannot instantiate an local Awk processing machine. "

The manual contains a translation of all of the Awk examples from the POSIX standard:

http://www.nongnu.org/txr/txr-manpage.html#N-03D16283

The (-> name form ...) syntax above is scoped to the surrounding awk macro. Like in Awk, the redirection is identified by string. If multiple such expressions appear with the same name, they denote the same stream (within the lexical scope of the awk macro instance to which they belong). These are implicitly kept in a hash table. When the macro terminates (normally or via non-local jump like an exception), these streams are all closed.

I'm not sure if it's a generational thing, but I thought that was interesting.

Anyways, are there any good resources to learn awk/sed effectively?

    for i in *.png; do pngtopnm < $i | cjpeg > `echo $i | sed 's/png$/jpeg/'`; done

  for i in in *.png ; do
    pngtopnm $i | cjpeg > ${i#.png}.jpeg
  done

I very much like awk, I prefer it over sed, because it's easy to read. Also proper man page is all one needs. But I find myself many times doing something like this:

  match($0, /regex/) {
    x = substr($0, RSTART, RLENGTH)
    if(match(x, /regex2/)) {
      ...
    } else if(match(x, /regex3/)) {
      ...

[0] http://doc.cat-v.org/bell_labs/structural_regexps/

I recently wrote a simple statistics tool using Awk to calculate median, variance, deviation, etc. and people say the code is readable and good for seeing the simplicity of Awk.

https://github.com/numcommand/num/blob/master/bin/num

https://github.com/mikebrennan000/mawk-2

In my perfect world mawk would have some of the gawk extensions, and it would have a csv reader mode to properly split csv into $1...$NF. Because that would be the killer tool.

I like to use awk when I need something a little more powerful than grep. Nevertheless, when I look at the examples and where the book is heading I prefer R for many of the tasks (in particular Rscript with a shebang).

Just to give an example: If you have to manipulate a CSV file, that would most certainly be possible with awk, but some day there might be a record which does contain the separator and your program will produce some garbage. R on the other hand comes with sophisticated algorithms to handle CSV file correct.

I truly respect awk for what it was and is but I also think that the use-cases where it is the best tool for the job has become very narrow over time.

(Can be hard to explain how this works without an image, so an (older) image is found in: https://twitter.com/smllmp/status/984173696448434176 )

The other person wrote it in awk, quite quickly. After writing my own version in Python (my version was waaaay over-engineered), I decided to blatantly rip-off the awk solution and re-implement it in Python.

It was almost as simple and as short.

Awk is much more compact as a language, but also way more limited. And it still has its quirks and a certain volume of information you have to gather. I'd say it's more worthwhile to learn Python instead, because you'll be able to use it for other purposes.

> Because it's a description of the complete language, the material is detailed, so we recommend that you skim it, then come back as necessary to check up on details.

Any book that recommends skimming is doing something right.

I don't have any references, sorry.

Why would one do that? It's destructive. A typeset book is much more than a text file.

Not to mention OCR is not perfect, especially when math / special symbols are involved.

For latest manual/book: https://www.gnu.org/software/gawk/manual/

And by that I mean sometimes it seems easy to solve a problem because you have the skill to do so. It looks easy but that's only because of the time invested in making it easy for you. For anyone else the challenge remains.

1. Run zero or more setup operations.

2. Loop over the lines of a text file and process its columns into an output format.

3. Run zero or more cleanup operations at the end.

You can feed in regexps and c code fragments and it will generate c code for you.

[1] https://www.tldp.org/HOWTO/Lex-YACC-HOWTO-3.html

  This is the kind of job that awk is meant for, so it's easy. Just type this command line:
  awk '$3 > 0 { print $1, $2 * $3 }' emp.data

https://news.ycombinator.com/item?id=14647022

http://i.imgur.com/e11d0aK.png

http://i.imgur.com/0Ysr7QQ.png

Look at the kerning on "Awk", it's not good. And look at the zoomed in version, the characters all have pixelisation and jaggies.

These were just viewed using Firefox's default pdf viewer. Is there a way to view them and see better a quality version of the document?

I don't get the Perl hate. Perl's unpopularity may have something to do with some of the languages design choices. I think what really killed it was Perl coders. Some of the worst code I've seen happened to be written in Perl. If you follow clean code principles Perl is fine. Mojolicious is an awesome framework. I like it a lot.

Today I code Python and C. I used to code Ruby and before that Perl. I loved Ruby's syntax but Ruby seems to be waning. I'm looking forward to coding in Go. I'll be coding Javascript but I'm not looking forward to it.

Use the tool that fits the job. I have no loyalties to any programming language.

Is a myth actually.

> Tom Radcliffe, recently presented a talk at YAPC North America titled “The Perl Paradox.” This concept refers to the fact that Perl has become virtually invisible in recent years while remaining one of the most important and critical modern programming languages. Little, if any, media attention has been paid to it, despite being ubiquitous in the backrooms of the enterprise.

> Yet at ActiveState, we have seen our Perl business continue to grow and thrive. Increasingly, our customers tell us that not only are they using more Perl but they’re doing more sophisticated things with it. Perl itself recently made it back into the Top 10 of the Tiobe rankings, and it remains one of the highest paying technologies. Therein lies the paradox.

https://www.activestate.com/blog/2016/07/perl-paradox

Similarly, I don't expect to see many jobs openings for wrenchers, but I fully expect a mechanic being hired someplace to be able to use a wrench.

That was exactly seen some twenty years ago. Perl as used to build infrastructure, like entire back-ends for sites and whatever.

You don't see those jobs anymore.

Try https://perl.careers/ for example.

I first heard of them from this slide deck that any developer, regardless of tech stack, should read: https://de.slideshare.net/perlcareers/how-to-write-a-develop...

Agreed -- I use awk all the time in shell pipeline.

> I don't get the Perl hate.

The problem is not with writing Perl -- I don't mind using Perl to write scripts and tools. In fact, I like it better than Python for writing glue code.

Reading Perl code -- especially if the code base is old and has been worked upon by multiple people -- now that is a real pain. Don't get me wrong, I have seen some really well written Perl code and have had the good fortune of working with some really smart Perl programmers. But the majority of Perl code I have encountered has been unreadable mess that makes me want to pull my hair out. There are times when I feel it is more productive to rewrite the code in Python than to spend time on the existing code.

> If you follow clean code principles Perl is fine.

Agreed -- except most Perl coders don't. Worse, they make a large chunk of people who contributed to CPAN -- something which was one of the major reasons behind the popularity of Perl.

> Use the tool that fits the job. I have no loyalties to any programming language.

Couldn't agree more.

That is funny, because one of objectives of OOP was to give code a better structure and therefore make it easier to understand.

I used to spend up to 40 hours in a Perl debugger trying to figure out what the program is doing, and after going 25 layers deep into the call stack, I'd come out one week later none the wiser as to what that damn spaghetti code was doing. Tracking down any bug would always turn into debugging of epic proportions. I never had such problems debugging machine code!

That is why Perl gets so much hate.

My favorite construct to hate was going through the logic and just as I was about finished trying to understand the state machine at that point hitting an unless{} clause which instantaneously wipes the slate clean. Oh how I hate Perl.

Second that. Worked with a guy who used (and adored) Perl for more than 15 years. His C++ code was so incomprehensible I had to rewrite many things in my spare time.

I would say hate is a strong word but I have 2 issues with Perl:

1) Regex choices made

2) Readability, if someone wrote something in Perl I normally would rewrite it if it took me less time then figuring out what they wrote and wasn't working as expected. Maybe it was just my poor skills but man Perl can be hard to tell what is actually going on.

There is a reason companies like Netflix, and Walmart are heavily invested in using Node.js for their internal services.

Really gets down to personal taste now more than it did in the past, but if a tool works, does the job and the alternatives don't offer any gains, then they stick with that. Not saying the alternatives are bad or worse in any way, or indeed better, they are just different and in some cases, maybe better. At least the core reason of them always being there instead of having to add another dependency and risk factor to a system have become moot these days.

TL;DR some older tools more guaranteed to be upon all systems as a lowest common denominator in the older days than now and legacy always outlives the machine, hence we still run COBOL today as it works, maybe better solutions but rebuilding rome overnight still avoided.

My favorite OS, FreeBSD, comes with neither bash, nor emacs, nor Perl, so I don’t consider them part of the “lowest common denominator” set even in 2018.

(Funnily enough, it does come with a C++ compiler!)