On Linux, after argc, argv and envp comes the even more mysterious auxv, a key-value store for binary data. The kernel shoves a lot of interesting stuff into auxv, including AT_RANDOM - 16 random bytes (used to construct stack canaries and function pointer encryption keys), AT_HWCAP (processor capability flags), and AT_SECURE (a flag specifying if the program is setuid and therefore security sensitive). Although a lot of it is meant just for internal C library use, it can be helpful for programmers too (e.g. being able to check hardware capabilities without a trip to cpuid).
This is true for ELF, not just Linux. And it need not be true for a.out (IIRC it's not true for a.out executables on Linux or any other Unix or Unix-like OS that supports a.out executable formats).
There are a few auxiliary vector keys that are standard per-ELF, and others that are platform-specific. The original purpose of the auxv was to provide the ELF interpreter (ld.so) with information it needs in order to bootstrap.
> IIRC it's not true for a.out executables on Linux
I was surprised to discover a.out support is still there in the kernel [1]. There has been talk on LKML about whether to remove it [2] – apparently it has accumulated regression bugs (particularly in the area of core dumping) which nobody has noticed and fixed, which is the inevitable fate of code that hangs around for decades after everybody stops using it – but it still hasn't been removed yet.
GDB also uses it to figure out where the binary is slid in memory (it reads /proc/pid/auxv, though, not the stack). AT_HWCAP is particularly useful on ARM because as far as I understand the feature registers are normally not readable by userspace and this is probably the easiest way to grab it on Linux.
Huh - I remember using `char env` as the third parameter to main. After a quick Googling, it would seem that's a GCC-specific thing. Surprised there's a Linux-specific (or ELF-specific, as another commented) convention other than that one.
It isn't specific to GCC at all. It's widespread among unixes historically (and they generally have their own C compiler which is not GCC), and it was mentioned in the C89 standard.
However, it is not part of standard C or POSIX/SUS.
It is nice to see some BSD asm for a change (I know, someone will probably say this is about C, not asm). If one is hoping to learn anything about assembly language from reading the internet, most material found will focus on Linux or Windows. Even the HN comments on a blog post about BSD asm are about Linux.
Modern Python lets you explicitly opt out of this by putting “/“ in your argument list iirc correctly:
def only_positional(a, b, /):
Pass
Similarly, you can prevent keyword args from being called as positional (which has the inverse problem of breaking client code by changing the order of keyword arguments):
The first feature pretty much only exists to make interop with C apis less error prone. It's not really something that should be used in idiomatic Python
> The first feature pretty much only exists to make interop with C apis less error prone.
It makes it so that certain built-ins and native APIs can be properly typed, sure, but positional/keyword interchangeability is arguably a violation of the Zen, too.
I disagree for the exact reason the thread here started: it’s part of your public API. If you don’t want keyword arguments in your public API contract, you absolutely should omit them from the contract.
I wonder if Python should still be the go to language for teaching programming. It's pretty simple and reads like English but I feel like if you can get over a few syntax bumps, Go is a lot easier to understand and is much more straightforward compared to Python which has a lot of quirks (like why can't spaces/tabs be mixed if they look the same on the editor, why are there so many different ways to write a loop)...
The only thing that might trip up beginners in Go is error handling but if they haven't learned any other way of doing it before then they might just accept it for what it is.
Go has a lot of quirks on things that matter far more. Like, arrays and slices, and how to use them as collections, is very much non-trivial compared to Python lists.
On the other hand, stuff like tabs vs spaces - when you would expect this to even come up? Whatever editor they'd use to start coding with, it'll be configured to use one or the other. In fact, these days, unless you get something exotic, it'll be spaces for sure.
And Python has exactly two ways to write a loop: one for conditional looping, and another when you want to iterate over a collection or a range. Is that really too many?
With tabs vs spaces stuff, the problem is that it's not exactly something that you can teach (like you said the editor will handle it for you). Then later if a beginner tries doing something on their own and runs into that error, it's not exactly intuitive how to fix it and they run into a brick wall that can't be solved no matter how much they look at their code.
C++11 is probably as close as you can get to something that's simple enough and widely used but also supports diving into how computers work because you will eventually have to manage your own memory (although I'm not sure if memory management is an important skill for someone just starting to learn how to code).
For a real deep understanding, you'd probably have to jump into the actual computer microchip architecture to learn how computers store memory and do things like pipelining, branch prediction, and caching. Then a primer on Operating Systems (I like "Three Easy Pieces") should fill in the gaps.
A bug is a problem that affects your users. If you're a library author then your users are developers of applications (and higher-level libraries), so an unintentional breaking change to your API is definitely a bug.
No compiler in Python. In combination with features like varargs, kwargs, unpacking dictionary into args, argument forwarding Python allows you to turn this into bugs.
Well, Python does get compiled to bytecode before it is run. And the compiler does tell you about some errors, like inconsistent indentation, even if that code is never executed.
Another example is Angular.js It used Javascript's toString() method to get the names of the arguments and then did dependency injection magic based on it.
Does anyone know if that is still the case for recent versions of Angular?
The name of method parameters in obj-c is not significant. The argument label and argument name are different things. In `- (void)frobulateThing:(id)firstThing andAlso:(id)secondThing;`, `firstThing` and `secondThing` are the argument names and can be changed with no impact on the callers, while `frobulateThing:andAlso:` is the part relevant to callers.
In Swift the argument labels are the same as the argument names by default, but you can also supply different values for them.
Java (sometimes) where reflection allows you to inspect the parameter names, and you can optionally retain the names in the compiled bytecode.
E.g. these days the Spring framework defaults to using the parameter name as the qualifying name to discriminate between otherwise identical types during autowiring.
Since it hasn't been mentioned yet, C#/.NET also has this dumb feature - the name of the parameter is compiled into the dll, and can be used for 'named arguments', where the caller can write `a: 20` in the argument list to assign the parameter called `a` to 20. If the library developer renames the parameter to something other than `a`, than that old code won't compile anymore (Though AFAIK the old compiled code is still ABI compatible with the new version, since the method arguments are resolved at compile time).
Named arguments in much of the Rails API are received by the method via a single hash parameter, not as individual lexically bound variables.
In most cases, Ruby handles this capture via a double-splat operator in the parameter list.
However, Ruby itself does have both positional and named method parameters. The former contribute to arity and may be overridden and renamed in subclasses, and this remains true for Rails methods accepting positional arguments.
It is also the case that a hash is implicitly congruent to named arguments, although this is controversial and may change in future. I’m sorry to report that no-one has ever described this as “arg hash” or “argh” for short.
Me too, I once knew it but looked it up again earlier today out of curiosity. I was actually quite surprised to see this thread popup on HN a few hours later!
I needed to understand linkers and object loaders for something at work recently, and I can't recommend Computer Systems: A Programmer's Perspective enough![1] It answered a lot of questions I'd always had about computer systems.
I jumped straight into Chapter 7: Linking and learned so much that I read a bunch of other chapters for the fun of it. E.g., Chapter 7 covered that the `_start` symbol mentioned in this post is just a symbol in your executable that the Linux executable loader knows to jump to, but it's not the only special symbol!
The chapter on linking also covered how loading and linking dynamic/shared libraries works, which was also really cool. I wrote up some of the things I learned about that stuff too.[2]
One neat thing I learned is that `argv` and `envp` are contiguous on Linux. You can change the process name that appears in `ps` by modifying memory that `argv` elements point to. If you need more space, you can also skip NULL-terminating `argv` so that it will read on into `envp`.
Modifying argv is a classic trick to implement an "error log of last resort" - How can you log an error message when the I/O itself has failed (e.g. the disk is full)? You can write you message to argv. A sysadmin running ps can quickly spot a "FATAL ERROR OCCURRED" argument appears next to the program name. Famously used by DJB in his mail server.
There are OSes where altering `argv[0]` changes the ps strings, and ones where it doesn't. Arranging to support this is tricky, as it can be a way to attack users of ps(1) and /proc!
Fun fact: iTerm recently had to redesign some of their APIs because the process name APIs would be susceptible to a malicious program overwriting its argv (the normal API for this has the kernel read out of the process's address space).
I can't find a reference for DEC's PDP processors, but the 68000 (Motorola) processor family had LINK and UNLINK instructions useful for constructing a (calling) stack frame. DEC's later VAX (Virtual Address eXtension) processor line included stack frames baked into the silicon. The CALLS instruction was called with the number of 32 bit arguments previously pushed onto the stack; the CALLG instruction was called with a pointer to a memory structure consisting of an argument count and an argument vector.
From the foregoing progression it can be inferred that this idiom was common and deemed a Good Thing by some faction in computer science of the time.
As for functions with specific names, when linking against libraries there are typically symbol tables for resolving references between separately compiled modules.
As a self-professed (I took out a small display ad in the print publications _Computer World_ and _Asian Computing Monthly_ in 1984) "VAX Hacker for Hire", I made use of C's utter lack of concern for such things by declaring a char* arg and using pointer arithmetic to get the actual count of args to implement optional parameters; I wasn't the only one. Between that and abusing the REF and VAL pragmas in pretty much all DEC programming languages, it was good times.
1) My recollection was that _Computer World_ was around $400 for two insertions, and _Asian Computer Monthly_ was slightly less for 6. They were small ads.
2) Yes, I did make the acquaintance of a local (!) company, which I ended up having a 10 year relationship with. I also had a few of entertaining inquiries, none of which got me in trouble with the law, but also didn't make me any money. (Know when to say "no".)
It was in 1984. ;-) For comparison, in 2018 I took out an ad in the _Yakima Herald_ prospecting for some ag tech work and didn't get a single nibble; that ad cost me over $1000 for 4 insertions.
I can see why they did it that way. No reason to waste cycles on something if you do not need it. Though most frameworks go and call it for you anyway these days.
I'd also add that CommandLineToArgvW doesn't do exactly the same thing that the C runtime does to parse arguments. They used to be the same but the C runtime was tweaked in 2008 whereas CommandLineToArgvW was kept the same for backwards compatibility. See http://daviddeley.com/autohotkey/parameters/parameters.htm#W...
Presumably they would have tested extensively for regressions. But I'm not sure what would break that wasn't already broken. It was an obscure undocumented case that was not intuitive.
You can also set what it is using that compiler option if you want. You usually do not want to as the default versions do quite a bit of stuff. I think the winmain one was the 3.x default. But changed when they added in other bits (I want to say threading, and console setup but I could be wrong).
Windows is a weird one as it has this odd DOS/VMS/OS2/Win16/32 heritage so entry into the program is one that has some bagage. Winmain is one of those leftovers. The params passed in need to use the functions associated to them.
My advice is always with this stuff. read the docs. There is a ton of info in there. It is kind of picky to get just right.
Even in a modern Unix, globs are expanded by the shell before invoking exec(3) with the individual arguments split out. They are not split/expanded by the C runtime.
‘Implement’ isn’t necessary. That code could be provided by a shared library.
That could be the superior option. Moving command line expansion into a shared library gets rid of the problem where the command line with globs expanded is too long, could have standardized option handling earlier (that system-provided library for globbing could easily grow to handle option flags), could better handle file names containing spaces, etc.
The shell can do whatever processing it likes but at the end of the day all that's passed to the application is a string. The string may or may not be what a user typed but the application has no knowledge of that.
The shell is whatever you use as a shell. Be that cmd, powershell or your own invention. My point is that it doesn't matter. The shell can process the command line however it likes but all the application receives is a single string.
The application can then do whatever it likes with the string. Sure you can use an entry point that parses the arguments in different ways but that's still the application parsing the string. You're just using a library function to do the parsing instead of doing it manually.
At no point does "Windows" do anything. As far as its concerned it's just a string being passed in at process creation.
At least on the toolchains I'm familiar with, the linker will insert crt0.o, a very basic part of the C runtime, which contains a _start implementation that pulls argc, argv, and envp from the stack:
Interestingly, macOS doesn't provide a crt0.o: execution begins in dyld for most programs, which calls the normal program entrypoint (usually main). Linux will do this too if you are dynamically linked.
* /usr/lib/x86_64-linux-gnu/crt1.o -- This file contains the "beginning" of your executable, and the definition of _start (the entry point to your entire application).
* /usr/lib/x86_64-linux-gnu/crti.o -- This contains the beginning of the .init and .fini instructions, which include the glue code for calling static initializers and destructors. This is more specifically linker magic to trigger ELF magic headers.
* /usr/lib/gcc/x86_64-linux-gnu/9/crtbegin.o -- More glue code for static initializers and destructors. There are a few different approaches to implementing static initializers that varies on a per-compiler basis.
* /tmp/test-4e14b0.o -- Your code! When you specify multiple object files, or multiple libraries, all of those links get dropped in this location.
* -lgcc, -lgcc_s -- These are compiler runtime libraries to handle supporting native C/C++ operations that are not implemented in hardware. For example, should you do a 64-bit division operation on an architecture that doesn't have such an instruction, the compiler generates a call to a library function implemented in these libraries.
* -lc -- This is the C runtime library. Here you may find the implementation of printf, for example.
* -lgcc, -lgcc_s (again!) -- The C runtime library's implementation may need to call out to these compiler-generated function calls, so include the libraries again in the search path because legacy linker order means you have to.
* /usr/lib/gcc/x86_64-linux-gnu/9/crtend.o -- This is the counterpart to crtbegin.o. Whereas crtbegin.o includes some stuff that will happen before all the user code is included, this includes some stuff that will happen afterwards.
* /usr/lib/x86_64-linux-gnu/crtn.o -- Ditto, but for crti.o instead.
On a modern glibc system, the linker will arrange for the entry point to be included in _start, defined in crt1.o. The Linux ABI formally specifies how the register state and stack will be arranged when you load a program image, and _start will translate this ABI into a function call invocation to __libc_start_main. __libc_start_main is implemented in libc.so, and will handle calling global initializers and the like before calling the main function implemented in your program.
In practice, most modern executables are dynamic executables, which means that executing your program means the Linux kernel actually executes another program instead (/lib64/ld-linux-x86-64.so.2 for x86-64), which will take on the burden of loading your program and then calling _start itself. This is to support loading shared libraries (such as the C library!) and implementing relocations, which the Linux kernel does not know how to do.
All of this tends to fall under the purview of linking and loading, which is generally not handled by compiler writers. Even most new programming languages tend to find it easier to link their programs using your platform's C compiler because it's nasty to worry about all of these details yourself.
They're part of the language specification - the main function has different semantics to a normal function in a few ways - so the compiler must handle it to some extent.
Not true. The kernel & shell makes the values available on the stack and the runtime can choose to link to them or not. The ABI also details how and where the values should be placed.
Yes true. Check 5.1.2.2.1 and 5.1.2.2.3 in the standard.
If you don't treat 'main' specially you won't have a conforming C compiler. It's unique in accepting different signatures, and for its return value syntax and semantics.
You cannot just compile it as a normal function and link it with a runtime routine. You need compiler support for it.
A fun fact related to the other points: you can legally call main in C, but not in C++ (it will usually work, though). Another fun fact is that on some platforms (Linux, macOS) you can "dynamically load" a (dynamically linked) program by calling dlopen on it and using dlsym to grab the address of main and just calling it :)
Good point. That makes it relatively difficult (not impossible) to compile `main` differently from other functions, other than diagnosing bad signatures if that's what the hosted environment requires, since it has to be callable like other functions.
There are probably implementations that require compiler support, but it is not required by the standard and not true of many common implementations. Support is by the caller, as described in others' parallel comments.
> You cannot just compile it as a normal function and link it with a runtime routine.
You can. As a concrete example, for the following two programs,
int maim(int argc, char *argv[]) { return 0; }
int main(int argc, char *argv[]) { return 0; }
the assembly code produced by gcc 9.3 x64 is identical except for the spelling of the identifier.
int maim(int argc, char *argv[]) { }
int main(int argc, char *argv[]) { }
They compile to different code. Can you figure out why?
A function called main is treated differently in this regard to other functions, which is enforced by the standard, and so has to be implemented by the compiler.
Damn, you're right: Currently, “reaching the } that terminates the main function returns a value of 0”. In my defense, that was different in C89, which was the last one I worked on implementing and knew completely: “If the main function executes a return that specifies no value, the termination status returned to the host environment is undefined” and “Reaching the } that terminates a function is equivalent to
executing a return statement without an expression.”
This is the C Standard specifying its VM to match the assumptions of a *nix environment. `main` isn't so special, IMO. Conformance to the C Standard is always optional, see C++, golang, Rust. However, conformance to ABI is less optional.
Technically, local C ABI may be callee-cleanup, and then main() must be special in that it's neither variadic nor fixed-prototype. But in a major compatible reality it's always cdecl, thus not much different from foo(void) being called with foo(int, ptr, ptr) via a prototype mismatch which a compiler is surely unable to detect (that's UB for that exact non-existent ABI reason, as it is for almost half of C). It is only special for a return statement really.
Note that on macOS there is an additional fourth argument to main, char *apple[] (I would write this as a double pointer, but Hacker News would eat it if I did). It contains things passed on from the kernel mostly on relevant to libc and dyld.
What's the point of this?
If you are writing a C program, get a C book and learn all about argc and argv.
Of you know a good C book, post about it to HN.
If your aren't writing a C program, this is irrelevant.
Why does HN every day pull a random page from a manual and vote it to the front page? This is a terrible way to learn anything.
This is the best way though to meet some guys who never wrote a book, but are competent enough to delve into details that barely any book mentions. If you want to know one thing deep, sure, get a book or two. But if you want to create your understanding on numerous topics (wide learning), your way is a very long journey that might not even end before you retire. I don't think someone learns here as in a class, we just discuss matters of our interests. You start with replying to tfa, and then it turns more and more whatever it turns into. Nothing new for a forum-like, really, many do not even need clickable headlines, since they do not click on them.
