Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist.

Just as the author thinks it's clunky to return HTTP error codes representing application errors, I think it's clunky to apply application logic to HTTP method semantics. GET, POST, and DELETE were designed as instructions telling a web server how to handle static-ish content, and it shows in their design. Why would a GET have a body? It's a request for the server to return a specific file. This leads to breaking REST standards - for example, search endpoints that logically are GET calls (they return entities), but are implemented as POST methods because the search criteria is complex enough that you wanted a request body. Or bulk delete calls that are similarly implemented as POST methods.

REST works best in a "rules are meant to be broken" manner in my opinion. It's not a bad system (which is why it is so common), but mixing application logic with HTTP transport logic does lead to oddities.

There have been some attempts to extend the list of http verbs to include something that fits those use cases more naturally - SEARCH and QUERY. QUERY got more traction, if I remember correctly, but I haven't heard much about it in a while.

https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Ver...

That’s probably part of the reason the more general one proposed for HTTP is “QUERY” and not “SEARCH”. Also, WebDAV, in following its apparent design philosophy of “more is more”, has multiple different GET-with-(required or optional)-body methods with different purposes: PROPFIND and REPORT as well as SEARCH.

A GET request to that path should return the current state of the resource at the path. As you note, the current state of that entity is that it does not exist.

Let's look at the spec:

>200 OK - The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

> GET an entity corresponding to the requested resource is sent in the response;

Did the request succeed?

Yes, we found the current state of the entity so we return 200

What do we return?

"An entity corresponding to the requested resource". In this case however we want to represent an entity that doesn't exist.

There is no resource at the path.

> As you note, the current state of that entity is that it does not exist.

A thing which does not exist does not have state. Existence (and even moreso non-existence) isn’t a state, existence is logically presupposed in any description of state.

Then please explain how you can write and I can understand the phrase "A thing which does not exist".

Who said you shouldn't? The shorthand is convenient. But it doesn't mean what other statements with the same shape mean. A thing which isn't doesn't have a state or representation.

Many online REST discussions focus too much on shallow linguistic analogies with verbs/nouns.

If you were doing a static website serving from a file system and said the same thing about a filename as a path component, you'd be seen as mad. But there is no difference when there is an id for a potential employee than a name for a potential file: if no employee/file exists when the server looks for the id/name, the correct result is 404 Not Found.

The id isn't the resource, it's part of the path to the resource (and if it is the resource in that URL, you need a different scheme for when you need to find the employee pointed to by the id and not info about the id.)

The content of the file `/api/v1/employees/100` need to be defined off-band somehow since you cant actually put an employee on disk.

moreover part of REST is that you are not responding with static files by path, but with representations of entities; the question then is whether the entity is the employee or the id.

I agree that `/api/v1/employees/100` is not the best name for the id entity, something like `/api/v1/employee-id-status/100` might be more descriptive.

In "true" REST, if you have a "search" endpoint, POSTing to it should create to a resource, which could be a reference to the list of results. The created resource should have its own endpoint (something like `/search/results/:query_id`), and the client could then cache it.

If you don't want to do that, but still want to say you are really following REST, you could have an endpoint to represent your resources (`/products`) and use GET with filter parameters in the query string (`/products?search=my+search+term`)

This is _specifically_ about HTTP APIs. REST is not a synonym for HTTP but there are much better resources out there that rant on about the important of hypertext and URL support etc.

This is mostly about the perspective of consumers.

>It does not matter too much which part of the URL is the one that is causing the URL to be wrong.

Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing. My argument is 404 is inappropriate because the web service exists, but the record doesn't.

> Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources.

I can't say this is wrong, but it doesn't feel right. `/api/v11` straight up resolves to nowhere. Maybe this an instance where Gone is better than Not Found?

> Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example.

100% agreed. It's just a thought I had while debating with my team.

I posted this here for this exact kind of feedback :D good points raised

So I will focus on HTTP then.

Say you install an nginx/apache and then have a static structure where you have the profiles of employees saved as PDFs directly on the disk.

Then you do `GET /public/v1/employees/1.pdf` and it works returns 200 with the content. Then you do `GET /public/v11/employees/1.pdf` in this case all servers will return 404. The same goes for `GET /public/v1/employees/100.pdf` will still be 404. What if someone asks for `GET /public/v1/employeea/1.pdf` the server will again respond with 404.

Then I go and I implement an webapp to replace that. I plan to keep the URLs the same but now there is an app that will return the .PDF as a datastream or file.

For me, I don't see any reason why to change the behaviour of the URLs just because I replaced a static app with a dynamic web app. Any HTTP server will respond in the same way thus the current one should respond the same.

Responding like this has a compatibility (let's say) reason behind that is not a personal nor related specifically to my project.

It’s worth being redundantly explicit that this does not extend to all cases. There are cases where a 204 is warranted. But I’m roughly convinced that it may not be as ubiquitous as I thought. Very rad.

>Then you do `GET /public/v1/employees/1.pdf` and it works returns 200 with the content. Then you do `GET /public/v11/employees/1.pdf` in this case all servers will return 404.

Which is a sensible default because the most nginx/apache can conclude is that the resource does not exist on the server. However, if we know that this server is the canonical record for these pdfs we can conclude that it doesn't exist if it's not on the server. So now we know the state (it doesn't exist) and can return its representation.

What does 200 means with regards to existence/non-existence.

I think 200 means something exists and 404 is the representation of non-existence.

You think (please correct me if I am wrong) that the existence or non-existence information should be in the body.

Actually I think the underlying (and more important point) is about how valuable the existence/non-existence information is for the client => how quick the client should have feedback about this?

I think existence is a very important information and thus should be a first class citizen of the data representation. Thus I want it in the status code on the same level with the body itself.

If you put it in the body then that means for me an extra step to parse the body and then see what is there. So then the existence/non-existence is on the second level.

In your case with responding with 200 + body then the 200 status becomes irrelevant and I always need to parse the body => time is lost to access the _first_ important information that should then guard my business logic to parse or not the body.

While in my case (using 200 and 404 status) the client receiving 404 knows directly (without any parsing of the body) that the request was not successful in retrieving existing data.

But that's not what the spec says:

>200 OK - The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

> GET an entity corresponding to the requested resource is sent in the response;

>The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

404 is not the representation of non-existence. It's the representation of not found. Something can be "not found" for many more reasons than non-existence. Which ultimately causes the person integrating your API much consternation because they have no idea if it's a "Everything worked" 404 or "My DNS is borked" 404 or "Your server's routing is borked" 404 or half a dozen other possibilities. Sure, you might add further information to your 404 response but that means you can't have generic 404=bad monitoring. Plus causes headaches for people that are working in systems that do assume 404=bad.

200 means that the request has succeeded. And in these cases it has. You requested a representation of employee 100 and you're getting one (it doesn't exist).

Even if you disagree with the word smithing the latter is far far easier to work with.

Originally, anything with a path was meant to simulate a directory tree of static files. We build it dynamically because that's easier to maintain. But making it look and act the same by returning 404s is historically correct.

Of course things evolve and move on. You're free to do as you wish. But to me you're making a bizarrely arbitrary distinction about what part of the application is allowed to return a 404 (routing code in the framework) and what aren't (your own code). Or did you not realize that a framework like Django isn't actually part of the webserver?

Your article is entitled "I've been abusing HTTP status codes" ... but... you're not "abusing" them, you're "not using" them for your APIs. (Or, said another way, you're leaving them to their normal usage for HTTP servers.)

Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST even if you avoided mentioning it.

...

Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".

It's doubly not. The REST Architectural style is (1) protocol neutral, rather than specific to HTTP and (2) emphasizes using the underlying protocol, whatever “as is”.

> Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".

But those are literally the same thing. A URL/URI is a “Uniform Resource Locator/Identifier”.

“I don't have a matching resource” is a 404 (unless you are distinguishing “I had a matching resource but you missed it and it's not coming back”, which is 410.) While you might use a body message to distinguish “I would never expect to have a resource with that shape URL” from “I have resources with URLs shaped like that, but not that particular URL”, both are within the usual, RFC-defined meaning of the 404 status code.

If you hit an endpoint and get a 404... did you do it wrong? Is my documentation outdated?

Even better: What recourse do you have? how do you figure out the answer?

Your only recourse is to email me. Send me cURL commands and screenshots and sit on your thumbs until I write back.

IMHO the REST folk were blinded by the existing 404 normalcy set up by web servers.

...

personally I think OP's idea isn't great. I think returning 200 and making me parse the response and hoping it's consistent between services is too much work compared to the simplicity of the HTTP response.

Instead, I'd change the default from 404 to 501.

    HTTP 501 - not implemented (URI is not working)
    HTTP 404 - resource not found (Joe doesn't exist in db)

So instead of

    GET /api/v1/employee/100
    Accept: application/json

    GetEmployee /1000
    X-API-VERSION: 1
    Accept: application/json

If making an RPC and restricting yourself to the known HTTP methods, the closest is

    GET /api/v1/employee?id=100
    Accept: application/json

Sure, you might want information in addition to that provided by the status code. And, again, rather than reinventing the wheel with some ad hoc mechanism, you can follow the HTTP spec for a solution: almost all HTTP status codes support a response body to communicate additional detail.

> Instead, I'd change the default from 404 to 501.

5xx errors indicate server problems, not request problems. If you wanted a different status code for “that path isn't structured in a way I understand” vs “I understand how I would look up something with that path but can't find it“, 400 or 421 for the former would be better than anything that is not a 4xx since they each (1) are in the correct class and reports a client error, and (2) have a definition which arguably fits the scenario, even if 404 arguably fits better.

The main "issue" is that 404 is the normalized response for web servers when an endpoint doesn't exist. So it feels like one is breaking the established paradigm by using something else, but I think it's absolutely worth doing.

Certainly a bigger fan of defaulting to returning a 409 than making my API consumers parse all my response bodies.

Oof, that's a hell of a good point. So much for that plan lol

> Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted". Seems like, for REST, you'd want to return a 400 (malformed request) or something if your URL was borked rather than overloading 404.

Yup, that's the headache I'm trying to muddle my way through.

Really it's less "this is how to build APIs" and more "have you considered your consumer when you return data?". But I think even in that context your point stands better.

Back to the drawing board it seems.

At least I can generate more content now :D

This is most appropriate when the URL does not make sense with the current state of the server, but other future operations could (in theory) change the server state such that it does make sense. This might make sense if you have a user-extensible data model, where some kind of relationship is being mapped into the path structure and you want to signal that this relationship is not currently known to the query system, but _could be_ in the future.

Now, you are faced with a decision for when this ephemeral status is a semantic conflict, when it is simply a resource not found, or when it is a forbidden request for the current user.

The last is subtle and depends on other security posture. Do you want to tell your user "this is possible/available for a sufficiently privileged user, just not for you" or do you want to avoid leaking information about higher privilege roles? This is similar to the debate about whether a login UX should tell you whether you have an invalid user id versus password or just say login failed without leaking more information to a potential attacker.

Why?

How often does this really come up?

Who is typing in URLs like this manually?

If you're typo'ing it in code, are you not doing any kind of validation/testing against any kind of spec that can catch this?

Why is it up to the actual webservice returning a 404 to catch these kind of typo errors?

And I'm not saying I disagree with the argument -- I fully get the argument that was made, but practically the fact that you're caring about it suggests you're missing other components in your stack. You're producing a URL request which is outside of the spec of legal URLs for the webservice. You can validate that before you ever make a real web request against a real server.

You can’t ignore the http verbs, so your article doesn’t make sens. You also shouldn’t ignore status codes.

You’ll also might get in trouble with caching.

You can easily use status codes, and provide all the detail + status field in the response. It makes consuming a lot easier

Gone means a resource identified by a URI existed, it no longer exists, and that resource (not the URI, necessarily) will never be available again.

The main idea of the article is exactly that he does not agree that http error codes be used for application level errors (as REST principles recommend).

I think at this point that ship has sailed. Most HTTP based API will use http error codes in various ways. I would be surprised to not receive a 404 when requesting a resource ID that does not exist for example when consuming a new API.

If you were talking about TCP/IP, then perhaps - because much of the internet's infrastructure is hard-coded (even burned into silicon) to use current standards. But application semantics aren't frozen in stone by any means.

Servers and clients built atop HTTP are still being written. Why not adopt better patterns that apply properly separate semantics for each layer?

I liked the REST ideas when they came out, particularly because they provided a much better and simpler alternative to SOAP. But I think improving patterns where we can is always a good idea.

Either way, a proper api doc/spec should make either approach a non-issue.

Personnally, I've switched to graphql where it makes sense. Application level error codes and handled on the graphql layer, not on the HTTP layer, so you could say I've adopted that approach!

This is actually more than a little scary.

If you're configuring the API client, and typo the base URL, your next automation run could decide that no resources exist anymore, and delete all kinds of things from your database.

I would really recommend designing APIs that communicate more than just the 404, and writing clients that check that extra part, to differentiate between "something is wrong" and "you requested item of type X from api Y, and that does not exist". If the client got a response that doesn't explicitly state X and Y, it should assume misconfiguration and give a more general error.

The response body could be used for this distinction, if you want to stick with 404.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

>Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid.

But then we've lost standardization which is the whole point of the error codes to begin with.

Returning 200 and then using response body to denote missing resource is no different. So you have to choose, either 404 can be invalid path and missing employee; or 200 can be valid data or missing employee. Personally I would prefer the former as 200/OK then indicates success.

404 Not found is not exactly correct. I argue that the server found a representation of the object with that representation being "It does not exist".

If you try to request a file `/public/pdfs/100.pdf` and that does not exists what does the server respond? 404

What the server responds if you try `/public/dpdfs/1.pdf`? Still 404 as that path does not exists on the local storage.

What is the difference for a client if 100.pdf should be an actually file or a data stream served from a web framework? There should be no difference.

Choosing to behave when building a dynamic app the same way as the static helps a lot with integrating with multiple other services (eg. caching, observability ...)

>The 200 (OK) status code indicates that the request has succeeded. The payload sent in a 200 response depends on the request method. For the methods defined by this specification, the intended meaning of the payload can be summarized as:

>GET a representation of the target resource;

>The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

If you to give the client the representation of the target resource (i.e. it doesn't exist) then send 200 and a body indicating it doesn't exist.

If you want to tell the client you couldn't find a representation for the target resource then send 404

Let's say that you have a nested resource/path that will return devices owned by the employee so you will have this path:

`GET /api/v1/employees/1/devices/1` and this might return the first device owned by the employee.

Now what do you think the following request should return:

`GET /api/v1/employees/100/devices/1` where the employee with ID 100 does not exists but there is a device with the ID 1 owned by some other employee?

or

`GET /api/v1/employees/100/devices/1000` where both the employee with id 100 and device with id 1000 does not exists?

What do you think the response should be? Still 200?

So you're asking for the device with id 1 owned by employee 100. The answer is that the device exists but is not owned by employee 100 because there's no employee 100. So return 200 plus however you want to represent "the device exists but is not owned by employee 100 because there's no employee 100".

>`GET /api/v1/employees/100/devices/1000` where both the employee with id 100 and device with id 1000 does not exists?

Same as above but subbing in "because employee 100 and device 1000 don't exist" as appropriate

Nonexistence is not a representation of the target resource. A resource that does not exist does not have a representation.

It means that the server did not find a current representation of the resource and that the server accepts responsibility for providing an authoritative answer to whether it exists (if the latter is not the case, the most correct response is 421.) Aside from that and the combination of being unwilling to provide a default representation, not having a representation consistent with the Accept header provided by the client, and preferring not to distinguish this case from non-existence with a 406 response (which, like the situations to which 421 applies, is an edge case), the reason for a resource not being found is overwhelmingly that it does not, in fact, exist.

It is true that there are some other things that a 404 might mean, but “does not exist” is not only within the space of things covered by “Not Found”, it is by far the most common reason for it.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

As others have mentioned, you can return anything you want in the body of a 404 to clarify. However the other issues you all mentioned should really be in the 500s with errors.

This isn't how the world works though. Much development happens on COTS platforms or internal frameworks or whatever. Many of those will have generic error handling, logging, alerting, etc. based off the response code.

Your pedantically correct 404 with a clarifying body ruins my day (really weeks) because now I have to chase whoever built those things to fix my problem. I am annoyed and curse your name.

One topic the article didn't even touch was flagging bad parameters in the request which didn't make sense on the application level. HTTP has 422 for that. I commonly write things like

    validate_my_param($c->req->params->{format})
       or $c->detach(HTTP => 422, ['Invalid format']);

I've used that for years and been quite happy with the downstream results.

Edit: I should elaborate on the downstream results.

On the javascript side, the ajax methods often contain separate success and failure callbacks. You often need to handle the failure callback to ensure the user knows that something broke. If you also have an error path in the success callback, it clutters the code.

    ajax({
      success(data, ...) {
        if (data.wasActuallyAnError) {
          // lines
          // of code
          // to handle
          // the error
        }
        else useTheData(data);
      },
      error(response, ...) {
        // lines
        // of code
        // to handle
        // the error
      }
    })

    ajax({
      success(data, ...) {
        useTheData(data);
      },
      error(response, ...) {
        // lines
        // of code
        // to handle
        // the error
      }
    })

Oh sure. I'll just call up the CTO and tell them to scrap the product they spent $10 million on because it's not pedantically correct in parsing HTML codes.

Yes, and then parse the appropriate response payload, in the context of the response code. Thats how eg. you get validation errors for a submitted form that just failed submission. If you're just assuming that everything !=200 is an error, it is your own assumption, not a framework shortcoming. As an example, most of the complex API systems I've developed would actually ignore completely the response code (so, basically the opposite of what you assumed), because the responses need to have further context information in case of errors, such as application-specific error code, context-specific error messages (eg. form field errors) or just translation-aware detailed error messages.

On a vaguely related note, most people working with HTTP API implementations seems to forget that GET requests can in fact have a request body. Most high-level clients/libraries will assume you won't use it, but it doesn't mean you can't use it.

No webserver will ever return a 404 for any of those cases.

Why, in all fairness, we always knew. It's an overloading of an existing technology, a bit hacky as it is, and stuff like GRPC are the answer to this, but still the simplicity of HTTP and REST are the demise of standardization.

REST is an architectural style, not a standard.

> It's an overloading of an existing technology, a bit hacky as it is

REST is not tied to a particular technology or protocol, and is not “overloading” whatever protocol(s) it is used with; the architectural style specifies using the subset of the available features of the underlying protocols that corresponds to REST semantics consistent with the specifications of those protocols.